2024-04-23T18:07:05.263 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T18:07:05.263 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} -- /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112/orig.config.yaml 2024-04-23T18:07:05.328 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T18:07:05.426 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7670112' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 57e0caf76fdb7ab8b1358588f292d08519163844 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5182 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 57e0caf76fdb7ab8b1358588f292d08519163844 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMtCpsB6fHHOWERpbqC88N84w83JhNicLksVn6GJEKBbciPrWOJ6s7iRJoaCO8meQ+jJVl/9z2r/kPIYr0WeTvI= smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSVB0DQq+JmjHsFTVlHc5+vgWW/txHUjTvJWMEyeQ4/gNLBLAMkjMc7qu1eq4qym1XvcczxFj1mWFEIbpRJ8= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDctVtOE1ycotqxHuvNAnBa3fEolSGV8vPpwsJsClYmYG3Fkg+E9SKzam0URIPIMw44k4iQ7itHePSrp1slMcE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_17:44:29 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T18:07:05.426 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa; will attempt to use it 2024-04-23T18:07:05.426 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks 2024-04-23T18:07:05.427 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T18:07:05.428 INFO:teuthology.task.internal:Checking packages... 2024-04-23T18:07:05.451 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-23T18:07:05.452 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T18:07:05.452 INFO:teuthology.packaging:ref: None 2024-04-23T18:07:05.452 INFO:teuthology.packaging:tag: None 2024-04-23T18:07:05.452 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-23T18:07:05.452 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:07:05.452 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-23T18:07:05.693 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-23T18:07:05.695 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T18:07:05.701 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T18:07:05.701 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T18:07:05.708 INFO:teuthology.task.internal:Saving configuration 2024-04-23T18:07:05.730 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T18:07:05.736 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T18:07:05.759 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:01:38.081400', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMtCpsB6fHHOWERpbqC88N84w83JhNicLksVn6GJEKBbciPrWOJ6s7iRJoaCO8meQ+jJVl/9z2r/kPIYr0WeTvI='} 2024-04-23T18:07:05.782 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:01:38.083181', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSVB0DQq+JmjHsFTVlHc5+vgWW/txHUjTvJWMEyeQ4/gNLBLAMkjMc7qu1eq4qym1XvcczxFj1mWFEIbpRJ8='} 2024-04-23T18:07:05.802 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:01:38.084741', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDctVtOE1ycotqxHuvNAnBa3fEolSGV8vPpwsJsClYmYG3Fkg+E9SKzam0URIPIMw44k4iQ7itHePSrp1slMcE='} 2024-04-23T18:07:05.802 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T18:07:05.809 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-23T18:07:05.809 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-23T18:07:05.809 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-23T18:07:05.809 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T18:07:05.919 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0f8712b0d0>, signals=[15]) 2024-04-23T18:07:05.919 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T18:07:05.925 INFO:teuthology.task.internal:Opening connections... 2024-04-23T18:07:05.925 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:07:05.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:07:06.010 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-04-23T18:07:06.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:07:06.084 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-04-23T18:07:06.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:07:06.158 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T18:07:06.165 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-04-23T18:07:06.183 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-04-23T18:07:06.183 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-04-23T18:07:06.241 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-04-23T18:07:06.242 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-04-23T18:07:06.242 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-04-23T18:07:06.242 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:07:06.242 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-04-23T18:07:06.242 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:07:06.242 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:07:06.242 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-04-23T18:07:06.242 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:07:06.242 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:07:06.243 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:07:06.243 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:07:06.243 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:07:06.243 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:07:06.243 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-04-23T18:07:06.279 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-04-23T18:07:06.297 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-04-23T18:07:06.298 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-04-23T18:07:06.356 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-04-23T18:07:06.356 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-04-23T18:07:06.356 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:07:06.357 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:07:06.357 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-04-23T18:07:06.387 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-04-23T18:07:06.405 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-04-23T18:07:06.406 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-04-23T18:07:06.463 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2024-04-23T18:07:06.463 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2024-04-23T18:07:06.463 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2024-04-23T18:07:06.463 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:07:06.463 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2024-04-23T18:07:06.463 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:07:06.464 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:07:06.464 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-04-23T18:07:06.464 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:07:06.464 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:07:06.464 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:07:06.464 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:07:06.464 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:07:06.464 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:07:06.464 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-04-23T18:07:06.495 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T18:07:06.506 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T18:07:06.512 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T18:07:06.512 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:07:06.515 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:07:06.517 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:07:06.533 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T18:07:06.539 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T18:07:06.539 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:07:06.572 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:07:06.575 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:07:06.659 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T18:07:06.734 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T18:07:06.750 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-23T18:07:06.750 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-23T18:07:06.750 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-23T18:07:06.751 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:07:06.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:07:06.768 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-04-23T18:07:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-437.el9.x86_64 2024-04-23T18:07:06.826 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-437.el9.x86_64 2024-04-23T18:07:06.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-04-23T18:07:09.082 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-23T18:07:12.166 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 19 MB 00:01 2024-04-23T18:07:24.249 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 752 kB/s | 6.2 MB 00:08 2024-04-23T18:07:25.984 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-23T18:07:27.967 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-23T18:07:33.251 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-23T18:07:35.157 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T18:07:35.157 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T18:07:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-23T18:07:35.261 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-04-23T18:07:35.261 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-23T18:07:35.371 DEBUG:teuthology.orchestra.run.smithi019:> echo no | sudo yum reinstall kernel || true 2024-04-23T18:07:35.934 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:02 ago on Tue 23 Apr 2024 06:07:33 PM UTC. 2024-04-23T18:07:36.064 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-23T18:07:36.072 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-23T18:07:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-04-23T18:07:36.072 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-23T18:07:36.072 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalling: 2024-04-23T18:07:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T18:07:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:07:36.072 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-23T18:07:36.072 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-23T18:07:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:07:36.073 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 255 k 2024-04-23T18:07:36.073 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 0 2024-04-23T18:07:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:Operation aborted. 2024-04-23T18:07:36.130 INFO:teuthology.orchestra.run.smithi019.stdout:Is this ok [y/N]: 2024-04-23T18:07:36.131 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum reinstall -y kernel || true 2024-04-23T18:07:36.644 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 06:07:33 PM UTC. 2024-04-23T18:07:36.765 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-23T18:07:36.766 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-23T18:07:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-04-23T18:07:36.766 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-23T18:07:36.766 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalling: 2024-04-23T18:07:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T18:07:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:07:36.766 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-23T18:07:36.766 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-23T18:07:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:07:36.766 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 255 k 2024-04-23T18:07:36.767 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 0 2024-04-23T18:07:36.767 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-04-23T18:07:37.086 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-23T18:07:37.086 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:07:37.087 INFO:teuthology.orchestra.run.smithi019.stdout:Total 797 kB/s | 255 kB 00:00 2024-04-23T18:07:37.107 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-04-23T18:07:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-04-23T18:07:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-04-23T18:07:37.304 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-04-23T18:07:37.304 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-04-23T18:07:37.442 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-04-23T18:07:37.520 INFO:teuthology.orchestra.run.smithi019.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T18:07:37.520 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:07:38.178 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:07:38.179 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T18:07:38.885 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:07:38.885 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:07:38.885 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalled: 2024-04-23T18:07:38.885 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T18:07:38.885 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:07:38.885 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-23T18:07:39.051 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T18:07:39.099 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T18:07:39.100 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T18:07:39.100 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T18:07:39.100 DEBUG:teuthology.parallel:result is None 2024-04-23T18:07:39.100 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T18:07:39.108 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T18:07:39.109 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:07:39.111 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:07:39.114 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:07:39.132 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T18:07:39.143 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T18:07:39.157 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T18:07:39.157 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:07:39.167 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:07:39.173 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:07:39.206 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T18:07:39.219 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T18:07:39.219 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:07:39.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:07:39.251 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:07:39.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:07:39.268 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:07:39.270 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:07:39.293 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:07:39.315 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:07:39.326 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:07:39.332 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:07:39.341 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:07:39.345 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:07:39.368 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:07:39.370 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T18:07:39.379 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T18:07:39.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:07:39.381 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:07:39.392 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:07:39.438 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T18:07:39.448 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T18:07:39.448 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:07:39.450 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:07:39.459 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:07:39.495 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:07:39.546 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:07:39.601 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:07:39.681 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:07:39.723 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:07:39.723 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:07:39.787 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:07:39.832 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:07:39.885 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:07:39.971 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:07:40.011 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:07:40.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:07:40.078 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:07:40.125 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:07:40.173 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:07:40.258 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:07:40.299 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:07:40.299 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:07:40.365 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-04-23T18:07:40.370 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-04-23T18:07:40.372 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-04-23T18:07:40.410 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:07:40.411 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:07:40.444 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:07:40.859 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T18:07:40.867 INFO:teuthology.task.internal:Starting timer... 2024-04-23T18:07:40.867 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T18:07:40.879 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T18:07:40.892 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-23T18:07:40.893 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-04-23T18:07:40.948 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-04-23T18:07:40.950 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-04-23T18:07:40.995 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-04-23T18:07:40.998 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-04-23T18:07:41.066 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-04-23T18:07:41.068 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T18:07:41.068 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-04-23T18:07:41.095 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-04-23T18:07:41.095 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-04-23T18:07:41.125 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-04-23T18:07:41.126 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-04-23T18:07:41.149 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-04-23T18:07:41.149 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2024-04-23T18:07:41.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:07:41.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:07:41.177 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:07:41.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:07:41.205 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:07:41.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:07:41.232 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T18:07:41.232 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:07:41.258 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:07:41.284 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:07:41.312 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T18:07:41.321 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T18:07:41.339 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T18:07:41.340 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi044.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T18:11:39.375 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-04-23T18:11:39.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-23T18:11:39.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:11:39.458 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-04-23T18:11:39.542 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-23T18:11:39.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-23T18:11:39.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:11:39.617 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-04-23T18:11:39.703 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-23T18:11:39.703 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-23T18:11:39.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:11:39.778 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-04-23T18:11:39.862 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-23T18:11:39.862 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T18:11:39.875 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T18:11:39.875 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:11:39.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:11:39.879 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:11:39.879 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:11:39.881 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:11:39.881 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:11:39.909 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:11:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:11:39.925 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:11:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:11:39.931 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:11:39.944 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:11:39.951 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-04-23T18:11:39.952 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-04-23T18:11:39.962 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-04-23T18:11:39.963 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-04-23T18:11:39.968 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-04-23T18:11:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:11:39.978 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:11:39.978 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-04-23T18:11:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:11:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:11:39.992 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:11:40.005 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:11:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:11:40.050 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:11:40.050 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-23T18:11:40.050 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.050 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.050 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.050 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.051 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:11:40.054 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:11:40.054 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-23T18:11:40.054 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.054 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.055 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.055 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.066 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:11:40.071 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:11:40.071 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-04-23T18:11:40.071 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.071 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.071 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.071 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:11:40.071 INFO:teuthology.run_tasks:Running task install... 2024-04-23T18:11:40.080 DEBUG:teuthology.task.install:project ceph 2024-04-23T18:11:40.080 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-23T18:11:40.080 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-23T18:11:40.081 INFO:teuthology.task.install:Using flavor: default 2024-04-23T18:11:40.087 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T18:11:40.087 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-23T18:11:40.087 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:11:40.088 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:11:40.090 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:11:40.090 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:11:40.091 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:11:40.091 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:11:40.318 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:11:40.318 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:11:40.322 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:11:40.322 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:11:40.325 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:11:40.325 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:11:40.524 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:11:40.524 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:11:40.524 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:11:40.549 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:11:40.549 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:11:40.590 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:11:40.590 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:11:40.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:11:40.597 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:11:40.597 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:11:40.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:11:40.618 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:11:40.623 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:11:40.624 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:11:40.625 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:11:40.626 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:11:40.691 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:11:40.693 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:11:40.694 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:11:40.755 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-04-23T18:11:40.756 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-04-23T18:11:40.771 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:11:40.772 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:11:40.836 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-04-23T18:11:40.837 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-04-23T18:11:40.838 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-04-23T18:11:40.840 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-04-23T18:11:40.980 INFO:teuthology.orchestra.run.smithi155.stdout:48 files removed 2024-04-23T18:11:41.007 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:11:41.075 INFO:teuthology.orchestra.run.smithi044.stdout:48 files removed 2024-04-23T18:11:41.083 INFO:teuthology.orchestra.run.smithi019.stdout:48 files removed 2024-04-23T18:11:41.102 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:11:41.122 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:11:41.697 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 194 kB/s | 83 kB 00:00 2024-04-23T18:11:41.898 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 159 kB/s | 83 kB 00:00 2024-04-23T18:11:41.911 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 162 kB/s | 83 kB 00:00 2024-04-23T18:11:42.182 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-23T18:11:42.382 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-04-23T18:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-23T18:11:42.749 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-04-23T18:11:42.837 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-23T18:11:42.882 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-04-23T18:11:43.365 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-23T18:11:43.466 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-23T18:11:45.617 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-23T18:11:49.684 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-23T18:11:51.018 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 16 kB 00:00 2024-04-23T18:11:51.920 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 911 kB/s | 8.0 MB 00:09 2024-04-23T18:11:53.436 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 21 MB 00:02 2024-04-23T18:11:53.738 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-23T18:11:58.364 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.2 MB 00:00 2024-04-23T18:11:58.502 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-23T18:12:00.524 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 16 kB 00:00 2024-04-23T18:12:00.587 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:12:00.588 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:12:00.599 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:12:00.600 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:12:00.600 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:12:00.601 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:12:00.603 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:12:00.605 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:12:00.606 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:12:00.665 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-23T18:12:00.681 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-04-23T18:12:00.681 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-04-23T18:12:00.681 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-04-23T18:12:00.681 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-23T18:12:00.681 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:12:00.681 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:12:00.681 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:12:00.681 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:12:00.681 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:12:00.681 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:12:00.682 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:12:00.683 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-04-23T18:12:00.684 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:12:00.685 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:12:00.686 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:12:00.687 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:12:00.688 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:12:00.689 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:12:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:12:00.691 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:12:00.692 INFO:teuthology.orchestra.run.smithi155.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout:Install 148 Packages 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 4 Packages 2024-04-23T18:12:00.693 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:12:00.694 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 222 M 2024-04-23T18:12:00.694 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-23T18:12:02.070 INFO:teuthology.orchestra.run.smithi155.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-23T18:12:02.454 INFO:teuthology.orchestra.run.smithi155.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.3 MB/s | 890 kB 00:00 2024-04-23T18:12:02.521 INFO:teuthology.orchestra.run.smithi155.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-23T18:12:03.113 INFO:teuthology.orchestra.run.smithi155.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 4.0 MB/s | 2.4 MB 00:00 2024-04-23T18:12:03.413 INFO:teuthology.orchestra.run.smithi155.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 5.0 MB/s | 1.5 MB 00:00 2024-04-23T18:12:04.015 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 21 MB 00:03 2024-04-23T18:12:04.505 INFO:teuthology.orchestra.run.smithi155.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 4.3 MB/s | 4.7 MB 00:01 2024-04-23T18:12:04.655 INFO:teuthology.orchestra.run.smithi155.stdout:(7/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.9 MB/s | 5.4 MB 00:02 2024-04-23T18:12:06.447 INFO:teuthology.orchestra.run.smithi155.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 4.5 MB/s | 21 MB 00:04 2024-04-23T18:12:06.514 INFO:teuthology.orchestra.run.smithi155.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 380 kB/s | 25 kB 00:00 2024-04-23T18:12:08.890 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 801 kB/s | 19 MB 00:24 2024-04-23T18:12:09.249 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-23T18:12:09.288 INFO:teuthology.orchestra.run.smithi155.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 3.5 MB/s | 17 MB 00:04 2024-04-23T18:12:09.413 INFO:teuthology.orchestra.run.smithi155.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.8 MB/s | 225 kB 00:00 2024-04-23T18:12:09.480 INFO:teuthology.orchestra.run.smithi155.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 406 kB/s | 27 kB 00:00 2024-04-23T18:12:09.540 INFO:teuthology.orchestra.run.smithi155.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 544 kB/s | 32 kB 00:00 2024-04-23T18:12:09.739 INFO:teuthology.orchestra.run.smithi155.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.6 MB/s | 747 kB 00:00 2024-04-23T18:12:09.798 INFO:teuthology.orchestra.run.smithi155.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 163 kB 00:00 2024-04-23T18:12:09.856 INFO:teuthology.orchestra.run.smithi155.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 2.1 MB/s | 126 kB 00:00 2024-04-23T18:12:09.990 INFO:teuthology.orchestra.run.smithi155.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 3.7 MB/s | 500 kB 00:00 2024-04-23T18:12:10.665 INFO:teuthology.orchestra.run.smithi155.stdout:(18/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 1.8 MB/s | 11 MB 00:06 2024-04-23T18:12:10.782 INFO:teuthology.orchestra.run.smithi155.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 387 kB/s | 45 kB 00:00 2024-04-23T18:12:10.932 INFO:teuthology.orchestra.run.smithi155.stdout:(20/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 5.7 MB/s | 5.4 MB 00:00 2024-04-23T18:12:10.958 INFO:teuthology.orchestra.run.smithi155.stdout:(21/152): python3-ceph-common-19.0.0-3156.gd69f 769 kB/s | 135 kB 00:00 2024-04-23T18:12:10.983 INFO:teuthology.orchestra.run.smithi155.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 3.1 MB/s | 161 kB 00:00 2024-04-23T18:12:11.075 INFO:teuthology.orchestra.run.smithi155.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 298 kB 00:00 2024-04-23T18:12:11.125 INFO:teuthology.orchestra.run.smithi155.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.9 MB/s | 99 kB 00:00 2024-04-23T18:12:11.192 INFO:teuthology.orchestra.run.smithi155.stdout:(25/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.3 MB/s | 320 kB 00:00 2024-04-23T18:12:11.217 INFO:teuthology.orchestra.run.smithi155.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 926 kB/s | 85 kB 00:00 2024-04-23T18:12:11.285 INFO:teuthology.orchestra.run.smithi155.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 170 kB 00:00 2024-04-23T18:12:11.335 INFO:teuthology.orchestra.run.smithi155.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 681 kB/s | 34 kB 00:00 2024-04-23T18:12:11.394 INFO:teuthology.orchestra.run.smithi155.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.4 MB/s | 142 kB 00:00 2024-04-23T18:12:11.510 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:12:11.510 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:12:11.522 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:12:11.522 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:12:11.523 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:12:11.524 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:12:11.526 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:12:11.528 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:12:11.529 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:12:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-23T18:12:11.602 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-23T18:12:11.602 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-04-23T18:12:11.602 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-23T18:12:11.602 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-04-23T18:12:11.602 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:12:11.603 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:12:11.604 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:12:11.605 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:12:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:12:11.607 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:12:11.609 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:12:11.610 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:12:11.611 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:12:11.612 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:12:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:12:11.614 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:12:11.614 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-04-23T18:12:11.614 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:12:11.614 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:12:11.614 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-23T18:12:11.614 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-23T18:12:11.614 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-04-23T18:12:11.614 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 4 Packages 2024-04-23T18:12:11.614 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:12:11.615 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 222 M 2024-04-23T18:12:11.615 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-04-23T18:12:12.136 INFO:teuthology.orchestra.run.smithi155.stdout:(30/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 8.7 MB/s | 49 MB 00:05 2024-04-23T18:12:12.203 INFO:teuthology.orchestra.run.smithi155.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.4 MB/s | 3.5 MB 00:00 2024-04-23T18:12:12.278 INFO:teuthology.orchestra.run.smithi155.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 3.2 MB/s | 249 kB 00:00 2024-04-23T18:12:12.328 INFO:teuthology.orchestra.run.smithi155.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 990 kB/s | 49 kB 00:00 2024-04-23T18:12:12.370 INFO:teuthology.orchestra.run.smithi155.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 395 kB/s | 16 kB 00:00 2024-04-23T18:12:12.471 INFO:teuthology.orchestra.run.smithi155.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.7 MB/s | 273 kB 00:00 2024-04-23T18:12:12.613 INFO:teuthology.orchestra.run.smithi155.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.3 MB/s | 762 kB 00:00 2024-04-23T18:12:12.663 INFO:teuthology.orchestra.run.smithi155.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 366 kB/s | 18 kB 00:00 2024-04-23T18:12:12.813 INFO:teuthology.orchestra.run.smithi155.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-23T18:12:12.838 INFO:teuthology.orchestra.run.smithi155.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 458 kB/s | 80 kB 00:00 2024-04-23T18:12:12.897 INFO:teuthology.orchestra.run.smithi155.stdout:(40/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 1.8 MB/s | 3.1 MB 00:01 2024-04-23T18:12:13.031 INFO:teuthology.orchestra.run.smithi155.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-23T18:12:13.056 INFO:teuthology.orchestra.run.smithi155.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-23T18:12:13.081 INFO:teuthology.orchestra.run.smithi155.stdout:(43/152): libquadmath-11.4.1-3.el9.x86_64.rpm 806 kB/s | 195 kB 00:00 2024-04-23T18:12:13.106 INFO:teuthology.orchestra.run.smithi155.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-23T18:12:13.132 INFO:teuthology.orchestra.run.smithi155.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-23T18:12:13.157 INFO:teuthology.orchestra.run.smithi155.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-23T18:12:13.191 INFO:teuthology.orchestra.run.smithi155.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.2 MB 00:00 2024-04-23T18:12:13.216 INFO:teuthology.orchestra.run.smithi155.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-23T18:12:13.516 INFO:teuthology.orchestra.run.smithi155.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 356 kB/s | 107 kB 00:00 2024-04-23T18:12:13.599 INFO:teuthology.orchestra.run.smithi155.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 258 kB/s | 21 kB 00:00 2024-04-23T18:12:13.700 INFO:teuthology.orchestra.run.smithi155.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 689 kB 00:00 2024-04-23T18:12:13.825 INFO:teuthology.orchestra.run.smithi155.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 943 kB 00:00 2024-04-23T18:12:13.850 INFO:teuthology.orchestra.run.smithi155.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 218 kB/s | 33 kB 00:00 2024-04-23T18:12:13.884 INFO:teuthology.orchestra.run.smithi155.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-04-23T18:12:13.935 INFO:teuthology.orchestra.run.smithi155.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 217 kB/s | 18 kB 00:00 2024-04-23T18:12:13.968 INFO:teuthology.orchestra.run.smithi155.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-04-23T18:12:14.011 INFO:teuthology.orchestra.run.smithi155.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-23T18:12:14.044 INFO:teuthology.orchestra.run.smithi155.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-04-23T18:12:14.202 INFO:teuthology.orchestra.run.smithi155.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-04-23T18:12:14.283 INFO:teuthology.orchestra.run.smithi019.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-23T18:12:14.294 INFO:teuthology.orchestra.run.smithi155.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 278 kB/s | 25 kB 00:00 2024-04-23T18:12:14.319 INFO:teuthology.orchestra.run.smithi155.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-04-23T18:12:14.378 INFO:teuthology.orchestra.run.smithi155.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-04-23T18:12:14.478 INFO:teuthology.orchestra.run.smithi155.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-23T18:12:14.504 INFO:teuthology.orchestra.run.smithi155.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-04-23T18:12:14.550 INFO:teuthology.orchestra.run.smithi019.stdout:(2/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 8.9 MB/s | 5.4 MB 00:00 2024-04-23T18:12:14.554 INFO:teuthology.orchestra.run.smithi155.stdout:(65/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.1 MB/s | 3.0 MB 00:00 2024-04-23T18:12:14.571 INFO:teuthology.orchestra.run.smithi155.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 383 kB/s | 35 kB 00:00 2024-04-23T18:12:14.600 INFO:teuthology.orchestra.run.smithi019.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.8 MB/s | 145 kB 00:00 2024-04-23T18:12:14.792 INFO:teuthology.orchestra.run.smithi019.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 12 MB/s | 2.4 MB 00:00 2024-04-23T18:12:14.834 INFO:teuthology.orchestra.run.smithi019.stdout:(5/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 890 kB 00:00 2024-04-23T18:12:14.934 INFO:teuthology.orchestra.run.smithi019.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 1.5 MB 00:00 2024-04-23T18:12:15.013 INFO:teuthology.orchestra.run.smithi155.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-04-23T18:12:15.230 INFO:teuthology.orchestra.run.smithi155.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 4.1 MB/s | 911 kB 00:00 2024-04-23T18:12:15.322 INFO:teuthology.orchestra.run.smithi155.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-23T18:12:15.447 INFO:teuthology.orchestra.run.smithi155.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-04-23T18:12:15.680 INFO:teuthology.orchestra.run.smithi155.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-04-23T18:12:15.747 INFO:teuthology.orchestra.run.smithi155.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-23T18:12:15.885 INFO:teuthology.orchestra.run.smithi019.stdout:(7/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 18 MB/s | 17 MB 00:00 2024-04-23T18:12:15.973 INFO:teuthology.orchestra.run.smithi155.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 8.9 MB/s | 5.8 MB 00:00 2024-04-23T18:12:16.039 INFO:teuthology.orchestra.run.smithi155.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-23T18:12:16.106 INFO:teuthology.orchestra.run.smithi155.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-23T18:12:16.173 INFO:teuthology.orchestra.run.smithi155.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 385 kB/s | 26 kB 00:00 2024-04-23T18:12:16.232 INFO:teuthology.orchestra.run.smithi155.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 330 kB/s | 19 kB 00:00 2024-04-23T18:12:16.335 INFO:teuthology.orchestra.run.smithi019.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.1 MB/s | 4.7 MB 00:01 2024-04-23T18:12:16.374 INFO:teuthology.orchestra.run.smithi155.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-04-23T18:12:16.458 INFO:teuthology.orchestra.run.smithi155.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 417 kB/s | 35 kB 00:00 2024-04-23T18:12:16.460 INFO:teuthology.orchestra.run.smithi019.stdout:(9/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 18 MB/s | 11 MB 00:00 2024-04-23T18:12:16.485 INFO:teuthology.orchestra.run.smithi019.stdout:(10/152): ceph-selinux-19.0.0-3156.gd69f13d8.el 168 kB/s | 25 kB 00:00 2024-04-23T18:12:16.800 INFO:teuthology.orchestra.run.smithi155.stdout:(80/152): openblas-openmp-0.3.21-2.el9.x86_64.r 2.1 MB/s | 4.7 MB 00:02 2024-04-23T18:12:17.058 INFO:teuthology.orchestra.run.smithi155.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 971 kB/s | 250 kB 00:00 2024-04-23T18:12:17.225 INFO:teuthology.orchestra.run.smithi155.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-04-23T18:12:17.317 INFO:teuthology.orchestra.run.smithi155.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 843 kB/s | 77 kB 00:00 2024-04-23T18:12:17.386 INFO:teuthology.orchestra.run.smithi155.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.5 MB/s | 5.1 MB 00:00 2024-04-23T18:12:17.427 INFO:teuthology.orchestra.run.smithi019.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 239 kB/s | 225 kB 00:00 2024-04-23T18:12:17.461 INFO:teuthology.orchestra.run.smithi155.stdout:(85/152): python3-babel-2.9.1-2.el9.noarch.rpm 3.5 MB/s | 6.0 MB 00:01 2024-04-23T18:12:17.486 INFO:teuthology.orchestra.run.smithi155.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 250 kB/s | 42 kB 00:00 2024-04-23T18:12:17.521 INFO:teuthology.orchestra.run.smithi155.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-04-23T18:12:17.527 INFO:teuthology.orchestra.run.smithi019.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 270 kB/s | 27 kB 00:00 2024-04-23T18:12:17.578 INFO:teuthology.orchestra.run.smithi155.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 876 kB/s | 51 kB 00:00 2024-04-23T18:12:17.604 INFO:teuthology.orchestra.run.smithi155.stdout:(89/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-04-23T18:12:17.628 INFO:teuthology.orchestra.run.smithi019.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 318 kB/s | 32 kB 00:00 2024-04-23T18:12:17.637 INFO:teuthology.orchestra.run.smithi155.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-04-23T18:12:17.696 INFO:teuthology.orchestra.run.smithi155.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-04-23T18:12:17.722 INFO:teuthology.orchestra.run.smithi155.stdout:(92/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.2 MB/s | 279 kB 00:00 2024-04-23T18:12:17.755 INFO:teuthology.orchestra.run.smithi155.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-04-23T18:12:17.822 INFO:teuthology.orchestra.run.smithi155.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 539 kB/s | 36 kB 00:00 2024-04-23T18:12:17.872 INFO:teuthology.orchestra.run.smithi155.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-04-23T18:12:17.914 INFO:teuthology.orchestra.run.smithi155.stdout:(96/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-04-23T18:12:17.981 INFO:teuthology.orchestra.run.smithi155.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-04-23T18:12:18.006 INFO:teuthology.orchestra.run.smithi155.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 629 kB/s | 58 kB 00:00 2024-04-23T18:12:18.040 INFO:teuthology.orchestra.run.smithi155.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-23T18:12:18.065 INFO:teuthology.orchestra.run.smithi155.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-23T18:12:18.107 INFO:teuthology.orchestra.run.smithi155.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-04-23T18:12:18.157 INFO:teuthology.orchestra.run.smithi155.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-04-23T18:12:18.658 INFO:teuthology.orchestra.run.smithi155.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.5 MB/s | 863 kB 00:00 2024-04-23T18:12:18.903 INFO:teuthology.orchestra.run.smithi019.stdout:(14/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 20 MB/s | 49 MB 00:02 2024-04-23T18:12:18.928 INFO:teuthology.orchestra.run.smithi019.stdout:(15/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 574 kB/s | 747 kB 00:01 2024-04-23T18:12:18.953 INFO:teuthology.orchestra.run.smithi019.stdout:(16/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 3.2 MB/s | 163 kB 00:00 2024-04-23T18:12:18.966 INFO:teuthology.orchestra.run.smithi155.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 678 kB/s | 548 kB 00:00 2024-04-23T18:12:18.991 INFO:teuthology.orchestra.run.smithi155.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 926 kB/s | 309 kB 00:00 2024-04-23T18:12:19.012 INFO:teuthology.orchestra.run.smithi019.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 8.4 MB/s | 500 kB 00:00 2024-04-23T18:12:19.050 INFO:teuthology.orchestra.run.smithi155.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 234 kB/s | 19 kB 00:00 2024-04-23T18:12:19.079 INFO:teuthology.orchestra.run.smithi019.stdout:(18/152): librados-devel-19.0.0-3156.gd69f13d8. 842 kB/s | 126 kB 00:00 2024-04-23T18:12:19.204 INFO:teuthology.orchestra.run.smithi019.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 361 kB/s | 45 kB 00:00 2024-04-23T18:12:19.305 INFO:teuthology.orchestra.run.smithi019.stdout:(20/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 18 MB/s | 5.4 MB 00:00 2024-04-23T18:12:19.355 INFO:teuthology.orchestra.run.smithi019.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 3.2 MB/s | 161 kB 00:00 2024-04-23T18:12:19.384 INFO:teuthology.orchestra.run.smithi155.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 78 kB/s | 26 kB 00:00 2024-04-23T18:12:19.414 INFO:teuthology.orchestra.run.smithi019.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 5.4 MB/s | 320 kB 00:00 2024-04-23T18:12:19.439 INFO:teuthology.orchestra.run.smithi019.stdout:(23/152): python3-ceph-common-19.0.0-3156.gd69f 574 kB/s | 135 kB 00:00 2024-04-23T18:12:19.450 INFO:teuthology.orchestra.run.smithi155.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-04-23T18:12:19.464 INFO:teuthology.orchestra.run.smithi019.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 5.8 MB/s | 298 kB 00:00 2024-04-23T18:12:19.476 INFO:teuthology.orchestra.run.smithi155.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 543 kB/s | 49 kB 00:00 2024-04-23T18:12:19.506 INFO:teuthology.orchestra.run.smithi019.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.0 MB/s | 85 kB 00:00 2024-04-23T18:12:19.534 INFO:teuthology.orchestra.run.smithi155.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 808 kB/s | 67 kB 00:00 2024-04-23T18:12:19.540 INFO:teuthology.orchestra.run.smithi019.stdout:(26/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 992 kB/s | 99 kB 00:00 2024-04-23T18:12:19.560 INFO:teuthology.orchestra.run.smithi155.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-23T18:12:19.635 INFO:teuthology.orchestra.run.smithi155.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 838 kB 00:00 2024-04-23T18:12:19.660 INFO:teuthology.orchestra.run.smithi155.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-04-23T18:12:19.682 INFO:teuthology.orchestra.run.smithi019.stdout:(27/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 18 MB/s | 3.1 MB 00:00 2024-04-23T18:12:19.719 INFO:teuthology.orchestra.run.smithi155.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 526 kB/s | 43 kB 00:00 2024-04-23T18:12:19.724 INFO:teuthology.orchestra.run.smithi019.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 825 kB/s | 34 kB 00:00 2024-04-23T18:12:19.744 INFO:teuthology.orchestra.run.smithi155.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 384 kB/s | 32 kB 00:00 2024-04-23T18:12:19.749 INFO:teuthology.orchestra.run.smithi019.stdout:(29/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 816 kB/s | 170 kB 00:00 2024-04-23T18:12:19.803 INFO:teuthology.orchestra.run.smithi155.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2024-04-23T18:12:19.828 INFO:teuthology.orchestra.run.smithi155.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-23T18:12:19.899 INFO:teuthology.orchestra.run.smithi019.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 812 kB/s | 142 kB 00:00 2024-04-23T18:12:19.903 INFO:teuthology.orchestra.run.smithi155.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-23T18:12:19.929 INFO:teuthology.orchestra.run.smithi155.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-23T18:12:19.979 INFO:teuthology.orchestra.run.smithi155.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 474 kB/s | 35 kB 00:00 2024-04-23T18:12:20.000 INFO:teuthology.orchestra.run.smithi019.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 14 MB/s | 3.5 MB 00:00 2024-04-23T18:12:20.004 INFO:teuthology.orchestra.run.smithi155.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-23T18:12:20.042 INFO:teuthology.orchestra.run.smithi019.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 5.8 MB/s | 249 kB 00:00 2024-04-23T18:12:20.075 INFO:teuthology.orchestra.run.smithi019.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 49 kB 00:00 2024-04-23T18:12:20.109 INFO:teuthology.orchestra.run.smithi019.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 493 kB/s | 16 kB 00:00 2024-04-23T18:12:20.113 INFO:teuthology.orchestra.run.smithi155.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-04-23T18:12:20.138 INFO:teuthology.orchestra.run.smithi155.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-23T18:12:20.151 INFO:teuthology.orchestra.run.smithi019.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 6.4 MB/s | 273 kB 00:00 2024-04-23T18:12:20.172 INFO:teuthology.orchestra.run.smithi155.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-04-23T18:12:20.214 INFO:teuthology.orchestra.run.smithi155.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-23T18:12:20.226 INFO:teuthology.orchestra.run.smithi019.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 10 MB/s | 762 kB 00:00 2024-04-23T18:12:20.248 INFO:teuthology.orchestra.run.smithi155.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-23T18:12:20.260 INFO:teuthology.orchestra.run.smithi019.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 547 kB/s | 18 kB 00:00 2024-04-23T18:12:20.289 INFO:teuthology.orchestra.run.smithi155.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-23T18:12:20.314 INFO:teuthology.orchestra.run.smithi155.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-04-23T18:12:20.390 INFO:teuthology.orchestra.run.smithi155.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-23T18:12:20.415 INFO:teuthology.orchestra.run.smithi155.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-04-23T18:12:20.465 INFO:teuthology.orchestra.run.smithi155.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-23T18:12:20.491 INFO:teuthology.orchestra.run.smithi155.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-04-23T18:12:20.541 INFO:teuthology.orchestra.run.smithi155.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-04-23T18:12:20.566 INFO:teuthology.orchestra.run.smithi155.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-23T18:12:20.616 INFO:teuthology.orchestra.run.smithi155.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-23T18:12:20.642 INFO:teuthology.orchestra.run.smithi155.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-23T18:12:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 189 kB/s | 80 kB 00:00 2024-04-23T18:12:20.692 INFO:teuthology.orchestra.run.smithi155.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-04-23T18:12:20.717 INFO:teuthology.orchestra.run.smithi155.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-23T18:12:20.767 INFO:teuthology.orchestra.run.smithi155.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 791 kB/s | 59 kB 00:00 2024-04-23T18:12:20.793 INFO:teuthology.orchestra.run.smithi155.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-04-23T18:12:20.843 INFO:teuthology.orchestra.run.smithi155.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-04-23T18:12:20.869 INFO:teuthology.orchestra.run.smithi155.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-23T18:12:20.919 INFO:teuthology.orchestra.run.smithi155.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-23T18:12:20.961 INFO:teuthology.orchestra.run.smithi155.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-23T18:12:20.995 INFO:teuthology.orchestra.run.smithi155.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-04-23T18:12:21.037 INFO:teuthology.orchestra.run.smithi155.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-23T18:12:21.112 INFO:teuthology.orchestra.run.smithi155.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-23T18:12:21.135 INFO:teuthology.orchestra.run.smithi019.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-04-23T18:12:21.244 INFO:teuthology.orchestra.run.smithi019.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-23T18:12:21.352 INFO:teuthology.orchestra.run.smithi019.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-23T18:12:21.604 INFO:teuthology.orchestra.run.smithi155.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 6.0 MB/s | 3.4 MB 00:00 2024-04-23T18:12:21.645 INFO:teuthology.orchestra.run.smithi019.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-04-23T18:12:21.761 INFO:teuthology.orchestra.run.smithi019.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 914 kB/s | 106 kB 00:00 2024-04-23T18:12:21.870 INFO:teuthology.orchestra.run.smithi019.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-23T18:12:21.940 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 678 kB/s | 6.2 MB 00:09 2024-04-23T18:12:21.954 INFO:teuthology.orchestra.run.smithi155.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.9 MB/s | 1.3 MB 00:00 2024-04-23T18:12:21.987 INFO:teuthology.orchestra.run.smithi019.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-23T18:12:22.095 INFO:teuthology.orchestra.run.smithi019.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-23T18:12:22.188 INFO:teuthology.orchestra.run.smithi155.stdout:(150/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.9 MB/s | 3.1 MB 00:01 2024-04-23T18:12:22.238 INFO:teuthology.orchestra.run.smithi155.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 2.4 MB 00:00 2024-04-23T18:12:22.404 INFO:teuthology.orchestra.run.smithi019.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-04-23T18:12:22.646 INFO:teuthology.orchestra.run.smithi019.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-04-23T18:12:22.705 INFO:teuthology.orchestra.run.smithi019.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-23T18:12:22.763 INFO:teuthology.orchestra.run.smithi019.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 370 kB/s | 21 kB 00:00 2024-04-23T18:12:22.855 INFO:teuthology.orchestra.run.smithi019.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-04-23T18:12:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 562 kB/s | 33 kB 00:00 2024-04-23T18:12:23.256 INFO:teuthology.orchestra.run.smithi019.stdout:(53/152): ceph-mgr-diskprediction-local-19.0.0- 2.2 MB/s | 7.4 MB 00:03 2024-04-23T18:12:23.356 INFO:teuthology.orchestra.run.smithi019.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-04-23T18:12:23.423 INFO:teuthology.orchestra.run.smithi019.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-23T18:12:23.581 INFO:teuthology.orchestra.run.smithi019.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-23T18:12:23.640 INFO:teuthology.orchestra.run.smithi019.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-04-23T18:12:23.799 INFO:teuthology.orchestra.run.smithi019.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.4 MB/s | 3.0 MB 00:00 2024-04-23T18:12:23.838 INFO:teuthology.orchestra.run.smithi155.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.6 MB/s | 16 MB 00:06 2024-04-23T18:12:23.842 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:12:23.842 INFO:teuthology.orchestra.run.smithi155.stdout:Total 9.6 MB/s | 222 MB 00:23 2024-04-23T18:12:23.857 INFO:teuthology.orchestra.run.smithi019.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-04-23T18:12:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-04-23T18:12:23.958 INFO:teuthology.orchestra.run.smithi019.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-04-23T18:12:23.983 INFO:teuthology.orchestra.run.smithi019.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 710 kB/s | 47 kB 00:00 2024-04-23T18:12:24.042 INFO:teuthology.orchestra.run.smithi019.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-23T18:12:24.070 INFO:teuthology.orchestra.run.smithi019.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-23T18:12:24.109 INFO:teuthology.orchestra.run.smithi019.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 525 kB/s | 35 kB 00:00 2024-04-23T18:12:24.176 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 16 kB 00:01 2024-04-23T18:12:24.904 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-23T18:12:25.046 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-23T18:12:25.046 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-23T18:12:25.134 INFO:teuthology.orchestra.run.smithi019.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-04-23T18:12:25.493 INFO:teuthology.orchestra.run.smithi019.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-04-23T18:12:25.710 INFO:teuthology.orchestra.run.smithi019.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 2.9 MB/s | 4.7 MB 00:01 2024-04-23T18:12:25.743 INFO:teuthology.orchestra.run.smithi019.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 3.6 MB/s | 911 kB 00:00 2024-04-23T18:12:25.802 INFO:teuthology.orchestra.run.smithi019.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-23T18:12:26.102 INFO:teuthology.orchestra.run.smithi019.stdout:(71/152): ceph-common-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 21 MB 00:12 2024-04-23T18:12:26.286 INFO:teuthology.orchestra.run.smithi019.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-04-23T18:12:26.353 INFO:teuthology.orchestra.run.smithi019.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2024-04-23T18:12:26.503 INFO:teuthology.orchestra.run.smithi019.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-04-23T18:12:26.645 INFO:teuthology.orchestra.run.smithi019.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-23T18:12:26.703 INFO:teuthology.orchestra.run.smithi019.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-04-23T18:12:26.771 INFO:teuthology.orchestra.run.smithi019.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-04-23T18:12:26.910 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-23T18:12:26.910 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-23T18:12:26.987 INFO:teuthology.orchestra.run.smithi019.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 794 kB/s | 172 kB 00:00 2024-04-23T18:12:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 522 kB/s | 35 kB 00:00 2024-04-23T18:12:27.313 INFO:teuthology.orchestra.run.smithi019.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 3.7 MB/s | 5.8 MB 00:01 2024-04-23T18:12:27.480 INFO:teuthology.orchestra.run.smithi019.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-04-23T18:12:27.597 INFO:teuthology.orchestra.run.smithi019.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-04-23T18:12:27.664 INFO:teuthology.orchestra.run.smithi019.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-23T18:12:27.731 INFO:teuthology.orchestra.run.smithi019.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 633 kB/s | 42 kB 00:00 2024-04-23T18:12:27.864 INFO:teuthology.orchestra.run.smithi019.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-04-23T18:12:27.931 INFO:teuthology.orchestra.run.smithi019.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-23T18:12:28.040 INFO:teuthology.orchestra.run.smithi019.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-23T18:12:28.174 INFO:teuthology.orchestra.run.smithi019.stdout:(88/152): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-04-23T18:12:28.190 INFO:teuthology.orchestra.run.smithi019.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 339 kB/s | 51 kB 00:00 2024-04-23T18:12:28.241 INFO:teuthology.orchestra.run.smithi019.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 805 kB/s | 54 kB 00:00 2024-04-23T18:12:28.308 INFO:teuthology.orchestra.run.smithi019.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 625 kB/s | 42 kB 00:00 2024-04-23T18:12:28.369 INFO:teuthology.orchestra.run.smithi019.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-04-23T18:12:28.467 INFO:teuthology.orchestra.run.smithi019.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-04-23T18:12:28.534 INFO:teuthology.orchestra.run.smithi019.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 536 kB/s | 36 kB 00:00 2024-04-23T18:12:28.601 INFO:teuthology.orchestra.run.smithi019.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 713 kB/s | 47 kB 00:00 2024-04-23T18:12:28.709 INFO:teuthology.orchestra.run.smithi019.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 305 kB 00:00 2024-04-23T18:12:28.778 INFO:teuthology.orchestra.run.smithi019.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 868 kB/s | 58 kB 00:00 2024-04-23T18:12:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-23T18:12:28.910 INFO:teuthology.orchestra.run.smithi019.stdout:(99/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.7 MB/s | 5.1 MB 00:01 2024-04-23T18:12:28.935 INFO:teuthology.orchestra.run.smithi019.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 694 kB/s | 64 kB 00:00 2024-04-23T18:12:28.977 INFO:teuthology.orchestra.run.smithi019.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-04-23T18:12:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 333 kB/s | 22 kB 00:00 2024-04-23T18:12:29.245 INFO:teuthology.orchestra.run.smithi019.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-04-23T18:12:29.278 INFO:teuthology.orchestra.run.smithi019.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-04-23T18:12:29.303 INFO:teuthology.orchestra.run.smithi019.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-04-23T18:12:29.404 INFO:teuthology.orchestra.run.smithi019.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-04-23T18:12:29.454 INFO:teuthology.orchestra.run.smithi019.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-04-23T18:12:29.512 INFO:teuthology.orchestra.run.smithi019.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 836 kB/s | 49 kB 00:00 2024-04-23T18:12:29.571 INFO:teuthology.orchestra.run.smithi019.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-23T18:12:29.663 INFO:teuthology.orchestra.run.smithi019.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-23T18:12:29.730 INFO:teuthology.orchestra.run.smithi019.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-23T18:12:29.797 INFO:teuthology.orchestra.run.smithi019.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-04-23T18:12:29.831 INFO:teuthology.orchestra.run.smithi019.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-23T18:12:29.856 INFO:teuthology.orchestra.run.smithi019.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 745 kB/s | 43 kB 00:00 2024-04-23T18:12:29.898 INFO:teuthology.orchestra.run.smithi019.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 483 kB/s | 32 kB 00:00 2024-04-23T18:12:29.923 INFO:teuthology.orchestra.run.smithi019.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 223 kB/s | 15 kB 00:00 2024-04-23T18:12:29.956 INFO:teuthology.orchestra.run.smithi019.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-23T18:12:29.982 INFO:teuthology.orchestra.run.smithi019.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-23T18:12:30.024 INFO:teuthology.orchestra.run.smithi019.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-23T18:12:30.049 INFO:teuthology.orchestra.run.smithi019.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 526 kB/s | 35 kB 00:00 2024-04-23T18:12:30.082 INFO:teuthology.orchestra.run.smithi019.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-23T18:12:30.175 INFO:teuthology.orchestra.run.smithi019.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-23T18:12:30.233 INFO:teuthology.orchestra.run.smithi019.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-04-23T18:12:30.284 INFO:teuthology.orchestra.run.smithi019.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-04-23T18:12:30.309 INFO:teuthology.orchestra.run.smithi019.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-23T18:12:30.343 INFO:teuthology.orchestra.run.smithi019.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-04-23T18:12:30.368 INFO:teuthology.orchestra.run.smithi019.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 333 kB/s | 19 kB 00:00 2024-04-23T18:12:30.393 INFO:teuthology.orchestra.run.smithi019.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-23T18:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 21 MB 00:06 2024-04-23T18:12:30.452 INFO:teuthology.orchestra.run.smithi019.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 796 kB/s | 46 kB 00:00 2024-04-23T18:12:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-23T18:12:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:(131/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.6 MB/s | 16 MB 00:02 2024-04-23T18:12:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 325 kB/s | 79 kB 00:00 2024-04-23T18:12:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 256 kB/s | 58 kB 00:00 2024-04-23T18:12:30.745 INFO:teuthology.orchestra.run.smithi019.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-04-23T18:12:30.770 INFO:teuthology.orchestra.run.smithi019.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-23T18:12:30.796 INFO:teuthology.orchestra.run.smithi019.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-04-23T18:12:30.821 INFO:teuthology.orchestra.run.smithi019.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-04-23T18:12:30.854 INFO:teuthology.orchestra.run.smithi019.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-23T18:12:30.871 INFO:teuthology.orchestra.run.smithi019.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-04-23T18:12:30.905 INFO:teuthology.orchestra.run.smithi019.stdout:(140/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-04-23T18:12:30.930 INFO:teuthology.orchestra.run.smithi019.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-23T18:12:30.955 INFO:teuthology.orchestra.run.smithi019.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-04-23T18:12:30.989 INFO:teuthology.orchestra.run.smithi019.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-23T18:12:31.014 INFO:teuthology.orchestra.run.smithi019.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-23T18:12:31.048 INFO:teuthology.orchestra.run.smithi019.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-04-23T18:12:31.073 INFO:teuthology.orchestra.run.smithi019.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-23T18:12:31.098 INFO:teuthology.orchestra.run.smithi019.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-23T18:12:31.232 INFO:teuthology.orchestra.run.smithi019.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-23T18:12:31.400 INFO:teuthology.orchestra.run.smithi019.stdout:(149/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 10 MB/s | 3.1 MB 00:00 2024-04-23T18:12:32.307 INFO:teuthology.orchestra.run.smithi019.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-04-23T18:12:32.558 INFO:teuthology.orchestra.run.smithi019.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-04-23T18:12:32.824 INFO:teuthology.orchestra.run.smithi019.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.9 MB/s | 3.4 MB 00:01 2024-04-23T18:12:32.827 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:12:32.827 INFO:teuthology.orchestra.run.smithi019.stdout:Total 10 MB/s | 222 MB 00:21 2024-04-23T18:12:32.870 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-23T18:12:32.949 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:12:32.996 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:12:33.141 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:12:33.184 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:12:33.209 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:12:33.252 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:12:33.590 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:12:33.710 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:12:33.832 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:12:33.861 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:12:33.952 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:12:33.963 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-04-23T18:12:33.982 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:12:34.122 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-04-23T18:12:34.122 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-04-23T18:12:34.130 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:12:34.179 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:12:34.215 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:12:34.318 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:12:34.350 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:12:34.395 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:12:34.426 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:12:34.447 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:12:34.626 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:12:34.667 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:12:34.783 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:12:34.879 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:12:34.908 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:12:34.942 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:12:35.008 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:12:35.041 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:12:35.085 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:12:35.118 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:12:35.145 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:12:35.214 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:12:35.268 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:12:35.298 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:12:35.342 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:12:35.418 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:12:35.453 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:12:35.496 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:12:35.526 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:12:35.594 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:12:35.639 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:12:35.683 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-23T18:12:35.713 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:12:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:12:35.908 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:12:35.993 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-04-23T18:12:35.994 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-04-23T18:12:36.434 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:12:36.528 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:12:36.558 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:12:36.600 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:12:36.638 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:12:36.680 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:12:36.843 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:12:36.908 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:12:36.971 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:12:37.043 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:12:37.075 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:12:37.120 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:12:37.166 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:12:37.202 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:12:37.246 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:12:37.287 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:12:37.432 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:12:37.476 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:12:37.507 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:12:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:12:37.642 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:12:37.673 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:12:37.713 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:12:37.747 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:12:37.780 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:12:37.854 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:12:37.877 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:12:37.878 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:12:37.889 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:12:37.890 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:12:37.891 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:12:37.892 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:12:37.893 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:12:37.896 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:12:37.897 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:12:37.963 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-23T18:12:37.981 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:12:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:12:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:12:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:12:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:12:37.988 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:12:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:12:37.990 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:12:37.991 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:12:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:12:37.993 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:12:37.994 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout:Install 148 Packages 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 4 Packages 2024-04-23T18:12:37.995 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:12:37.996 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 222 M 2024-04-23T18:12:37.996 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-23T18:12:38.012 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:12:38.039 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:12:38.074 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:12:38.113 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:12:38.145 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:12:38.186 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:12:38.226 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:12:38.259 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:12:38.300 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:12:38.335 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:12:38.422 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:12:38.465 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:12:38.505 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:12:38.560 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:12:38.658 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:12:38.706 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:12:39.432 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:12:39.487 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:12:39.528 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:12:39.627 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:12:39.686 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:12:39.773 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:12:39.805 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:12:40.000 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:12:40.036 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:12:40.123 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:12:40.765 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:12:40.901 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:12:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-04-23T18:12:42.103 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:12:42.123 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:12:42.142 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:12:42.194 INFO:teuthology.orchestra.run.smithi044.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-23T18:12:42.196 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:12:42.206 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:12:42.234 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:12:42.238 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:12:42.263 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:12:42.267 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:12:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:12:42.501 INFO:teuthology.orchestra.run.smithi044.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.8 MB/s | 890 kB 00:00 2024-04-23T18:12:42.561 INFO:teuthology.orchestra.run.smithi044.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.4 MB/s | 145 kB 00:00 2024-04-23T18:12:42.581 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:12:42.622 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:12:42.652 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:12:42.703 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:12:42.735 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:12:42.779 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:12:42.780 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:12:42.878 INFO:teuthology.orchestra.run.smithi044.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 7.5 MB/s | 2.4 MB 00:00 2024-04-23T18:12:42.905 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:12:42.940 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:12:43.023 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:12:43.061 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 7.8 MB/s | 1.5 MB 00:00 2024-04-23T18:12:43.227 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:12:43.267 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:12:43.283 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:12:43.299 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:12:43.310 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:12:43.371 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:12:43.401 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:12:43.415 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:12:43.444 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:12:43.507 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:12:43.530 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:12:43.550 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 8.3 MB/s | 4.7 MB 00:00 2024-04-23T18:12:43.738 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:12:43.770 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:12:43.882 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:12:43.978 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 11 MB/s | 21 MB 00:01 2024-04-23T18:12:44.011 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:12:44.046 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:12:44.122 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:12:44.170 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:12:44.214 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:12:44.246 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:12:44.273 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:12:44.345 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:12:44.397 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:12:44.426 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:12:44.462 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:12:44.487 INFO:teuthology.orchestra.run.smithi044.stdout:(8/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.2 MB/s | 5.4 MB 00:02 2024-04-23T18:12:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:12:44.572 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:12:44.616 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:12:44.629 INFO:teuthology.orchestra.run.smithi044.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 178 kB/s | 25 kB 00:00 2024-04-23T18:12:44.644 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:12:44.719 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:12:44.768 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:12:44.813 INFO:teuthology.orchestra.run.smithi044.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 13 MB/s | 11 MB 00:00 2024-04-23T18:12:44.845 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:12:44.896 INFO:teuthology.orchestra.run.smithi044.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.6 MB/s | 225 kB 00:00 2024-04-23T18:12:44.963 INFO:teuthology.orchestra.run.smithi044.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 407 kB/s | 27 kB 00:00 2024-04-23T18:12:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:12:45.038 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:12:45.130 INFO:teuthology.orchestra.run.smithi044.stdout:(13/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 11 MB/s | 17 MB 00:01 2024-04-23T18:12:45.155 INFO:teuthology.orchestra.run.smithi044.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 165 kB/s | 32 kB 00:00 2024-04-23T18:12:45.206 INFO:teuthology.orchestra.run.smithi044.stdout:(15/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 9.7 MB/s | 747 kB 00:00 2024-04-23T18:12:45.231 INFO:teuthology.orchestra.run.smithi044.stdout:(16/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.1 MB/s | 163 kB 00:00 2024-04-23T18:12:45.256 INFO:teuthology.orchestra.run.smithi044.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 2.5 MB/s | 126 kB 00:00 2024-04-23T18:12:45.281 INFO:teuthology.orchestra.run.smithi044.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 9.8 MB/s | 500 kB 00:00 2024-04-23T18:12:45.348 INFO:teuthology.orchestra.run.smithi044.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 676 kB/s | 45 kB 00:00 2024-04-23T18:12:45.423 INFO:teuthology.orchestra.run.smithi044.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.8 MB/s | 135 kB 00:00 2024-04-23T18:12:45.540 INFO:teuthology.orchestra.run.smithi044.stdout:(21/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 19 MB/s | 5.4 MB 00:00 2024-04-23T18:12:45.557 INFO:teuthology.orchestra.run.smithi044.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.2 MB/s | 161 kB 00:00 2024-04-23T18:12:45.591 INFO:teuthology.orchestra.run.smithi044.stdout:(23/152): python3-rados-19.0.0-3156.gd69f13d8.e 6.3 MB/s | 320 kB 00:00 2024-04-23T18:12:45.600 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:12:45.616 INFO:teuthology.orchestra.run.smithi044.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 5.0 MB/s | 298 kB 00:00 2024-04-23T18:12:45.641 INFO:teuthology.orchestra.run.smithi044.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.9 MB/s | 99 kB 00:00 2024-04-23T18:12:45.667 INFO:teuthology.orchestra.run.smithi044.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.7 MB/s | 85 kB 00:00 2024-04-23T18:12:45.685 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:12:45.696 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:12:45.719 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:12:45.739 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:12:45.752 INFO:teuthology.orchestra.run.smithi044.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 170 kB 00:00 2024-04-23T18:12:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:12:45.792 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:12:45.819 INFO:teuthology.orchestra.run.smithi044.stdout:(28/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 17 MB/s | 3.1 MB 00:00 2024-04-23T18:12:45.843 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:12:45.844 INFO:teuthology.orchestra.run.smithi044.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 372 kB/s | 34 kB 00:00 2024-04-23T18:12:45.870 INFO:teuthology.orchestra.run.smithi044.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.8 MB/s | 142 kB 00:00 2024-04-23T18:12:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:12:46.070 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:12:46.113 INFO:teuthology.orchestra.run.smithi044.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 13 MB/s | 3.5 MB 00:00 2024-04-23T18:12:46.127 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:12:46.187 INFO:teuthology.orchestra.run.smithi044.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 3.2 MB/s | 249 kB 00:00 2024-04-23T18:12:46.200 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:12:46.237 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:12:46.274 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:12:46.279 INFO:teuthology.orchestra.run.smithi044.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 539 kB/s | 49 kB 00:00 2024-04-23T18:12:46.282 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:12:46.321 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:12:46.325 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:12:46.365 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:12:46.371 INFO:teuthology.orchestra.run.smithi044.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-04-23T18:12:46.397 INFO:teuthology.orchestra.run.smithi044.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 140 kB/s | 16 kB 00:00 2024-04-23T18:12:46.408 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:12:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:12:46.416 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:12:46.416 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:12:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:12:46.421 INFO:teuthology.orchestra.run.smithi044.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 5.3 MB/s | 273 kB 00:00 2024-04-23T18:12:46.443 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:12:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:12:46.464 INFO:teuthology.orchestra.run.smithi044.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 439 kB/s | 18 kB 00:00 2024-04-23T18:12:46.489 INFO:teuthology.orchestra.run.smithi044.stdout:(38/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 8.1 MB/s | 762 kB 00:00 2024-04-23T18:12:46.604 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:12:46.614 INFO:teuthology.orchestra.run.smithi044.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-04-23T18:12:46.647 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:12:46.677 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:12:46.681 INFO:teuthology.orchestra.run.smithi044.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-23T18:12:46.719 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:12:46.731 INFO:teuthology.orchestra.run.smithi044.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-23T18:12:46.765 INFO:teuthology.orchestra.run.smithi044.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-04-23T18:12:46.816 INFO:teuthology.orchestra.run.smithi044.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-23T18:12:46.818 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:12:46.869 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:12:46.909 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:12:46.942 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:12:46.967 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:12:47.045 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:12:47.149 INFO:teuthology.orchestra.run.smithi044.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-04-23T18:12:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:12:47.199 INFO:teuthology.orchestra.run.smithi044.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-23T18:12:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:12:47.225 INFO:teuthology.orchestra.run.smithi044.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 331 kB/s | 135 kB 00:00 2024-04-23T18:12:47.250 INFO:teuthology.orchestra.run.smithi044.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-23T18:12:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:12:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:12:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:12:47.366 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:12:47.396 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:12:47.400 INFO:teuthology.orchestra.run.smithi044.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-04-23T18:12:47.438 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:12:47.442 INFO:teuthology.orchestra.run.smithi044.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-23T18:12:47.468 INFO:teuthology.orchestra.run.smithi044.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-04-23T18:12:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:12:47.493 INFO:teuthology.orchestra.run.smithi044.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-04-23T18:12:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:12:47.518 INFO:teuthology.orchestra.run.smithi044.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-23T18:12:47.543 INFO:teuthology.orchestra.run.smithi044.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-04-23T18:12:47.548 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 549 kB/s | 18 kB 00:00 2024-04-23T18:12:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-23T18:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-04-23T18:12:47.686 INFO:teuthology.orchestra.run.smithi044.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-23T18:12:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:12:47.712 INFO:teuthology.orchestra.run.smithi044.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-04-23T18:12:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:12:47.745 INFO:teuthology.orchestra.run.smithi044.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-23T18:12:47.774 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:12:47.779 INFO:teuthology.orchestra.run.smithi044.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-23T18:12:47.804 INFO:teuthology.orchestra.run.smithi044.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-04-23T18:12:47.829 INFO:teuthology.orchestra.run.smithi044.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-23T18:12:47.855 INFO:teuthology.orchestra.run.smithi044.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-23T18:12:47.886 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:12:47.888 INFO:teuthology.orchestra.run.smithi044.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-23T18:12:47.913 INFO:teuthology.orchestra.run.smithi044.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 605 kB/s | 35 kB 00:00 2024-04-23T18:12:47.937 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:12:47.997 INFO:teuthology.orchestra.run.smithi044.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 44 MB/s | 4.7 MB 00:00 2024-04-23T18:12:48.073 INFO:teuthology.orchestra.run.smithi044.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-04-23T18:12:48.114 INFO:teuthology.orchestra.run.smithi044.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-23T18:12:48.148 INFO:teuthology.orchestra.run.smithi044.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-23T18:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-23T18:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-04-23T18:12:48.299 INFO:teuthology.orchestra.run.smithi044.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 38 MB/s | 5.8 MB 00:00 2024-04-23T18:12:48.324 INFO:teuthology.orchestra.run.smithi044.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-04-23T18:12:48.366 INFO:teuthology.orchestra.run.smithi044.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-23T18:12:48.450 INFO:teuthology.orchestra.run.smithi044.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-04-23T18:12:48.475 INFO:teuthology.orchestra.run.smithi044.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-23T18:12:48.500 INFO:teuthology.orchestra.run.smithi044.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-04-23T18:12:48.525 INFO:teuthology.orchestra.run.smithi044.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-23T18:12:48.551 INFO:teuthology.orchestra.run.smithi044.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-23T18:12:48.576 INFO:teuthology.orchestra.run.smithi044.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-04-23T18:12:48.618 INFO:teuthology.orchestra.run.smithi044.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-23T18:12:48.693 INFO:teuthology.orchestra.run.smithi044.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-04-23T18:12:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:12:48.719 INFO:teuthology.orchestra.run.smithi044.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-23T18:12:48.753 INFO:teuthology.orchestra.run.smithi044.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-23T18:12:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:12:48.778 INFO:teuthology.orchestra.run.smithi044.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-04-23T18:12:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:12:48.803 INFO:teuthology.orchestra.run.smithi044.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-23T18:12:48.828 INFO:teuthology.orchestra.run.smithi044.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-23T18:12:48.854 INFO:teuthology.orchestra.run.smithi044.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-23T18:12:48.870 INFO:teuthology.orchestra.run.smithi044.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-23T18:12:48.887 INFO:teuthology.orchestra.run.smithi044.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-23T18:12:48.895 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:12:48.921 INFO:teuthology.orchestra.run.smithi044.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-04-23T18:12:48.950 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:12:48.955 INFO:teuthology.orchestra.run.smithi044.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-04-23T18:12:48.997 INFO:teuthology.orchestra.run.smithi044.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-23T18:12:49.031 INFO:teuthology.orchestra.run.smithi044.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-04-23T18:12:49.046 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:12:49.077 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:12:49.198 INFO:teuthology.orchestra.run.smithi044.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 50 MB/s | 16 MB 00:00 2024-04-23T18:12:49.239 INFO:teuthology.orchestra.run.smithi044.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 227 kB/s | 47 kB 00:00 2024-04-23T18:12:49.262 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:12:49.265 INFO:teuthology.orchestra.run.smithi044.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-23T18:12:49.290 INFO:teuthology.orchestra.run.smithi044.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-23T18:12:49.291 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:12:49.315 INFO:teuthology.orchestra.run.smithi044.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-23T18:12:49.340 INFO:teuthology.orchestra.run.smithi044.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-23T18:12:49.365 INFO:teuthology.orchestra.run.smithi044.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-04-23T18:12:49.371 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:12:49.391 INFO:teuthology.orchestra.run.smithi044.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-04-23T18:12:49.433 INFO:teuthology.orchestra.run.smithi044.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-23T18:12:49.661 INFO:teuthology.orchestra.run.smithi044.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-23T18:12:49.683 INFO:teuthology.orchestra.run.smithi044.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.4 MB/s | 309 kB 00:00 2024-04-23T18:12:49.709 INFO:teuthology.orchestra.run.smithi044.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-04-23T18:12:49.759 INFO:teuthology.orchestra.run.smithi044.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-04-23T18:12:49.809 INFO:teuthology.orchestra.run.smithi044.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 972 kB/s | 49 kB 00:00 2024-04-23T18:12:49.868 INFO:teuthology.orchestra.run.smithi044.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-23T18:12:49.943 INFO:teuthology.orchestra.run.smithi044.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-23T18:12:50.052 INFO:teuthology.orchestra.run.smithi044.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-04-23T18:12:50.065 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:12:50.168 INFO:teuthology.orchestra.run.smithi044.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-04-23T18:12:50.177 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:12:50.194 INFO:teuthology.orchestra.run.smithi044.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-04-23T18:12:50.228 INFO:teuthology.orchestra.run.smithi044.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 738 kB/s | 43 kB 00:00 2024-04-23T18:12:50.253 INFO:teuthology.orchestra.run.smithi044.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 546 kB/s | 32 kB 00:00 2024-04-23T18:12:50.278 INFO:teuthology.orchestra.run.smithi044.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-04-23T18:12:50.303 INFO:teuthology.orchestra.run.smithi044.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-23T18:12:50.346 INFO:teuthology.orchestra.run.smithi044.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-23T18:12:50.371 INFO:teuthology.orchestra.run.smithi044.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-23T18:12:50.396 INFO:teuthology.orchestra.run.smithi044.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 706 kB/s | 35 kB 00:00 2024-04-23T18:12:50.421 INFO:teuthology.orchestra.run.smithi044.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-23T18:12:50.472 INFO:teuthology.orchestra.run.smithi044.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-23T18:12:50.522 INFO:teuthology.orchestra.run.smithi044.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-04-23T18:12:50.564 INFO:teuthology.orchestra.run.smithi044.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-23T18:12:50.589 INFO:teuthology.orchestra.run.smithi044.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-23T18:12:50.614 INFO:teuthology.orchestra.run.smithi044.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-04-23T18:12:50.640 INFO:teuthology.orchestra.run.smithi044.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-04-23T18:12:50.665 INFO:teuthology.orchestra.run.smithi044.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-04-23T18:12:50.715 INFO:teuthology.orchestra.run.smithi044.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 926 kB/s | 46 kB 00:00 2024-04-23T18:12:50.748 INFO:teuthology.orchestra.run.smithi044.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-23T18:12:50.774 INFO:teuthology.orchestra.run.smithi044.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-23T18:12:50.799 INFO:teuthology.orchestra.run.smithi044.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-23T18:12:50.824 INFO:teuthology.orchestra.run.smithi044.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-23T18:12:50.850 INFO:teuthology.orchestra.run.smithi044.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-04-23T18:12:50.883 INFO:teuthology.orchestra.run.smithi044.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-23T18:12:50.908 INFO:teuthology.orchestra.run.smithi044.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-04-23T18:12:50.934 INFO:teuthology.orchestra.run.smithi044.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-04-23T18:12:50.959 INFO:teuthology.orchestra.run.smithi044.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-23T18:12:50.984 INFO:teuthology.orchestra.run.smithi044.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-23T18:12:51.009 INFO:teuthology.orchestra.run.smithi044.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-04-23T18:12:51.035 INFO:teuthology.orchestra.run.smithi044.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-23T18:12:51.060 INFO:teuthology.orchestra.run.smithi044.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-23T18:12:51.078 INFO:teuthology.orchestra.run.smithi044.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-04-23T18:12:51.110 INFO:teuthology.orchestra.run.smithi044.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 8.4 MB/s | 427 kB 00:00 2024-04-23T18:12:51.128 INFO:teuthology.orchestra.run.smithi044.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-04-23T18:12:51.153 INFO:teuthology.orchestra.run.smithi044.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 4.4 MB/s | 192 kB 00:00 2024-04-23T18:12:51.237 INFO:teuthology.orchestra.run.smithi044.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-23T18:12:51.487 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:12:51.504 INFO:teuthology.orchestra.run.smithi044.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 9.7 MB/s | 3.4 MB 00:00 2024-04-23T18:12:51.569 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:12:51.597 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:12:51.630 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:12:51.646 INFO:teuthology.orchestra.run.smithi044.stdout:(149/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 7.7 MB/s | 3.1 MB 00:00 2024-04-23T18:12:51.729 INFO:teuthology.orchestra.run.smithi044.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 1.3 MB 00:00 2024-04-23T18:12:51.960 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:12:52.001 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:12:52.093 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:12:52.123 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:12:52.160 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:12:52.271 INFO:teuthology.orchestra.run.smithi044.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 2.4 MB 00:00 2024-04-23T18:12:52.665 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:12:52.704 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:12:52.775 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:12:52.806 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:12:55.147 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:12:55.177 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:12:55.747 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:12:55.797 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:12:55.895 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:12:55.896 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:12:55.896 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:12:55.896 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:12:55.923 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:12:57.789 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:12:57.789 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-04-23T18:12:57.789 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-04-23T18:12:57.789 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-04-23T18:12:57.789 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-04-23T18:12:57.790 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-04-23T18:12:57.790 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /root 2024-04-23T18:12:57.790 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-04-23T18:12:57.790 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:12:57.970 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:12:57.986 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:12:57.986 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:12:57.986 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:12:57.986 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:12:59.481 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:12:59.507 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:12:59.507 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:12:59.507 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:12:59.508 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:12:59.508 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:12:59.508 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:12:59.732 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:12:59.754 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:12:59.754 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:12:59.754 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:12:59.754 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:12:59.755 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:12:59.755 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:13:00.199 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:13:00.221 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:13:00.221 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:00.221 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:13:00.221 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:13:00.221 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:13:00.221 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:13:00.321 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:13:00.377 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:13:00.433 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:13:00.627 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:13:00.675 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:13:00.752 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:13:00.841 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:13:00.980 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:13:01.024 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:13:01.056 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:13:01.084 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:13:01.553 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:13:01.582 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:13:01.615 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:13:01.676 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:13:01.706 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:13:02.743 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:13:02.790 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:13:03.755 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:13:03.783 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:13:03.884 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:13:04.054 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:13:04.093 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:13:04.116 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:13:04.116 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:04.116 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:13:04.116 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:13:04.116 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:13:04.116 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:13:04.160 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:13:04.171 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:13:04.269 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:13:04.317 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:13:05.243 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:13:05.279 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:13:05.302 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:13:05.302 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:05.302 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:13:05.302 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:13:05.303 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:13:05.303 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:13:05.347 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:13:05.368 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:13:05.369 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:05.369 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:13:05.369 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:13:05.414 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:13:05.436 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:13:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:13:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:13:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:13:05.437 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:13:05.740 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:13:05.762 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:13:05.762 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:05.762 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:13:05.762 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:13:05.762 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:13:05.762 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:13:05.812 INFO:teuthology.orchestra.run.smithi044.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 2.3 MB/s | 49 MB 00:21 2024-04-23T18:13:05.818 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:13:05.819 INFO:teuthology.orchestra.run.smithi044.stdout:Total 8.0 MB/s | 222 MB 00:27 2024-04-23T18:13:06.952 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-23T18:13:07.112 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-23T18:13:07.112 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-23T18:13:07.844 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:13:07.844 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-04-23T18:13:07.844 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-04-23T18:13:07.845 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-04-23T18:13:07.845 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-04-23T18:13:07.845 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-04-23T18:13:07.845 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-04-23T18:13:07.845 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-04-23T18:13:07.845 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:08.018 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:13:08.034 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:13:08.034 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:08.034 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:13:08.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:08.986 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-23T18:13:08.987 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-23T18:13:09.569 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:13:09.596 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:13:09.596 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:09.596 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:13:09.596 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:13:09.596 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:13:09.596 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:09.829 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:13:09.853 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:13:09.853 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:09.853 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:13:09.853 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:13:09.853 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:13:09.853 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:13:10.345 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:13:10.345 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:10.346 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:13:10.346 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:13:10.346 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:13:10.346 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:10.351 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:13:10.393 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:13:10.433 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:13:10.453 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:13:10.465 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:13:10.492 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:13:10.511 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:13:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:13:10.683 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:13:10.730 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:13:10.768 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:13:10.771 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:13:10.824 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:13:10.905 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:13:10.994 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:13:11.139 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:13:11.182 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:13:11.213 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:13:11.241 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:13:11.379 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:13:11.416 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:13:11.435 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:13:11.703 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:13:11.731 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:13:11.764 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:13:11.828 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:13:11.862 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:13:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:13:12.370 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:13:12.411 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:13:12.491 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:13:12.523 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:13:12.524 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:13:12.607 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:13:12.632 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:13:12.632 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:13:12.719 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:13:12.798 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:13:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:13:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:13:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:13:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:13:14.150 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:13:14.326 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:13:14.349 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:13:14.374 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:13:14.374 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:14.374 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:13:14.374 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:13:14.374 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:13:14.374 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:14.417 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:13:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:13:14.521 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:13:14.574 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:13:15.221 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-23T18:13:15.507 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:13:15.546 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:13:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:13:15.578 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:13:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:13:15.602 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:15.603 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:13:15.603 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:13:15.603 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:13:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:15.608 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:13:15.645 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:13:15.650 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:13:15.668 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:13:15.668 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:15.668 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:13:15.669 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:15.675 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:13:15.719 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:13:15.720 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:13:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:13:15.744 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:15.744 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:13:15.745 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:13:15.745 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:13:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:16.063 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:13:16.063 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:13:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:13:16.087 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:16.087 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:13:16.087 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:13:16.087 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:13:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:16.168 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:13:16.283 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:13:16.310 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:13:16.386 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:13:16.415 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:13:16.637 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:13:16.664 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:13:16.770 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:13:16.799 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:13:16.862 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:13:16.883 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:13:16.908 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:13:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:13:17.142 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:13:17.254 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:13:17.358 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:13:17.399 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:13:17.434 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:13:17.511 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:13:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:13:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:13:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:13:17.662 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:13:17.735 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:13:17.786 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:13:17.815 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:13:17.859 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:13:17.937 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:13:17.970 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:13:18.013 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:13:18.041 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:13:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:13:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:13:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:13:18.387 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:13:18.442 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:13:18.811 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:13:18.811 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:13:18.811 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:13:18.811 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:13:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:13:18.813 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:13:18.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:13:18.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:13:18.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:13:18.817 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:13:18.818 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:13:18.819 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:13:18.820 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:13:18.821 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:13:18.822 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:13:18.822 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:13:18.822 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:13:18.822 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:13:18.822 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:13:18.822 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:13:18.822 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:13:19.018 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:13:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:13:19.150 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:13:19.192 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:13:19.223 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:13:19.438 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:13:19.483 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:13:19.540 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:13:19.623 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:13:19.659 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:13:19.696 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:13:19.744 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:13:19.778 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:13:19.822 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:13:19.864 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:13:20.007 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:13:20.052 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:13:20.082 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:13:20.124 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:13:20.223 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:13:20.274 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:13:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:13:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:13:20.444 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:13:20.572 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:13:20.604 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:13:20.633 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:13:20.667 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:13:20.706 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:13:20.738 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:13:20.767 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:13:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:13:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:13:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:13:20.911 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:13:20.915 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:13:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.917 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.917 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.917 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:13:20.919 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:13:20.920 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:13:20.920 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:13:20.920 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:13:20.920 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:13:20.920 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:13:20.920 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.920 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.920 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:13:20.921 INFO:teuthology.orchestra.run.smithi155.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:13:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:13:20.923 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:13:20.924 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:13:20.925 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.926 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:13:20.927 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:13:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:13:20.929 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-23T18:13:20.944 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:13:20.985 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:13:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:13:21.026 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:13:21.058 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:13:21.059 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:13:21.077 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:13:21.098 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:13:21.146 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:13:21.257 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:13:21.271 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:13:21.309 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:13:21.314 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:13:21.340 DEBUG:teuthology.parallel:result is None 2024-04-23T18:13:21.344 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:13:22.058 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:13:22.085 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:13:22.100 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:13:22.121 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:13:22.138 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:13:22.163 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:13:22.270 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:13:22.323 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:13:22.413 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:13:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:13:22.634 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:13:22.662 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:13:22.742 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:13:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:13:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:13:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:13:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:13:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:13:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:13:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:13:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:13:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:13:23.435 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:13:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:13:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:13:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:13:24.874 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:13:24.959 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:13:24.987 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:13:25.019 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:13:25.348 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:13:25.390 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:13:25.481 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:13:25.512 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:13:25.548 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:13:26.052 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:13:26.076 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:13:26.147 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:13:26.170 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:13:28.566 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:13:28.599 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:13:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:13:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:13:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:13:29.305 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:13:29.305 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:13:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:13:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:13:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:13:29.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:13:29.358 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:13:29.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:13:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:13:29.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:13:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:13:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:13:29.364 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:13:29.365 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:13:29.366 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:13:29.366 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:13:31.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:13:31.413 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:31.413 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-04-23T18:13:31.413 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.414 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.415 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:13:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:13:31.418 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:13:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:13:31.421 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:13:31.421 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:13:31.421 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:13:31.421 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:13:31.421 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:13:31.421 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:13:31.421 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:13:31.421 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:13:31.421 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:13:31.421 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:13:31.422 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:13:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:13:31.425 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:13:31.425 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:13:31.425 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:13:31.425 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:13:31.425 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:13:31.425 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:13:31.425 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:13:31.425 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:13:31.425 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-23T18:13:31.839 DEBUG:teuthology.parallel:result is None 2024-04-23T18:13:41.370 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:13:41.370 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-04-23T18:13:41.370 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-04-23T18:13:41.370 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-04-23T18:13:41.370 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-04-23T18:13:41.371 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-04-23T18:13:41.371 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-04-23T18:13:41.371 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-04-23T18:13:41.371 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:41.550 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:13:41.565 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:13:41.565 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:41.566 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:13:41.566 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:13:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:13:43.130 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:43.130 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:13:43.130 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:13:43.130 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:13:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:13:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:13:43.394 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:43.394 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:13:43.394 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:13:43.394 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:13:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:13:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:13:43.886 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:43.886 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:13:43.887 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:13:43.887 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:13:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:13:44.052 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:13:44.115 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:13:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:13:44.372 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:13:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:13:44.542 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:13:44.687 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:13:44.722 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:13:44.754 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:13:44.789 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:13:45.219 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:13:45.249 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:13:45.280 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:13:45.343 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:13:45.378 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:13:46.492 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:13:46.529 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:13:47.554 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:13:47.705 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:13:47.815 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:13:47.985 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:13:48.015 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:13:48.039 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:13:48.039 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:48.039 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:13:48.039 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:13:48.039 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:13:48.040 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:48.091 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:13:48.102 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:13:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:13:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:13:49.238 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:13:49.269 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:13:49.293 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:13:49.293 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:49.293 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:13:49.293 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:13:49.293 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:13:49.293 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:49.336 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:13:49.359 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:13:49.359 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:49.360 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:13:49.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:49.412 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:13:49.436 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:13:49.436 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:49.436 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:13:49.436 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:13:49.437 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:13:49.437 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:49.762 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:13:49.787 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:13:49.787 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:13:49.787 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:13:49.788 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:13:49.788 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:13:49.788 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:13:54.674 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:13:54.716 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:13:54.758 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:13:54.789 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:13:54.816 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:13:55.019 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:13:55.062 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:13:55.083 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:13:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:13:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:13:55.810 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:13:56.811 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:13:56.854 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:13:56.947 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:13:56.982 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:13:56.982 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:13:57.066 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:13:57.099 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:13:57.099 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:13:57.258 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:14:03.293 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:14:03.294 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:14:03.294 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:14:03.294 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:14:03.294 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:14:03.294 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:14:03.294 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:14:03.296 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:14:03.311 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:14:03.311 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:14:03.311 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:14:03.311 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:14:03.311 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:14:03.311 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:14:03.312 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:14:03.313 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:14:03.314 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:14:03.315 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:14:03.316 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:14:03.317 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:14:03.318 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:14:03.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:14:03.320 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:14:03.320 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:14:05.386 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:14:05.386 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:14:05.386 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-04-23T18:14:05.386 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.386 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.386 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:14:05.386 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:14:05.386 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:14:05.387 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.388 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.389 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:14:05.391 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:14:05.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:14:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:14:05.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:14:05.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.396 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:14:05.397 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:14:05.397 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:14:05.397 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:14:05.397 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:14:05.397 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:14:05.397 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:14:05.397 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:14:05.397 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-23T18:14:05.398 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-23T18:14:05.768 DEBUG:teuthology.parallel:result is None 2024-04-23T18:14:05.769 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:14:05.914 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:14:05.941 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:14:05.941 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:14:05.941 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:14:05.944 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:14:06.080 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:14:06.102 INFO:teuthology.orchestra.run.smithi044.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:14:06.102 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:14:06.102 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:14:06.104 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:14:06.443 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:14:06.469 INFO:teuthology.orchestra.run.smithi155.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:14:06.469 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:14:06.469 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:14:06.471 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T18:14:06.471 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:14:06.471 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:14:06.503 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:14:06.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:14:06.530 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:14:06.530 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:14:06.559 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T18:14:06.560 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:14:06.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:14:06.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:14:06.659 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:14:06.659 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:14:06.683 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:14:06.748 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:14:06.748 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:14:06.776 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:14:06.844 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T18:14:06.845 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:14:06.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:14:06.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:14:06.943 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:14:06.943 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:14:06.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:14:07.033 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:14:07.033 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:14:07.057 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:14:07.121 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T18:14:07.121 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:14:07.121 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:14:07.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:14:07.219 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:14:07.219 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:14:07.244 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:14:07.308 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:14:07.309 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:14:07.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:14:07.398 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-23T18:14:07.503 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2}} 2024-04-23T18:14:07.503 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:14:07.504 INFO:tasks.cephadm:Cluster fsid is 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:14:07.504 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-23T18:14:07.504 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.19', 'mon.b': '172.21.15.44', 'mon.c': '172.21.15.155'} 2024-04-23T18:14:07.504 INFO:tasks.cephadm:First mon is mon.a on smithi019 2024-04-23T18:14:07.504 INFO:tasks.cephadm:First mgr is x 2024-04-23T18:14:07.504 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-23T18:14:07.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-04-23T18:14:07.534 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname $(hostname -s) 2024-04-23T18:14:07.559 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-04-23T18:14:07.584 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-23T18:14:07.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:14:07.729 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-23T18:14:07.866 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:14:07.868 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-23T18:14:07.868 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:14:08.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:14 /home/ubuntu/cephtest/cephadm 2024-04-23T18:14:08.431 DEBUG:teuthology.orchestra.run.smithi044:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:14:08.889 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:14 /home/ubuntu/cephtest/cephadm 2024-04-23T18:14:08.889 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:14:09.495 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:14 /home/ubuntu/cephtest/cephadm 2024-04-23T18:14:09.495 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:14:09.515 DEBUG:teuthology.orchestra.run.smithi044:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:14:09.534 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:14:09.570 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-23T18:14:09.571 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:14:09.574 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:14:09.577 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:14:11.068 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:14:11.179 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:14:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:14:43.734 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-23T18:14:43.734 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:14:43.734 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:14:43.734 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2024-04-23T18:14:43.734 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:14:43.735 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-04-23T18:14:43.735 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-23T18:15:19.299 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-23T18:15:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:15:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:15:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "repo_digests": [ 2024-04-23T18:15:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:15:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-23T18:15:19.300 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-23T18:15:22.122 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:15:22.122 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:15:22.122 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:15:22.122 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-04-23T18:15:22.123 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:15:22.123 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:15:22.123 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:15:22.148 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-04-23T18:15:22.188 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph 2024-04-23T18:15:22.221 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-04-23T18:15:22.254 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-04-23T18:15:22.278 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /etc/ceph 2024-04-23T18:15:22.307 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-04-23T18:15:22.335 INFO:tasks.cephadm:Writing seed config... 2024-04-23T18:15:22.336 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-23T18:15:22.336 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-23T18:15:22.336 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-23T18:15:22.336 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-23T18:15:22.336 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-23T18:15:22.336 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-23T18:15:22.336 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-23T18:15:22.337 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-23T18:15:22.338 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-23T18:15:22.339 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-23T18:15:22.340 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:15:22.340 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-23T18:15:22.355 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 46706838-019d-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-23T18:15:22.355 DEBUG:teuthology.orchestra.run.smithi019:mon.a> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mon.a.service 2024-04-23T18:15:22.398 DEBUG:teuthology.orchestra.run.smithi019:mgr.x> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.x.service 2024-04-23T18:15:22.441 INFO:tasks.cephadm:Bootstrapping... 2024-04-23T18:15:22.441 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 46706838-019d-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:15:22.638 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:15:22.638 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '46706838-019d-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-04-23T18:15:22.638 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-23T18:15:22.639 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-04-23T18:15:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T18:15:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-04-23T18:15:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-04-23T18:15:22.673 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T18:15:22.673 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T18:15:22.679 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T18:15:22.679 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-23T18:15:22.686 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-04-23T18:15:22.692 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-04-23T18:15:22.692 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-04-23T18:15:22.692 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-04-23T18:15:22.716 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T18:15:22.716 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-23T18:15:22.716 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-04-23T18:15:22.716 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-04-23T18:15:22.722 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T18:15:22.723 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T18:15:22.728 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T18:15:22.729 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-23T18:15:22.735 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-04-23T18:15:22.741 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-04-23T18:15:22.742 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-04-23T18:15:22.742 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-04-23T18:15:22.742 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:22.742 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 140227624397984 on /run/cephadm/46706838-019d-11ef-bc93-c7b262605968.lock 2024-04-23T18:15:22.742 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 140227624397984 acquired on /run/cephadm/46706838-019d-11ef-bc93-c7b262605968.lock 2024-04-23T18:15:22.742 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-04-23T18:15:22.742 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-04-23T18:15:22.743 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-04-23T18:15:22.745 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-04-23T18:15:22.746 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-04-23T18:15:22.748 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-23T18:15:22.748 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-23T18:15:22.748 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-23T18:15:22.751 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-23T18:15:22.751 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-23T18:15:22.751 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T18:15:22.751 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-23T18:15:22.751 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-23T18:15:22.751 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T18:15:22.752 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-04-23T18:15:22.752 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-04-23T18:15:22.752 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-23T18:15:22.752 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-23T18:15:22.753 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:15:23.241 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-23T18:15:23.241 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:15:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-04-23T18:15:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-23T18:15:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-23T18:15:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-23T18:15:23.242 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-23T18:15:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:15:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:15:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-04-23T18:15:25.398 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-04-23T18:15:25.398 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-04-23T18:15:26.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQC9+idm7ZaHLBAA3YsVD9HqJ1oZ697asDgjxQ== 2024-04-23T18:15:27.534 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQC++idm9HT4MRAA8LnKHuzzByn0/N35uQyDxw== 2024-04-23T18:15:28.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQC/+idmEWCyNhAACe+/WkIaIOuhRgdcwqUs4A== 2024-04-23T18:15:28.520 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-04-23T18:15:29.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T18:15:29.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-23T18:15:29.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:29.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T18:15:29.485 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for a [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T18:15:29.485 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-04-23T18:15:29.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:29.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T18:15:29.485 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:15:29.485 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-04-23T18:15:30.465 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.a on 2024-04-23T18:15:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-23T18:15:30.874 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-23T18:15:31.079 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-46706838-019d-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-46706838-019d-11ef-bc93-c7b262605968.target. 2024-04-23T18:15:31.079 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-46706838-019d-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-46706838-019d-11ef-bc93-c7b262605968.target. 2024-04-23T18:15:31.317 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-46706838-019d-11ef-bc93-c7b262605968@mon.a 2024-04-23T18:15:31.317 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-46706838-019d-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-46706838-019d-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-23T18:15:31.492 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-46706838-019d-11ef-bc93-c7b262605968.target.wants/ceph-46706838-019d-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-46706838-019d-11ef-bc93-c7b262605968@.service. 2024-04-23T18:15:31.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:31 smithi019 podman[28820]: 2024-04-23 18:15:31.76318492 +0000 UTC m=+0.125108644 container create a73c679be8ddef1c9e1c40e3a10e281f729acb8f2c2f0cb838f931ff8eb7be77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:15:32.091 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:15:32.091 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-23T18:15:32.097 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:15:32.097 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-23T18:15:32.098 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-23T18:15:32.098 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T18:15:32.098 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-04-23T18:15:32.098 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-04-23T18:15:32.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:32 smithi019 podman[28820]: 2024-04-23 18:15:32.0003893 +0000 UTC m=+0.362313039 container init a73c679be8ddef1c9e1c40e3a10e281f729acb8f2c2f0cb838f931ff8eb7be77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:15:32.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:32 smithi019 podman[28820]: 2024-04-23 18:15:32.003481565 +0000 UTC m=+0.365405289 container start a73c679be8ddef1c9e1c40e3a10e281f729acb8f2c2f0cb838f931ff8eb7be77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:15:32.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:32 smithi019 ceph-mon[28839]: mkfs 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:32.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:32 smithi019 ceph-mon[28839]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:15:32.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:32 smithi019 bash[28820]: a73c679be8ddef1c9e1c40e3a10e281f729acb8f2c2f0cb838f931ff8eb7be77 2024-04-23T18:15:32.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:32 smithi019 systemd[1]: Started Ceph mon.a for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: monmap epoch 1 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: last_changed 2024-04-23T18:15:28.877188+0000 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: min_mon_release 19 (squid) 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: election_strategy: 1 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: fsmap 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T18:15:33.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: mgrmap e1: no daemons active 2024-04-23T18:15:33.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:33 smithi019 ceph-mon[28839]: from='client.? 172.21.15.19:0/68805795' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-23T18:15:33.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-04-23T18:15:33.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:33.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-23T18:15:33.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:33.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-04-23T18:15:33.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.573306s) 2024-04-23T18:15:33.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-23T18:15:33.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-23T18:15:33.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:33.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-04-23T18:15:33.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.537+0000 7fa10672e700 1 Processor -- start 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.537+0000 7fa10672e700 1 -- start start 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.537+0000 7fa10672e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa100107830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.537+0000 7fa10672e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa100107e00 con 0x7fa100107410 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.538+0000 7fa0fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa100107830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.538+0000 7fa0fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa100107830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43484/0 (socket says 172.21.15.19:43484) 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.538+0000 7fa0fffff700 1 -- 172.21.15.19:0/1266480027 learned_addr learned my addr 172.21.15.19:0/1266480027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.538+0000 7fa0fffff700 1 -- 172.21.15.19:0/1266480027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa100108610 con 0x7fa100107410 2024-04-23T18:15:33.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.538+0000 7fa0fffff700 1 --2- 172.21.15.19:0/1266480027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa100107830 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00098a0 tx=0x7fa0f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed4cbf02de3705 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.538+0000 7fa0feffd700 1 -- 172.21.15.19:0/1266480027 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f0004160 con 0x7fa100107410 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.538+0000 7fa0feffd700 1 -- 172.21.15.19:0/1266480027 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0004300 con 0x7fa100107410 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.538+0000 7fa0feffd700 1 -- 172.21.15.19:0/1266480027 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f00036a0 con 0x7fa100107410 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.539+0000 7fa10672e700 1 -- 172.21.15.19:0/1266480027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 msgr2=0x7fa100107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.539+0000 7fa10672e700 1 --2- 172.21.15.19:0/1266480027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa100107830 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00098a0 tx=0x7fa0f0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.539+0000 7fa10672e700 1 -- 172.21.15.19:0/1266480027 shutdown_connections 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.539+0000 7fa10672e700 1 --2- 172.21.15.19:0/1266480027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa100107830 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.539+0000 7fa10672e700 1 -- 172.21.15.19:0/1266480027 >> 172.21.15.19:0/1266480027 conn(0x7fa100102a70 msgr2=0x7fa100104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.539+0000 7fa10672e700 1 -- 172.21.15.19:0/1266480027 shutdown_connections 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.539+0000 7fa10672e700 1 -- 172.21.15.19:0/1266480027 wait complete. 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.539+0000 7fa10672e700 1 Processor -- start 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.540+0000 7fa10672e700 1 -- start start 2024-04-23T18:15:33.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.540+0000 7fa10672e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa10019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.540+0000 7fa10672e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa100108b40 con 0x7fa100107410 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.540+0000 7fa0fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa10019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.540+0000 7fa0fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa10019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43500/0 (socket says 172.21.15.19:43500) 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.540+0000 7fa0fffff700 1 -- 172.21.15.19:0/68805795 learned_addr learned my addr 172.21.15.19:0/68805795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.540+0000 7fa0fffff700 1 -- 172.21.15.19:0/68805795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10019b330 con 0x7fa100107410 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.540+0000 7fa0fffff700 1 --2- 172.21.15.19:0/68805795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa10019adf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000c00 tx=0x7fa0f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.540+0000 7fa0fd7fa700 1 -- 172.21.15.19:0/68805795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f0003db0 con 0x7fa100107410 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.541+0000 7fa0fd7fa700 1 -- 172.21.15.19:0/68805795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0011630 con 0x7fa100107410 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.541+0000 7fa10672e700 1 -- 172.21.15.19:0/68805795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10019b5c0 con 0x7fa100107410 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.541+0000 7fa0fd7fa700 1 -- 172.21.15.19:0/68805795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f0011950 con 0x7fa100107410 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.541+0000 7fa10672e700 1 -- 172.21.15.19:0/68805795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10019e2b0 con 0x7fa100107410 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.541+0000 7fa0fd7fa700 1 -- 172.21.15.19:0/68805795 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa0f0011af0 con 0x7fa100107410 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.541+0000 7fa0fd7fa700 1 -- 172.21.15.19:0/68805795 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa0f001c450 con 0x7fa100107410 2024-04-23T18:15:33.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.542+0000 7fa10672e700 1 -- 172.21.15.19:0/68805795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1000640a0 con 0x7fa100107410 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.544+0000 7fa0fd7fa700 1 -- 172.21.15.19:0/68805795 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa0f0032020 con 0x7fa100107410 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.618+0000 7fa10672e700 1 -- 172.21.15.19:0/68805795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa10002df80 con 0x7fa100107410 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.618+0000 7fa0fd7fa700 1 -- 172.21.15.19:0/68805795 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fa0f0004260 con 0x7fa100107410 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.619+0000 7fa10672e700 1 -- 172.21.15.19:0/68805795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 msgr2=0x7fa10019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.619+0000 7fa10672e700 1 --2- 172.21.15.19:0/68805795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa10019adf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000c00 tx=0x7fa0f0004750 comp rx=0 tx=0).stop 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.619+0000 7fa10672e700 1 -- 172.21.15.19:0/68805795 shutdown_connections 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.619+0000 7fa10672e700 1 --2- 172.21.15.19:0/68805795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100107410 0x7fa10019adf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.619+0000 7fa10672e700 1 -- 172.21.15.19:0/68805795 >> 172.21.15.19:0/68805795 conn(0x7fa100102a70 msgr2=0x7fa100191770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.619+0000 7fa10672e700 1 -- 172.21.15.19:0/68805795 shutdown_connections 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:32.619+0000 7fa10672e700 1 -- 172.21.15.19:0/68805795 wait complete. 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-04-23T18:15:33.316 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-04-23T18:15:34.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:34 smithi019 ceph-mon[28839]: from='client.? 172.21.15.19:0/2887350731' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T18:15:34.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:34 smithi019 ceph-mon[28839]: from='client.? 172.21.15.19:0/2887350731' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-23T18:15:34.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T18:15:34.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.753+0000 7f9f1504a700 1 Processor -- start 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.753+0000 7f9f1504a700 1 -- start start 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.753+0000 7f9f1504a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1010b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.753+0000 7f9f1504a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1007be80 con 0x7f9f10108fc0 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.753+0000 7f9f0ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1010b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.753+0000 7f9f0ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1010b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43512/0 (socket says 172.21.15.19:43512) 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.753+0000 7f9f0ed9d700 1 -- 172.21.15.19:0/1961545229 learned_addr learned my addr 172.21.15.19:0/1961545229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.754+0000 7f9f0ed9d700 1 -- 172.21.15.19:0/1961545229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1010b8f0 con 0x7f9f10108fc0 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.754+0000 7f9f0ed9d700 1 --2- 172.21.15.19:0/1961545229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1010b3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9f000098a0 tx=0x7f9f00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18198c968a540160 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:34.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.754+0000 7f9f0dd9b700 1 -- 172.21.15.19:0/1961545229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f00004160 con 0x7f9f10108fc0 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.754+0000 7f9f0dd9b700 1 -- 172.21.15.19:0/1961545229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f00004300 con 0x7f9f10108fc0 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.754+0000 7f9f0dd9b700 1 -- 172.21.15.19:0/1961545229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f000036a0 con 0x7f9f10108fc0 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.754+0000 7f9f1504a700 1 -- 172.21.15.19:0/1961545229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 msgr2=0x7f9f1010b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.754+0000 7f9f1504a700 1 --2- 172.21.15.19:0/1961545229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1010b3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9f000098a0 tx=0x7f9f00009bb0 comp rx=0 tx=0).stop 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.755+0000 7f9f1504a700 1 -- 172.21.15.19:0/1961545229 shutdown_connections 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.755+0000 7f9f1504a700 1 --2- 172.21.15.19:0/1961545229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1010b3b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.755+0000 7f9f1504a700 1 -- 172.21.15.19:0/1961545229 >> 172.21.15.19:0/1961545229 conn(0x7f9f10102a70 msgr2=0x7f9f10104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.755+0000 7f9f1504a700 1 -- 172.21.15.19:0/1961545229 shutdown_connections 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.755+0000 7f9f1504a700 1 -- 172.21.15.19:0/1961545229 wait complete. 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.755+0000 7f9f1504a700 1 Processor -- start 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.756+0000 7f9f1504a700 1 -- start start 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.756+0000 7f9f1504a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.756+0000 7f9f1504a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1010bd60 con 0x7f9f10108fc0 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.756+0000 7f9f0ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.756+0000 7f9f0ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1019bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43516/0 (socket says 172.21.15.19:43516) 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.756+0000 7f9f0ed9d700 1 -- 172.21.15.19:0/2887350731 learned_addr learned my addr 172.21.15.19:0/2887350731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.756+0000 7f9f0ed9d700 1 -- 172.21.15.19:0/2887350731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1019c450 con 0x7f9f10108fc0 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.756+0000 7f9f0ed9d700 1 --2- 172.21.15.19:0/2887350731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1019bf10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000c00 tx=0x7f9f00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.756+0000 7f9ef7fff700 1 -- 172.21.15.19:0/2887350731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f00003db0 con 0x7f9f10108fc0 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.757+0000 7f9ef7fff700 1 -- 172.21.15.19:0/2887350731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f00011630 con 0x7f9f10108fc0 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.757+0000 7f9f1504a700 1 -- 172.21.15.19:0/2887350731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1019c6e0 con 0x7f9f10108fc0 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.757+0000 7f9ef7fff700 1 -- 172.21.15.19:0/2887350731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f00011950 con 0x7f9f10108fc0 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.757+0000 7f9f1504a700 1 -- 172.21.15.19:0/2887350731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1019f380 con 0x7f9f10108fc0 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.757+0000 7f9ef7fff700 1 -- 172.21.15.19:0/2887350731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f00011af0 con 0x7f9f10108fc0 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.757+0000 7f9ef7fff700 1 -- 172.21.15.19:0/2887350731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f0001c450 con 0x7f9f10108fc0 2024-04-23T18:15:34.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.758+0000 7f9f1504a700 1 -- 172.21.15.19:0/2887350731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f100640a0 con 0x7f9f10108fc0 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.760+0000 7f9ef7fff700 1 -- 172.21.15.19:0/2887350731 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9f00033080 con 0x7f9f10108fc0 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.830+0000 7f9f1504a700 1 -- 172.21.15.19:0/2887350731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9f10107f30 con 0x7f9f10108fc0 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.833+0000 7f9ef7fff700 1 -- 172.21.15.19:0/2887350731 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f9f00004260 con 0x7f9f10108fc0 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.834+0000 7f9ef7fff700 1 -- 172.21.15.19:0/2887350731 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f0001c650 con 0x7f9f10108fc0 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.834+0000 7f9f1504a700 1 -- 172.21.15.19:0/2887350731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 msgr2=0x7f9f1019bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.834+0000 7f9f1504a700 1 --2- 172.21.15.19:0/2887350731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1019bf10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000c00 tx=0x7f9f00004750 comp rx=0 tx=0).stop 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.834+0000 7f9f1504a700 1 -- 172.21.15.19:0/2887350731 shutdown_connections 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.834+0000 7f9f1504a700 1 --2- 172.21.15.19:0/2887350731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10108fc0 0x7f9f1019bf10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.834+0000 7f9f1504a700 1 -- 172.21.15.19:0/2887350731 >> 172.21.15.19:0/2887350731 conn(0x7f9f10102a70 msgr2=0x7f9f10104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.834+0000 7f9f1504a700 1 -- 172.21.15.19:0/2887350731 shutdown_connections 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:33.834+0000 7f9f1504a700 1 -- 172.21.15.19:0/2887350731 wait complete. 2024-04-23T18:15:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-04-23T18:15:35.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:35 smithi019 ceph-mon[28839]: from='client.? 172.21.15.19:0/2987751736' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:35.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.015+0000 7f8afde5b700 1 Processor -- start 2024-04-23T18:15:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.015+0000 7f8afde5b700 1 -- start start 2024-04-23T18:15:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.016+0000 7f8afde5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af807c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.016+0000 7f8afde5b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8af807c8e0 con 0x7f8af807de60 2024-04-23T18:15:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.016+0000 7f8af77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af807c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.016+0000 7f8af77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af807c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43526/0 (socket says 172.21.15.19:43526) 2024-04-23T18:15:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.016+0000 7f8af77fe700 1 -- 172.21.15.19:0/1263658034 learned_addr learned my addr 172.21.15.19:0/1263658034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.016+0000 7f8af77fe700 1 -- 172.21.15.19:0/1263658034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af807ca60 con 0x7f8af807de60 2024-04-23T18:15:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.016+0000 7f8af77fe700 1 --2- 172.21.15.19:0/1263658034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af807c310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80098a0 tx=0x7f8ae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafc80d9c2f0a09a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.016+0000 7f8af67fc700 1 -- 172.21.15.19:0/1263658034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae8004160 con 0x7f8af807de60 2024-04-23T18:15:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.017+0000 7f8af67fc700 1 -- 172.21.15.19:0/1263658034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8ae8004300 con 0x7f8af807de60 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.017+0000 7f8af67fc700 1 -- 172.21.15.19:0/1263658034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae8011af0 con 0x7f8af807de60 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.017+0000 7f8afde5b700 1 -- 172.21.15.19:0/1263658034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 msgr2=0x7f8af807c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.017+0000 7f8afde5b700 1 --2- 172.21.15.19:0/1263658034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af807c310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80098a0 tx=0x7f8ae8009bb0 comp rx=0 tx=0).stop 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.017+0000 7f8afde5b700 1 -- 172.21.15.19:0/1263658034 shutdown_connections 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.017+0000 7f8afde5b700 1 --2- 172.21.15.19:0/1263658034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af807c310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.017+0000 7f8afde5b700 1 -- 172.21.15.19:0/1263658034 >> 172.21.15.19:0/1263658034 conn(0x7f8af8103b10 msgr2=0x7f8af8105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.017+0000 7f8afde5b700 1 -- 172.21.15.19:0/1263658034 shutdown_connections 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.017+0000 7f8afde5b700 1 -- 172.21.15.19:0/1263658034 wait complete. 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.018+0000 7f8afde5b700 1 Processor -- start 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.018+0000 7f8afde5b700 1 -- start start 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.018+0000 7f8afde5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.018+0000 7f8afde5b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8af810ea40 con 0x7f8af807de60 2024-04-23T18:15:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.018+0000 7f8af77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:35.729 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.018+0000 7f8af77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43542/0 (socket says 172.21.15.19:43542) 2024-04-23T18:15:35.729 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.018+0000 7f8af77fe700 1 -- 172.21.15.19:0/2987751736 learned_addr learned my addr 172.21.15.19:0/2987751736 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:35.729 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.019+0000 7f8af77fe700 1 -- 172.21.15.19:0/2987751736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af819ee30 con 0x7f8af807de60 2024-04-23T18:15:35.729 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.019+0000 7f8af77fe700 1 --2- 172.21.15.19:0/2987751736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af819e8f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8011ac0 tx=0x7f8ae8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:35.729 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.019+0000 7f8af4ff9700 1 -- 172.21.15.19:0/2987751736 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae8003b20 con 0x7f8af807de60 2024-04-23T18:15:35.729 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.019+0000 7f8af4ff9700 1 -- 172.21.15.19:0/2987751736 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8ae801a8c0 con 0x7f8af807de60 2024-04-23T18:15:35.729 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.019+0000 7f8afde5b700 1 -- 172.21.15.19:0/2987751736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af819f0c0 con 0x7f8af807de60 2024-04-23T18:15:35.729 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.019+0000 7f8af4ff9700 1 -- 172.21.15.19:0/2987751736 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae80248d0 con 0x7f8af807de60 2024-04-23T18:15:35.729 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.019+0000 7f8afde5b700 1 -- 172.21.15.19:0/2987751736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af81a1db0 con 0x7f8af807de60 2024-04-23T18:15:35.730 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.019+0000 7f8af4ff9700 1 -- 172.21.15.19:0/2987751736 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8ae8022020 con 0x7f8af807de60 2024-04-23T18:15:35.730 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.019+0000 7f8af4ff9700 1 -- 172.21.15.19:0/2987751736 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8ae80193c0 con 0x7f8af807de60 2024-04-23T18:15:35.730 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.020+0000 7f8afde5b700 1 -- 172.21.15.19:0/2987751736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af80640a0 con 0x7f8af807de60 2024-04-23T18:15:35.730 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.023+0000 7f8af4ff9700 1 -- 172.21.15.19:0/2987751736 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8ae801aa70 con 0x7f8af807de60 2024-04-23T18:15:35.730 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.094+0000 7f8afde5b700 1 -- 172.21.15.19:0/2987751736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f8af802df80 con 0x7f8af807de60 2024-04-23T18:15:35.730 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.094+0000 7f8af4ff9700 1 -- 172.21.15.19:0/2987751736 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f8ae801a430 con 0x7f8af807de60 2024-04-23T18:15:35.730 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.095+0000 7f8afde5b700 1 -- 172.21.15.19:0/2987751736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 msgr2=0x7f8af819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:35.730 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.095+0000 7f8afde5b700 1 --2- 172.21.15.19:0/2987751736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af819e8f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8011ac0 tx=0x7f8ae8011ba0 comp rx=0 tx=0).stop 2024-04-23T18:15:35.730 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.095+0000 7f8afde5b700 1 -- 172.21.15.19:0/2987751736 shutdown_connections 2024-04-23T18:15:35.730 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.095+0000 7f8afde5b700 1 --2- 172.21.15.19:0/2987751736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af807de60 0x7f8af819e8f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:35.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.095+0000 7f8afde5b700 1 -- 172.21.15.19:0/2987751736 >> 172.21.15.19:0/2987751736 conn(0x7f8af8103b10 msgr2=0x7f8af8104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:35.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.095+0000 7f8afde5b700 1 -- 172.21.15.19:0/2987751736 shutdown_connections 2024-04-23T18:15:35.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:35.095+0000 7f8afde5b700 1 -- 172.21.15.19:0/2987751736 wait complete. 2024-04-23T18:15:35.731 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-04-23T18:15:36.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:35 smithi019 systemd[1]: Stopping Ceph mon.a for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:15:36.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a[28835]: 2024-04-23T18:15:36.069+0000 7f0488ede700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:15:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a[28835]: 2024-04-23T18:15:36.069+0000 7f0488ede700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-23T18:15:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:36 smithi019 podman[29085]: 2024-04-23 18:15:36.274287032 +0000 UTC m=+0.376182854 container died a73c679be8ddef1c9e1c40e3a10e281f729acb8f2c2f0cb838f931ff8eb7be77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True) 2024-04-23T18:15:36.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:36 smithi019 podman[29085]: 2024-04-23 18:15:36.627155423 +0000 UTC m=+0.729051246 container cleanup a73c679be8ddef1c9e1c40e3a10e281f729acb8f2c2f0cb838f931ff8eb7be77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-23T18:15:36.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:36 smithi019 bash[29085]: ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a 2024-04-23T18:15:36.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:36 smithi019 podman[29099]: 2024-04-23 18:15:36.794572486 +0000 UTC m=+0.521585315 container remove a73c679be8ddef1c9e1c40e3a10e281f729acb8f2c2f0cb838f931ff8eb7be77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-23T18:15:37.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-23T18:15:37.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 systemd[1]: Stopped Ceph mon.a for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:15:37.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 systemd[1]: Starting Ceph mon.a for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:15:37.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 podman[29196]: 2024-04-23 18:15:37.131787826 +0000 UTC m=+0.018070275 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:15:37.483 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 podman[29196]: 2024-04-23 18:15:37.237182713 +0000 UTC m=+0.123465191 container create c42d35d9ae5c8e74c58c49a5fa587e4e57c1562907de4a1a8f7b28bb7525f015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-04-23T18:15:37.483 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 podman[29196]: 2024-04-23 18:15:37.489572468 +0000 UTC m=+0.375854904 container init c42d35d9ae5c8e74c58c49a5fa587e4e57c1562907de4a1a8f7b28bb7525f015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:15:37.610 INFO:teuthology.orchestra.run.smithi019.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-23T18:15:37.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 podman[29196]: 2024-04-23 18:15:37.492708662 +0000 UTC m=+0.378991099 container start c42d35d9ae5c8e74c58c49a5fa587e4e57c1562907de4a1a8f7b28bb7525f015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: pidfile_write: ignore empty --pid-file 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: load: jerasure load: lrc 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: RocksDB version: 7.9.2 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Git sha 0 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: DB SUMMARY 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: DB Session ID: BCF3169PE50N43NHJ2ZV 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: CURRENT file: CURRENT 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-23T18:15:37.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105674 ; 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.error_if_exists: 0 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.create_if_missing: 0 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.paranoid_checks: 1 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.env: 0x560e4d34fcc0 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.info_log: 0x560e4df6bbe0 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.statistics: (nil) 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.use_fsync: 0 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_log_file_size: 0 2024-04-23T18:15:37.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T18:15:37.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T18:15:37.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T18:15:37.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T18:15:37.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.allow_fallocate: 1 2024-04-23T18:15:37.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T18:15:37.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T18:15:37.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.use_direct_reads: 0 2024-04-23T18:15:37.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T18:15:37.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.db_log_dir: 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.wal_dir: 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.write_buffer_manager: 0x560e4deff0e0 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T18:15:37.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.unordered_write: 0 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.row_cache: None 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.wal_filter: None 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T18:15:37.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.two_write_queues: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.wal_compression: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.atomic_flush: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.log_readahead_size: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T18:15:37.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_background_jobs: 2 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_background_compactions: -1 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_subcompactions: 1 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_open_files: -1 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T18:15:37.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_background_flushes: -1 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Compression algorithms supported: 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: kZSTD supported: 0 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: kXpressCompression supported: 0 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: kZlibCompression supported: 1 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: kSnappyCompression supported: 1 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: kLZ4Compression supported: 1 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: kBZip2Compression supported: 0 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T18:15:37.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.merge_operator: 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_filter: None 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e4df6bdc0) 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-04-23T18:15:37.746 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_type: 0 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_index_type: 0 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_shortening: 1 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: checksum: 4 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: no_block_cache: 0 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache: 0x560e4df311f0 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_options: 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: capacity : 536870912 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_shard_bits : 4 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: strict_capacity_limit : 0 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_compressed: (nil) 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: persistent_cache: (nil) 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size: 4096 2024-04-23T18:15:37.747 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size_deviation: 10 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_restart_interval: 16 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_block_restart_interval: 1 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: metadata_block_size: 4096 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: partition_filters: 0 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: use_delta_encoding: 1 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: filter_policy: bloomfilter 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: whole_key_filtering: 1 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: verify_compression: 0 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: format_version: 5 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_index_compression: 1 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_align: 0 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: max_auto_readahead_size: 262144 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: prepopulate_block_cache: 0 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: initial_auto_readahead_size: 8192 2024-04-23T18:15:37.748 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compression: NoCompression 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.num_levels: 7 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T18:15:37.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T18:15:37.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T18:15:37.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T18:15:37.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.inplace_update_support: 0 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.bloom_locality: 0 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.max_successive_merges: 0 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.ttl: 2592000 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T18:15:37.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.enable_blob_files: false 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.min_blob_size: 0 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-23T18:15:37.754 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 36cf2cbb-a328-41c8-bb9b-23aa0e22103a 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896137521650, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896137522774, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100781, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98669, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14302, "raw_average_key_size": 49, "raw_value_size": 90887, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713896137, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "36cf2cbb-a328-41c8-bb9b-23aa0e22103a", "db_session_id": "BCF3169PE50N43NHJ2ZV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896137522859, "job": 1, "event": "recovery_finished"} 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560e4e046000 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: DB pointer 0x560e4e030000 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** DB Stats ** 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:15:37.755 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: L0 2/0 100.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: Sum 2/0 100.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-04-23T18:15:37.756 INFO:journalctl@ceph.mon.a.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 136.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative compaction: 0.00 GB write, 29.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval compaction: 0.00 GB write, 29.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:15:37.757 INFO:journalctl@ceph.mon.a.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache BinnedLRUCache@0x560e4df311f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: starting mon.a rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: mon.a@-1(???) e1 preinit fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: mon.a@-1(???).mds e1 new map 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: mon.a@-1(???).mds e1 print_map 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout: e1 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout: legacy client fscid: -1 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-23T18:15:37.758 INFO:journalctl@ceph.mon.a.smithi019.stdout: No filesystems configured 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: monmap epoch 1 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: last_changed 2024-04-23T18:15:28.877188+0000 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: min_mon_release 19 (squid) 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: election_strategy: 1 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: fsmap 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T18:15:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 ceph-mon[29215]: mgrmap e1: no daemons active 2024-04-23T18:15:37.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 bash[29196]: c42d35d9ae5c8e74c58c49a5fa587e4e57c1562907de4a1a8f7b28bb7525f015 2024-04-23T18:15:37.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:37 smithi019 systemd[1]: Started Ceph mon.a for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:15:42.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:41 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3375720467' entity='client.admin' 2024-04-23T18:15:46.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.670+0000 7f2b111d1700 1 Processor -- start 2024-04-23T18:15:46.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.670+0000 7f2b111d1700 1 -- start start 2024-04-23T18:15:46.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.670+0000 7f2b111d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:46.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.670+0000 7f2b111d1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c108f30 con 0x7f2b0c108540 2024-04-23T18:15:46.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.671+0000 7f2b0ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.671+0000 7f2b0ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49940/0 (socket says 172.21.15.19:49940) 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.671+0000 7f2b0ad9d700 1 -- 172.21.15.19:0/245386496 learned_addr learned my addr 172.21.15.19:0/245386496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.671+0000 7f2b0ad9d700 1 -- 172.21.15.19:0/245386496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c1096f0 con 0x7f2b0c108540 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.671+0000 7f2b0ad9d700 1 --2- 172.21.15.19:0/245386496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c108960 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009e80 tx=0x7f2afc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f7acd7810ae810 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.672+0000 7f2b09d9b700 1 -- 172.21.15.19:0/245386496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2afc017070 con 0x7f2b0c108540 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.672+0000 7f2b09d9b700 1 -- 172.21.15.19:0/245386496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2afc00b8c0 con 0x7f2b0c108540 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.672+0000 7f2b09d9b700 1 -- 172.21.15.19:0/245386496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2afc004490 con 0x7f2b0c108540 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.672+0000 7f2b111d1700 1 -- 172.21.15.19:0/245386496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 msgr2=0x7f2b0c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.672+0000 7f2b111d1700 1 --2- 172.21.15.19:0/245386496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c108960 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009e80 tx=0x7f2afc00b330 comp rx=0 tx=0).stop 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.672+0000 7f2b111d1700 1 -- 172.21.15.19:0/245386496 shutdown_connections 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.672+0000 7f2b111d1700 1 --2- 172.21.15.19:0/245386496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c108960 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.672+0000 7f2b111d1700 1 -- 172.21.15.19:0/245386496 >> 172.21.15.19:0/245386496 conn(0x7f2b0c103ba0 msgr2=0x7f2b0c105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.672+0000 7f2b111d1700 1 -- 172.21.15.19:0/245386496 shutdown_connections 2024-04-23T18:15:46.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.672+0000 7f2b111d1700 1 -- 172.21.15.19:0/245386496 wait complete. 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.673+0000 7f2b111d1700 1 Processor -- start 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.673+0000 7f2b111d1700 1 -- start start 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b111d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c07db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b111d1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c10c920 con 0x7f2b0c108540 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b0ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c07db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b0ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c07db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49944/0 (socket says 172.21.15.19:49944) 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b0ad9d700 1 -- 172.21.15.19:0/3375720467 learned_addr learned my addr 172.21.15.19:0/3375720467 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b0ad9d700 1 -- 172.21.15.19:0/3375720467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c07e0c0 con 0x7f2b0c108540 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b0ad9d700 1 --2- 172.21.15.19:0/3375720467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c07db80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b7b0 tx=0x7f2afc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b03fff700 1 -- 172.21.15.19:0/3375720467 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2afc020020 con 0x7f2b0c108540 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b03fff700 1 -- 172.21.15.19:0/3375720467 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2afc012430 con 0x7f2b0c108540 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b111d1700 1 -- 172.21.15.19:0/3375720467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c07c2b0 con 0x7f2b0c108540 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b03fff700 1 -- 172.21.15.19:0/3375720467 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2afc017040 con 0x7f2b0c108540 2024-04-23T18:15:46.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.674+0000 7f2b111d1700 1 -- 172.21.15.19:0/3375720467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c07c7f0 con 0x7f2b0c108540 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.675+0000 7f2b03fff700 1 -- 172.21.15.19:0/3375720467 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2afc023020 con 0x7f2b0c108540 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.675+0000 7f2b03fff700 1 -- 172.21.15.19:0/3375720467 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2afc00f050 con 0x7f2b0c108540 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.675+0000 7f2b111d1700 1 -- 172.21.15.19:0/3375720467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0c10caa0 con 0x7f2b0c108540 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.678+0000 7f2b03fff700 1 -- 172.21.15.19:0/3375720467 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2afc02a020 con 0x7f2b0c108540 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.753+0000 7f2b111d1700 1 -- 172.21.15.19:0/3375720467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f2b0c07c9c0 con 0x7f2b0c108540 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.755+0000 7f2b03fff700 1 -- 172.21.15.19:0/3375720467 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f2b0c07c9c0 con 0x7f2b0c108540 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.756+0000 7f2b111d1700 1 -- 172.21.15.19:0/3375720467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 msgr2=0x7f2b0c07db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.756+0000 7f2b111d1700 1 --2- 172.21.15.19:0/3375720467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c07db80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b7b0 tx=0x7f2afc00be20 comp rx=0 tx=0).stop 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.756+0000 7f2b111d1700 1 -- 172.21.15.19:0/3375720467 shutdown_connections 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.756+0000 7f2b111d1700 1 --2- 172.21.15.19:0/3375720467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b0c108540 0x7f2b0c07db80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.756+0000 7f2b111d1700 1 -- 172.21.15.19:0/3375720467 >> 172.21.15.19:0/3375720467 conn(0x7f2b0c103ba0 msgr2=0x7f2b0c104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.756+0000 7f2b111d1700 1 -- 172.21.15.19:0/3375720467 shutdown_connections 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:40.756+0000 7f2b111d1700 1 -- 172.21.15.19:0/3375720467 wait complete. 2024-04-23T18:15:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-23T18:15:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:15:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-04-23T18:15:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-23T18:15:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-23T18:15:47.790 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.x 2024-04-23T18:15:47.790 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-23T18:15:48.019 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-46706838-019d-11ef-bc93-c7b262605968.target.wants/ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-46706838-019d-11ef-bc93-c7b262605968@.service. 2024-04-23T18:15:48.260 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:48 smithi019 systemd[1]: Starting Ceph mgr.x for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:15:48.544 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:48 smithi019 podman[29433]: 2024-04-23 18:15:48.185690687 +0000 UTC m=+0.029663700 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:15:48.544 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:48 smithi019 podman[29433]: 2024-04-23 18:15:48.298695565 +0000 UTC m=+0.142668614 container create 6f128d5987ef75bda4e61740cb0bdf0d336b5002288d42c8a4eaf135526aeef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True) 2024-04-23T18:15:48.544 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:48 smithi019 podman[29433]: 2024-04-23 18:15:48.484139086 +0000 UTC m=+0.328112090 container init 6f128d5987ef75bda4e61740cb0bdf0d336b5002288d42c8a4eaf135526aeef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:15:48.544 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:48 smithi019 podman[29433]: 2024-04-23 18:15:48.48661092 +0000 UTC m=+0.330583924 container start 6f128d5987ef75bda4e61740cb0bdf0d336b5002288d42c8a4eaf135526aeef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-23T18:15:48.544 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:48 smithi019 bash[29433]: 6f128d5987ef75bda4e61740cb0bdf0d336b5002288d42c8a4eaf135526aeef2 2024-04-23T18:15:48.559 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:15:48.559 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-23T18:15:48.567 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:15:48.567 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-23T18:15:48.567 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-23T18:15:48.567 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T18:15:48.577 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:15:48.577 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-23T18:15:48.585 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:15:48.585 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-23T18:15:48.586 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-23T18:15:48.586 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-23T18:15:48.586 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-04-23T18:15:48.586 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-04-23T18:15:48.832 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:48 smithi019 systemd[1]: Started Ceph mgr.x for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:15:48.832 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:48 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:48.657+0000 7fd26c148200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:15:49.157 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:48 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:48.854+0000 7fd26c148200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:15:49.157 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:48 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:48.960+0000 7fd26c148200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:15:49.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:49 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2989698069' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:15:49.511 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:49 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:49.309+0000 7fd26c148200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:15:49.511 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:49 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:49.397+0000 7fd26c148200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:15:49.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:15:49.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:15:49.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:15:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:49.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:15:32.047092+0000", 2024-04-23T18:15:49.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:15:49.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:49.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:15:49.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:15:49.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.072+0000 7f378d539700 1 Processor -- start 2024-04-23T18:15:49.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.072+0000 7f378d539700 1 -- start start 2024-04-23T18:15:49.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.073+0000 7f378d539700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f378810b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:49.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.073+0000 7f378d539700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f378807bec0 con 0x7f3788108f80 2024-04-23T18:15:49.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.073+0000 7f3786ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f378810b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:49.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.073+0000 7f3786ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f378810b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49976/0 (socket says 172.21.15.19:49976) 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.073+0000 7f3786ffd700 1 -- 172.21.15.19:0/1179159239 learned_addr learned my addr 172.21.15.19:0/1179159239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.073+0000 7f3786ffd700 1 -- 172.21.15.19:0/1179159239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378810b8b0 con 0x7f3788108f80 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.073+0000 7f3786ffd700 1 --2- 172.21.15.19:0/1179159239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f378810b370 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f37780098a0 tx=0x7f3778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e70088800c1665e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.074+0000 7f3785ffb700 1 -- 172.21.15.19:0/1179159239 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3778004160 con 0x7f3788108f80 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.074+0000 7f3785ffb700 1 -- 172.21.15.19:0/1179159239 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3778004300 con 0x7f3788108f80 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.074+0000 7f3785ffb700 1 -- 172.21.15.19:0/1179159239 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3778011af0 con 0x7f3788108f80 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.074+0000 7f378d539700 1 -- 172.21.15.19:0/1179159239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 msgr2=0x7f378810b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.074+0000 7f378d539700 1 --2- 172.21.15.19:0/1179159239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f378810b370 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f37780098a0 tx=0x7f3778009bb0 comp rx=0 tx=0).stop 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.074+0000 7f378d539700 1 -- 172.21.15.19:0/1179159239 shutdown_connections 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.074+0000 7f378d539700 1 --2- 172.21.15.19:0/1179159239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f378810b370 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.074+0000 7f378d539700 1 -- 172.21.15.19:0/1179159239 >> 172.21.15.19:0/1179159239 conn(0x7f3788102a70 msgr2=0x7f3788104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.074+0000 7f378d539700 1 -- 172.21.15.19:0/1179159239 shutdown_connections 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.074+0000 7f378d539700 1 -- 172.21.15.19:0/1179159239 wait complete. 2024-04-23T18:15:49.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.075+0000 7f378d539700 1 Processor -- start 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.075+0000 7f378d539700 1 -- start start 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.075+0000 7f378d539700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f3788108800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.075+0000 7f378d539700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f378810c880 con 0x7f3788108f80 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.075+0000 7f3786ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f3788108800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.075+0000 7f3786ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f3788108800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49982/0 (socket says 172.21.15.19:49982) 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.075+0000 7f3786ffd700 1 -- 172.21.15.19:0/2989698069 learned_addr learned my addr 172.21.15.19:0/2989698069 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.076+0000 7f3786ffd700 1 -- 172.21.15.19:0/2989698069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3788108d40 con 0x7f3788108f80 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.076+0000 7f3786ffd700 1 --2- 172.21.15.19:0/2989698069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f3788108800 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3778011ac0 tx=0x7f3778011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.076+0000 7f377ffff700 1 -- 172.21.15.19:0/2989698069 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3778003b20 con 0x7f3788108f80 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.076+0000 7f377ffff700 1 -- 172.21.15.19:0/2989698069 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f377801a8c0 con 0x7f3788108f80 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.076+0000 7f378d539700 1 -- 172.21.15.19:0/2989698069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3788106f30 con 0x7f3788108f80 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.076+0000 7f377ffff700 1 -- 172.21.15.19:0/2989698069 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37780248d0 con 0x7f3788108f80 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.076+0000 7f378d539700 1 -- 172.21.15.19:0/2989698069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3788107470 con 0x7f3788108f80 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.076+0000 7f377ffff700 1 -- 172.21.15.19:0/2989698069 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3778022020 con 0x7f3788108f80 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.076+0000 7f377ffff700 1 -- 172.21.15.19:0/2989698069 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f37780193c0 con 0x7f3788108f80 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.077+0000 7f378d539700 1 -- 172.21.15.19:0/2989698069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37880640a0 con 0x7f3788108f80 2024-04-23T18:15:49.833 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.080+0000 7f377ffff700 1 -- 172.21.15.19:0/2989698069 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3778042e50 con 0x7f3788108f80 2024-04-23T18:15:49.834 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.163+0000 7f378d539700 1 -- 172.21.15.19:0/2989698069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3788195de0 con 0x7f3788108f80 2024-04-23T18:15:49.834 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.164+0000 7f377ffff700 1 -- 172.21.15.19:0/2989698069 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f3778031020 con 0x7f3788108f80 2024-04-23T18:15:49.834 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.164+0000 7f378d539700 1 -- 172.21.15.19:0/2989698069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 msgr2=0x7f3788108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:49.834 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.164+0000 7f378d539700 1 --2- 172.21.15.19:0/2989698069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f3788108800 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3778011ac0 tx=0x7f3778011ba0 comp rx=0 tx=0).stop 2024-04-23T18:15:49.834 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.165+0000 7f378d539700 1 -- 172.21.15.19:0/2989698069 shutdown_connections 2024-04-23T18:15:49.834 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.165+0000 7f378d539700 1 --2- 172.21.15.19:0/2989698069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3788108f80 0x7f3788108800 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:49.834 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.165+0000 7f378d539700 1 -- 172.21.15.19:0/2989698069 >> 172.21.15.19:0/2989698069 conn(0x7f3788102a70 msgr2=0x7f37881036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:49.834 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.165+0000 7f378d539700 1 -- 172.21.15.19:0/2989698069 shutdown_connections 2024-04-23T18:15:49.834 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:49.165+0000 7f378d539700 1 -- 172.21.15.19:0/2989698069 wait complete. 2024-04-23T18:15:49.834 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-04-23T18:15:51.000 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:50 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:50.655+0000 7fd26c148200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:15:51.001 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:50 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:50.831+0000 7fd26c148200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:15:51.001 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:51 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:51.006+0000 7fd26c148200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:15:51.604 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:51 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:51.305+0000 7fd26c148200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:15:51.604 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:51 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:51.400+0000 7fd26c148200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:15:51.943 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:51 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:51.610+0000 7fd26c148200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:15:51.943 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:51 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:51.809+0000 7fd26c148200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:15:52.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:52 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1430028179' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:15:52.513 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:52 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:52.458+0000 7fd26c148200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:15:53.214 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:52 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:52.908+0000 7fd26c148200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:15:53.214 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:53 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:53.009+0000 7fd26c148200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:15:53.214 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:53 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:53.220+0000 7fd26c148200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:15:54.110 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:53 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:53.768+0000 7fd26c148200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:15:54.110 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:53 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:53.863+0000 7fd26c148200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:15:54.164 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:54.164 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:15:54.164 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:15:54.164 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:15:54.164 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:15:54.164 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:15:54.164 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:15:54.164 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:54.164 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:15:54.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:15:54.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:15:54.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:15:54.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:15:54.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:15:54.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:15:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:15:32.047092+0000", 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:15:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.261+0000 7fc449cf8700 1 Processor -- start 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.262+0000 7fc449cf8700 1 -- start start 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.262+0000 7fc449cf8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44410b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.262+0000 7fc449cf8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc44407bec0 con 0x7fc444108fa0 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.262+0000 7fc4437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44410b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.262+0000 7fc4437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44410b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33998/0 (socket says 172.21.15.19:33998) 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.262+0000 7fc4437fe700 1 -- 172.21.15.19:0/1841364622 learned_addr learned my addr 172.21.15.19:0/1841364622 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.262+0000 7fc4437fe700 1 -- 172.21.15.19:0/1841364622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44410b8d0 con 0x7fc444108fa0 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.262+0000 7fc4437fe700 1 --2- 172.21.15.19:0/1841364622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44410b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4340098a0 tx=0x7fc434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bc14f0ef161f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.263+0000 7fc4427fc700 1 -- 172.21.15.19:0/1841364622 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc434004160 con 0x7fc444108fa0 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.263+0000 7fc4427fc700 1 -- 172.21.15.19:0/1841364622 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc434004300 con 0x7fc444108fa0 2024-04-23T18:15:54.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.263+0000 7fc4427fc700 1 -- 172.21.15.19:0/1841364622 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc434011af0 con 0x7fc444108fa0 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.263+0000 7fc449cf8700 1 -- 172.21.15.19:0/1841364622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 msgr2=0x7fc44410b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.263+0000 7fc449cf8700 1 --2- 172.21.15.19:0/1841364622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44410b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4340098a0 tx=0x7fc434009bb0 comp rx=0 tx=0).stop 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.263+0000 7fc449cf8700 1 -- 172.21.15.19:0/1841364622 shutdown_connections 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.263+0000 7fc449cf8700 1 --2- 172.21.15.19:0/1841364622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44410b390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.263+0000 7fc449cf8700 1 -- 172.21.15.19:0/1841364622 >> 172.21.15.19:0/1841364622 conn(0x7fc444102a10 msgr2=0x7fc444104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.264+0000 7fc449cf8700 1 -- 172.21.15.19:0/1841364622 shutdown_connections 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.264+0000 7fc449cf8700 1 -- 172.21.15.19:0/1841364622 wait complete. 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.264+0000 7fc449cf8700 1 Processor -- start 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.264+0000 7fc449cf8700 1 -- start start 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc449cf8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc449cf8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc44410c830 con 0x7fc444108fa0 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc4437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:54.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc4437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34000/0 (socket says 172.21.15.19:34000) 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc4437fe700 1 -- 172.21.15.19:0/1430028179 learned_addr learned my addr 172.21.15.19:0/1430028179 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc4437fe700 1 -- 172.21.15.19:0/1430028179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44419c970 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc4437fe700 1 --2- 172.21.15.19:0/1430028179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44419c430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc434009fb0 tx=0x7fc434011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc440ff9700 1 -- 172.21.15.19:0/1430028179 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4340036a0 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc449cf8700 1 -- 172.21.15.19:0/1430028179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44419cc00 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc440ff9700 1 -- 172.21.15.19:0/1430028179 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc43401a430 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc440ff9700 1 -- 172.21.15.19:0/1430028179 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc434019550 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.265+0000 7fc449cf8700 1 -- 172.21.15.19:0/1430028179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44419f8f0 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.266+0000 7fc440ff9700 1 -- 172.21.15.19:0/1430028179 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc434019b90 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.266+0000 7fc440ff9700 1 -- 172.21.15.19:0/1430028179 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc434024730 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.267+0000 7fc449cf8700 1 -- 172.21.15.19:0/1430028179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4440640a0 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.271+0000 7fc440ff9700 1 -- 172.21.15.19:0/1430028179 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc43401a5e0 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.346+0000 7fc449cf8700 1 -- 172.21.15.19:0/1430028179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc444107160 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.346+0000 7fc440ff9700 1 -- 172.21.15.19:0/1430028179 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fc43401a8e0 con 0x7fc444108fa0 2024-04-23T18:15:54.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.347+0000 7fc449cf8700 1 -- 172.21.15.19:0/1430028179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 msgr2=0x7fc44419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:54.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.347+0000 7fc449cf8700 1 --2- 172.21.15.19:0/1430028179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44419c430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc434009fb0 tx=0x7fc434011940 comp rx=0 tx=0).stop 2024-04-23T18:15:54.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.347+0000 7fc449cf8700 1 -- 172.21.15.19:0/1430028179 shutdown_connections 2024-04-23T18:15:54.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.347+0000 7fc449cf8700 1 --2- 172.21.15.19:0/1430028179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc444108fa0 0x7fc44419c430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:54.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.347+0000 7fc449cf8700 1 -- 172.21.15.19:0/1430028179 >> 172.21.15.19:0/1430028179 conn(0x7fc444102a10 msgr2=0x7fc444103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:54.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.347+0000 7fc449cf8700 1 -- 172.21.15.19:0/1430028179 shutdown_connections 2024-04-23T18:15:54.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:52.347+0000 7fc449cf8700 1 -- 172.21.15.19:0/1430028179 wait complete. 2024-04-23T18:15:54.180 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-04-23T18:15:54.475 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:54.116+0000 7fd26c148200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:15:54.475 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:54.260+0000 7fd26c148200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:15:54.476 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:54.360+0000 7fd26c148200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:15:54.476 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:54.481+0000 7fd26c148200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:15:55.260 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:54.793+0000 7fd26c148200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:15:55.693 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:55.420+0000 7fd26c148200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:15:55.694 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:55.644+0000 7fd26c148200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: Activating manager daemon x 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: mgrmap e2: x(active, starting, since 0.00327552s) 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: from='mgr.14100 172.21.15.19:0/110717398' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: from='mgr.14100 172.21.15.19:0/110717398' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: from='mgr.14100 172.21.15.19:0/110717398' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: from='mgr.14100 172.21.15.19:0/110717398' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: from='mgr.14100 172.21.15.19:0/110717398' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: Manager daemon x is now available 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: from='mgr.14100 172.21.15.19:0/110717398' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: from='mgr.14100 172.21.15.19:0/110717398' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:15:56.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: from='mgr.14100 172.21.15.19:0/110717398' entity='mgr.x' 2024-04-23T18:15:56.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: from='mgr.14100 172.21.15.19:0/110717398' entity='mgr.x' 2024-04-23T18:15:56.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:55 smithi019 ceph-mon[29215]: from='mgr.14100 172.21.15.19:0/110717398' entity='mgr.x' 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:15:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 19, 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:15:57.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:15:57.344 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:15:57.344 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:15:57.344 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:15:57.344 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:15:57.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:15:57.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:15:32.047092+0000", 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.633+0000 7ff2c462a700 1 Processor -- start 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.633+0000 7ff2c462a700 1 -- start start 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.634+0000 7ff2c462a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.634+0000 7ff2c462a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc108f00 con 0x7ff2bc108510 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.634+0000 7ff2c23c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.634+0000 7ff2c23c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34096/0 (socket says 172.21.15.19:34096) 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.634+0000 7ff2c23c6700 1 -- 172.21.15.19:0/1430228078 learned_addr learned my addr 172.21.15.19:0/1430228078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.634+0000 7ff2c23c6700 1 -- 172.21.15.19:0/1430228078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc1096c0 con 0x7ff2bc108510 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.634+0000 7ff2c23c6700 1 --2- 172.21.15.19:0/1430228078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc108930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009e80 tx=0x7ff2ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3a464fced61371 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.635+0000 7ff2c13c4700 1 -- 172.21.15.19:0/1430228078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac017070 con 0x7ff2bc108510 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.635+0000 7ff2c13c4700 1 -- 172.21.15.19:0/1430228078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff2ac00b8c0 con 0x7ff2bc108510 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.635+0000 7ff2c13c4700 1 -- 172.21.15.19:0/1430228078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac004490 con 0x7ff2bc108510 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.635+0000 7ff2c462a700 1 -- 172.21.15.19:0/1430228078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 msgr2=0x7ff2bc108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.635+0000 7ff2c462a700 1 --2- 172.21.15.19:0/1430228078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc108930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009e80 tx=0x7ff2ac00b330 comp rx=0 tx=0).stop 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.635+0000 7ff2c462a700 1 -- 172.21.15.19:0/1430228078 shutdown_connections 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.635+0000 7ff2c462a700 1 --2- 172.21.15.19:0/1430228078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc108930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.635+0000 7ff2c462a700 1 -- 172.21.15.19:0/1430228078 >> 172.21.15.19:0/1430228078 conn(0x7ff2bc103b70 msgr2=0x7ff2bc105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.635+0000 7ff2c462a700 1 -- 172.21.15.19:0/1430228078 shutdown_connections 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.635+0000 7ff2c462a700 1 -- 172.21.15.19:0/1430228078 wait complete. 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.636+0000 7ff2c462a700 1 Processor -- start 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.636+0000 7ff2c462a700 1 -- start start 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.636+0000 7ff2c462a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.636+0000 7ff2c462a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc10ca70 con 0x7ff2bc108510 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.636+0000 7ff2c23c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:57.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.636+0000 7ff2c23c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34102/0 (socket says 172.21.15.19:34102) 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.636+0000 7ff2c23c6700 1 -- 172.21.15.19:0/334881149 learned_addr learned my addr 172.21.15.19:0/334881149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.637+0000 7ff2c23c6700 1 -- 172.21.15.19:0/334881149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc19cc00 con 0x7ff2bc108510 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.637+0000 7ff2c23c6700 1 --2- 172.21.15.19:0/334881149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc19c6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac004a60 tx=0x7ff2ac00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.637+0000 7ff2bb7fe700 1 -- 172.21.15.19:0/334881149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac020020 con 0x7ff2bc108510 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.637+0000 7ff2bb7fe700 1 -- 172.21.15.19:0/334881149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff2ac003c60 con 0x7ff2bc108510 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.637+0000 7ff2bb7fe700 1 -- 172.21.15.19:0/334881149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac017040 con 0x7ff2bc108510 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.637+0000 7ff2c462a700 1 -- 172.21.15.19:0/334881149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2bc19ce90 con 0x7ff2bc108510 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.637+0000 7ff2c462a700 1 -- 172.21.15.19:0/334881149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2bc19fb80 con 0x7ff2bc108510 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.638+0000 7ff2bb7fe700 1 -- 172.21.15.19:0/334881149 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7ff2ac023020 con 0x7ff2bc108510 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.638+0000 7ff2c462a700 1 -- 172.21.15.19:0/334881149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2bc0640a0 con 0x7ff2bc108510 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.638+0000 7ff2bb7fe700 1 -- 172.21.15.19:0/334881149 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff2ac00f050 con 0x7ff2bc108510 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.640+0000 7ff2bb7fe700 1 -- 172.21.15.19:0/334881149 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff2ac02a020 con 0x7ff2bc108510 2024-04-23T18:15:57.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.650+0000 7ff2bb7fe700 1 -- 172.21.15.19:0/334881149 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7ff2ac012950 con 0x7ff2bc108510 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.650+0000 7ff2bb7fe700 1 --2- 172.21.15.19:0/334881149 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7ff2a403a220 0x7ff2a403c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.650+0000 7ff2c1bc5700 1 --2- 172.21.15.19:0/334881149 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7ff2a403a220 0x7ff2a403c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.651+0000 7ff2c1bc5700 1 --2- 172.21.15.19:0/334881149 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7ff2a403a220 0x7ff2a403c610 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0006fd0 tx=0x7ff2b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.716+0000 7ff2c462a700 1 -- 172.21.15.19:0/334881149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff2bc02dfb0 con 0x7ff2bc108510 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.716+0000 7ff2bb7fe700 1 -- 172.21.15.19:0/334881149 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff2ac017c20 con 0x7ff2bc108510 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.717+0000 7ff2c462a700 1 -- 172.21.15.19:0/334881149 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7ff2a403a220 msgr2=0x7ff2a403c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.717+0000 7ff2c462a700 1 --2- 172.21.15.19:0/334881149 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7ff2a403a220 0x7ff2a403c610 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0006fd0 tx=0x7ff2b0006e40 comp rx=0 tx=0).stop 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.717+0000 7ff2c462a700 1 -- 172.21.15.19:0/334881149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 msgr2=0x7ff2bc19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.717+0000 7ff2c462a700 1 --2- 172.21.15.19:0/334881149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc19c6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac004a60 tx=0x7ff2ac00bd50 comp rx=0 tx=0).stop 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.717+0000 7ff2c462a700 1 -- 172.21.15.19:0/334881149 shutdown_connections 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.717+0000 7ff2c462a700 1 --2- 172.21.15.19:0/334881149 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7ff2a403a220 0x7ff2a403c610 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.717+0000 7ff2c462a700 1 --2- 172.21.15.19:0/334881149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2bc108510 0x7ff2bc19c6c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.717+0000 7ff2c462a700 1 -- 172.21.15.19:0/334881149 >> 172.21.15.19:0/334881149 conn(0x7ff2bc103b70 msgr2=0x7ff2bc104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.717+0000 7ff2c462a700 1 -- 172.21.15.19:0/334881149 shutdown_connections 2024-04-23T18:15:57.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:56.717+0000 7ff2c462a700 1 -- 172.21.15.19:0/334881149 wait complete. 2024-04-23T18:15:57.351 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-04-23T18:15:58.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:57 smithi019 ceph-mon[29215]: mgrmap e3: x(active, since 1.00699s) 2024-04-23T18:15:58.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:57 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/334881149' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:15:58.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:58.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T18:15:58.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:58.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-23T18:15:58.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T18:15:58.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T18:15:58.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.824+0000 7faf4953e700 1 Processor -- start 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.824+0000 7faf4953e700 1 -- start start 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.824+0000 7faf4953e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:58.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.825+0000 7faf4953e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf4407c830 con 0x7faf4407de40 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.825+0000 7faf42ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.825+0000 7faf42ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34108/0 (socket says 172.21.15.19:34108) 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.825+0000 7faf42ffd700 1 -- 172.21.15.19:0/385805571 learned_addr learned my addr 172.21.15.19:0/385805571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.825+0000 7faf42ffd700 1 -- 172.21.15.19:0/385805571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4407c9b0 con 0x7faf4407de40 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.825+0000 7faf42ffd700 1 --2- 172.21.15.19:0/385805571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4407c2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faf340098a0 tx=0x7faf34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa79ad2f8b02ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.825+0000 7faf41ffb700 1 -- 172.21.15.19:0/385805571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf34004160 con 0x7faf4407de40 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.825+0000 7faf41ffb700 1 -- 172.21.15.19:0/385805571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faf34004300 con 0x7faf4407de40 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.825+0000 7faf41ffb700 1 -- 172.21.15.19:0/385805571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf34011af0 con 0x7faf4407de40 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.826+0000 7faf4953e700 1 -- 172.21.15.19:0/385805571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 msgr2=0x7faf4407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.826+0000 7faf4953e700 1 --2- 172.21.15.19:0/385805571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4407c2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faf340098a0 tx=0x7faf34009bb0 comp rx=0 tx=0).stop 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.826+0000 7faf4953e700 1 -- 172.21.15.19:0/385805571 shutdown_connections 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.826+0000 7faf4953e700 1 --2- 172.21.15.19:0/385805571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4407c2f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.826+0000 7faf4953e700 1 -- 172.21.15.19:0/385805571 >> 172.21.15.19:0/385805571 conn(0x7faf44103b70 msgr2=0x7faf44105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.826+0000 7faf4953e700 1 -- 172.21.15.19:0/385805571 shutdown_connections 2024-04-23T18:15:58.705 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.826+0000 7faf4953e700 1 -- 172.21.15.19:0/385805571 wait complete. 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.827+0000 7faf4953e700 1 Processor -- start 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.827+0000 7faf4953e700 1 -- start start 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.827+0000 7faf4953e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.827+0000 7faf4953e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf4410ea80 con 0x7faf4407de40 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.827+0000 7faf42ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.827+0000 7faf42ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34114/0 (socket says 172.21.15.19:34114) 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.827+0000 7faf42ffd700 1 -- 172.21.15.19:0/3734126136 learned_addr learned my addr 172.21.15.19:0/3734126136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.828+0000 7faf42ffd700 1 -- 172.21.15.19:0/3734126136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4419ee60 con 0x7faf4407de40 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.828+0000 7faf42ffd700 1 --2- 172.21.15.19:0/3734126136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4419e920 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf34003f40 tx=0x7faf34011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.828+0000 7faf2bfff700 1 -- 172.21.15.19:0/3734126136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf340038b0 con 0x7faf4407de40 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.828+0000 7faf4953e700 1 -- 172.21.15.19:0/3734126136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4419f0f0 con 0x7faf4407de40 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.828+0000 7faf2bfff700 1 -- 172.21.15.19:0/3734126136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faf34003a50 con 0x7faf4407de40 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.828+0000 7faf2bfff700 1 -- 172.21.15.19:0/3734126136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf3401a5c0 con 0x7faf4407de40 2024-04-23T18:15:58.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.828+0000 7faf4953e700 1 -- 172.21.15.19:0/3734126136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf441a1de0 con 0x7faf4407de40 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.829+0000 7faf2bfff700 1 -- 172.21.15.19:0/3734126136 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7faf3401a8c0 con 0x7faf4407de40 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.829+0000 7faf2bfff700 1 --2- 172.21.15.19:0/3734126136 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7faf2c039510 0x7faf2c03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.829+0000 7faf4953e700 1 -- 172.21.15.19:0/3734126136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf440640a0 con 0x7faf4407de40 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.829+0000 7faf2bfff700 1 -- 172.21.15.19:0/3734126136 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faf3404dbd0 con 0x7faf4407de40 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.829+0000 7faf427fc700 1 --2- 172.21.15.19:0/3734126136 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7faf2c039510 0x7faf2c03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.830+0000 7faf427fc700 1 --2- 172.21.15.19:0/3734126136 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7faf2c039510 0x7faf2c03b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf38006fd0 tx=0x7faf38006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:57.835+0000 7faf2bfff700 1 -- 172.21.15.19:0/3734126136 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf34019c70 con 0x7faf4407de40 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.041+0000 7faf4953e700 1 -- 172.21.15.19:0/3734126136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7faf441a3130 con 0x7faf4407de40 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.042+0000 7faf2bfff700 1 -- 172.21.15.19:0/3734126136 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7faf3404d5f0 con 0x7faf4407de40 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.044+0000 7faf4953e700 1 -- 172.21.15.19:0/3734126136 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7faf2c039510 msgr2=0x7faf2c03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.044+0000 7faf4953e700 1 --2- 172.21.15.19:0/3734126136 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7faf2c039510 0x7faf2c03b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf38006fd0 tx=0x7faf38006e40 comp rx=0 tx=0).stop 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.044+0000 7faf4953e700 1 -- 172.21.15.19:0/3734126136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 msgr2=0x7faf4419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.044+0000 7faf4953e700 1 --2- 172.21.15.19:0/3734126136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4419e920 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf34003f40 tx=0x7faf34011890 comp rx=0 tx=0).stop 2024-04-23T18:15:58.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.044+0000 7faf4953e700 1 -- 172.21.15.19:0/3734126136 shutdown_connections 2024-04-23T18:15:58.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.044+0000 7faf4953e700 1 --2- 172.21.15.19:0/3734126136 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7faf2c039510 0x7faf2c03b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:58.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.044+0000 7faf4953e700 1 --2- 172.21.15.19:0/3734126136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4407de40 0x7faf4419e920 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:58.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.044+0000 7faf4953e700 1 -- 172.21.15.19:0/3734126136 >> 172.21.15.19:0/3734126136 conn(0x7faf44103b70 msgr2=0x7faf441048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:58.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.044+0000 7faf4953e700 1 -- 172.21.15.19:0/3734126136 shutdown_connections 2024-04-23T18:15:58.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:58.044+0000 7faf4953e700 1 -- 172.21.15.19:0/3734126136 wait complete. 2024-04-23T18:15:58.708 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-04-23T18:15:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:58 smithi019 ceph-mon[29215]: mgrmap e4: x(active, since 2s) 2024-04-23T18:15:58.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:58 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3734126136' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T18:15:59.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:15:59 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4154918298' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-23T18:15:59.974 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:59 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: ignoring --setuser ceph since I am not root 2024-04-23T18:15:59.974 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:59 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: ignoring --setgroup ceph since I am not root 2024-04-23T18:15:59.974 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:15:59 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:15:59.836+0000 7fd170fdc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:16:00.260 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:00.028+0000 7fd170fdc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:16:00.260 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:00.133+0000 7fd170fdc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:16:00.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.135+0000 7f598260e700 1 Processor -- start 2024-04-23T18:16:00.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.135+0000 7f598260e700 1 -- start start 2024-04-23T18:16:00.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.135+0000 7f598260e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c1077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:00.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.135+0000 7f598260e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f597c107d80 con 0x7f597c107390 2024-04-23T18:16:00.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.135+0000 7f597bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c1077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:00.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.135+0000 7f597bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c1077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34122/0 (socket says 172.21.15.19:34122) 2024-04-23T18:16:00.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.135+0000 7f597bfff700 1 -- 172.21.15.19:0/3470363620 learned_addr learned my addr 172.21.15.19:0/3470363620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:00.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.136+0000 7f597bfff700 1 -- 172.21.15.19:0/3470363620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c108590 con 0x7f597c107390 2024-04-23T18:16:00.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.136+0000 7f597bfff700 1 --2- 172.21.15.19:0/3470363620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c1077b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098a0 tx=0x7f596c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e568540eb32f7524 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.136+0000 7f597affd700 1 -- 172.21.15.19:0/3470363620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f596c004160 con 0x7f597c107390 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.136+0000 7f597affd700 1 -- 172.21.15.19:0/3470363620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f596c004300 con 0x7f597c107390 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.136+0000 7f597affd700 1 -- 172.21.15.19:0/3470363620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f596c011af0 con 0x7f597c107390 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.137+0000 7f598260e700 1 -- 172.21.15.19:0/3470363620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 msgr2=0x7f597c1077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.137+0000 7f598260e700 1 --2- 172.21.15.19:0/3470363620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c1077b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098a0 tx=0x7f596c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.137+0000 7f598260e700 1 -- 172.21.15.19:0/3470363620 shutdown_connections 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.137+0000 7f598260e700 1 --2- 172.21.15.19:0/3470363620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c1077b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.137+0000 7f598260e700 1 -- 172.21.15.19:0/3470363620 >> 172.21.15.19:0/3470363620 conn(0x7f597c1029b0 msgr2=0x7f597c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.137+0000 7f598260e700 1 -- 172.21.15.19:0/3470363620 shutdown_connections 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.137+0000 7f598260e700 1 -- 172.21.15.19:0/3470363620 wait complete. 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.137+0000 7f598260e700 1 Processor -- start 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.137+0000 7f598260e700 1 -- start start 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.138+0000 7f598260e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c192460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:00.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.138+0000 7f598260e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f597c10b940 con 0x7f597c107390 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.138+0000 7f597bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c192460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.138+0000 7f597bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c192460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34132/0 (socket says 172.21.15.19:34132) 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.138+0000 7f597bfff700 1 -- 172.21.15.19:0/4154918298 learned_addr learned my addr 172.21.15.19:0/4154918298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.138+0000 7f597bfff700 1 -- 172.21.15.19:0/4154918298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c1929a0 con 0x7f597c107390 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.138+0000 7f597bfff700 1 --2- 172.21.15.19:0/4154918298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c192460 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f596c009fb0 tx=0x7f596c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.139+0000 7f59797fa700 1 -- 172.21.15.19:0/4154918298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f596c0036a0 con 0x7f597c107390 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.139+0000 7f59797fa700 1 -- 172.21.15.19:0/4154918298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f596c01a430 con 0x7f597c107390 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.139+0000 7f598260e700 1 -- 172.21.15.19:0/4154918298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c190c30 con 0x7f597c107390 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.139+0000 7f59797fa700 1 -- 172.21.15.19:0/4154918298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f596c019550 con 0x7f597c107390 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.139+0000 7f598260e700 1 -- 172.21.15.19:0/4154918298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c191110 con 0x7f597c107390 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.139+0000 7f59797fa700 1 -- 172.21.15.19:0/4154918298 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f596c019850 con 0x7f597c107390 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.140+0000 7f59797fa700 1 --2- 172.21.15.19:0/4154918298 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f5964039510 0x7f596403b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.140+0000 7f59797fa700 1 -- 172.21.15.19:0/4154918298 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f596c04d8a0 con 0x7f597c107390 2024-04-23T18:16:00.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.140+0000 7f597b7fe700 1 --2- 172.21.15.19:0/4154918298 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f5964039510 0x7f596403b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.140+0000 7f598260e700 1 -- 172.21.15.19:0/4154918298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597c0640a0 con 0x7f597c107390 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.140+0000 7f597b7fe700 1 --2- 172.21.15.19:0/4154918298 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f5964039510 0x7f596403b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5970006fd0 tx=0x7f5970006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.145+0000 7f59797fa700 1 -- 172.21.15.19:0/4154918298 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f596c016dd0 con 0x7f597c107390 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.414+0000 7f598260e700 1 -- 172.21.15.19:0/4154918298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f597c196110 con 0x7f597c107390 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.656+0000 7f59797fa700 1 -- 172.21.15.19:0/4154918298 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f596c032020 con 0x7f597c107390 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.658+0000 7f59797fa700 1 -- 172.21.15.19:0/4154918298 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f596c02d960 con 0x7f597c107390 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.659+0000 7f598260e700 1 -- 172.21.15.19:0/4154918298 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f5964039510 msgr2=0x7f596403b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.659+0000 7f598260e700 1 --2- 172.21.15.19:0/4154918298 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f5964039510 0x7f596403b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5970006fd0 tx=0x7f5970006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.659+0000 7f598260e700 1 -- 172.21.15.19:0/4154918298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 msgr2=0x7f597c192460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.659+0000 7f598260e700 1 --2- 172.21.15.19:0/4154918298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c192460 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f596c009fb0 tx=0x7f596c011940 comp rx=0 tx=0).stop 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.659+0000 7f598260e700 1 -- 172.21.15.19:0/4154918298 shutdown_connections 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.659+0000 7f598260e700 1 --2- 172.21.15.19:0/4154918298 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f5964039510 0x7f596403b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.659+0000 7f598260e700 1 --2- 172.21.15.19:0/4154918298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f597c107390 0x7f597c192460 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.659+0000 7f598260e700 1 -- 172.21.15.19:0/4154918298 >> 172.21.15.19:0/4154918298 conn(0x7f597c1029b0 msgr2=0x7f597c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.660+0000 7f598260e700 1 -- 172.21.15.19:0/4154918298 shutdown_connections 2024-04-23T18:16:00.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:15:59.660+0000 7f598260e700 1 -- 172.21.15.19:0/4154918298 wait complete. 2024-04-23T18:16:00.547 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:00.463+0000 7fd170fdc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:16:00.548 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:00.554+0000 7fd170fdc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:16:00.913 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:00 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4154918298' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-23T18:16:00.913 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:00 smithi019 ceph-mon[29215]: mgrmap e5: x(active, since 4s) 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.711+0000 7f5102079700 1 Processor -- start 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.711+0000 7f5102079700 1 -- start start 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.711+0000 7f5102079700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc10b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.711+0000 7f5102079700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc07be60 con 0x7f50fc108fb0 2024-04-23T18:16:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.711+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc10b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.711+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc10b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37636/0 (socket says 172.21.15.19:37636) 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.711+0000 7f50fb7fe700 1 -- 172.21.15.19:0/225558447 learned_addr learned my addr 172.21.15.19:0/225558447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.711+0000 7f50fb7fe700 1 -- 172.21.15.19:0/225558447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc10b8e0 con 0x7f50fc108fb0 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.711+0000 7f50fb7fe700 1 --2- 172.21.15.19:0/225558447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc10b3a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0098a0 tx=0x7f50ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4557af0be9ecc98c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.712+0000 7f50fa7fc700 1 -- 172.21.15.19:0/225558447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ec004160 con 0x7f50fc108fb0 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.712+0000 7f50fa7fc700 1 -- 172.21.15.19:0/225558447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f50ec004300 con 0x7f50fc108fb0 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.712+0000 7f50fa7fc700 1 -- 172.21.15.19:0/225558447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ec011af0 con 0x7f50fc108fb0 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.712+0000 7f5102079700 1 -- 172.21.15.19:0/225558447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 msgr2=0x7f50fc10b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.712+0000 7f5102079700 1 --2- 172.21.15.19:0/225558447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc10b3a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0098a0 tx=0x7f50ec009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.712+0000 7f5102079700 1 -- 172.21.15.19:0/225558447 shutdown_connections 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.712+0000 7f5102079700 1 --2- 172.21.15.19:0/225558447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc10b3a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.712+0000 7f5102079700 1 -- 172.21.15.19:0/225558447 >> 172.21.15.19:0/225558447 conn(0x7f50fc1029c0 msgr2=0x7f50fc104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.712+0000 7f5102079700 1 -- 172.21.15.19:0/225558447 shutdown_connections 2024-04-23T18:16:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.713+0000 7f5102079700 1 -- 172.21.15.19:0/225558447 wait complete. 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.713+0000 7f5102079700 1 Processor -- start 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.713+0000 7f5102079700 1 -- start start 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.714+0000 7f5102079700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc108830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.714+0000 7f5102079700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc10c850 con 0x7f50fc108fb0 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.714+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.714+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc108830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37652/0 (socket says 172.21.15.19:37652) 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.714+0000 7f50fb7fe700 1 -- 172.21.15.19:0/3094033927 learned_addr learned my addr 172.21.15.19:0/3094033927 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.714+0000 7f50fb7fe700 1 -- 172.21.15.19:0/3094033927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc108d70 con 0x7f50fc108fb0 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.714+0000 7f50fb7fe700 1 --2- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc108830 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f50ec011ac0 tx=0x7f50ec011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.714+0000 7f50f8ff9700 1 -- 172.21.15.19:0/3094033927 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ec003b20 con 0x7f50fc108fb0 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.715+0000 7f50f8ff9700 1 -- 172.21.15.19:0/3094033927 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f50ec01a8c0 con 0x7f50fc108fb0 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.715+0000 7f5102079700 1 -- 172.21.15.19:0/3094033927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc106f60 con 0x7f50fc108fb0 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.715+0000 7f50f8ff9700 1 -- 172.21.15.19:0/3094033927 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ec0248d0 con 0x7f50fc108fb0 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.715+0000 7f5102079700 1 -- 172.21.15.19:0/3094033927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc1074a0 con 0x7f50fc108fb0 2024-04-23T18:16:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.716+0000 7f50f8ff9700 1 -- 172.21.15.19:0/3094033927 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f50ec022020 con 0x7f50fc108fb0 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.716+0000 7f5102079700 1 -- 172.21.15.19:0/3094033927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50fc0640a0 con 0x7f50fc108fb0 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.716+0000 7f50f8ff9700 1 --2- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f50e40395b0 0x7f50e403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.716+0000 7f50f8ff9700 1 -- 172.21.15.19:0/3094033927 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f50ec04d570 con 0x7f50fc108fb0 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.716+0000 7f50faffd700 1 -- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f50e40395b0 msgr2=0x7f50e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1085063871 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.716+0000 7f50faffd700 1 --2- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f50e40395b0 0x7f50e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.721+0000 7f50f8ff9700 1 -- 172.21.15.19:0/3094033927 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50ec04d2d0 con 0x7f50fc108fb0 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.917+0000 7f50faffd700 1 -- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f50e40395b0 msgr2=0x7f50e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1085063871 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:00.917+0000 7f50faffd700 1 --2- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f50e40395b0 0x7f50e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.035+0000 7f5102079700 1 -- 172.21.15.19:0/3094033927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f50fc195ee0 con 0x7f50fc108fb0 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.035+0000 7f50f8ff9700 1 -- 172.21.15.19:0/3094033927 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f50ec00b5a0 con 0x7f50fc108fb0 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.038+0000 7f5102079700 1 -- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f50e40395b0 msgr2=0x7f50e403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.038+0000 7f5102079700 1 --2- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f50e40395b0 0x7f50e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:01.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.038+0000 7f5102079700 1 -- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 msgr2=0x7f50fc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:01.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.038+0000 7f5102079700 1 --2- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc108830 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f50ec011ac0 tx=0x7f50ec011ba0 comp rx=0 tx=0).stop 2024-04-23T18:16:01.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.038+0000 7f5102079700 1 -- 172.21.15.19:0/3094033927 shutdown_connections 2024-04-23T18:16:01.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.038+0000 7f5102079700 1 --2- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f50e40395b0 0x7f50e403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:01.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.038+0000 7f5102079700 1 --2- 172.21.15.19:0/3094033927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50fc108fb0 0x7f50fc108830 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:01.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.038+0000 7f5102079700 1 -- 172.21.15.19:0/3094033927 >> 172.21.15.19:0/3094033927 conn(0x7f50fc1029c0 msgr2=0x7f50fc103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:01.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.038+0000 7f5102079700 1 -- 172.21.15.19:0/3094033927 shutdown_connections 2024-04-23T18:16:01.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:01.038+0000 7f5102079700 1 -- 172.21.15.19:0/3094033927 wait complete. 2024-04-23T18:16:01.643 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-04-23T18:16:01.643 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-04-23T18:16:01.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:01 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3094033927' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T18:16:01.885 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:01.813+0000 7fd170fdc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:16:02.166 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:01.994+0000 7fd170fdc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:16:02.167 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:02.173+0000 7fd170fdc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:16:02.741 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:02.473+0000 7fd170fdc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:16:02.741 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:02.569+0000 7fd170fdc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:16:03.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:02.781+0000 7fd170fdc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:16:03.011 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:02.981+0000 7fd170fdc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:16:04.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:03.610+0000 7fd170fdc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:16:04.356 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:04.051+0000 7fd170fdc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:16:04.357 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:04.151+0000 7fd170fdc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:16:04.357 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:04.362+0000 7fd170fdc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:16:05.260 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:04.929+0000 7fd170fdc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:16:05.261 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:05.027+0000 7fd170fdc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:16:05.635 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:05.282+0000 7fd170fdc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:16:05.635 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:05.420+0000 7fd170fdc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:16:05.635 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:05.519+0000 7fd170fdc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:16:05.635 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:05.641+0000 7fd170fdc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:16:06.260 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:05.959+0000 7fd170fdc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:16:06.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:06.587+0000 7fd170fdc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:16:06.863 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:06.815+0000 7fd170fdc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: Active manager daemon x restarted 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: Activating manager daemon x 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: osdmap e2: 0 total, 0 up, 0 in 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: mgrmap e6: x(active, starting, since 0.00379897s) 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: Manager daemon x is now available 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:06 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:08.128 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:07 smithi019 ceph-mon[29215]: Found migration_current of "None". Setting to last migration. 2024-04-23T18:16:08.128 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:07 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:16:08.128 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:07 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:16:08.128 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:07 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:08.128 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:07 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:08.128 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:07 smithi019 ceph-mon[29215]: mgrmap e7: x(active, since 1.0056s) 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.086+0000 7f8cc8cfb700 1 Processor -- start 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.086+0000 7f8cc8cfb700 1 -- start start 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.086+0000 7f8cc8cfb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc410b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.086+0000 7f8cc8cfb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc407be40 con 0x7f8cc4108f80 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.087+0000 7f8cc259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc410b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.087+0000 7f8cc259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc410b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37668/0 (socket says 172.21.15.19:37668) 2024-04-23T18:16:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.087+0000 7f8cc259c700 1 -- 172.21.15.19:0/4212037903 learned_addr learned my addr 172.21.15.19:0/4212037903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.087+0000 7f8cc259c700 1 -- 172.21.15.19:0/4212037903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc410b8b0 con 0x7f8cc4108f80 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.087+0000 7f8cc259c700 1 --2- 172.21.15.19:0/4212037903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc410b370 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40098a0 tx=0x7f8cb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac179ba1d37aa73 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.087+0000 7f8cc159a700 1 -- 172.21.15.19:0/4212037903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb4004160 con 0x7f8cc4108f80 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.087+0000 7f8cc159a700 1 -- 172.21.15.19:0/4212037903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8cb4004300 con 0x7f8cc4108f80 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.087+0000 7f8cc159a700 1 -- 172.21.15.19:0/4212037903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb4011af0 con 0x7f8cc4108f80 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.088+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/4212037903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 msgr2=0x7f8cc410b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.088+0000 7f8cc8cfb700 1 --2- 172.21.15.19:0/4212037903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc410b370 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40098a0 tx=0x7f8cb4009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.088+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/4212037903 shutdown_connections 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.088+0000 7f8cc8cfb700 1 --2- 172.21.15.19:0/4212037903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc410b370 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.088+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/4212037903 >> 172.21.15.19:0/4212037903 conn(0x7f8cc41029b0 msgr2=0x7f8cc4104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.088+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/4212037903 shutdown_connections 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.088+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/4212037903 wait complete. 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.089+0000 7f8cc8cfb700 1 Processor -- start 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.089+0000 7f8cc8cfb700 1 -- start start 2024-04-23T18:16:08.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.089+0000 7f8cc8cfb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.089+0000 7f8cc8cfb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc410c810 con 0x7f8cc4108f80 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.089+0000 7f8cc259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.090+0000 7f8cc259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37684/0 (socket says 172.21.15.19:37684) 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.090+0000 7f8cc259c700 1 -- 172.21.15.19:0/1083985013 learned_addr learned my addr 172.21.15.19:0/1083985013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.090+0000 7f8cc259c700 1 -- 172.21.15.19:0/1083985013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc419c9c0 con 0x7f8cc4108f80 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.090+0000 7f8cc259c700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc419c480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009fb0 tx=0x7f8cb4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.090+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb40036a0 con 0x7f8cc4108f80 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.090+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/1083985013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc419cc50 con 0x7f8cc4108f80 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.090+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8cb401a430 con 0x7f8cc4108f80 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.090+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb4019550 con 0x7f8cc4108f80 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.090+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/1083985013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc419f940 con 0x7f8cc4108f80 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.091+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f8cb4019850 con 0x7f8cc4108f80 2024-04-23T18:16:08.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.092+0000 7f8cb37fe700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 0x7f8cac03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.092+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 --> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8cac03c130 con 0x7f8cac039560 2024-04-23T18:16:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.092+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8cb404d930 con 0x7f8cc4108f80 2024-04-23T18:16:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.092+0000 7f8cc1d9b700 1 -- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 msgr2=0x7f8cac03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1085063871 2024-04-23T18:16:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.092+0000 7f8cc1d9b700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 0x7f8cac03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:16:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.293+0000 7f8cc1d9b700 1 -- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 msgr2=0x7f8cac03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1085063871 2024-04-23T18:16:08.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.293+0000 7f8cc1d9b700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 0x7f8cac03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:16:08.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.694+0000 7f8cc1d9b700 1 -- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 msgr2=0x7f8cac03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1085063871 2024-04-23T18:16:08.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:02.694+0000 7f8cc1d9b700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 0x7f8cac03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:16:08.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:03.495+0000 7f8cc1d9b700 1 -- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 msgr2=0x7f8cac03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1085063871 2024-04-23T18:16:08.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:03.495+0000 7f8cc1d9b700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 0x7f8cac03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T18:16:08.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:05.096+0000 7f8cc1d9b700 1 -- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 msgr2=0x7f8cac03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1085063871 2024-04-23T18:16:08.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:05.096+0000 7f8cc1d9b700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 0x7f8cac03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T18:16:08.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:06.819+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f8cb4019b90 con 0x7f8cc4108f80 2024-04-23T18:16:08.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:06.820+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 msgr2=0x7f8cac03ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:16:08.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:06.820+0000 7f8cb37fe700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 0x7f8cac03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:08.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.820+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f8cb404f6a0 con 0x7f8cc4108f80 2024-04-23T18:16:08.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.820+0000 7f8cb37fe700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f8cac03ce50 0x7f8cac03f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:08.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.820+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 --> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8cb401fec0 con 0x7f8cac03ce50 2024-04-23T18:16:08.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.821+0000 7f8cc1d9b700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f8cac03ce50 0x7f8cac03f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:08.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.821+0000 7f8cc1d9b700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f8cac03ce50 0x7f8cac03f240 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8003a10 tx=0x7f8cb8009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:08.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.822+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 <== mgr.14118 v2:172.21.15.19:6800/2431053142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f8cb401fec0 con 0x7f8cac03ce50 2024-04-23T18:16:08.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/1083985013 --> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8cc4069b90 con 0x7f8cac03ce50 2024-04-23T18:16:08.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cb37fe700 1 -- 172.21.15.19:0/1083985013 <== mgr.14118 v2:172.21.15.19:6800/2431053142 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8cc4069b90 con 0x7f8cac03ce50 2024-04-23T18:16:08.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f8cac03ce50 msgr2=0x7f8cac03f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:08.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f8cac03ce50 0x7f8cac03f240 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8003a10 tx=0x7f8cb8009290 comp rx=0 tx=0).stop 2024-04-23T18:16:08.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 msgr2=0x7f8cc419c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:08.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc419c480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009fb0 tx=0x7f8cb4011940 comp rx=0 tx=0).stop 2024-04-23T18:16:08.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/1083985013 shutdown_connections 2024-04-23T18:16:08.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc4108f80 0x7f8cc419c480 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:08.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f8cac03ce50 0x7f8cac03f240 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:08.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 --2- 172.21.15.19:0/1083985013 >> [v2:172.21.15.19:6800/1085063871,v1:172.21.15.19:6801/1085063871] conn(0x7f8cac039560 0x7f8cac03ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:08.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/1083985013 >> 172.21.15.19:0/1083985013 conn(0x7f8cc41029b0 msgr2=0x7f8cc4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:08.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/1083985013 shutdown_connections 2024-04-23T18:16:08.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:07.829+0000 7f8cc8cfb700 1 -- 172.21.15.19:0/1083985013 wait complete. 2024-04-23T18:16:08.455 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-04-23T18:16:08.455 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-04-23T18:16:09.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:08 smithi019 ceph-mon[29215]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T18:16:09.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:08 smithi019 ceph-mon[29215]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T18:16:09.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:08 smithi019 ceph-mon[29215]: [23/Apr/2024:18:16:07] ENGINE Bus STARTING 2024-04-23T18:16:09.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:08 smithi019 ceph-mon[29215]: [23/Apr/2024:18:16:07] ENGINE Serving on https://172.21.15.19:7150 2024-04-23T18:16:09.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:08 smithi019 ceph-mon[29215]: [23/Apr/2024:18:16:08] ENGINE Serving on http://172.21.15.19:8765 2024-04-23T18:16:09.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:08 smithi019 ceph-mon[29215]: [23/Apr/2024:18:16:08] ENGINE Bus STARTED 2024-04-23T18:16:09.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:08 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:09.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.919+0000 7fe6af279700 1 Processor -- start 2024-04-23T18:16:09.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.919+0000 7fe6af279700 1 -- start start 2024-04-23T18:16:09.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.919+0000 7fe6af279700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a8108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.919+0000 7fe6af279700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8108f00 con 0x7fe6a8108510 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.919+0000 7fe6ad015700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a8108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.920+0000 7fe6ad015700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a8108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37762/0 (socket says 172.21.15.19:37762) 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.920+0000 7fe6ad015700 1 -- 172.21.15.19:0/3881676963 learned_addr learned my addr 172.21.15.19:0/3881676963 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.920+0000 7fe6ad015700 1 -- 172.21.15.19:0/3881676963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a81096c0 con 0x7fe6a8108510 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.920+0000 7fe6ad015700 1 --2- 172.21.15.19:0/3881676963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a8108930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40098a0 tx=0x7fe6a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e529f7d137f5529d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.920+0000 7fe69ffff700 1 -- 172.21.15.19:0/3881676963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a4004160 con 0x7fe6a8108510 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.920+0000 7fe69ffff700 1 -- 172.21.15.19:0/3881676963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe6a4004300 con 0x7fe6a8108510 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.920+0000 7fe69ffff700 1 -- 172.21.15.19:0/3881676963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a4011af0 con 0x7fe6a8108510 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.921+0000 7fe6af279700 1 -- 172.21.15.19:0/3881676963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 msgr2=0x7fe6a8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.921+0000 7fe6af279700 1 --2- 172.21.15.19:0/3881676963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a8108930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40098a0 tx=0x7fe6a4009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.921+0000 7fe6af279700 1 -- 172.21.15.19:0/3881676963 shutdown_connections 2024-04-23T18:16:09.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.921+0000 7fe6af279700 1 --2- 172.21.15.19:0/3881676963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a8108930 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.921+0000 7fe6af279700 1 -- 172.21.15.19:0/3881676963 >> 172.21.15.19:0/3881676963 conn(0x7fe6a8103b70 msgr2=0x7fe6a8105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.921+0000 7fe6af279700 1 -- 172.21.15.19:0/3881676963 shutdown_connections 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.921+0000 7fe6af279700 1 -- 172.21.15.19:0/3881676963 wait complete. 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.921+0000 7fe6af279700 1 Processor -- start 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.922+0000 7fe6af279700 1 -- start start 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.922+0000 7fe6af279700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a807db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.922+0000 7fe6af279700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a810ca70 con 0x7fe6a8108510 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.922+0000 7fe6ad015700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a807db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.922+0000 7fe6ad015700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a807db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37774/0 (socket says 172.21.15.19:37774) 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.922+0000 7fe6ad015700 1 -- 172.21.15.19:0/2185788543 learned_addr learned my addr 172.21.15.19:0/2185788543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.922+0000 7fe6ad015700 1 -- 172.21.15.19:0/2185788543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a807e070 con 0x7fe6a8108510 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.922+0000 7fe6ad015700 1 --2- 172.21.15.19:0/2185788543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a807db30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4003f40 tx=0x7fe6a4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.922+0000 7fe69e7fc700 1 -- 172.21.15.19:0/2185788543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a40038b0 con 0x7fe6a8108510 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.923+0000 7fe6af279700 1 -- 172.21.15.19:0/2185788543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a807c2b0 con 0x7fe6a8108510 2024-04-23T18:16:09.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.923+0000 7fe69e7fc700 1 -- 172.21.15.19:0/2185788543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe6a4003a50 con 0x7fe6a8108510 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.923+0000 7fe69e7fc700 1 -- 172.21.15.19:0/2185788543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a401a5c0 con 0x7fe6a8108510 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.923+0000 7fe6af279700 1 -- 172.21.15.19:0/2185788543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a807c790 con 0x7fe6a8108510 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.924+0000 7fe69e7fc700 1 -- 172.21.15.19:0/2185788543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe6a4024430 con 0x7fe6a8108510 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.924+0000 7fe6af279700 1 -- 172.21.15.19:0/2185788543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a80640a0 con 0x7fe6a8108510 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.924+0000 7fe69e7fc700 1 --2- 172.21.15.19:0/2185788543 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fe694039560 0x7fe69403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.924+0000 7fe69e7fc700 1 -- 172.21.15.19:0/2185788543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe6a404dad0 con 0x7fe6a8108510 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.924+0000 7fe6ac814700 1 --2- 172.21.15.19:0/2185788543 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fe694039560 0x7fe69403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.924+0000 7fe6ac814700 1 --2- 172.21.15.19:0/2185788543 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fe694039560 0x7fe69403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe698006fd0 tx=0x7fe698006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:08.929+0000 7fe69e7fc700 1 -- 172.21.15.19:0/2185788543 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6a402d910 con 0x7fe6a8108510 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.157+0000 7fe6af279700 1 -- 172.21.15.19:0/2185788543 --> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fe6a8195ef0 con 0x7fe694039560 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.161+0000 7fe69e7fc700 1 -- 172.21.15.19:0/2185788543 <== mgr.14118 v2:172.21.15.19:6800/2431053142 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe6a8195ef0 con 0x7fe694039560 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.164+0000 7fe6af279700 1 -- 172.21.15.19:0/2185788543 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fe694039560 msgr2=0x7fe69403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.164+0000 7fe6af279700 1 --2- 172.21.15.19:0/2185788543 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fe694039560 0x7fe69403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe698006fd0 tx=0x7fe698006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.164+0000 7fe6af279700 1 -- 172.21.15.19:0/2185788543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 msgr2=0x7fe6a807db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:09.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.164+0000 7fe6af279700 1 --2- 172.21.15.19:0/2185788543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a807db30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4003f40 tx=0x7fe6a4011890 comp rx=0 tx=0).stop 2024-04-23T18:16:09.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.164+0000 7fe6af279700 1 -- 172.21.15.19:0/2185788543 shutdown_connections 2024-04-23T18:16:09.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.164+0000 7fe6af279700 1 --2- 172.21.15.19:0/2185788543 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fe694039560 0x7fe69403ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:09.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.164+0000 7fe6af279700 1 --2- 172.21.15.19:0/2185788543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6a8108510 0x7fe6a807db30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:09.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.164+0000 7fe6af279700 1 -- 172.21.15.19:0/2185788543 >> 172.21.15.19:0/2185788543 conn(0x7fe6a8103b70 msgr2=0x7fe6a8104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:09.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.164+0000 7fe6af279700 1 -- 172.21.15.19:0/2185788543 shutdown_connections 2024-04-23T18:16:09.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:09.164+0000 7fe6af279700 1 -- 172.21.15.19:0/2185788543 wait complete. 2024-04-23T18:16:10.093 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:09 smithi019 ceph-mon[29215]: mgrmap e8: x(active, since 2s) 2024-04-23T18:16:10.093 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:09 smithi019 ceph-mon[29215]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:10.093 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:09 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:10.093 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:09 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:11.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-23T18:16:11.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.308+0000 7f4fc0850700 1 Processor -- start 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.308+0000 7f4fc0850700 1 -- start start 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.308+0000 7f4fc0850700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.308+0000 7f4fc0850700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc108f00 con 0x7f4fbc108510 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.308+0000 7f4fba59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.308+0000 7f4fba59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36314/0 (socket says 172.21.15.19:36314) 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.308+0000 7f4fba59c700 1 -- 172.21.15.19:0/3175048170 learned_addr learned my addr 172.21.15.19:0/3175048170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.309+0000 7f4fba59c700 1 -- 172.21.15.19:0/3175048170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc1096c0 con 0x7f4fbc108510 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.309+0000 7f4fba59c700 1 --2- 172.21.15.19:0/3175048170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc108930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0098a0 tx=0x7f4fac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7b42d0ced5b610 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.309+0000 7f4fb959a700 1 -- 172.21.15.19:0/3175048170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fac004160 con 0x7f4fbc108510 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.309+0000 7f4fb959a700 1 -- 172.21.15.19:0/3175048170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4fac004300 con 0x7f4fbc108510 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.309+0000 7f4fb959a700 1 -- 172.21.15.19:0/3175048170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fac011af0 con 0x7f4fbc108510 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.310+0000 7f4fc0850700 1 -- 172.21.15.19:0/3175048170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 msgr2=0x7f4fbc108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.310+0000 7f4fc0850700 1 --2- 172.21.15.19:0/3175048170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc108930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0098a0 tx=0x7f4fac009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:11.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.310+0000 7f4fc0850700 1 -- 172.21.15.19:0/3175048170 shutdown_connections 2024-04-23T18:16:11.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.310+0000 7f4fc0850700 1 --2- 172.21.15.19:0/3175048170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc108930 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:11.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.310+0000 7f4fc0850700 1 -- 172.21.15.19:0/3175048170 >> 172.21.15.19:0/3175048170 conn(0x7f4fbc103b70 msgr2=0x7f4fbc105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:11.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.310+0000 7f4fc0850700 1 -- 172.21.15.19:0/3175048170 shutdown_connections 2024-04-23T18:16:11.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.310+0000 7f4fc0850700 1 -- 172.21.15.19:0/3175048170 wait complete. 2024-04-23T18:16:11.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.311+0000 7f4fc0850700 1 Processor -- start 2024-04-23T18:16:11.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.311+0000 7f4fc0850700 1 -- start start 2024-04-23T18:16:11.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.311+0000 7f4fc0850700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:11.126 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.311+0000 7f4fc0850700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc10ca70 con 0x7f4fbc108510 2024-04-23T18:16:11.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.311+0000 7f4fba59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:11.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.311+0000 7f4fba59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc19c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36328/0 (socket says 172.21.15.19:36328) 2024-04-23T18:16:11.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.311+0000 7f4fba59c700 1 -- 172.21.15.19:0/2836877035 learned_addr learned my addr 172.21.15.19:0/2836877035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:11.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.311+0000 7f4fba59c700 1 -- 172.21.15.19:0/2836877035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc19cca0 con 0x7f4fbc108510 2024-04-23T18:16:11.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.312+0000 7f4fba59c700 1 --2- 172.21.15.19:0/2836877035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc19c760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000c00 tx=0x7f4fac011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:11.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.312+0000 7f4fab7fe700 1 -- 172.21.15.19:0/2836877035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fac003b20 con 0x7f4fbc108510 2024-04-23T18:16:11.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.312+0000 7f4fab7fe700 1 -- 172.21.15.19:0/2836877035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4fac01a8c0 con 0x7f4fbc108510 2024-04-23T18:16:11.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.312+0000 7f4fc0850700 1 -- 172.21.15.19:0/2836877035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc19cf30 con 0x7f4fbc108510 2024-04-23T18:16:11.127 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.312+0000 7f4fab7fe700 1 -- 172.21.15.19:0/2836877035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fac024940 con 0x7f4fbc108510 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.312+0000 7f4fc0850700 1 -- 172.21.15.19:0/2836877035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc19fbd0 con 0x7f4fbc108510 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.313+0000 7f4fab7fe700 1 -- 172.21.15.19:0/2836877035 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4fac022020 con 0x7f4fbc108510 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.313+0000 7f4fab7fe700 1 --2- 172.21.15.19:0/2836877035 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4fa4039600 0x7f4fa403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.313+0000 7f4fc0850700 1 -- 172.21.15.19:0/2836877035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fbc069b90 con 0x7f4fbc108510 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.313+0000 7f4fab7fe700 1 -- 172.21.15.19:0/2836877035 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4fac04d7e0 con 0x7f4fbc108510 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.313+0000 7f4fb9d9b700 1 --2- 172.21.15.19:0/2836877035 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4fa4039600 0x7f4fa403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.313+0000 7f4fb9d9b700 1 --2- 172.21.15.19:0/2836877035 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4fa4039600 0x7f4fa403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0006fd0 tx=0x7f4fb0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.318+0000 7f4fab7fe700 1 -- 172.21.15.19:0/2836877035 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fac00be50 con 0x7f4fbc108510 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.525+0000 7f4fc0850700 1 -- 172.21.15.19:0/2836877035 --> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f4fbc062e80 con 0x7f4fa4039600 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.526+0000 7f4fab7fe700 1 -- 172.21.15.19:0/2836877035 <== mgr.14118 v2:172.21.15.19:6800/2431053142 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f4fbc062e80 con 0x7f4fa4039600 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.528+0000 7f4fc0850700 1 -- 172.21.15.19:0/2836877035 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4fa4039600 msgr2=0x7f4fa403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.528+0000 7f4fc0850700 1 --2- 172.21.15.19:0/2836877035 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4fa4039600 0x7f4fa403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0006fd0 tx=0x7f4fb0006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.528+0000 7f4fc0850700 1 -- 172.21.15.19:0/2836877035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 msgr2=0x7f4fbc19c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.528+0000 7f4fc0850700 1 --2- 172.21.15.19:0/2836877035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc19c760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000c00 tx=0x7f4fac011b70 comp rx=0 tx=0).stop 2024-04-23T18:16:11.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.528+0000 7f4fc0850700 1 -- 172.21.15.19:0/2836877035 shutdown_connections 2024-04-23T18:16:11.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.528+0000 7f4fc0850700 1 --2- 172.21.15.19:0/2836877035 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4fa4039600 0x7f4fa403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:11.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.528+0000 7f4fc0850700 1 --2- 172.21.15.19:0/2836877035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fbc108510 0x7f4fbc19c760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:11.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.528+0000 7f4fc0850700 1 -- 172.21.15.19:0/2836877035 >> 172.21.15.19:0/2836877035 conn(0x7f4fbc103b70 msgr2=0x7f4fbc104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:11.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.528+0000 7f4fc0850700 1 -- 172.21.15.19:0/2836877035 shutdown_connections 2024-04-23T18:16:11.129 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:10.528+0000 7f4fc0850700 1 -- 172.21.15.19:0/2836877035 wait complete. 2024-04-23T18:16:11.129 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-04-23T18:16:11.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:10 smithi019 ceph-mon[29215]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:12.738 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: Generating public/private rsa key pair. 2024-04-23T18:16:12.738 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: Your identification has been saved in /tmp/tmp83rd72mg/key. 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: Your public key has been saved in /tmp/tmp83rd72mg/key.pub. 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: The key fingerprint is: 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: SHA256:pGA29k6UnP+4cjvPOnVUt3i7axBFfdfNvbjljOe5MwY ceph-46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: The key's randomart image is: 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: +---[RSA 3072]----+ 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: | .o=| 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: | . o ..X| 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: | * = . .+.=| 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: | + = + .+ = | 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: | + S . O .| 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: | o o. .E = | 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: | . .... = o| 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: | . +o O | 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: | o+*o oo=| 2024-04-23T18:16:12.739 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: +----[SHA256]-----+ 2024-04-23T18:16:13.063 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.551+0000 7f33607fa700 1 Processor -- start 2024-04-23T18:16:13.063 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.551+0000 7f33607fa700 1 -- start start 2024-04-23T18:16:13.063 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.552+0000 7f33607fa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f3358108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:13.063 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.552+0000 7f33607fa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3358108f00 con 0x7f3358108510 2024-04-23T18:16:13.063 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.552+0000 7f335e596700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f3358108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:13.063 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.552+0000 7f335e596700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f3358108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36338/0 (socket says 172.21.15.19:36338) 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.552+0000 7f335e596700 1 -- 172.21.15.19:0/2329889470 learned_addr learned my addr 172.21.15.19:0/2329889470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.552+0000 7f335e596700 1 -- 172.21.15.19:0/2329889470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33581096c0 con 0x7f3358108510 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.552+0000 7f335e596700 1 --2- 172.21.15.19:0/2329889470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f3358108930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f33540098a0 tx=0x7f3354009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2c805d968fbfaf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.553+0000 7f335d594700 1 -- 172.21.15.19:0/2329889470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3354004160 con 0x7f3358108510 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.553+0000 7f335d594700 1 -- 172.21.15.19:0/2329889470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3354004300 con 0x7f3358108510 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.553+0000 7f335d594700 1 -- 172.21.15.19:0/2329889470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3354011af0 con 0x7f3358108510 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.553+0000 7f33607fa700 1 -- 172.21.15.19:0/2329889470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 msgr2=0x7f3358108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.553+0000 7f33607fa700 1 --2- 172.21.15.19:0/2329889470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f3358108930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f33540098a0 tx=0x7f3354009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.553+0000 7f33607fa700 1 -- 172.21.15.19:0/2329889470 shutdown_connections 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.553+0000 7f33607fa700 1 --2- 172.21.15.19:0/2329889470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f3358108930 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.553+0000 7f33607fa700 1 -- 172.21.15.19:0/2329889470 >> 172.21.15.19:0/2329889470 conn(0x7f3358103b70 msgr2=0x7f3358105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.553+0000 7f33607fa700 1 -- 172.21.15.19:0/2329889470 shutdown_connections 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.553+0000 7f33607fa700 1 -- 172.21.15.19:0/2329889470 wait complete. 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.554+0000 7f33607fa700 1 Processor -- start 2024-04-23T18:16:13.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.554+0000 7f33607fa700 1 -- start start 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.554+0000 7f33607fa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f335807db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.554+0000 7f33607fa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f335810ca70 con 0x7f3358108510 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.554+0000 7f335e596700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f335807db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.554+0000 7f335e596700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f335807db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36350/0 (socket says 172.21.15.19:36350) 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.554+0000 7f335e596700 1 -- 172.21.15.19:0/2037665199 learned_addr learned my addr 172.21.15.19:0/2037665199 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.555+0000 7f335e596700 1 -- 172.21.15.19:0/2037665199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335807e070 con 0x7f3358108510 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.555+0000 7f335e596700 1 --2- 172.21.15.19:0/2037665199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f335807db30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3354000c00 tx=0x7f3354011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.555+0000 7f334f7fe700 1 -- 172.21.15.19:0/2037665199 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3354003b20 con 0x7f3358108510 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.555+0000 7f334f7fe700 1 -- 172.21.15.19:0/2037665199 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f335401a8c0 con 0x7f3358108510 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.555+0000 7f33607fa700 1 -- 172.21.15.19:0/2037665199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335807c2b0 con 0x7f3358108510 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.555+0000 7f334f7fe700 1 -- 172.21.15.19:0/2037665199 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3354024940 con 0x7f3358108510 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.555+0000 7f33607fa700 1 -- 172.21.15.19:0/2037665199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335807c790 con 0x7f3358108510 2024-04-23T18:16:13.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.556+0000 7f33607fa700 1 -- 172.21.15.19:0/2037665199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33580640a0 con 0x7f3358108510 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.556+0000 7f334f7fe700 1 -- 172.21.15.19:0/2037665199 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3354022020 con 0x7f3358108510 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.556+0000 7f334f7fe700 1 --2- 172.21.15.19:0/2037665199 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f3344039600 0x7f334403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.556+0000 7f334f7fe700 1 -- 172.21.15.19:0/2037665199 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f335404d9f0 con 0x7f3358108510 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.556+0000 7f335dd95700 1 --2- 172.21.15.19:0/2037665199 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f3344039600 0x7f334403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.557+0000 7f335dd95700 1 --2- 172.21.15.19:0/2037665199 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f3344039600 0x7f334403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3348006fd0 tx=0x7f3348006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.561+0000 7f334f7fe700 1 -- 172.21.15.19:0/2037665199 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f335400b5a0 con 0x7f3358108510 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:11.770+0000 7f33607fa700 1 -- 172.21.15.19:0/2037665199 --> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f3358195ef0 con 0x7f3344039600 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.428+0000 7f334f7fe700 1 -- 172.21.15.19:0/2037665199 <== mgr.14118 v2:172.21.15.19:6800/2431053142 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3358195ef0 con 0x7f3344039600 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.430+0000 7f33607fa700 1 -- 172.21.15.19:0/2037665199 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f3344039600 msgr2=0x7f334403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.430+0000 7f33607fa700 1 --2- 172.21.15.19:0/2037665199 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f3344039600 0x7f334403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3348006fd0 tx=0x7f3348006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.430+0000 7f33607fa700 1 -- 172.21.15.19:0/2037665199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 msgr2=0x7f335807db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.430+0000 7f33607fa700 1 --2- 172.21.15.19:0/2037665199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f335807db30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3354000c00 tx=0x7f3354011b70 comp rx=0 tx=0).stop 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.430+0000 7f33607fa700 1 -- 172.21.15.19:0/2037665199 shutdown_connections 2024-04-23T18:16:13.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.430+0000 7f33607fa700 1 --2- 172.21.15.19:0/2037665199 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f3344039600 0x7f334403bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:13.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.430+0000 7f33607fa700 1 --2- 172.21.15.19:0/2037665199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3358108510 0x7f335807db30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:13.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.430+0000 7f33607fa700 1 -- 172.21.15.19:0/2037665199 >> 172.21.15.19:0/2037665199 conn(0x7f3358103b70 msgr2=0x7f33581048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:13.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.430+0000 7f33607fa700 1 -- 172.21.15.19:0/2037665199 shutdown_connections 2024-04-23T18:16:13.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:12.430+0000 7f33607fa700 1 -- 172.21.15.19:0/2037665199 wait complete. 2024-04-23T18:16:13.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:13 smithi019 ceph-mon[29215]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:13.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:13 smithi019 ceph-mon[29215]: Generating ssh key... 2024-04-23T18:16:13.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:13 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:13.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:13 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2UpI8vamfYH8oFaSv1UuFEJ5ea+Uj+bVdn86+Cd9NmffRPMDzCsXlu/KYeLqXIAKrbOO+i+O53DqTCiGVbSZPoYrcAde0p7l/vK0PBJECzk/2MQH16gAKlD26TchLhnkCTUK9PL4GVCCfcPXgnKOt9CTwa3EDMSL4xx1w+JTa7gUxxlnEy6iD0pRQ03okpse7RqYpn9445Gp1GNCB6rq4ZbtVuNtuOmsp4z0Yk487qsuRYBxI7eu8A37vgRY4Y+uoUWpIIBTksRQ3fqYXE9zynVeYuX2vf2ckN9Q5XFz52f+QfmskeldvU9jq8ozR28hrhJElkUnRbi1jfQscNndZZvrBVGsy1dA9G3z7JCQf+6zzYzGls/rChbm2ehvaJg/cDfcWI8A5VyuP76DYzPiuY7ucobpzZKdvoGDiWQhtnaa0fQx1GFGp5CszK0H7KOkb+Bn7nutL2Sx3ZHhQv192/AiODQTlGjLq/AB961osYayhX18EkCqWkbueBDFFNt8= ceph-46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.537+0000 7f499e4cb700 1 Processor -- start 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.537+0000 7f499e4cb700 1 -- start start 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.537+0000 7f499e4cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f49981077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.537+0000 7f499e4cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4998107d80 con 0x7f4998107390 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.538+0000 7f4997fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f49981077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.538+0000 7f4997fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f49981077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36366/0 (socket says 172.21.15.19:36366) 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.538+0000 7f4997fff700 1 -- 172.21.15.19:0/4185728920 learned_addr learned my addr 172.21.15.19:0/4185728920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.538+0000 7f4997fff700 1 -- 172.21.15.19:0/4185728920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4998108590 con 0x7f4998107390 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.538+0000 7f4997fff700 1 --2- 172.21.15.19:0/4185728920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f49981077b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f498800bb40 tx=0x7f498800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ffb93e1fb9538c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:14.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.538+0000 7f4996ffd700 1 -- 172.21.15.19:0/4185728920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f498800dbe0 con 0x7f4998107390 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.538+0000 7f4996ffd700 1 -- 172.21.15.19:0/4185728920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49880044c0 con 0x7f4998107390 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.538+0000 7f4996ffd700 1 -- 172.21.15.19:0/4185728920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4988010470 con 0x7f4998107390 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.539+0000 7f499e4cb700 1 -- 172.21.15.19:0/4185728920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 msgr2=0x7f49981077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.539+0000 7f499e4cb700 1 --2- 172.21.15.19:0/4185728920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f49981077b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f498800bb40 tx=0x7f498800be50 comp rx=0 tx=0).stop 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.539+0000 7f499e4cb700 1 -- 172.21.15.19:0/4185728920 shutdown_connections 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.539+0000 7f499e4cb700 1 --2- 172.21.15.19:0/4185728920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f49981077b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.539+0000 7f499e4cb700 1 -- 172.21.15.19:0/4185728920 >> 172.21.15.19:0/4185728920 conn(0x7f49981029b0 msgr2=0x7f4998104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.539+0000 7f499e4cb700 1 -- 172.21.15.19:0/4185728920 shutdown_connections 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.539+0000 7f499e4cb700 1 -- 172.21.15.19:0/4185728920 wait complete. 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.540+0000 7f499e4cb700 1 Processor -- start 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.540+0000 7f499e4cb700 1 -- start start 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.540+0000 7f499e4cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f499819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.540+0000 7f499e4cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f499807afe0 con 0x7f4998107390 2024-04-23T18:16:14.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.540+0000 7f4997fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f499819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.540+0000 7f4997fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f499819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36372/0 (socket says 172.21.15.19:36372) 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.540+0000 7f4997fff700 1 -- 172.21.15.19:0/1430913359 learned_addr learned my addr 172.21.15.19:0/1430913359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.540+0000 7f4997fff700 1 -- 172.21.15.19:0/1430913359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499819eb60 con 0x7f4998107390 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.541+0000 7f4997fff700 1 --2- 172.21.15.19:0/1430913359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f499819e620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f498800b340 tx=0x7f4988003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.541+0000 7f49957fa700 1 -- 172.21.15.19:0/1430913359 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4988010720 con 0x7f4998107390 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.541+0000 7f49957fa700 1 -- 172.21.15.19:0/1430913359 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f498801d070 con 0x7f4998107390 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.541+0000 7f499e4cb700 1 -- 172.21.15.19:0/1430913359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499819edf0 con 0x7f4998107390 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.541+0000 7f49957fa700 1 -- 172.21.15.19:0/1430913359 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4988021ba0 con 0x7f4998107390 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.541+0000 7f499e4cb700 1 -- 172.21.15.19:0/1430913359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49981a1ae0 con 0x7f4998107390 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.542+0000 7f49957fa700 1 -- 172.21.15.19:0/1430913359 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f498801f030 con 0x7f4998107390 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.542+0000 7f49957fa700 1 --2- 172.21.15.19:0/1430913359 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f49800395b0 0x7f498003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.542+0000 7f499e4cb700 1 -- 172.21.15.19:0/1430913359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4984005360 con 0x7f4998107390 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.542+0000 7f49957fa700 1 -- 172.21.15.19:0/1430913359 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f498804d9c0 con 0x7f4998107390 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.542+0000 7f49977fe700 1 --2- 172.21.15.19:0/1430913359 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f49800395b0 0x7f498003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:14.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.543+0000 7f49977fe700 1 --2- 172.21.15.19:0/1430913359 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f49800395b0 0x7f498003ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f498c006fd0 tx=0x7f498c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.547+0000 7f49957fa700 1 -- 172.21.15.19:0/1430913359 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4988015070 con 0x7f4998107390 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.758+0000 7f499e4cb700 1 -- 172.21.15.19:0/1430913359 --> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4984000bf0 con 0x7f49800395b0 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.759+0000 7f49957fa700 1 -- 172.21.15.19:0/1430913359 <== mgr.14118 v2:172.21.15.19:6800/2431053142 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4984000bf0 con 0x7f49800395b0 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.761+0000 7f499e4cb700 1 -- 172.21.15.19:0/1430913359 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f49800395b0 msgr2=0x7f498003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.761+0000 7f499e4cb700 1 --2- 172.21.15.19:0/1430913359 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f49800395b0 0x7f498003ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f498c006fd0 tx=0x7f498c006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.761+0000 7f499e4cb700 1 -- 172.21.15.19:0/1430913359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 msgr2=0x7f499819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.761+0000 7f499e4cb700 1 --2- 172.21.15.19:0/1430913359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f499819e620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f498800b340 tx=0x7f4988003dc0 comp rx=0 tx=0).stop 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.761+0000 7f499e4cb700 1 -- 172.21.15.19:0/1430913359 shutdown_connections 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.761+0000 7f499e4cb700 1 --2- 172.21.15.19:0/1430913359 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f49800395b0 0x7f498003ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.761+0000 7f499e4cb700 1 --2- 172.21.15.19:0/1430913359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4998107390 0x7f499819e620 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.761+0000 7f499e4cb700 1 -- 172.21.15.19:0/1430913359 >> 172.21.15.19:0/1430913359 conn(0x7f49981029b0 msgr2=0x7f4998190830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.761+0000 7f499e4cb700 1 -- 172.21.15.19:0/1430913359 shutdown_connections 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:13.761+0000 7f499e4cb700 1 -- 172.21.15.19:0/1430913359 wait complete. 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-04-23T18:16:14.381 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-04-23T18:16:15.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:14 smithi019 ceph-mon[29215]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:15.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:15 smithi019 ceph-mon[29215]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:15.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:15 smithi019 ceph-mon[29215]: Deploying cephadm binary to smithi019 2024-04-23T18:16:19.918 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-04-23T18:16:19.918 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.838+0000 7f6a96051700 1 Processor -- start 2024-04-23T18:16:19.918 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.838+0000 7f6a96051700 1 -- start start 2024-04-23T18:16:19.918 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.839+0000 7f6a96051700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a90108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:19.918 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.839+0000 7f6a96051700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90108f30 con 0x7f6a90108540 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.839+0000 7f6a8f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a90108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.839+0000 7f6a8f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a90108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36378/0 (socket says 172.21.15.19:36378) 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.839+0000 7f6a8f7fe700 1 -- 172.21.15.19:0/2333743532 learned_addr learned my addr 172.21.15.19:0/2333743532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.839+0000 7f6a8f7fe700 1 -- 172.21.15.19:0/2333743532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a901096f0 con 0x7f6a90108540 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.839+0000 7f6a8f7fe700 1 --2- 172.21.15.19:0/2333743532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a90108960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6a800098a0 tx=0x7f6a80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568e05ebb216d99a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.840+0000 7f6a8e7fc700 1 -- 172.21.15.19:0/2333743532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a80004160 con 0x7f6a90108540 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.840+0000 7f6a8e7fc700 1 -- 172.21.15.19:0/2333743532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6a80004300 con 0x7f6a90108540 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.840+0000 7f6a8e7fc700 1 -- 172.21.15.19:0/2333743532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a80011af0 con 0x7f6a90108540 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.840+0000 7f6a96051700 1 -- 172.21.15.19:0/2333743532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 msgr2=0x7f6a90108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.840+0000 7f6a96051700 1 --2- 172.21.15.19:0/2333743532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a90108960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6a800098a0 tx=0x7f6a80009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.840+0000 7f6a96051700 1 -- 172.21.15.19:0/2333743532 shutdown_connections 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.840+0000 7f6a96051700 1 --2- 172.21.15.19:0/2333743532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a90108960 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.840+0000 7f6a96051700 1 -- 172.21.15.19:0/2333743532 >> 172.21.15.19:0/2333743532 conn(0x7f6a90103ba0 msgr2=0x7f6a90105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.840+0000 7f6a96051700 1 -- 172.21.15.19:0/2333743532 shutdown_connections 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.840+0000 7f6a96051700 1 -- 172.21.15.19:0/2333743532 wait complete. 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.841+0000 7f6a96051700 1 Processor -- start 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.841+0000 7f6a96051700 1 -- start start 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.841+0000 7f6a96051700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a9019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.841+0000 7f6a96051700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9010c920 con 0x7f6a90108540 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.841+0000 7f6a8f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a9019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.841+0000 7f6a8f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a9019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36388/0 (socket says 172.21.15.19:36388) 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.841+0000 7f6a8f7fe700 1 -- 172.21.15.19:0/817439174 learned_addr learned my addr 172.21.15.19:0/817439174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.842+0000 7f6a8f7fe700 1 -- 172.21.15.19:0/817439174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9019c990 con 0x7f6a90108540 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.842+0000 7f6a8f7fe700 1 --2- 172.21.15.19:0/817439174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a9019c450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009fb0 tx=0x7f6a80011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.842+0000 7f6a8cff9700 1 -- 172.21.15.19:0/817439174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a800036a0 con 0x7f6a90108540 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.842+0000 7f6a8cff9700 1 -- 172.21.15.19:0/817439174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6a8001a430 con 0x7f6a90108540 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.842+0000 7f6a96051700 1 -- 172.21.15.19:0/817439174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9019cc20 con 0x7f6a90108540 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.842+0000 7f6a8cff9700 1 -- 172.21.15.19:0/817439174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a80019550 con 0x7f6a90108540 2024-04-23T18:16:19.920 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.842+0000 7f6a96051700 1 -- 172.21.15.19:0/817439174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9019f910 con 0x7f6a90108540 2024-04-23T18:16:19.921 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.843+0000 7f6a8cff9700 1 -- 172.21.15.19:0/817439174 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6a80019850 con 0x7f6a90108540 2024-04-23T18:16:19.921 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.843+0000 7f6a8cff9700 1 --2- 172.21.15.19:0/817439174 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f6a78039560 0x7f6a7803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:19.921 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.843+0000 7f6a8cff9700 1 -- 172.21.15.19:0/817439174 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6a8004d8d0 con 0x7f6a90108540 2024-04-23T18:16:19.921 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.843+0000 7f6a96051700 1 -- 172.21.15.19:0/817439174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9010caa0 con 0x7f6a90108540 2024-04-23T18:16:19.921 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.843+0000 7f6a8effd700 1 --2- 172.21.15.19:0/817439174 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f6a78039560 0x7f6a7803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:19.921 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.844+0000 7f6a8effd700 1 --2- 172.21.15.19:0/817439174 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f6a78039560 0x7f6a7803ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6a84006fd0 tx=0x7f6a84006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:19.921 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:14.848+0000 7f6a8cff9700 1 -- 172.21.15.19:0/817439174 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a80016dd0 con 0x7f6a90108540 2024-04-23T18:16:19.921 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:15.064+0000 7f6a96051700 1 -- 172.21.15.19:0/817439174 --> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) -- 0x7f6a90104e20 con 0x7f6a78039560 2024-04-23T18:16:19.922 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.306+0000 7f6a8cff9700 1 -- 172.21.15.19:0/817439174 <== mgr.14118 v2:172.21.15.19:6800/2431053142 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6a90104e20 con 0x7f6a78039560 2024-04-23T18:16:19.922 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.309+0000 7f6a96051700 1 -- 172.21.15.19:0/817439174 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f6a78039560 msgr2=0x7f6a7803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:19.922 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.309+0000 7f6a96051700 1 --2- 172.21.15.19:0/817439174 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f6a78039560 0x7f6a7803ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6a84006fd0 tx=0x7f6a84006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:19.922 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.309+0000 7f6a96051700 1 -- 172.21.15.19:0/817439174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 msgr2=0x7f6a9019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:19.922 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.309+0000 7f6a96051700 1 --2- 172.21.15.19:0/817439174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a9019c450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009fb0 tx=0x7f6a80011940 comp rx=0 tx=0).stop 2024-04-23T18:16:19.922 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.309+0000 7f6a96051700 1 -- 172.21.15.19:0/817439174 shutdown_connections 2024-04-23T18:16:19.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.309+0000 7f6a96051700 1 --2- 172.21.15.19:0/817439174 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f6a78039560 0x7f6a7803ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:19.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.309+0000 7f6a96051700 1 --2- 172.21.15.19:0/817439174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a90108540 0x7f6a9019c450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:19.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.309+0000 7f6a96051700 1 -- 172.21.15.19:0/817439174 >> 172.21.15.19:0/817439174 conn(0x7f6a90103ba0 msgr2=0x7f6a90104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:19.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.309+0000 7f6a96051700 1 -- 172.21.15.19:0/817439174 shutdown_connections 2024-04-23T18:16:19.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:19.309+0000 7f6a96051700 1 -- 172.21.15.19:0/817439174 wait complete. 2024-04-23T18:16:19.923 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mon service... 2024-04-23T18:16:20.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:20 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:20.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:20 smithi019 ceph-mon[29215]: Added host smithi019 2024-04-23T18:16:20.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:20 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:21.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-23T18:16:21.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.426+0000 7f27661e0700 1 Processor -- start 2024-04-23T18:16:21.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.426+0000 7f27661e0700 1 -- start start 2024-04-23T18:16:21.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.426+0000 7f27661e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276010b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:21.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.426+0000 7f27661e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f276007be80 con 0x7f2760108fc0 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.426+0000 7f275f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276010b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.426+0000 7f275f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276010b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58616/0 (socket says 172.21.15.19:58616) 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.426+0000 7f275f7fe700 1 -- 172.21.15.19:0/177057951 learned_addr learned my addr 172.21.15.19:0/177057951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.426+0000 7f275f7fe700 1 -- 172.21.15.19:0/177057951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276010b8f0 con 0x7f2760108fc0 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f275f7fe700 1 --2- 172.21.15.19:0/177057951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276010b3b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f27500098a0 tx=0x7f2750009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682d627b56851ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f275e7fc700 1 -- 172.21.15.19:0/177057951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2750004160 con 0x7f2760108fc0 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f275e7fc700 1 -- 172.21.15.19:0/177057951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2750004300 con 0x7f2760108fc0 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f275e7fc700 1 -- 172.21.15.19:0/177057951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2750011af0 con 0x7f2760108fc0 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f27661e0700 1 -- 172.21.15.19:0/177057951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 msgr2=0x7f276010b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f27661e0700 1 --2- 172.21.15.19:0/177057951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276010b3b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f27500098a0 tx=0x7f2750009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f27661e0700 1 -- 172.21.15.19:0/177057951 shutdown_connections 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f27661e0700 1 --2- 172.21.15.19:0/177057951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276010b3b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:21.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f27661e0700 1 -- 172.21.15.19:0/177057951 >> 172.21.15.19:0/177057951 conn(0x7f2760102a70 msgr2=0x7f2760104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f27661e0700 1 -- 172.21.15.19:0/177057951 shutdown_connections 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.427+0000 7f27661e0700 1 -- 172.21.15.19:0/177057951 wait complete. 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.428+0000 7f27661e0700 1 Processor -- start 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.428+0000 7f27661e0700 1 -- start start 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.428+0000 7f27661e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.428+0000 7f27661e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f276010c8b0 con 0x7f2760108fc0 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.428+0000 7f275f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.428+0000 7f275f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276019c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58620/0 (socket says 172.21.15.19:58620) 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.428+0000 7f275f7fe700 1 -- 172.21.15.19:0/670228594 learned_addr learned my addr 172.21.15.19:0/670228594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.429+0000 7f275f7fe700 1 -- 172.21.15.19:0/670228594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276019cca0 con 0x7f2760108fc0 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.429+0000 7f275f7fe700 1 --2- 172.21.15.19:0/670228594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276019c760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2750003f40 tx=0x7f2750011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.429+0000 7f275cff9700 1 -- 172.21.15.19:0/670228594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27500038b0 con 0x7f2760108fc0 2024-04-23T18:16:21.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.429+0000 7f27661e0700 1 -- 172.21.15.19:0/670228594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276019cf30 con 0x7f2760108fc0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.429+0000 7f275cff9700 1 -- 172.21.15.19:0/670228594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2750003a50 con 0x7f2760108fc0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.429+0000 7f275cff9700 1 -- 172.21.15.19:0/670228594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f275001a5c0 con 0x7f2760108fc0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.429+0000 7f27661e0700 1 -- 172.21.15.19:0/670228594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276019fbd0 con 0x7f2760108fc0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.430+0000 7f275cff9700 1 -- 172.21.15.19:0/670228594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f275001a8c0 con 0x7f2760108fc0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.430+0000 7f27661e0700 1 -- 172.21.15.19:0/670228594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27600640a0 con 0x7f2760108fc0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.430+0000 7f275cff9700 1 --2- 172.21.15.19:0/670228594 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f27480395b0 0x7f274803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.430+0000 7f275cff9700 1 -- 172.21.15.19:0/670228594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f275004dd10 con 0x7f2760108fc0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.430+0000 7f275effd700 1 --2- 172.21.15.19:0/670228594 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f27480395b0 0x7f274803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.431+0000 7f275effd700 1 --2- 172.21.15.19:0/670228594 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f27480395b0 0x7f274803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2754006fd0 tx=0x7f2754006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.435+0000 7f275cff9700 1 -- 172.21.15.19:0/670228594 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f275002d910 con 0x7f2760108fc0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.662+0000 7f27661e0700 1 -- 172.21.15.19:0/670228594 --> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f276002dff0 con 0x7f27480395b0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.667+0000 7f275cff9700 1 -- 172.21.15.19:0/670228594 <== mgr.14118 v2:172.21.15.19:6800/2431053142 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f276002dff0 con 0x7f27480395b0 2024-04-23T18:16:21.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.669+0000 7f27661e0700 1 -- 172.21.15.19:0/670228594 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f27480395b0 msgr2=0x7f274803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.669+0000 7f27661e0700 1 --2- 172.21.15.19:0/670228594 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f27480395b0 0x7f274803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2754006fd0 tx=0x7f2754006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.669+0000 7f27661e0700 1 -- 172.21.15.19:0/670228594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 msgr2=0x7f276019c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.669+0000 7f27661e0700 1 --2- 172.21.15.19:0/670228594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276019c760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2750003f40 tx=0x7f2750011890 comp rx=0 tx=0).stop 2024-04-23T18:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.669+0000 7f27661e0700 1 -- 172.21.15.19:0/670228594 shutdown_connections 2024-04-23T18:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.669+0000 7f27661e0700 1 --2- 172.21.15.19:0/670228594 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f27480395b0 0x7f274803ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.669+0000 7f27661e0700 1 --2- 172.21.15.19:0/670228594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2760108fc0 0x7f276019c760 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.669+0000 7f27661e0700 1 -- 172.21.15.19:0/670228594 >> 172.21.15.19:0/670228594 conn(0x7f2760102a70 msgr2=0x7f2760103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.669+0000 7f27661e0700 1 -- 172.21.15.19:0/670228594 shutdown_connections 2024-04-23T18:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:20.669+0000 7f27661e0700 1 -- 172.21.15.19:0/670228594 wait complete. 2024-04-23T18:16:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mgr service... 2024-04-23T18:16:21.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:21 smithi019 ceph-mon[29215]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:21.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:21 smithi019 ceph-mon[29215]: Saving service mon spec with placement count:5 2024-04-23T18:16:21.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:21 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:23.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:23 smithi019 ceph-mon[29215]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:23.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:23 smithi019 ceph-mon[29215]: Saving service mgr spec with placement count:2 2024-04-23T18:16:23.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:23 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:23.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:23 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:23.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-23T18:16:23.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.807+0000 7fb8d3ebc700 1 Processor -- start 2024-04-23T18:16:23.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.807+0000 7fb8d3ebc700 1 -- start start 2024-04-23T18:16:23.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.807+0000 7fb8d3ebc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:23.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.807+0000 7fb8d3ebc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1075a0 con 0x7fb8cc106bb0 2024-04-23T18:16:23.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.807+0000 7fb8d1c58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.807+0000 7fb8d1c58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc106fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58634/0 (socket says 172.21.15.19:58634) 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.807+0000 7fb8d1c58700 1 -- 172.21.15.19:0/3492183533 learned_addr learned my addr 172.21.15.19:0/3492183533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.807+0000 7fb8d1c58700 1 -- 172.21.15.19:0/3492183533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc107db0 con 0x7fb8cc106bb0 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.808+0000 7fb8d1c58700 1 --2- 172.21.15.19:0/3492183533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc106fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0098a0 tx=0x7fb8bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f3f75f481e23d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.808+0000 7fb8d0c56700 1 -- 172.21.15.19:0/3492183533 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8bc004160 con 0x7fb8cc106bb0 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.808+0000 7fb8d0c56700 1 -- 172.21.15.19:0/3492183533 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb8bc004300 con 0x7fb8cc106bb0 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.808+0000 7fb8d0c56700 1 -- 172.21.15.19:0/3492183533 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8bc011af0 con 0x7fb8cc106bb0 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.808+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/3492183533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 msgr2=0x7fb8cc106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.808+0000 7fb8d3ebc700 1 --2- 172.21.15.19:0/3492183533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc106fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0098a0 tx=0x7fb8bc009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.808+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/3492183533 shutdown_connections 2024-04-23T18:16:23.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.808+0000 7fb8d3ebc700 1 --2- 172.21.15.19:0/3492183533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc106fd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.808+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/3492183533 >> 172.21.15.19:0/3492183533 conn(0x7fb8cc1021b0 msgr2=0x7fb8cc1045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.809+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/3492183533 shutdown_connections 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.809+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/3492183533 wait complete. 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.809+0000 7fb8d3ebc700 1 Processor -- start 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.809+0000 7fb8d3ebc700 1 -- start start 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.809+0000 7fb8d3ebc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.809+0000 7fb8d3ebc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc10b160 con 0x7fb8cc106bb0 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.809+0000 7fb8d1c58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.810+0000 7fb8d1c58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58646/0 (socket says 172.21.15.19:58646) 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.810+0000 7fb8d1c58700 1 -- 172.21.15.19:0/751272437 learned_addr learned my addr 172.21.15.19:0/751272437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.810+0000 7fb8d1c58700 1 -- 172.21.15.19:0/751272437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc19cd00 con 0x7fb8cc106bb0 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.810+0000 7fb8d1c58700 1 --2- 172.21.15.19:0/751272437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc19c7c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc003f40 tx=0x7fb8bc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:23.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.810+0000 7fb8caffd700 1 -- 172.21.15.19:0/751272437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8bc0038b0 con 0x7fb8cc106bb0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.810+0000 7fb8caffd700 1 -- 172.21.15.19:0/751272437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb8bc003a50 con 0x7fb8cc106bb0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.810+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/751272437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc19cf90 con 0x7fb8cc106bb0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.810+0000 7fb8caffd700 1 -- 172.21.15.19:0/751272437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8bc01a5c0 con 0x7fb8cc106bb0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.810+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/751272437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc19fc80 con 0x7fb8cc106bb0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.811+0000 7fb8caffd700 1 -- 172.21.15.19:0/751272437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb8bc024430 con 0x7fb8cc106bb0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.811+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/751272437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8cc0640a0 con 0x7fb8cc106bb0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.811+0000 7fb8caffd700 1 --2- 172.21.15.19:0/751272437 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fb8b40395b0 0x7fb8b403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.812+0000 7fb8caffd700 1 -- 172.21.15.19:0/751272437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb8bc04db50 con 0x7fb8cc106bb0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.812+0000 7fb8d1457700 1 --2- 172.21.15.19:0/751272437 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fb8b40395b0 0x7fb8b403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.812+0000 7fb8d1457700 1 --2- 172.21.15.19:0/751272437 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fb8b40395b0 0x7fb8b403ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0006fd0 tx=0x7fb8c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:21.817+0000 7fb8caffd700 1 -- 172.21.15.19:0/751272437 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8bc02cc00 con 0x7fb8cc106bb0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.042+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/751272437 --> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb8cc103d90 con 0x7fb8b40395b0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.046+0000 7fb8caffd700 1 -- 172.21.15.19:0/751272437 <== mgr.14118 v2:172.21.15.19:6800/2431053142 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb8cc103d90 con 0x7fb8b40395b0 2024-04-23T18:16:23.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.048+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/751272437 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fb8b40395b0 msgr2=0x7fb8b403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:23.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.048+0000 7fb8d3ebc700 1 --2- 172.21.15.19:0/751272437 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fb8b40395b0 0x7fb8b403ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0006fd0 tx=0x7fb8c0006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:23.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.048+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/751272437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 msgr2=0x7fb8cc19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:23.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.048+0000 7fb8d3ebc700 1 --2- 172.21.15.19:0/751272437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc19c7c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc003f40 tx=0x7fb8bc011890 comp rx=0 tx=0).stop 2024-04-23T18:16:23.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.048+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/751272437 shutdown_connections 2024-04-23T18:16:23.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.048+0000 7fb8d3ebc700 1 --2- 172.21.15.19:0/751272437 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fb8b40395b0 0x7fb8b403ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:23.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.048+0000 7fb8d3ebc700 1 --2- 172.21.15.19:0/751272437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8cc106bb0 0x7fb8cc19c7c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:23.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.048+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/751272437 >> 172.21.15.19:0/751272437 conn(0x7fb8cc1021b0 msgr2=0x7fb8cc103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:23.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.049+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/751272437 shutdown_connections 2024-04-23T18:16:23.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:22.049+0000 7fb8d3ebc700 1 -- 172.21.15.19:0/751272437 wait complete. 2024-04-23T18:16:24.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:24 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:24.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:24 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2681943848' entity='client.admin' 2024-04-23T18:16:25.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.891+0000 7f4559ac9700 1 Processor -- start 2024-04-23T18:16:25.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.891+0000 7f4559ac9700 1 -- start start 2024-04-23T18:16:25.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.891+0000 7f4559ac9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455410b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:25.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.891+0000 7f4559ac9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f455407bec0 con 0x7f4554108f80 2024-04-23T18:16:25.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.892+0000 7f45537fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455410b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:25.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.892+0000 7f45537fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455410b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58660/0 (socket says 172.21.15.19:58660) 2024-04-23T18:16:25.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.892+0000 7f45537fe700 1 -- 172.21.15.19:0/3530172257 learned_addr learned my addr 172.21.15.19:0/3530172257 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.892+0000 7f45537fe700 1 -- 172.21.15.19:0/3530172257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455410b8b0 con 0x7f4554108f80 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.892+0000 7f45537fe700 1 --2- 172.21.15.19:0/3530172257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455410b370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45440098a0 tx=0x7f4544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b918bb8f29237109 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.892+0000 7f45527fc700 1 -- 172.21.15.19:0/3530172257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4544004160 con 0x7f4554108f80 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.892+0000 7f45527fc700 1 -- 172.21.15.19:0/3530172257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4544004300 con 0x7f4554108f80 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.892+0000 7f45527fc700 1 -- 172.21.15.19:0/3530172257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4544011a20 con 0x7f4554108f80 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.893+0000 7f4559ac9700 1 -- 172.21.15.19:0/3530172257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 msgr2=0x7f455410b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.893+0000 7f4559ac9700 1 --2- 172.21.15.19:0/3530172257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455410b370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45440098a0 tx=0x7f4544009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.893+0000 7f4559ac9700 1 -- 172.21.15.19:0/3530172257 shutdown_connections 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.893+0000 7f4559ac9700 1 --2- 172.21.15.19:0/3530172257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455410b370 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.893+0000 7f4559ac9700 1 -- 172.21.15.19:0/3530172257 >> 172.21.15.19:0/3530172257 conn(0x7f4554102a70 msgr2=0x7f4554104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.893+0000 7f4559ac9700 1 -- 172.21.15.19:0/3530172257 shutdown_connections 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.893+0000 7f4559ac9700 1 -- 172.21.15.19:0/3530172257 wait complete. 2024-04-23T18:16:25.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.894+0000 7f4559ac9700 1 Processor -- start 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.894+0000 7f4559ac9700 1 -- start start 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f4559ac9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f4559ac9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f455410c8c0 con 0x7f4554108f80 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f45537fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f45537fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58676/0 (socket says 172.21.15.19:58676) 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f45537fe700 1 -- 172.21.15.19:0/2681943848 learned_addr learned my addr 172.21.15.19:0/2681943848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f45537fe700 1 -- 172.21.15.19:0/2681943848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455419cc30 con 0x7f4554108f80 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f45537fe700 1 --2- 172.21.15.19:0/2681943848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455419c6f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f45440116e0 tx=0x7f4544011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f4550ff9700 1 -- 172.21.15.19:0/2681943848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4544003a60 con 0x7f4554108f80 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f4559ac9700 1 -- 172.21.15.19:0/2681943848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455419cec0 con 0x7f4554108f80 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f4550ff9700 1 -- 172.21.15.19:0/2681943848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454401b8f0 con 0x7f4554108f80 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f4550ff9700 1 -- 172.21.15.19:0/2681943848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4544019860 con 0x7f4554108f80 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.895+0000 7f4559ac9700 1 -- 172.21.15.19:0/2681943848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455419fb60 con 0x7f4554108f80 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.896+0000 7f4550ff9700 1 -- 172.21.15.19:0/2681943848 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4544019b60 con 0x7f4554108f80 2024-04-23T18:16:25.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.897+0000 7f4550ff9700 1 --2- 172.21.15.19:0/2681943848 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4540039560 0x7f454003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.897+0000 7f4550ff9700 1 -- 172.21.15.19:0/2681943848 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f454404ec80 con 0x7f4554108f80 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.897+0000 7f4559ac9700 1 -- 172.21.15.19:0/2681943848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45540640a0 con 0x7f4554108f80 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.897+0000 7f4552ffd700 1 --2- 172.21.15.19:0/2681943848 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4540039560 0x7f454003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.897+0000 7f4552ffd700 1 --2- 172.21.15.19:0/2681943848 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4540039560 0x7f454003ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f453c006fd0 tx=0x7f453c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:23.902+0000 7f4550ff9700 1 -- 172.21.15.19:0/2681943848 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f454401f030 con 0x7f4554108f80 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.119+0000 7f4559ac9700 1 -- 172.21.15.19:0/2681943848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f455402df80 con 0x7f4554108f80 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.121+0000 7f4550ff9700 1 -- 172.21.15.19:0/2681943848 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4544029260 con 0x7f4554108f80 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.124+0000 7f4559ac9700 1 -- 172.21.15.19:0/2681943848 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4540039560 msgr2=0x7f454003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.124+0000 7f4559ac9700 1 --2- 172.21.15.19:0/2681943848 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4540039560 0x7f454003ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f453c006fd0 tx=0x7f453c006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.124+0000 7f4559ac9700 1 -- 172.21.15.19:0/2681943848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 msgr2=0x7f455419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.124+0000 7f4559ac9700 1 --2- 172.21.15.19:0/2681943848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455419c6f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f45440116e0 tx=0x7f4544011aa0 comp rx=0 tx=0).stop 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.124+0000 7f4559ac9700 1 -- 172.21.15.19:0/2681943848 shutdown_connections 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.124+0000 7f4559ac9700 1 --2- 172.21.15.19:0/2681943848 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f4540039560 0x7f454003ba20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.124+0000 7f4559ac9700 1 --2- 172.21.15.19:0/2681943848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554108f80 0x7f455419c6f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:25.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.124+0000 7f4559ac9700 1 -- 172.21.15.19:0/2681943848 >> 172.21.15.19:0/2681943848 conn(0x7f4554102a70 msgr2=0x7f4554103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:25.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.124+0000 7f4559ac9700 1 -- 172.21.15.19:0/2681943848 shutdown_connections 2024-04-23T18:16:25.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:24.124+0000 7f4559ac9700 1 -- 172.21.15.19:0/2681943848 wait complete. 2024-04-23T18:16:25.927 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:25 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:26.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.932+0000 7f70c2fd9700 1 Processor -- start 2024-04-23T18:16:26.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.933+0000 7f70c2fd9700 1 -- start start 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.933+0000 7f70c2fd9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc10b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.933+0000 7f70c2fd9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc07be60 con 0x7f70bc108f60 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.933+0000 7f70c0d75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc10b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.933+0000 7f70c0d75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc10b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58692/0 (socket says 172.21.15.19:58692) 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.933+0000 7f70c0d75700 1 -- 172.21.15.19:0/4077476869 learned_addr learned my addr 172.21.15.19:0/4077476869 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.933+0000 7f70c0d75700 1 -- 172.21.15.19:0/4077476869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70bc10b890 con 0x7f70bc108f60 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.934+0000 7f70c0d75700 1 --2- 172.21.15.19:0/4077476869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc10b350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70b00098a0 tx=0x7f70b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872753aeb132d21e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.934+0000 7f70bb7fe700 1 -- 172.21.15.19:0/4077476869 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b0004160 con 0x7f70bc108f60 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.934+0000 7f70bb7fe700 1 -- 172.21.15.19:0/4077476869 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b0004300 con 0x7f70bc108f60 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.934+0000 7f70bb7fe700 1 -- 172.21.15.19:0/4077476869 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b0011a20 con 0x7f70bc108f60 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.935+0000 7f70c2fd9700 1 -- 172.21.15.19:0/4077476869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 msgr2=0x7f70bc10b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.935+0000 7f70c2fd9700 1 --2- 172.21.15.19:0/4077476869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc10b350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70b00098a0 tx=0x7f70b0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:26.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.935+0000 7f70c2fd9700 1 -- 172.21.15.19:0/4077476869 shutdown_connections 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.935+0000 7f70c2fd9700 1 --2- 172.21.15.19:0/4077476869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc10b350 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.935+0000 7f70c2fd9700 1 -- 172.21.15.19:0/4077476869 >> 172.21.15.19:0/4077476869 conn(0x7f70bc1029b0 msgr2=0x7f70bc104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.935+0000 7f70c2fd9700 1 -- 172.21.15.19:0/4077476869 shutdown_connections 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.935+0000 7f70c2fd9700 1 -- 172.21.15.19:0/4077476869 wait complete. 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.936+0000 7f70c2fd9700 1 Processor -- start 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.936+0000 7f70c2fd9700 1 -- start start 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.936+0000 7f70c2fd9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.936+0000 7f70c2fd9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc10c840 con 0x7f70bc108f60 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.936+0000 7f70c0d75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.936+0000 7f70c0d75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc19c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58702/0 (socket says 172.21.15.19:58702) 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.936+0000 7f70c0d75700 1 -- 172.21.15.19:0/1950041311 learned_addr learned my addr 172.21.15.19:0/1950041311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.936+0000 7f70c0d75700 1 -- 172.21.15.19:0/1950041311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70bc19c920 con 0x7f70bc108f60 2024-04-23T18:16:26.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.936+0000 7f70c0d75700 1 --2- 172.21.15.19:0/1950041311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc19c3e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f70b0004660 tx=0x7f70b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.937+0000 7f70b9ffb700 1 -- 172.21.15.19:0/1950041311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b0003aa0 con 0x7f70bc108f60 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.937+0000 7f70b9ffb700 1 -- 172.21.15.19:0/1950041311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b001b8f0 con 0x7f70bc108f60 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.937+0000 7f70b9ffb700 1 -- 172.21.15.19:0/1950041311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b0023900 con 0x7f70bc108f60 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.937+0000 7f70c2fd9700 1 -- 172.21.15.19:0/1950041311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70bc19cbb0 con 0x7f70bc108f60 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.937+0000 7f70c2fd9700 1 -- 172.21.15.19:0/1950041311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70bc19f8a0 con 0x7f70bc108f60 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.938+0000 7f70b9ffb700 1 -- 172.21.15.19:0/1950041311 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f70b001baa0 con 0x7f70bc108f60 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.938+0000 7f70b9ffb700 1 --2- 172.21.15.19:0/1950041311 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f70ac039600 0x7f70ac03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.938+0000 7f70c2fd9700 1 -- 172.21.15.19:0/1950041311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70bc0640a0 con 0x7f70bc108f60 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.938+0000 7f70b9ffb700 1 -- 172.21.15.19:0/1950041311 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f70b004ddf0 con 0x7f70bc108f60 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.938+0000 7f70bbfff700 1 --2- 172.21.15.19:0/1950041311 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f70ac039600 0x7f70ac03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.939+0000 7f70bbfff700 1 --2- 172.21.15.19:0/1950041311 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f70ac039600 0x7f70ac03bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70a8006fd0 tx=0x7f70a8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:26.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:25.943+0000 7f70b9ffb700 1 -- 172.21.15.19:0/1950041311 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70b001f030 con 0x7f70bc108f60 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.154+0000 7f70c2fd9700 1 -- 172.21.15.19:0/1950041311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f70bc19fec0 con 0x7f70bc108f60 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.157+0000 7f70b9ffb700 1 -- 172.21.15.19:0/1950041311 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f70b00202b0 con 0x7f70bc108f60 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.160+0000 7f70c2fd9700 1 -- 172.21.15.19:0/1950041311 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f70ac039600 msgr2=0x7f70ac03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.160+0000 7f70c2fd9700 1 --2- 172.21.15.19:0/1950041311 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f70ac039600 0x7f70ac03bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70a8006fd0 tx=0x7f70a8006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.160+0000 7f70c2fd9700 1 -- 172.21.15.19:0/1950041311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 msgr2=0x7f70bc19c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.160+0000 7f70c2fd9700 1 --2- 172.21.15.19:0/1950041311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc19c3e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f70b0004660 tx=0x7f70b0011aa0 comp rx=0 tx=0).stop 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.160+0000 7f70c2fd9700 1 -- 172.21.15.19:0/1950041311 shutdown_connections 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.160+0000 7f70c2fd9700 1 --2- 172.21.15.19:0/1950041311 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f70ac039600 0x7f70ac03bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.160+0000 7f70c2fd9700 1 --2- 172.21.15.19:0/1950041311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70bc108f60 0x7f70bc19c3e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.160+0000 7f70c2fd9700 1 -- 172.21.15.19:0/1950041311 >> 172.21.15.19:0/1950041311 conn(0x7f70bc1029b0 msgr2=0x7f70bc1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.160+0000 7f70c2fd9700 1 -- 172.21.15.19:0/1950041311 shutdown_connections 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:26.160+0000 7f70c2fd9700 1 -- 172.21.15.19:0/1950041311 wait complete. 2024-04-23T18:16:26.727 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-04-23T18:16:27.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:27 smithi019 ceph-mon[29215]: from='mgr.14118 172.21.15.19:0/2011608078' entity='mgr.x' 2024-04-23T18:16:27.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:27 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1950041311' entity='client.admin' 2024-04-23T18:16:28.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:28 smithi019 ceph-mon[29215]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:16:28.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:28 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1994633418' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-23T18:16:28.260 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: ignoring --setuser ceph since I am not root 2024-04-23T18:16:28.260 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: ignoring --setgroup ceph since I am not root 2024-04-23T18:16:28.260 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:28.266+0000 7f88bd928200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:16:28.510 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:28.460+0000 7f88bd928200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:16:28.883 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:28.567+0000 7f88bd928200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:16:29.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.181+0000 7f28535ee700 1 Processor -- start 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.181+0000 7f28535ee700 1 -- start start 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.181+0000 7f28535ee700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c106c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.181+0000 7f28535ee700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f284c107200 con 0x7f284c106810 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.181+0000 7f285138a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c106c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.181+0000 7f285138a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c106c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58714/0 (socket says 172.21.15.19:58714) 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.181+0000 7f285138a700 1 -- 172.21.15.19:0/3372407662 learned_addr learned my addr 172.21.15.19:0/3372407662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.182+0000 7f285138a700 1 -- 172.21.15.19:0/3372407662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c1097b0 con 0x7f284c106810 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.182+0000 7f285138a700 1 --2- 172.21.15.19:0/3372407662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c106c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f28480098a0 tx=0x7f2848009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2548b58a7beb0314 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.182+0000 7f2843fff700 1 -- 172.21.15.19:0/3372407662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2848004160 con 0x7f284c106810 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.182+0000 7f2843fff700 1 -- 172.21.15.19:0/3372407662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2848004300 con 0x7f284c106810 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.182+0000 7f2843fff700 1 -- 172.21.15.19:0/3372407662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2848011a20 con 0x7f284c106810 2024-04-23T18:16:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.183+0000 7f28535ee700 1 -- 172.21.15.19:0/3372407662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 msgr2=0x7f284c106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.183+0000 7f28535ee700 1 --2- 172.21.15.19:0/3372407662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c106c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f28480098a0 tx=0x7f2848009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.183+0000 7f28535ee700 1 -- 172.21.15.19:0/3372407662 shutdown_connections 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.183+0000 7f28535ee700 1 --2- 172.21.15.19:0/3372407662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c106c30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.183+0000 7f28535ee700 1 -- 172.21.15.19:0/3372407662 >> 172.21.15.19:0/3372407662 conn(0x7f284c102110 msgr2=0x7f284c104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.183+0000 7f28535ee700 1 -- 172.21.15.19:0/3372407662 shutdown_connections 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.183+0000 7f28535ee700 1 -- 172.21.15.19:0/3372407662 wait complete. 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.183+0000 7f28535ee700 1 Processor -- start 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.184+0000 7f28535ee700 1 -- start start 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.184+0000 7f28535ee700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.184+0000 7f28535ee700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f284c10cb60 con 0x7f284c106810 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.184+0000 7f285138a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.184+0000 7f285138a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c19c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58718/0 (socket says 172.21.15.19:58718) 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.184+0000 7f285138a700 1 -- 172.21.15.19:0/1994633418 learned_addr learned my addr 172.21.15.19:0/1994633418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.184+0000 7f285138a700 1 -- 172.21.15.19:0/1994633418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c19c920 con 0x7f284c106810 2024-04-23T18:16:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.184+0000 7f285138a700 1 --2- 172.21.15.19:0/1994633418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c19c3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2848004660 tx=0x7f2848011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.184+0000 7f28427fc700 1 -- 172.21.15.19:0/1994633418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2848003aa0 con 0x7f284c106810 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.185+0000 7f28427fc700 1 -- 172.21.15.19:0/1994633418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f284801b8f0 con 0x7f284c106810 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.185+0000 7f28535ee700 1 -- 172.21.15.19:0/1994633418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c19cbb0 con 0x7f284c106810 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.185+0000 7f28427fc700 1 -- 172.21.15.19:0/1994633418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2848023900 con 0x7f284c106810 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.185+0000 7f28535ee700 1 -- 172.21.15.19:0/1994633418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c19f8a0 con 0x7f284c106810 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.185+0000 7f28427fc700 1 -- 172.21.15.19:0/1994633418 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2848023bc0 con 0x7f284c106810 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.186+0000 7f28427fc700 1 --2- 172.21.15.19:0/1994633418 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f28380395b0 0x7f283803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.186+0000 7f28427fc700 1 -- 172.21.15.19:0/1994633418 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f284804dee0 con 0x7f284c106810 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.186+0000 7f2850b89700 1 --2- 172.21.15.19:0/1994633418 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f28380395b0 0x7f283803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.186+0000 7f28535ee700 1 -- 172.21.15.19:0/1994633418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284c0640a0 con 0x7f284c106810 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.186+0000 7f2850b89700 1 --2- 172.21.15.19:0/1994633418 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f28380395b0 0x7f283803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f283c006fd0 tx=0x7f283c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.191+0000 7f28427fc700 1 -- 172.21.15.19:0/1994633418 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f284801f030 con 0x7f284c106810 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.464+0000 7f28535ee700 1 -- 172.21.15.19:0/1994633418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f284c19fd60 con 0x7f284c106810 2024-04-23T18:16:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.078+0000 7f28427fc700 1 -- 172.21.15.19:0/1994633418 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f2848010380 con 0x7f284c106810 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.080+0000 7f28427fc700 1 -- 172.21.15.19:0/1994633418 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f2848048070 con 0x7f284c106810 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.081+0000 7f28535ee700 1 -- 172.21.15.19:0/1994633418 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f28380395b0 msgr2=0x7f283803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.081+0000 7f28535ee700 1 --2- 172.21.15.19:0/1994633418 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f28380395b0 0x7f283803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f283c006fd0 tx=0x7f283c006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.081+0000 7f28535ee700 1 -- 172.21.15.19:0/1994633418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 msgr2=0x7f284c19c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.081+0000 7f28535ee700 1 --2- 172.21.15.19:0/1994633418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c19c3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2848004660 tx=0x7f2848011aa0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.081+0000 7f28535ee700 1 -- 172.21.15.19:0/1994633418 shutdown_connections 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.081+0000 7f28535ee700 1 --2- 172.21.15.19:0/1994633418 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7f28380395b0 0x7f283803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.081+0000 7f28535ee700 1 --2- 172.21.15.19:0/1994633418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c106810 0x7f284c19c3e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.081+0000 7f28535ee700 1 -- 172.21.15.19:0/1994633418 >> 172.21.15.19:0/1994633418 conn(0x7f284c102110 msgr2=0x7f284c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.081+0000 7f28535ee700 1 -- 172.21.15.19:0/1994633418 shutdown_connections 2024-04-23T18:16:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:28.081+0000 7f28535ee700 1 -- 172.21.15.19:0/1994633418 wait complete. 2024-04-23T18:16:29.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:29 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1994633418' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-23T18:16:29.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:29 smithi019 ceph-mon[29215]: mgrmap e9: x(active, since 21s) 2024-04-23T18:16:29.153 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:28.889+0000 7f88bd928200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:16:29.153 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:28.979+0000 7f88bd928200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:16:30.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:30 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/33111930' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T18:16:30.367 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:30.262+0000 7f88bd928200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.519+0000 7ff2c756c700 1 Processor -- start 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.520+0000 7ff2c756c700 1 -- start start 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.520+0000 7ff2c756c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c0108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.520+0000 7ff2c756c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0108f60 con 0x7ff2c0108570 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.520+0000 7ff2c5308700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c0108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.520+0000 7ff2c5308700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c0108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58740/0 (socket says 172.21.15.19:58740) 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.520+0000 7ff2c5308700 1 -- 172.21.15.19:0/2670738106 learned_addr learned my addr 172.21.15.19:0/2670738106 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.520+0000 7ff2c5308700 1 -- 172.21.15.19:0/2670738106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c0109720 con 0x7ff2c0108570 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.521+0000 7ff2c5308700 1 --2- 172.21.15.19:0/2670738106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c0108990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00098a0 tx=0x7ff2b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff7a14b351f073d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.521+0000 7ff2b7fff700 1 -- 172.21.15.19:0/2670738106 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0004160 con 0x7ff2c0108570 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.521+0000 7ff2b7fff700 1 -- 172.21.15.19:0/2670738106 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b0004300 con 0x7ff2c0108570 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.521+0000 7ff2b7fff700 1 -- 172.21.15.19:0/2670738106 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0011a20 con 0x7ff2c0108570 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.521+0000 7ff2c756c700 1 -- 172.21.15.19:0/2670738106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 msgr2=0x7ff2c0108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.521+0000 7ff2c756c700 1 --2- 172.21.15.19:0/2670738106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c0108990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00098a0 tx=0x7ff2b0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.522+0000 7ff2c756c700 1 -- 172.21.15.19:0/2670738106 shutdown_connections 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.522+0000 7ff2c756c700 1 --2- 172.21.15.19:0/2670738106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c0108990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.522+0000 7ff2c756c700 1 -- 172.21.15.19:0/2670738106 >> 172.21.15.19:0/2670738106 conn(0x7ff2c0103bd0 msgr2=0x7ff2c0105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.522+0000 7ff2c756c700 1 -- 172.21.15.19:0/2670738106 shutdown_connections 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.522+0000 7ff2c756c700 1 -- 172.21.15.19:0/2670738106 wait complete. 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.522+0000 7ff2c756c700 1 Processor -- start 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.522+0000 7ff2c756c700 1 -- start start 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2c756c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2c756c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c010cad0 con 0x7ff2c0108570 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2c5308700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2c5308700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58750/0 (socket says 172.21.15.19:58750) 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2c5308700 1 -- 172.21.15.19:0/33111930 learned_addr learned my addr 172.21.15.19:0/33111930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2c5308700 1 -- 172.21.15.19:0/33111930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c019c920 con 0x7ff2c0108570 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2c5308700 1 --2- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c019c3e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00119f0 tx=0x7ff2b0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2b67fc700 1 -- 172.21.15.19:0/33111930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b00039e0 con 0x7ff2c0108570 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2b67fc700 1 -- 172.21.15.19:0/33111930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b001b8f0 con 0x7ff2c0108570 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2c756c700 1 -- 172.21.15.19:0/33111930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c019cbb0 con 0x7ff2c0108570 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.523+0000 7ff2b67fc700 1 -- 172.21.15.19:0/33111930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b00237a0 con 0x7ff2c0108570 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.524+0000 7ff2c756c700 1 -- 172.21.15.19:0/33111930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c019f8a0 con 0x7ff2c0108570 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.525+0000 7ff2b67fc700 1 -- 172.21.15.19:0/33111930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ff2b001b440 con 0x7ff2c0108570 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.525+0000 7ff2c756c700 1 -- 172.21.15.19:0/33111930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c010cc50 con 0x7ff2c0108570 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.525+0000 7ff2b67fc700 1 --2- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7ff2ac0395b0 0x7ff2ac03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.525+0000 7ff2b67fc700 1 -- 172.21.15.19:0/33111930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff2b004deb0 con 0x7ff2c0108570 2024-04-23T18:16:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.525+0000 7ff2c4b07700 1 -- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7ff2ac0395b0 msgr2=0x7ff2ac03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2431053142 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.525+0000 7ff2c4b07700 1 --2- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7ff2ac0395b0 0x7ff2ac03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.530+0000 7ff2b67fc700 1 -- 172.21.15.19:0/33111930 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2b001f030 con 0x7ff2c0108570 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.726+0000 7ff2c4b07700 1 -- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7ff2ac0395b0 msgr2=0x7ff2ac03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2431053142 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.726+0000 7ff2c4b07700 1 --2- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7ff2ac0395b0 0x7ff2ac03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.797+0000 7ff2c756c700 1 -- 172.21.15.19:0/33111930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff2c019fe50 con 0x7ff2c0108570 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.797+0000 7ff2b67fc700 1 -- 172.21.15.19:0/33111930 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7ff2b0018350 con 0x7ff2c0108570 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.800+0000 7ff2c756c700 1 -- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7ff2ac0395b0 msgr2=0x7ff2ac03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.800+0000 7ff2c756c700 1 --2- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7ff2ac0395b0 0x7ff2ac03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.800+0000 7ff2c756c700 1 -- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 msgr2=0x7ff2c019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.800+0000 7ff2c756c700 1 --2- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c019c3e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00119f0 tx=0x7ff2b0011ad0 comp rx=0 tx=0).stop 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.800+0000 7ff2c756c700 1 -- 172.21.15.19:0/33111930 shutdown_connections 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.800+0000 7ff2c756c700 1 --2- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7ff2ac0395b0 0x7ff2ac03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.800+0000 7ff2c756c700 1 --2- 172.21.15.19:0/33111930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108570 0x7ff2c019c3e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.800+0000 7ff2c756c700 1 -- 172.21.15.19:0/33111930 >> 172.21.15.19:0/33111930 conn(0x7ff2c0103bd0 msgr2=0x7ff2c01046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.800+0000 7ff2c756c700 1 -- 172.21.15.19:0/33111930 shutdown_connections 2024-04-23T18:16:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:29.800+0000 7ff2c756c700 1 -- 172.21.15.19:0/33111930 wait complete. 2024-04-23T18:16:30.379 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-04-23T18:16:30.379 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-04-23T18:16:30.617 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:30.445+0000 7f88bd928200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:16:30.617 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:30.623+0000 7f88bd928200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:16:31.233 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:30.931+0000 7f88bd928200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:16:31.233 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:31 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:31.028+0000 7f88bd928200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:16:31.233 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:31 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:31.239+0000 7f88bd928200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:16:31.496 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:31 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:31.441+0000 7f88bd928200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:16:32.506 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:32 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:32.069+0000 7f88bd928200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:16:32.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:32 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:32.512+0000 7f88bd928200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:16:32.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:32 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:32.612+0000 7f88bd928200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:16:33.260 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:32 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:32.827+0000 7f88bd928200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:16:33.724 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:33.377+0000 7f88bd928200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:16:33.724 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:33.474+0000 7f88bd928200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:16:34.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:33.729+0000 7f88bd928200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:16:34.011 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:33.869+0000 7f88bd928200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:16:34.011 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:33.968+0000 7f88bd928200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:16:34.397 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:34.089+0000 7f88bd928200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:16:34.397 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:34.402+0000 7f88bd928200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:16:35.311 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:35.035+0000 7f88bd928200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:16:35.311 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:16:35.260+0000 7f88bd928200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:16:35.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: Active manager daemon x restarted 2024-04-23T18:16:35.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: Activating manager daemon x 2024-04-23T18:16:35.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: osdmap e3: 0 total, 0 up, 0 in 2024-04-23T18:16:35.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: mgrmap e10: x(active, starting, since 0.00593214s) 2024-04-23T18:16:35.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:16:35.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:16:35.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:16:35.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:16:35.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:16:35.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: Manager daemon x is now available 2024-04-23T18:16:35.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:36.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:36 smithi019 ceph-mon[29215]: mgrmap e11: x(active, since 1.00684s) 2024-04-23T18:16:36.936 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.895+0000 7fc2b055a700 1 Processor -- start 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.895+0000 7fc2b055a700 1 -- start start 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.896+0000 7fc2b055a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a810b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.896+0000 7fc2b055a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a807be80 con 0x7fc2a8108f40 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.896+0000 7fc2ae2f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a810b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.896+0000 7fc2ae2f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a810b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35778/0 (socket says 172.21.15.19:35778) 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.896+0000 7fc2ae2f6700 1 -- 172.21.15.19:0/3623137291 learned_addr learned my addr 172.21.15.19:0/3623137291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.896+0000 7fc2ae2f6700 1 -- 172.21.15.19:0/3623137291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a810b870 con 0x7fc2a8108f40 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.896+0000 7fc2ae2f6700 1 --2- 172.21.15.19:0/3623137291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a810b330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc2980098a0 tx=0x7fc298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5ec84b07e85f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.896+0000 7fc2ad2f4700 1 -- 172.21.15.19:0/3623137291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc298004160 con 0x7fc2a8108f40 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.897+0000 7fc2ad2f4700 1 -- 172.21.15.19:0/3623137291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc298004300 con 0x7fc2a8108f40 2024-04-23T18:16:36.937 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.897+0000 7fc2ad2f4700 1 -- 172.21.15.19:0/3623137291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc298011a20 con 0x7fc2a8108f40 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.897+0000 7fc2b055a700 1 -- 172.21.15.19:0/3623137291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 msgr2=0x7fc2a810b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.897+0000 7fc2b055a700 1 --2- 172.21.15.19:0/3623137291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a810b330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc2980098a0 tx=0x7fc298009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.897+0000 7fc2b055a700 1 -- 172.21.15.19:0/3623137291 shutdown_connections 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.897+0000 7fc2b055a700 1 --2- 172.21.15.19:0/3623137291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a810b330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.897+0000 7fc2b055a700 1 -- 172.21.15.19:0/3623137291 >> 172.21.15.19:0/3623137291 conn(0x7fc2a81029b0 msgr2=0x7fc2a8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.897+0000 7fc2b055a700 1 -- 172.21.15.19:0/3623137291 shutdown_connections 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.897+0000 7fc2b055a700 1 -- 172.21.15.19:0/3623137291 wait complete. 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.898+0000 7fc2b055a700 1 Processor -- start 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.898+0000 7fc2b055a700 1 -- start start 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.898+0000 7fc2b055a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.898+0000 7fc2b055a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a810c870 con 0x7fc2a8108f40 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.898+0000 7fc2ae2f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.898+0000 7fc2ae2f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35784/0 (socket says 172.21.15.19:35784) 2024-04-23T18:16:36.938 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.898+0000 7fc2ae2f6700 1 -- 172.21.15.19:0/215765358 learned_addr learned my addr 172.21.15.19:0/215765358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.899+0000 7fc2ae2f6700 1 -- 172.21.15.19:0/215765358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a819c940 con 0x7fc2a8108f40 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.899+0000 7fc2ae2f6700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a819c400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc2980116e0 tx=0x7fc298011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.899+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc298003b20 con 0x7fc2a8108f40 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.899+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc29801b8f0 con 0x7fc2a8108f40 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.899+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2980238c0 con 0x7fc2a8108f40 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.899+0000 7fc2b055a700 1 -- 172.21.15.19:0/215765358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a819cbd0 con 0x7fc2a8108f40 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.899+0000 7fc2b055a700 1 -- 172.21.15.19:0/215765358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a819f8c0 con 0x7fc2a8108f40 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.900+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fc29801baa0 con 0x7fc2a8108f40 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.900+0000 7fc29f7fe700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 0x7fc29403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.900+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 --> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc29403c1e0 con 0x7fc294039610 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.901+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc29804e040 con 0x7fc2a8108f40 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.901+0000 7fc2adaf5700 1 -- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 msgr2=0x7fc29403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2431053142 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.901+0000 7fc2adaf5700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 0x7fc29403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.102+0000 7fc2adaf5700 1 -- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 msgr2=0x7fc29403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2431053142 2024-04-23T18:16:36.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.102+0000 7fc2adaf5700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 0x7fc29403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.503+0000 7fc2adaf5700 1 -- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 msgr2=0x7fc29403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2431053142 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.503+0000 7fc2adaf5700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 0x7fc29403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:32.304+0000 7fc2adaf5700 1 -- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 msgr2=0x7fc29403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2431053142 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:32.304+0000 7fc2adaf5700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 0x7fc29403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:33.904+0000 7fc2adaf5700 1 -- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 msgr2=0x7fc29403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2431053142 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:33.904+0000 7fc2adaf5700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 0x7fc29403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:35.267+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fc298023a60 con 0x7fc2a8108f40 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:35.267+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 msgr2=0x7fc29403bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:35.267+0000 7fc29f7fe700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 0x7fc29403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.266+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fc29804f6f0 con 0x7fc2a8108f40 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.266+0000 7fc29f7fe700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc29403cff0 0x7fc29403f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.267+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc29403c1e0 con 0x7fc29403cff0 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.267+0000 7fc2adaf5700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc29403cff0 0x7fc29403f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.268+0000 7fc2adaf5700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc29403cff0 0x7fc29403f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4003a10 tx=0x7fc2a4009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:36.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.268+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fc29403c1e0 con 0x7fc29403cff0 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.275+0000 7fc2b055a700 1 -- 172.21.15.19:0/215765358 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc2a8069b90 con 0x7fc29403cff0 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.275+0000 7fc29f7fe700 1 -- 172.21.15.19:0/215765358 <== mgr.14150 v2:172.21.15.19:6800/1993809272 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fc2a8069b90 con 0x7fc29403cff0 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 -- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc29403cff0 msgr2=0x7fc29403f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc29403cff0 0x7fc29403f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4003a10 tx=0x7fc2a4009270 comp rx=0 tx=0).stop 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 -- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 msgr2=0x7fc2a819c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a819c400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc2980116e0 tx=0x7fc298011aa0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 -- 172.21.15.19:0/215765358 shutdown_connections 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc29403cff0 0x7fc29403f3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a8108f40 0x7fc2a819c400 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 --2- 172.21.15.19:0/215765358 >> [v2:172.21.15.19:6800/2431053142,v1:172.21.15.19:6801/2431053142] conn(0x7fc294039610 0x7fc29403bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 -- 172.21.15.19:0/215765358 >> 172.21.15.19:0/215765358 conn(0x7fc2a81029b0 msgr2=0x7fc2a81034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:36.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 -- 172.21.15.19:0/215765358 shutdown_connections 2024-04-23T18:16:36.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.276+0000 7fc2b055a700 1 -- 172.21.15.19:0/215765358 wait complete. 2024-04-23T18:16:36.942 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-04-23T18:16:36.942 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-04-23T18:16:37.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:37 smithi019 ceph-mon[29215]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T18:16:37.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:37 smithi019 ceph-mon[29215]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T18:16:38.351 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:38 smithi019 ceph-mon[29215]: mgrmap e12: x(active, since 2s) 2024-04-23T18:16:38.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:38.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:38.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:38 smithi019 ceph-mon[29215]: [23/Apr/2024:18:16:37] ENGINE Bus STARTING 2024-04-23T18:16:38.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:38.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:38 smithi019 ceph-mon[29215]: [23/Apr/2024:18:16:37] ENGINE Serving on http://172.21.15.19:8765 2024-04-23T18:16:38.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:38 smithi019 ceph-mon[29215]: [23/Apr/2024:18:16:37] ENGINE Serving on https://172.21.15.19:7150 2024-04-23T18:16:38.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:38 smithi019 ceph-mon[29215]: [23/Apr/2024:18:16:37] ENGINE Bus STARTED 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.370+0000 7f60fca9f700 1 Processor -- start 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.370+0000 7f60fca9f700 1 -- start start 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.370+0000 7f60fca9f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f810b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.370+0000 7f60fca9f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60f807bec0 con 0x7f60f8108f00 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.370+0000 7f60f659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f810b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.370+0000 7f60f659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f810b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35844/0 (socket says 172.21.15.19:35844) 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.370+0000 7f60f659c700 1 -- 172.21.15.19:0/3445430234 learned_addr learned my addr 172.21.15.19:0/3445430234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.370+0000 7f60f659c700 1 -- 172.21.15.19:0/3445430234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f810b830 con 0x7f60f8108f00 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.371+0000 7f60f659c700 1 --2- 172.21.15.19:0/3445430234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f810b2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60e80098a0 tx=0x7f60e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3986766470d47d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:38.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.371+0000 7f60f559a700 1 -- 172.21.15.19:0/3445430234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60e8004160 con 0x7f60f8108f00 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.371+0000 7f60f559a700 1 -- 172.21.15.19:0/3445430234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60e8004300 con 0x7f60f8108f00 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.371+0000 7f60f559a700 1 -- 172.21.15.19:0/3445430234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60e8011a20 con 0x7f60f8108f00 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.371+0000 7f60fca9f700 1 -- 172.21.15.19:0/3445430234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 msgr2=0x7f60f810b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.371+0000 7f60fca9f700 1 --2- 172.21.15.19:0/3445430234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f810b2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60e80098a0 tx=0x7f60e8009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.372+0000 7f60fca9f700 1 -- 172.21.15.19:0/3445430234 shutdown_connections 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.372+0000 7f60fca9f700 1 --2- 172.21.15.19:0/3445430234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f810b2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.372+0000 7f60fca9f700 1 -- 172.21.15.19:0/3445430234 >> 172.21.15.19:0/3445430234 conn(0x7f60f81029b0 msgr2=0x7f60f8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.372+0000 7f60fca9f700 1 -- 172.21.15.19:0/3445430234 shutdown_connections 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.372+0000 7f60fca9f700 1 -- 172.21.15.19:0/3445430234 wait complete. 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.372+0000 7f60fca9f700 1 Processor -- start 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.372+0000 7f60fca9f700 1 -- start start 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60fca9f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60fca9f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60f810c840 con 0x7f60f8108f00 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60f659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:38.360 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60f659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35848/0 (socket says 172.21.15.19:35848) 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60f659c700 1 -- 172.21.15.19:0/4235682946 learned_addr learned my addr 172.21.15.19:0/4235682946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60f659c700 1 -- 172.21.15.19:0/4235682946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f819eb50 con 0x7f60f8108f00 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60f659c700 1 --2- 172.21.15.19:0/4235682946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f819e610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60e80116e0 tx=0x7f60e8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60e77fe700 1 -- 172.21.15.19:0/4235682946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60e8003b20 con 0x7f60f8108f00 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60e77fe700 1 -- 172.21.15.19:0/4235682946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60e801b8f0 con 0x7f60f8108f00 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60fca9f700 1 -- 172.21.15.19:0/4235682946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f819ede0 con 0x7f60f8108f00 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60fca9f700 1 -- 172.21.15.19:0/4235682946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f81a1ad0 con 0x7f60f8108f00 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.373+0000 7f60e77fe700 1 -- 172.21.15.19:0/4235682946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60e80238c0 con 0x7f60f8108f00 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.374+0000 7f60e77fe700 1 -- 172.21.15.19:0/4235682946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f60e801baa0 con 0x7f60f8108f00 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.374+0000 7f60fca9f700 1 -- 172.21.15.19:0/4235682946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f80640a0 con 0x7f60f8108f00 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.375+0000 7f60e77fe700 1 --2- 172.21.15.19:0/4235682946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60e00395b0 0x7f60e003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.375+0000 7f60e77fe700 1 -- 172.21.15.19:0/4235682946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f60e804e180 con 0x7f60f8108f00 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.375+0000 7f60f5d9b700 1 --2- 172.21.15.19:0/4235682946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60e00395b0 0x7f60e003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:38.361 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.375+0000 7f60f5d9b700 1 --2- 172.21.15.19:0/4235682946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60e00395b0 0x7f60e003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f60ec006fd0 tx=0x7f60ec006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.380+0000 7f60e77fe700 1 -- 172.21.15.19:0/4235682946 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60e801f030 con 0x7f60f8108f00 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.595+0000 7f60fca9f700 1 -- 172.21.15.19:0/4235682946 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f60f8103b30 con 0x7f60e00395b0 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.689+0000 7f60e77fe700 1 -- 172.21.15.19:0/4235682946 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f60f8103b30 con 0x7f60e00395b0 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.691+0000 7f60fca9f700 1 -- 172.21.15.19:0/4235682946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60e00395b0 msgr2=0x7f60e003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.691+0000 7f60fca9f700 1 --2- 172.21.15.19:0/4235682946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60e00395b0 0x7f60e003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f60ec006fd0 tx=0x7f60ec006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.691+0000 7f60fca9f700 1 -- 172.21.15.19:0/4235682946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 msgr2=0x7f60f819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.691+0000 7f60fca9f700 1 --2- 172.21.15.19:0/4235682946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f819e610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60e80116e0 tx=0x7f60e8011aa0 comp rx=0 tx=0).stop 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.692+0000 7f60fca9f700 1 -- 172.21.15.19:0/4235682946 shutdown_connections 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.692+0000 7f60fca9f700 1 --2- 172.21.15.19:0/4235682946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60e00395b0 0x7f60e003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.692+0000 7f60fca9f700 1 --2- 172.21.15.19:0/4235682946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8108f00 0x7f60f819e610 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.692+0000 7f60fca9f700 1 -- 172.21.15.19:0/4235682946 >> 172.21.15.19:0/4235682946 conn(0x7f60f81029b0 msgr2=0x7f60f8103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.692+0000 7f60fca9f700 1 -- 172.21.15.19:0/4235682946 shutdown_connections 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:37.692+0000 7f60fca9f700 1 -- 172.21.15.19:0/4235682946 wait complete. 2024-04-23T18:16:38.362 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-04-23T18:16:39.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:39 smithi019 ceph-mon[29215]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:39.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$5/bcqCguAj2/r6Dsw5F2jOQ84YgoZKbOG7QXBnUinYZV8kCbxh6JS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713896199, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-23T18:16:39.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.842+0000 7f5adf16e700 1 Processor -- start 2024-04-23T18:16:39.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.842+0000 7f5adf16e700 1 -- start start 2024-04-23T18:16:39.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.843+0000 7f5adf16e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad8108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:39.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.843+0000 7f5adf16e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8108f00 con 0x7f5ad8108510 2024-04-23T18:16:39.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.843+0000 7f5adcf0a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad8108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.843+0000 7f5adcf0a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad8108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35862/0 (socket says 172.21.15.19:35862) 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.843+0000 7f5adcf0a700 1 -- 172.21.15.19:0/2160468194 learned_addr learned my addr 172.21.15.19:0/2160468194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.843+0000 7f5adcf0a700 1 -- 172.21.15.19:0/2160468194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad81096c0 con 0x7f5ad8108510 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.843+0000 7f5adcf0a700 1 --2- 172.21.15.19:0/2160468194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad8108930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0098a0 tx=0x7f5acc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1472c098a1ba0c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.843+0000 7f5ad77fe700 1 -- 172.21.15.19:0/2160468194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5acc004160 con 0x7f5ad8108510 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.844+0000 7f5ad77fe700 1 -- 172.21.15.19:0/2160468194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5acc004300 con 0x7f5ad8108510 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.844+0000 7f5ad77fe700 1 -- 172.21.15.19:0/2160468194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5acc011a20 con 0x7f5ad8108510 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.844+0000 7f5adf16e700 1 -- 172.21.15.19:0/2160468194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 msgr2=0x7f5ad8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.844+0000 7f5adf16e700 1 --2- 172.21.15.19:0/2160468194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad8108930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0098a0 tx=0x7f5acc009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.844+0000 7f5adf16e700 1 -- 172.21.15.19:0/2160468194 shutdown_connections 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.844+0000 7f5adf16e700 1 --2- 172.21.15.19:0/2160468194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad8108930 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.844+0000 7f5adf16e700 1 -- 172.21.15.19:0/2160468194 >> 172.21.15.19:0/2160468194 conn(0x7f5ad8103b70 msgr2=0x7f5ad8105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.844+0000 7f5adf16e700 1 -- 172.21.15.19:0/2160468194 shutdown_connections 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.844+0000 7f5adf16e700 1 -- 172.21.15.19:0/2160468194 wait complete. 2024-04-23T18:16:39.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.845+0000 7f5adf16e700 1 Processor -- start 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.845+0000 7f5adf16e700 1 -- start start 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.845+0000 7f5adf16e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.845+0000 7f5adf16e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad810ca70 con 0x7f5ad8108510 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.845+0000 7f5adcf0a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.845+0000 7f5adcf0a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad819e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35874/0 (socket says 172.21.15.19:35874) 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.845+0000 7f5adcf0a700 1 -- 172.21.15.19:0/2809511871 learned_addr learned my addr 172.21.15.19:0/2809511871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.845+0000 7f5adcf0a700 1 -- 172.21.15.19:0/2809511871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad819ee90 con 0x7f5ad8108510 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.845+0000 7f5adcf0a700 1 --2- 172.21.15.19:0/2809511871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad819e950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0116e0 tx=0x7f5acc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.846+0000 7f5ad5ffb700 1 -- 172.21.15.19:0/2809511871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5acc0039e0 con 0x7f5ad8108510 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.846+0000 7f5ad5ffb700 1 -- 172.21.15.19:0/2809511871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5acc01b8f0 con 0x7f5ad8108510 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.846+0000 7f5ad5ffb700 1 -- 172.21.15.19:0/2809511871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5acc023760 con 0x7f5ad8108510 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.846+0000 7f5adf16e700 1 -- 172.21.15.19:0/2809511871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad819f120 con 0x7f5ad8108510 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.846+0000 7f5adf16e700 1 -- 172.21.15.19:0/2809511871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad81a1dc0 con 0x7f5ad8108510 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.847+0000 7f5ad5ffb700 1 -- 172.21.15.19:0/2809511871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f5acc01b440 con 0x7f5ad8108510 2024-04-23T18:16:39.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.847+0000 7f5ad5ffb700 1 --2- 172.21.15.19:0/2809511871 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5ac8039610 0x7f5ac803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.847+0000 7f5adf16e700 1 -- 172.21.15.19:0/2809511871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad80640a0 con 0x7f5ad8108510 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.847+0000 7f5ad5ffb700 1 -- 172.21.15.19:0/2809511871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5acc04dfb0 con 0x7f5ad8108510 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.847+0000 7f5ad7fff700 1 --2- 172.21.15.19:0/2809511871 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5ac8039610 0x7f5ac803bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.848+0000 7f5ad7fff700 1 --2- 172.21.15.19:0/2809511871 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5ac8039610 0x7f5ac803bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4006fd0 tx=0x7f5ac4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.852+0000 7f5ad5ffb700 1 -- 172.21.15.19:0/2809511871 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5acc01f030 con 0x7f5ad8108510 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.088+0000 7f5adf16e700 1 -- 172.21.15.19:0/2809511871 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f5ad8196fc0 con 0x7f5ac8039610 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.334+0000 7f5ad5ffb700 1 -- 172.21.15.19:0/2809511871 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f5ad8196fc0 con 0x7f5ac8039610 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.336+0000 7f5adf16e700 1 -- 172.21.15.19:0/2809511871 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5ac8039610 msgr2=0x7f5ac803bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.336+0000 7f5adf16e700 1 --2- 172.21.15.19:0/2809511871 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5ac8039610 0x7f5ac803bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4006fd0 tx=0x7f5ac4006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.336+0000 7f5adf16e700 1 -- 172.21.15.19:0/2809511871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 msgr2=0x7f5ad819e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.336+0000 7f5adf16e700 1 --2- 172.21.15.19:0/2809511871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad819e950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0116e0 tx=0x7f5acc011aa0 comp rx=0 tx=0).stop 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.336+0000 7f5adf16e700 1 -- 172.21.15.19:0/2809511871 shutdown_connections 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.336+0000 7f5adf16e700 1 --2- 172.21.15.19:0/2809511871 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5ac8039610 0x7f5ac803bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.336+0000 7f5adf16e700 1 --2- 172.21.15.19:0/2809511871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad8108510 0x7f5ad819e950 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.336+0000 7f5adf16e700 1 -- 172.21.15.19:0/2809511871 >> 172.21.15.19:0/2809511871 conn(0x7f5ad8103b70 msgr2=0x7f5ad8104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:39.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.337+0000 7f5adf16e700 1 -- 172.21.15.19:0/2809511871 shutdown_connections 2024-04-23T18:16:39.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:39.337+0000 7f5adf16e700 1 -- 172.21.15.19:0/2809511871 wait complete. 2024-04-23T18:16:39.945 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-04-23T18:16:40.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:40.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:41.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.395+0000 7fdd74617700 1 Processor -- start 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.395+0000 7fdd74617700 1 -- start start 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.396+0000 7fdd74617700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.396+0000 7fdd74617700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c108f30 con 0x7fdd6c108540 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.396+0000 7fdd723b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.396+0000 7fdd723b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36754/0 (socket says 172.21.15.19:36754) 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.396+0000 7fdd723b3700 1 -- 172.21.15.19:0/863401375 learned_addr learned my addr 172.21.15.19:0/863401375 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.396+0000 7fdd723b3700 1 -- 172.21.15.19:0/863401375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c1096f0 con 0x7fdd6c108540 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.396+0000 7fdd723b3700 1 --2- 172.21.15.19:0/863401375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c108960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0098a0 tx=0x7fdd5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e876042b98eddc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.396+0000 7fdd713b1700 1 -- 172.21.15.19:0/863401375 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd5c004160 con 0x7fdd6c108540 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.397+0000 7fdd713b1700 1 -- 172.21.15.19:0/863401375 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd5c004300 con 0x7fdd6c108540 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.397+0000 7fdd713b1700 1 -- 172.21.15.19:0/863401375 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd5c011a20 con 0x7fdd6c108540 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.397+0000 7fdd74617700 1 -- 172.21.15.19:0/863401375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 msgr2=0x7fdd6c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.397+0000 7fdd74617700 1 --2- 172.21.15.19:0/863401375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c108960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0098a0 tx=0x7fdd5c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.397+0000 7fdd74617700 1 -- 172.21.15.19:0/863401375 shutdown_connections 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.397+0000 7fdd74617700 1 --2- 172.21.15.19:0/863401375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c108960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.397+0000 7fdd74617700 1 -- 172.21.15.19:0/863401375 >> 172.21.15.19:0/863401375 conn(0x7fdd6c103ba0 msgr2=0x7fdd6c105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.397+0000 7fdd74617700 1 -- 172.21.15.19:0/863401375 shutdown_connections 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.397+0000 7fdd74617700 1 -- 172.21.15.19:0/863401375 wait complete. 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.398+0000 7fdd74617700 1 Processor -- start 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.398+0000 7fdd74617700 1 -- start start 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.398+0000 7fdd74617700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.398+0000 7fdd74617700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c10caa0 con 0x7fdd6c108540 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.398+0000 7fdd723b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.398+0000 7fdd723b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36762/0 (socket says 172.21.15.19:36762) 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.398+0000 7fdd723b3700 1 -- 172.21.15.19:0/1304054480 learned_addr learned my addr 172.21.15.19:0/1304054480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.398+0000 7fdd723b3700 1 -- 172.21.15.19:0/1304054480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c19c930 con 0x7fdd6c108540 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.399+0000 7fdd723b3700 1 --2- 172.21.15.19:0/1304054480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c19c3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0119f0 tx=0x7fdd5c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:41.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.399+0000 7fdd637fe700 1 -- 172.21.15.19:0/1304054480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd5c0039e0 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.399+0000 7fdd637fe700 1 -- 172.21.15.19:0/1304054480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd5c01b8f0 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.399+0000 7fdd74617700 1 -- 172.21.15.19:0/1304054480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c19cbc0 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.399+0000 7fdd637fe700 1 -- 172.21.15.19:0/1304054480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd5c0237a0 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.399+0000 7fdd74617700 1 -- 172.21.15.19:0/1304054480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c19f8b0 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.400+0000 7fdd637fe700 1 -- 172.21.15.19:0/1304054480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fdd5c023a50 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.400+0000 7fdd74617700 1 -- 172.21.15.19:0/1304054480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6c10cc20 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.400+0000 7fdd637fe700 1 --2- 172.21.15.19:0/1304054480 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdd58039610 0x7fdd5803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.400+0000 7fdd637fe700 1 -- 172.21.15.19:0/1304054480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdd5c04e150 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.400+0000 7fdd71bb2700 1 --2- 172.21.15.19:0/1304054480 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdd58039610 0x7fdd5803bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.401+0000 7fdd71bb2700 1 --2- 172.21.15.19:0/1304054480 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdd58039610 0x7fdd5803bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006fd0 tx=0x7fdd68006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.405+0000 7fdd637fe700 1 -- 172.21.15.19:0/1304054480 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd5c01f030 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.631+0000 7fdd74617700 1 -- 172.21.15.19:0/1304054480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fdd6c19fe60 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.632+0000 7fdd637fe700 1 -- 172.21.15.19:0/1304054480 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fdd5c01bb10 con 0x7fdd6c108540 2024-04-23T18:16:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.634+0000 7fdd74617700 1 -- 172.21.15.19:0/1304054480 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdd58039610 msgr2=0x7fdd5803bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:41.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.634+0000 7fdd74617700 1 --2- 172.21.15.19:0/1304054480 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdd58039610 0x7fdd5803bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006fd0 tx=0x7fdd68006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:41.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.634+0000 7fdd74617700 1 -- 172.21.15.19:0/1304054480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 msgr2=0x7fdd6c19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:41.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.634+0000 7fdd74617700 1 --2- 172.21.15.19:0/1304054480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c19c3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0119f0 tx=0x7fdd5c011ad0 comp rx=0 tx=0).stop 2024-04-23T18:16:41.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.634+0000 7fdd74617700 1 -- 172.21.15.19:0/1304054480 shutdown_connections 2024-04-23T18:16:41.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.634+0000 7fdd74617700 1 --2- 172.21.15.19:0/1304054480 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdd58039610 0x7fdd5803bad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:41.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.634+0000 7fdd74617700 1 --2- 172.21.15.19:0/1304054480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd6c108540 0x7fdd6c19c3f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:41.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.634+0000 7fdd74617700 1 -- 172.21.15.19:0/1304054480 >> 172.21.15.19:0/1304054480 conn(0x7fdd6c103ba0 msgr2=0x7fdd6c1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:41.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.635+0000 7fdd74617700 1 -- 172.21.15.19:0/1304054480 shutdown_connections 2024-04-23T18:16:41.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:40.635+0000 7fdd74617700 1 -- 172.21.15.19:0/1304054480 wait complete. 2024-04-23T18:16:41.385 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:16:41.385 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-23T18:16:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:16:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-23T18:16:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-23T18:16:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-23T18:16:41.395 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-04-23T18:16:41.395 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:16:41.395 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019:8443/ 2024-04-23T18:16:41.395 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-04-23T18:16:41.395 INFO:teuthology.orchestra.run.smithi019.stdout: Password: 36ydxf29sv 2024-04-23T18:16:41.395 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:16:41.396 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config directory 2024-04-23T18:16:41.396 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling autotune for osd_memory_target 2024-04-23T18:16:41.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:41 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1304054480' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-23T18:16:42.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:42 smithi019 ceph-mon[29215]: mgrmap e13: x(active, since 6s) 2024-04-23T18:16:43.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.828+0000 7fd54f386700 1 Processor -- start 2024-04-23T18:16:43.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.828+0000 7fd54f386700 1 -- start start 2024-04-23T18:16:43.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.828+0000 7fd54f386700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:43.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.828+0000 7fd54f386700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd54807c830 con 0x7fd54807de40 2024-04-23T18:16:43.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.828+0000 7fd54d122700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:43.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.828+0000 7fd54d122700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36772/0 (socket says 172.21.15.19:36772) 2024-04-23T18:16:43.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.828+0000 7fd54d122700 1 -- 172.21.15.19:0/4046426384 learned_addr learned my addr 172.21.15.19:0/4046426384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:43.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.829+0000 7fd54d122700 1 -- 172.21.15.19:0/4046426384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54807c9b0 con 0x7fd54807de40 2024-04-23T18:16:43.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.829+0000 7fd54d122700 1 --2- 172.21.15.19:0/4046426384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54807c2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd5440098a0 tx=0x7fd544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503c72199fdd60c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:43.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.829+0000 7fd53ffff700 1 -- 172.21.15.19:0/4046426384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd544004160 con 0x7fd54807de40 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.829+0000 7fd53ffff700 1 -- 172.21.15.19:0/4046426384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd544004300 con 0x7fd54807de40 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.829+0000 7fd53ffff700 1 -- 172.21.15.19:0/4046426384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd544011a20 con 0x7fd54807de40 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.830+0000 7fd54f386700 1 -- 172.21.15.19:0/4046426384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 msgr2=0x7fd54807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.830+0000 7fd54f386700 1 --2- 172.21.15.19:0/4046426384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54807c2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd5440098a0 tx=0x7fd544009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.830+0000 7fd54f386700 1 -- 172.21.15.19:0/4046426384 shutdown_connections 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.830+0000 7fd54f386700 1 --2- 172.21.15.19:0/4046426384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54807c2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.830+0000 7fd54f386700 1 -- 172.21.15.19:0/4046426384 >> 172.21.15.19:0/4046426384 conn(0x7fd548103b70 msgr2=0x7fd548105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.830+0000 7fd54f386700 1 -- 172.21.15.19:0/4046426384 shutdown_connections 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.830+0000 7fd54f386700 1 -- 172.21.15.19:0/4046426384 wait complete. 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.830+0000 7fd54f386700 1 Processor -- start 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd54f386700 1 -- start start 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd54f386700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd54f386700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd54810ea70 con 0x7fd54807de40 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd54d122700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd54d122700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54819e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36786/0 (socket says 172.21.15.19:36786) 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd54d122700 1 -- 172.21.15.19:0/3384349776 learned_addr learned my addr 172.21.15.19:0/3384349776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:43.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd54d122700 1 -- 172.21.15.19:0/3384349776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54819ee20 con 0x7fd54807de40 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd54d122700 1 --2- 172.21.15.19:0/3384349776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54819e8e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd5440116e0 tx=0x7fd544011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd53e7fc700 1 -- 172.21.15.19:0/3384349776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd544003a60 con 0x7fd54807de40 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd53e7fc700 1 -- 172.21.15.19:0/3384349776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd54401b8f0 con 0x7fd54807de40 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd54f386700 1 -- 172.21.15.19:0/3384349776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54819f0b0 con 0x7fd54807de40 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.831+0000 7fd53e7fc700 1 -- 172.21.15.19:0/3384349776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd544019860 con 0x7fd54807de40 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.832+0000 7fd54f386700 1 -- 172.21.15.19:0/3384349776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5481a1d50 con 0x7fd54807de40 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.832+0000 7fd53e7fc700 1 -- 172.21.15.19:0/3384349776 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd544019b60 con 0x7fd54807de40 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.832+0000 7fd54f386700 1 -- 172.21.15.19:0/3384349776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5480640a0 con 0x7fd54807de40 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.833+0000 7fd53e7fc700 1 --2- 172.21.15.19:0/3384349776 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd534039650 0x7fd53403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.833+0000 7fd53e7fc700 1 -- 172.21.15.19:0/3384349776 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd54404dff0 con 0x7fd54807de40 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.833+0000 7fd54c921700 1 --2- 172.21.15.19:0/3384349776 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd534039650 0x7fd53403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.833+0000 7fd54c921700 1 --2- 172.21.15.19:0/3384349776 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd534039650 0x7fd53403bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd538006fd0 tx=0x7fd538006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:42.838+0000 7fd53e7fc700 1 -- 172.21.15.19:0/3384349776 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd54401f030 con 0x7fd54807de40 2024-04-23T18:16:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.043+0000 7fd54f386700 1 -- 172.21.15.19:0/3384349776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fd54802df80 con 0x7fd54807de40 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.043+0000 7fd53e7fc700 1 -- 172.21.15.19:0/3384349776 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fd54401bda0 con 0x7fd54807de40 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.045+0000 7fd54f386700 1 -- 172.21.15.19:0/3384349776 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd534039650 msgr2=0x7fd53403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.045+0000 7fd54f386700 1 --2- 172.21.15.19:0/3384349776 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd534039650 0x7fd53403bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd538006fd0 tx=0x7fd538006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.045+0000 7fd54f386700 1 -- 172.21.15.19:0/3384349776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 msgr2=0x7fd54819e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.045+0000 7fd54f386700 1 --2- 172.21.15.19:0/3384349776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54819e8e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd5440116e0 tx=0x7fd544011aa0 comp rx=0 tx=0).stop 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.045+0000 7fd54f386700 1 -- 172.21.15.19:0/3384349776 shutdown_connections 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.045+0000 7fd54f386700 1 --2- 172.21.15.19:0/3384349776 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd534039650 0x7fd53403bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.045+0000 7fd54f386700 1 --2- 172.21.15.19:0/3384349776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd54807de40 0x7fd54819e8e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.045+0000 7fd54f386700 1 -- 172.21.15.19:0/3384349776 >> 172.21.15.19:0/3384349776 conn(0x7fd548103b70 msgr2=0x7fd548104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.046+0000 7fd54f386700 1 -- 172.21.15.19:0/3384349776 shutdown_connections 2024-04-23T18:16:43.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:43.046+0000 7fd54f386700 1 -- 172.21.15.19:0/3384349776 wait complete. 2024-04-23T18:16:45.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.117+0000 7fc478b11700 1 Processor -- start 2024-04-23T18:16:45.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.117+0000 7fc478b11700 1 -- start start 2024-04-23T18:16:45.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.117+0000 7fc478b11700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc474108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:45.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.117+0000 7fc478b11700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc474108f00 con 0x7fc474108510 2024-04-23T18:16:45.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.117+0000 7fc47259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc474108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:45.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.117+0000 7fc47259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc474108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36800/0 (socket says 172.21.15.19:36800) 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.117+0000 7fc47259c700 1 -- 172.21.15.19:0/821139896 learned_addr learned my addr 172.21.15.19:0/821139896 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.117+0000 7fc47259c700 1 -- 172.21.15.19:0/821139896 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4741096c0 con 0x7fc474108510 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.118+0000 7fc47259c700 1 --2- 172.21.15.19:0/821139896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc474108930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4640098a0 tx=0x7fc464009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bd492f112fab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.118+0000 7fc47159a700 1 -- 172.21.15.19:0/821139896 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc464004160 con 0x7fc474108510 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.118+0000 7fc47159a700 1 -- 172.21.15.19:0/821139896 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc464004300 con 0x7fc474108510 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.118+0000 7fc47159a700 1 -- 172.21.15.19:0/821139896 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc464011a20 con 0x7fc474108510 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.118+0000 7fc478b11700 1 -- 172.21.15.19:0/821139896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 msgr2=0x7fc474108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.118+0000 7fc478b11700 1 --2- 172.21.15.19:0/821139896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc474108930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4640098a0 tx=0x7fc464009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.119+0000 7fc478b11700 1 -- 172.21.15.19:0/821139896 shutdown_connections 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.119+0000 7fc478b11700 1 --2- 172.21.15.19:0/821139896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc474108930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.119+0000 7fc478b11700 1 -- 172.21.15.19:0/821139896 >> 172.21.15.19:0/821139896 conn(0x7fc474103b70 msgr2=0x7fc474105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.119+0000 7fc478b11700 1 -- 172.21.15.19:0/821139896 shutdown_connections 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.119+0000 7fc478b11700 1 -- 172.21.15.19:0/821139896 wait complete. 2024-04-23T18:16:45.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.119+0000 7fc478b11700 1 Processor -- start 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.119+0000 7fc478b11700 1 -- start start 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc478b11700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc47419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc478b11700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410ca70 con 0x7fc474108510 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc47259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc47419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc47259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc47419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36802/0 (socket says 172.21.15.19:36802) 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc47259c700 1 -- 172.21.15.19:0/896488760 learned_addr learned my addr 172.21.15.19:0/896488760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc47259c700 1 -- 172.21.15.19:0/896488760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47419cbe0 con 0x7fc474108510 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc47259c700 1 --2- 172.21.15.19:0/896488760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc47419c6a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc464009520 tx=0x7fc464011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc4637fe700 1 -- 172.21.15.19:0/896488760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4640039e0 con 0x7fc474108510 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc4637fe700 1 -- 172.21.15.19:0/896488760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc46401b8f0 con 0x7fc474108510 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc4637fe700 1 -- 172.21.15.19:0/896488760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc464019770 con 0x7fc474108510 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc478b11700 1 -- 172.21.15.19:0/896488760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47419ce70 con 0x7fc474108510 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.120+0000 7fc478b11700 1 -- 172.21.15.19:0/896488760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47419fb60 con 0x7fc474108510 2024-04-23T18:16:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.121+0000 7fc4637fe700 1 -- 172.21.15.19:0/896488760 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc464019a20 con 0x7fc474108510 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.121+0000 7fc478b11700 1 -- 172.21.15.19:0/896488760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4740640a0 con 0x7fc474108510 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.122+0000 7fc4637fe700 1 --2- 172.21.15.19:0/896488760 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc45c0396f0 0x7fc45c03bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.122+0000 7fc4637fe700 1 -- 172.21.15.19:0/896488760 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc46404e1f0 con 0x7fc474108510 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.122+0000 7fc471d9b700 1 --2- 172.21.15.19:0/896488760 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc45c0396f0 0x7fc45c03bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.122+0000 7fc471d9b700 1 --2- 172.21.15.19:0/896488760 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc45c0396f0 0x7fc45c03bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc468006fd0 tx=0x7fc468006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.127+0000 7fc4637fe700 1 -- 172.21.15.19:0/896488760 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc46401f030 con 0x7fc474108510 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.452+0000 7fc478b11700 1 -- 172.21.15.19:0/896488760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc47402df80 con 0x7fc474108510 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.454+0000 7fc4637fe700 1 -- 172.21.15.19:0/896488760 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fc464010a10 con 0x7fc474108510 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.456+0000 7fc478b11700 1 -- 172.21.15.19:0/896488760 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc45c0396f0 msgr2=0x7fc45c03bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.456+0000 7fc478b11700 1 --2- 172.21.15.19:0/896488760 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc45c0396f0 0x7fc45c03bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc468006fd0 tx=0x7fc468006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.456+0000 7fc478b11700 1 -- 172.21.15.19:0/896488760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 msgr2=0x7fc47419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.456+0000 7fc478b11700 1 --2- 172.21.15.19:0/896488760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc47419c6a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc464009520 tx=0x7fc464011aa0 comp rx=0 tx=0).stop 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.456+0000 7fc478b11700 1 -- 172.21.15.19:0/896488760 shutdown_connections 2024-04-23T18:16:45.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.456+0000 7fc478b11700 1 --2- 172.21.15.19:0/896488760 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc45c0396f0 0x7fc45c03bbb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.456+0000 7fc478b11700 1 --2- 172.21.15.19:0/896488760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474108510 0x7fc47419c6a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.456+0000 7fc478b11700 1 -- 172.21.15.19:0/896488760 >> 172.21.15.19:0/896488760 conn(0x7fc474103b70 msgr2=0x7fc474104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.456+0000 7fc478b11700 1 -- 172.21.15.19:0/896488760 shutdown_connections 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.456+0000 7fc478b11700 1 -- 172.21.15.19:0/896488760 wait complete. 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:16:45.055 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-23T18:16:45.056 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:16:45.056 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-04-23T18:16:45.056 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:16:45.056 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-04-23T18:16:45.056 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:16:45.056 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-23T18:16:45.056 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:16:45.056 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-04-23T18:16:45.096 INFO:tasks.cephadm:Fetching config... 2024-04-23T18:16:45.096 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:16:45.097 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-23T18:16:45.117 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-23T18:16:45.117 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:16:45.117 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-23T18:16:45.171 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-23T18:16:45.172 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:16:45.172 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-23T18:16:45.237 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-23T18:16:45.237 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:16:45.237 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-23T18:16:45.294 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-23T18:16:45.294 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2UpI8vamfYH8oFaSv1UuFEJ5ea+Uj+bVdn86+Cd9NmffRPMDzCsXlu/KYeLqXIAKrbOO+i+O53DqTCiGVbSZPoYrcAde0p7l/vK0PBJECzk/2MQH16gAKlD26TchLhnkCTUK9PL4GVCCfcPXgnKOt9CTwa3EDMSL4xx1w+JTa7gUxxlnEy6iD0pRQ03okpse7RqYpn9445Gp1GNCB6rq4ZbtVuNtuOmsp4z0Yk487qsuRYBxI7eu8A37vgRY4Y+uoUWpIIBTksRQ3fqYXE9zynVeYuX2vf2ckN9Q5XFz52f+QfmskeldvU9jq8ozR28hrhJElkUnRbi1jfQscNndZZvrBVGsy1dA9G3z7JCQf+6zzYzGls/rChbm2ehvaJg/cDfcWI8A5VyuP76DYzPiuY7ucobpzZKdvoGDiWQhtnaa0fQx1GFGp5CszK0H7KOkb+Bn7nutL2Sx3ZHhQv192/AiODQTlGjLq/AB961osYayhX18EkCqWkbueBDFFNt8= ceph-46706838-019d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:16:45.372 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2UpI8vamfYH8oFaSv1UuFEJ5ea+Uj+bVdn86+Cd9NmffRPMDzCsXlu/KYeLqXIAKrbOO+i+O53DqTCiGVbSZPoYrcAde0p7l/vK0PBJECzk/2MQH16gAKlD26TchLhnkCTUK9PL4GVCCfcPXgnKOt9CTwa3EDMSL4xx1w+JTa7gUxxlnEy6iD0pRQ03okpse7RqYpn9445Gp1GNCB6rq4ZbtVuNtuOmsp4z0Yk487qsuRYBxI7eu8A37vgRY4Y+uoUWpIIBTksRQ3fqYXE9zynVeYuX2vf2ckN9Q5XFz52f+QfmskeldvU9jq8ozR28hrhJElkUnRbi1jfQscNndZZvrBVGsy1dA9G3z7JCQf+6zzYzGls/rChbm2ehvaJg/cDfcWI8A5VyuP76DYzPiuY7ucobpzZKdvoGDiWQhtnaa0fQx1GFGp5CszK0H7KOkb+Bn7nutL2Sx3ZHhQv192/AiODQTlGjLq/AB961osYayhX18EkCqWkbueBDFFNt8= ceph-46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:45.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2UpI8vamfYH8oFaSv1UuFEJ5ea+Uj+bVdn86+Cd9NmffRPMDzCsXlu/KYeLqXIAKrbOO+i+O53DqTCiGVbSZPoYrcAde0p7l/vK0PBJECzk/2MQH16gAKlD26TchLhnkCTUK9PL4GVCCfcPXgnKOt9CTwa3EDMSL4xx1w+JTa7gUxxlnEy6iD0pRQ03okpse7RqYpn9445Gp1GNCB6rq4ZbtVuNtuOmsp4z0Yk487qsuRYBxI7eu8A37vgRY4Y+uoUWpIIBTksRQ3fqYXE9zynVeYuX2vf2ckN9Q5XFz52f+QfmskeldvU9jq8ozR28hrhJElkUnRbi1jfQscNndZZvrBVGsy1dA9G3z7JCQf+6zzYzGls/rChbm2ehvaJg/cDfcWI8A5VyuP76DYzPiuY7ucobpzZKdvoGDiWQhtnaa0fQx1GFGp5CszK0H7KOkb+Bn7nutL2Sx3ZHhQv192/AiODQTlGjLq/AB961osYayhX18EkCqWkbueBDFFNt8= ceph-46706838-019d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:16:45.426 INFO:teuthology.orchestra.run.smithi044.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2UpI8vamfYH8oFaSv1UuFEJ5ea+Uj+bVdn86+Cd9NmffRPMDzCsXlu/KYeLqXIAKrbOO+i+O53DqTCiGVbSZPoYrcAde0p7l/vK0PBJECzk/2MQH16gAKlD26TchLhnkCTUK9PL4GVCCfcPXgnKOt9CTwa3EDMSL4xx1w+JTa7gUxxlnEy6iD0pRQ03okpse7RqYpn9445Gp1GNCB6rq4ZbtVuNtuOmsp4z0Yk487qsuRYBxI7eu8A37vgRY4Y+uoUWpIIBTksRQ3fqYXE9zynVeYuX2vf2ckN9Q5XFz52f+QfmskeldvU9jq8ozR28hrhJElkUnRbi1jfQscNndZZvrBVGsy1dA9G3z7JCQf+6zzYzGls/rChbm2ehvaJg/cDfcWI8A5VyuP76DYzPiuY7ucobpzZKdvoGDiWQhtnaa0fQx1GFGp5CszK0H7KOkb+Bn7nutL2Sx3ZHhQv192/AiODQTlGjLq/AB961osYayhX18EkCqWkbueBDFFNt8= ceph-46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:45.438 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2UpI8vamfYH8oFaSv1UuFEJ5ea+Uj+bVdn86+Cd9NmffRPMDzCsXlu/KYeLqXIAKrbOO+i+O53DqTCiGVbSZPoYrcAde0p7l/vK0PBJECzk/2MQH16gAKlD26TchLhnkCTUK9PL4GVCCfcPXgnKOt9CTwa3EDMSL4xx1w+JTa7gUxxlnEy6iD0pRQ03okpse7RqYpn9445Gp1GNCB6rq4ZbtVuNtuOmsp4z0Yk487qsuRYBxI7eu8A37vgRY4Y+uoUWpIIBTksRQ3fqYXE9zynVeYuX2vf2ckN9Q5XFz52f+QfmskeldvU9jq8ozR28hrhJElkUnRbi1jfQscNndZZvrBVGsy1dA9G3z7JCQf+6zzYzGls/rChbm2ehvaJg/cDfcWI8A5VyuP76DYzPiuY7ucobpzZKdvoGDiWQhtnaa0fQx1GFGp5CszK0H7KOkb+Bn7nutL2Sx3ZHhQv192/AiODQTlGjLq/AB961osYayhX18EkCqWkbueBDFFNt8= ceph-46706838-019d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:16:45.477 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2UpI8vamfYH8oFaSv1UuFEJ5ea+Uj+bVdn86+Cd9NmffRPMDzCsXlu/KYeLqXIAKrbOO+i+O53DqTCiGVbSZPoYrcAde0p7l/vK0PBJECzk/2MQH16gAKlD26TchLhnkCTUK9PL4GVCCfcPXgnKOt9CTwa3EDMSL4xx1w+JTa7gUxxlnEy6iD0pRQ03okpse7RqYpn9445Gp1GNCB6rq4ZbtVuNtuOmsp4z0Yk487qsuRYBxI7eu8A37vgRY4Y+uoUWpIIBTksRQ3fqYXE9zynVeYuX2vf2ckN9Q5XFz52f+QfmskeldvU9jq8ozR28hrhJElkUnRbi1jfQscNndZZvrBVGsy1dA9G3z7JCQf+6zzYzGls/rChbm2ehvaJg/cDfcWI8A5VyuP76DYzPiuY7ucobpzZKdvoGDiWQhtnaa0fQx1GFGp5CszK0H7KOkb+Bn7nutL2Sx3ZHhQv192/AiODQTlGjLq/AB961osYayhX18EkCqWkbueBDFFNt8= ceph-46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:45.489 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-23T18:16:45.737 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:16:45.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:45 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/896488760' entity='client.admin' 2024-04-23T18:16:48.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.050+0000 7fcb82a05700 1 -- 172.21.15.19:0/3225755302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb7c1024c0 msgr2=0x7fcb7c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.050+0000 7fcb82a05700 1 --2- 172.21.15.19:0/3225755302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb7c1024c0 0x7fcb7c1028e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009910 tx=0x7fcb6c009c20 comp rx=0 tx=0).stop 2024-04-23T18:16:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.051+0000 7fcb82a05700 1 -- 172.21.15.19:0/3225755302 shutdown_connections 2024-04-23T18:16:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.051+0000 7fcb82a05700 1 --2- 172.21.15.19:0/3225755302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb7c1024c0 0x7fcb7c1028e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.051+0000 7fcb82a05700 1 -- 172.21.15.19:0/3225755302 >> 172.21.15.19:0/3225755302 conn(0x7fcb7c0fda60 msgr2=0x7fcb7c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.051+0000 7fcb82a05700 1 -- 172.21.15.19:0/3225755302 shutdown_connections 2024-04-23T18:16:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.051+0000 7fcb82a05700 1 -- 172.21.15.19:0/3225755302 wait complete. 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.051+0000 7fcb82a05700 1 Processor -- start 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.051+0000 7fcb82a05700 1 -- start start 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.052+0000 7fcb82a05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb7c1024c0 0x7fcb7c0766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.052+0000 7fcb82a05700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c108400 con 0x7fcb7c1024c0 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.052+0000 7fcb7bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb7c1024c0 0x7fcb7c0766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.052+0000 7fcb7bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb7c1024c0 0x7fcb7c0766b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36820/0 (socket says 172.21.15.19:36820) 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.052+0000 7fcb7bfff700 1 -- 172.21.15.19:0/2674055524 learned_addr learned my addr 172.21.15.19:0/2674055524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.052+0000 7fcb7bfff700 1 -- 172.21.15.19:0/2674055524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c076bf0 con 0x7fcb7c1024c0 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.052+0000 7fcb7bfff700 1 --2- 172.21.15.19:0/2674055524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb7c1024c0 0x7fcb7c0766b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c005800 tx=0x7fcb6c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.052+0000 7fcb797fa700 1 -- 172.21.15.19:0/2674055524 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb6c027070 con 0x7fcb7c1024c0 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.052+0000 7fcb82a05700 1 -- 172.21.15.19:0/2674055524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c074e30 con 0x7fcb7c1024c0 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.052+0000 7fcb797fa700 1 -- 172.21.15.19:0/2674055524 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb6c00be10 con 0x7fcb7c1024c0 2024-04-23T18:16:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.053+0000 7fcb797fa700 1 -- 172.21.15.19:0/2674055524 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb6c00f460 con 0x7fcb7c1024c0 2024-04-23T18:16:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.053+0000 7fcb82a05700 1 -- 172.21.15.19:0/2674055524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c075310 con 0x7fcb7c1024c0 2024-04-23T18:16:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.053+0000 7fcb797fa700 1 -- 172.21.15.19:0/2674055524 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcb6c02a070 con 0x7fcb7c1024c0 2024-04-23T18:16:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.053+0000 7fcb82a05700 1 -- 172.21.15.19:0/2674055524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7c108b80 con 0x7fcb7c1024c0 2024-04-23T18:16:48.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.054+0000 7fcb797fa700 1 --2- 172.21.15.19:0/2674055524 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb64039650 0x7fcb6403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:48.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.054+0000 7fcb797fa700 1 -- 172.21.15.19:0/2674055524 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcb6c01e030 con 0x7fcb7c1024c0 2024-04-23T18:16:48.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.054+0000 7fcb7b7fe700 1 --2- 172.21.15.19:0/2674055524 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb64039650 0x7fcb6403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:48.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.054+0000 7fcb7b7fe700 1 --2- 172.21.15.19:0/2674055524 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb64039650 0x7fcb6403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcb70006fd0 tx=0x7fcb70006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:48.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.059+0000 7fcb797fa700 1 -- 172.21.15.19:0/2674055524 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb6c013070 con 0x7fcb7c1024c0 2024-04-23T18:16:48.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.270+0000 7fcb82a05700 1 -- 172.21.15.19:0/2674055524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fcb7c190e60 con 0x7fcb7c1024c0 2024-04-23T18:16:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.273+0000 7fcb797fa700 1 -- 172.21.15.19:0/2674055524 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fcb6c0310c0 con 0x7fcb7c1024c0 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.276+0000 7fcb82a05700 1 -- 172.21.15.19:0/2674055524 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb64039650 msgr2=0x7fcb6403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.276+0000 7fcb82a05700 1 --2- 172.21.15.19:0/2674055524 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb64039650 0x7fcb6403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcb70006fd0 tx=0x7fcb70006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.276+0000 7fcb82a05700 1 -- 172.21.15.19:0/2674055524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb7c1024c0 msgr2=0x7fcb7c0766b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.276+0000 7fcb82a05700 1 --2- 172.21.15.19:0/2674055524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb7c1024c0 0x7fcb7c0766b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c005800 tx=0x7fcb6c009f60 comp rx=0 tx=0).stop 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.276+0000 7fcb82a05700 1 -- 172.21.15.19:0/2674055524 shutdown_connections 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.276+0000 7fcb82a05700 1 --2- 172.21.15.19:0/2674055524 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb64039650 0x7fcb6403bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.276+0000 7fcb82a05700 1 --2- 172.21.15.19:0/2674055524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb7c1024c0 0x7fcb7c0766b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.276+0000 7fcb82a05700 1 -- 172.21.15.19:0/2674055524 >> 172.21.15.19:0/2674055524 conn(0x7fcb7c0fda60 msgr2=0x7fcb7c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.276+0000 7fcb82a05700 1 -- 172.21.15.19:0/2674055524 shutdown_connections 2024-04-23T18:16:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:48.276+0000 7fcb82a05700 1 -- 172.21.15.19:0/2674055524 wait complete. 2024-04-23T18:16:48.888 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-23T18:16:48.888 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-23T18:16:49.174 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:16:49.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:49 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2674055524' entity='client.admin' 2024-04-23T18:16:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.554+0000 7fcd495af700 1 -- 172.21.15.19:0/2772351229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd441013c0 msgr2=0x7fcd441017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.554+0000 7fcd495af700 1 --2- 172.21.15.19:0/2772351229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd441013c0 0x7fcd441017e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009910 tx=0x7fcd34009c20 comp rx=0 tx=0).stop 2024-04-23T18:16:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.554+0000 7fcd495af700 1 -- 172.21.15.19:0/2772351229 shutdown_connections 2024-04-23T18:16:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.554+0000 7fcd495af700 1 --2- 172.21.15.19:0/2772351229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd441013c0 0x7fcd441017e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.554+0000 7fcd495af700 1 -- 172.21.15.19:0/2772351229 >> 172.21.15.19:0/2772351229 conn(0x7fcd440fc980 msgr2=0x7fcd440feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.554+0000 7fcd495af700 1 -- 172.21.15.19:0/2772351229 shutdown_connections 2024-04-23T18:16:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.554+0000 7fcd495af700 1 -- 172.21.15.19:0/2772351229 wait complete. 2024-04-23T18:16:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.555+0000 7fcd495af700 1 Processor -- start 2024-04-23T18:16:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.555+0000 7fcd495af700 1 -- start start 2024-04-23T18:16:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.555+0000 7fcd495af700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd441013c0 0x7fcd44197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.555+0000 7fcd495af700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd44107300 con 0x7fcd441013c0 2024-04-23T18:16:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.555+0000 7fcd42ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd441013c0 0x7fcd44197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.555+0000 7fcd42ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd441013c0 0x7fcd44197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50812/0 (socket says 172.21.15.19:50812) 2024-04-23T18:16:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.555+0000 7fcd42ffd700 1 -- 172.21.15.19:0/2346575607 learned_addr learned my addr 172.21.15.19:0/2346575607 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.555+0000 7fcd42ffd700 1 -- 172.21.15.19:0/2346575607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd441980a0 con 0x7fcd441013c0 2024-04-23T18:16:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.555+0000 7fcd42ffd700 1 --2- 172.21.15.19:0/2346575607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd441013c0 0x7fcd44197b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcd34005950 tx=0x7fcd340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.556+0000 7fcd2bfff700 1 -- 172.21.15.19:0/2346575607 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd34026070 con 0x7fcd441013c0 2024-04-23T18:16:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.556+0000 7fcd2bfff700 1 -- 172.21.15.19:0/2346575607 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd3400bc00 con 0x7fcd441013c0 2024-04-23T18:16:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.556+0000 7fcd495af700 1 -- 172.21.15.19:0/2346575607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd44198330 con 0x7fcd441013c0 2024-04-23T18:16:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.556+0000 7fcd2bfff700 1 -- 172.21.15.19:0/2346575607 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd34003a80 con 0x7fcd441013c0 2024-04-23T18:16:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.556+0000 7fcd495af700 1 -- 172.21.15.19:0/2346575607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4419b020 con 0x7fcd441013c0 2024-04-23T18:16:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.557+0000 7fcd2bfff700 1 -- 172.21.15.19:0/2346575607 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcd34029070 con 0x7fcd441013c0 2024-04-23T18:16:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.557+0000 7fcd2bfff700 1 --2- 172.21.15.19:0/2346575607 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd2c0396b0 0x7fcd2c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.557+0000 7fcd2bfff700 1 -- 172.21.15.19:0/2346575607 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcd3401e070 con 0x7fcd441013c0 2024-04-23T18:16:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.557+0000 7fcd427fc700 1 --2- 172.21.15.19:0/2346575607 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd2c0396b0 0x7fcd2c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.557+0000 7fcd495af700 1 -- 172.21.15.19:0/2346575607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd44107a80 con 0x7fcd441013c0 2024-04-23T18:16:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.557+0000 7fcd427fc700 1 --2- 172.21.15.19:0/2346575607 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd2c0396b0 0x7fcd2c03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd38006fd0 tx=0x7fcd38006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.562+0000 7fcd2bfff700 1 -- 172.21.15.19:0/2346575607 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd34030070 con 0x7fcd441013c0 2024-04-23T18:16:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.775+0000 7fcd495af700 1 -- 172.21.15.19:0/2346575607 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fcd4402dff0 con 0x7fcd2c0396b0 2024-04-23T18:16:51.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.779+0000 7fcd2bfff700 1 -- 172.21.15.19:0/2346575607 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcd4402dff0 con 0x7fcd2c0396b0 2024-04-23T18:16:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.781+0000 7fcd495af700 1 -- 172.21.15.19:0/2346575607 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd2c0396b0 msgr2=0x7fcd2c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.781+0000 7fcd495af700 1 --2- 172.21.15.19:0/2346575607 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd2c0396b0 0x7fcd2c03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd38006fd0 tx=0x7fcd38006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.781+0000 7fcd495af700 1 -- 172.21.15.19:0/2346575607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd441013c0 msgr2=0x7fcd44197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.781+0000 7fcd495af700 1 --2- 172.21.15.19:0/2346575607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd441013c0 0x7fcd44197b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcd34005950 tx=0x7fcd340057f0 comp rx=0 tx=0).stop 2024-04-23T18:16:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.781+0000 7fcd495af700 1 -- 172.21.15.19:0/2346575607 shutdown_connections 2024-04-23T18:16:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.781+0000 7fcd495af700 1 --2- 172.21.15.19:0/2346575607 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd2c0396b0 0x7fcd2c03bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.781+0000 7fcd495af700 1 --2- 172.21.15.19:0/2346575607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd441013c0 0x7fcd44197b60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.781+0000 7fcd495af700 1 -- 172.21.15.19:0/2346575607 >> 172.21.15.19:0/2346575607 conn(0x7fcd440fc980 msgr2=0x7fcd4406ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.781+0000 7fcd495af700 1 -- 172.21.15.19:0/2346575607 shutdown_connections 2024-04-23T18:16:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:51.781+0000 7fcd495af700 1 -- 172.21.15.19:0/2346575607 wait complete. 2024-04-23T18:16:52.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:52.601 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi044 2024-04-23T18:16:52.741 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:16:52.741 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.conf 2024-04-23T18:16:52.761 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:16:52.761 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:16:52.819 INFO:tasks.cephadm:Adding host smithi044 to orchestrator... 2024-04-23T18:16:52.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch host add smithi044 2024-04-23T18:16:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:16:53.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:53 smithi019 ceph-mon[29215]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:53.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:53 smithi019 ceph-mon[29215]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-23T18:16:53.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:53 smithi019 ceph-mon[29215]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:16:53.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:53.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:53.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:16:54.350 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:54 smithi019 ceph-mon[29215]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:16:54.350 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:54 smithi019 ceph-mon[29215]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:16:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.739+0000 7f10256d2700 1 -- 172.21.15.19:0/3746206804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10201036d0 msgr2=0x7f1020103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.739+0000 7f10256d2700 1 --2- 172.21.15.19:0/3746206804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10201036d0 0x7f1020103ab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1010009910 tx=0x7f1010009c20 comp rx=0 tx=0).stop 2024-04-23T18:16:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.739+0000 7f10256d2700 1 -- 172.21.15.19:0/3746206804 shutdown_connections 2024-04-23T18:16:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.739+0000 7f10256d2700 1 --2- 172.21.15.19:0/3746206804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10201036d0 0x7f1020103ab0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.739+0000 7f10256d2700 1 -- 172.21.15.19:0/3746206804 >> 172.21.15.19:0/3746206804 conn(0x7f10200fce40 msgr2=0x7f10200ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.739+0000 7f10256d2700 1 -- 172.21.15.19:0/3746206804 shutdown_connections 2024-04-23T18:16:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.739+0000 7f10256d2700 1 -- 172.21.15.19:0/3746206804 wait complete. 2024-04-23T18:16:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.740+0000 7f10256d2700 1 Processor -- start 2024-04-23T18:16:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.740+0000 7f10256d2700 1 -- start start 2024-04-23T18:16:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.740+0000 7f10256d2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10201036d0 0x7f1020197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.740+0000 7f10256d2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f102010c9a0 con 0x7f10201036d0 2024-04-23T18:16:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.740+0000 7f101effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10201036d0 0x7f1020197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.740+0000 7f101effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10201036d0 0x7f1020197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50830/0 (socket says 172.21.15.19:50830) 2024-04-23T18:16:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.740+0000 7f101effd700 1 -- 172.21.15.19:0/1362438901 learned_addr learned my addr 172.21.15.19:0/1362438901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:16:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.740+0000 7f101effd700 1 -- 172.21.15.19:0/1362438901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1020198450 con 0x7f10201036d0 2024-04-23T18:16:54.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.741+0000 7f101effd700 1 --2- 172.21.15.19:0/1362438901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10201036d0 0x7f1020197f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1010006010 tx=0x7f1010005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:54.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.741+0000 7f1007fff700 1 -- 172.21.15.19:0/1362438901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1010026070 con 0x7f10201036d0 2024-04-23T18:16:54.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.741+0000 7f1007fff700 1 -- 172.21.15.19:0/1362438901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f101000bc00 con 0x7f10201036d0 2024-04-23T18:16:54.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.741+0000 7f10256d2700 1 -- 172.21.15.19:0/1362438901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10201986e0 con 0x7f10201036d0 2024-04-23T18:16:54.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.741+0000 7f1007fff700 1 -- 172.21.15.19:0/1362438901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1010003910 con 0x7f10201036d0 2024-04-23T18:16:54.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.741+0000 7f10256d2700 1 -- 172.21.15.19:0/1362438901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1020198b40 con 0x7f10201036d0 2024-04-23T18:16:54.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.742+0000 7f1007fff700 1 -- 172.21.15.19:0/1362438901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1010029070 con 0x7f10201036d0 2024-04-23T18:16:54.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.742+0000 7f10256d2700 1 -- 172.21.15.19:0/1362438901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102006b0d0 con 0x7f10201036d0 2024-04-23T18:16:54.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.742+0000 7f1007fff700 1 --2- 172.21.15.19:0/1362438901 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1008039650 0x7f100803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:54.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.742+0000 7f1007fff700 1 -- 172.21.15.19:0/1362438901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f101001e070 con 0x7f10201036d0 2024-04-23T18:16:54.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.742+0000 7f101e7fc700 1 --2- 172.21.15.19:0/1362438901 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1008039650 0x7f100803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:54.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.743+0000 7f101e7fc700 1 --2- 172.21.15.19:0/1362438901 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1008039650 0x7f100803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1014006fd0 tx=0x7f1014006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:54.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.747+0000 7f1007fff700 1 -- 172.21.15.19:0/1362438901 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1010013070 con 0x7f10201036d0 2024-04-23T18:16:54.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:54.958+0000 7f10256d2700 1 -- 172.21.15.19:0/1362438901 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}) -- 0x7f10201992e0 con 0x7f1008039650 2024-04-23T18:16:55.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:55 smithi019 ceph-mon[29215]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:56.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:56 smithi019 ceph-mon[29215]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:16:56.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:56 smithi019 ceph-mon[29215]: Deploying cephadm binary to smithi044 2024-04-23T18:16:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:16:58 smithi019 ceph-mon[29215]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:16:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.202+0000 7f1007fff700 1 -- 172.21.15.19:0/1362438901 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f10201992e0 con 0x7f1008039650 2024-04-23T18:16:59.196 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi044' with addr '172.21.15.44' 2024-04-23T18:16:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.205+0000 7f10256d2700 1 -- 172.21.15.19:0/1362438901 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1008039650 msgr2=0x7f100803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.205+0000 7f10256d2700 1 --2- 172.21.15.19:0/1362438901 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1008039650 0x7f100803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1014006fd0 tx=0x7f1014006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.205+0000 7f10256d2700 1 -- 172.21.15.19:0/1362438901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10201036d0 msgr2=0x7f1020197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.205+0000 7f10256d2700 1 --2- 172.21.15.19:0/1362438901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10201036d0 0x7f1020197f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1010006010 tx=0x7f1010005780 comp rx=0 tx=0).stop 2024-04-23T18:16:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.205+0000 7f10256d2700 1 -- 172.21.15.19:0/1362438901 shutdown_connections 2024-04-23T18:16:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.205+0000 7f10256d2700 1 --2- 172.21.15.19:0/1362438901 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1008039650 0x7f100803bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.205+0000 7f10256d2700 1 --2- 172.21.15.19:0/1362438901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10201036d0 0x7f1020197f10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.205+0000 7f10256d2700 1 -- 172.21.15.19:0/1362438901 >> 172.21.15.19:0/1362438901 conn(0x7f10200fce40 msgr2=0x7f1020104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.205+0000 7f10256d2700 1 -- 172.21.15.19:0/1362438901 shutdown_connections 2024-04-23T18:16:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:16:59.205+0000 7f10256d2700 1 -- 172.21.15.19:0/1362438901 wait complete. 2024-04-23T18:16:59.936 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T18:17:00.184 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:00.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:00 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:00.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:00 smithi019 ceph-mon[29215]: Added host smithi044 2024-04-23T18:17:00.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:00 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:00.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:00 smithi019 ceph-mon[29215]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:00.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:00 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:01.618 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:01 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:01.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.728+0000 7fceef4e3700 1 -- 172.21.15.19:0/3027465821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcee810a660 msgr2=0x7fcee810aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:01.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.728+0000 7fceef4e3700 1 --2- 172.21.15.19:0/3027465821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcee810a660 0x7fcee810aa40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fced8009910 tx=0x7fced8009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:01.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.728+0000 7fceef4e3700 1 -- 172.21.15.19:0/3027465821 shutdown_connections 2024-04-23T18:17:01.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.728+0000 7fceef4e3700 1 --2- 172.21.15.19:0/3027465821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcee810a660 0x7fcee810aa40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:01.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.728+0000 7fceef4e3700 1 -- 172.21.15.19:0/3027465821 >> 172.21.15.19:0/3027465821 conn(0x7fcee8100190 msgr2=0x7fcee81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:01.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.728+0000 7fceef4e3700 1 -- 172.21.15.19:0/3027465821 shutdown_connections 2024-04-23T18:17:01.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.728+0000 7fceef4e3700 1 -- 172.21.15.19:0/3027465821 wait complete. 2024-04-23T18:17:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.729+0000 7fceef4e3700 1 Processor -- start 2024-04-23T18:17:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.729+0000 7fceef4e3700 1 -- start start 2024-04-23T18:17:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.729+0000 7fceef4e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcee810a660 0x7fcee819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.729+0000 7fceef4e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcee810eb50 con 0x7fcee810a660 2024-04-23T18:17:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.729+0000 7fceed27f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcee810a660 0x7fcee819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.729+0000 7fceed27f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcee810a660 0x7fcee819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37640/0 (socket says 172.21.15.19:37640) 2024-04-23T18:17:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.729+0000 7fceed27f700 1 -- 172.21.15.19:0/3435856047 learned_addr learned my addr 172.21.15.19:0/3435856047 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:17:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.730+0000 7fceed27f700 1 -- 172.21.15.19:0/3435856047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee819a5f0 con 0x7fcee810a660 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.730+0000 7fceed27f700 1 --2- 172.21.15.19:0/3435856047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcee810a660 0x7fcee819a0b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fced80056d0 tx=0x7fced80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.730+0000 7fcede7fc700 1 -- 172.21.15.19:0/3435856047 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fced8026070 con 0x7fcee810a660 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.730+0000 7fcede7fc700 1 -- 172.21.15.19:0/3435856047 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced800bba0 con 0x7fcee810a660 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.730+0000 7fcede7fc700 1 -- 172.21.15.19:0/3435856047 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fced8003920 con 0x7fcee810a660 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.730+0000 7fceef4e3700 1 -- 172.21.15.19:0/3435856047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee819a8e0 con 0x7fcee810a660 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.730+0000 7fceef4e3700 1 -- 172.21.15.19:0/3435856047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee819e980 con 0x7fcee810a660 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.731+0000 7fcede7fc700 1 -- 172.21.15.19:0/3435856047 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fced8029070 con 0x7fcee810a660 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.731+0000 7fceef4e3700 1 -- 172.21.15.19:0/3435856047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee8105da0 con 0x7fcee810a660 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.731+0000 7fcede7fc700 1 --2- 172.21.15.19:0/3435856047 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fced4039650 0x7fced403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.731+0000 7fcede7fc700 1 -- 172.21.15.19:0/3435856047 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fced801e070 con 0x7fcee810a660 2024-04-23T18:17:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.731+0000 7fceeca7e700 1 --2- 172.21.15.19:0/3435856047 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fced4039650 0x7fced403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.732+0000 7fceeca7e700 1 --2- 172.21.15.19:0/3435856047 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fced4039650 0x7fced403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcee4006fd0 tx=0x7fcee4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.736+0000 7fcede7fc700 1 -- 172.21.15.19:0/3435856047 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fced8013070 con 0x7fcee810a660 2024-04-23T18:17:01.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.955+0000 7fceef4e3700 1 -- 172.21.15.19:0/3435856047 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcee819b410 con 0x7fced4039650 2024-04-23T18:17:01.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.956+0000 7fcede7fc700 1 -- 172.21.15.19:0/3435856047 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fcee819b410 con 0x7fced4039650 2024-04-23T18:17:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:17:01.950 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}] 2024-04-23T18:17:01.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.959+0000 7fceef4e3700 1 -- 172.21.15.19:0/3435856047 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fced4039650 msgr2=0x7fced403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:01.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.959+0000 7fceef4e3700 1 --2- 172.21.15.19:0/3435856047 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fced4039650 0x7fced403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcee4006fd0 tx=0x7fcee4006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:01.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.959+0000 7fceef4e3700 1 -- 172.21.15.19:0/3435856047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcee810a660 msgr2=0x7fcee819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:01.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.959+0000 7fceef4e3700 1 --2- 172.21.15.19:0/3435856047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcee810a660 0x7fcee819a0b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fced80056d0 tx=0x7fced80057b0 comp rx=0 tx=0).stop 2024-04-23T18:17:01.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.959+0000 7fceef4e3700 1 -- 172.21.15.19:0/3435856047 shutdown_connections 2024-04-23T18:17:01.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.959+0000 7fceef4e3700 1 --2- 172.21.15.19:0/3435856047 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fced4039650 0x7fced403bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:01.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.959+0000 7fceef4e3700 1 --2- 172.21.15.19:0/3435856047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcee810a660 0x7fcee819a0b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:01.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.959+0000 7fceef4e3700 1 -- 172.21.15.19:0/3435856047 >> 172.21.15.19:0/3435856047 conn(0x7fcee8100190 msgr2=0x7fcee810c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:01.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.959+0000 7fceef4e3700 1 -- 172.21.15.19:0/3435856047 shutdown_connections 2024-04-23T18:17:01.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:01.959+0000 7fceef4e3700 1 -- 172.21.15.19:0/3435856047 wait complete. 2024-04-23T18:17:02.570 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-04-23T18:17:02.570 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:17:02.570 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-04-23T18:17:02.588 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:17:02.588 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:02.645 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-04-23T18:17:02.645 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch host add smithi155 2024-04-23T18:17:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:03.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:02 smithi019 ceph-mon[29215]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:03.225 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:03.225 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:02 smithi019 ceph-mon[29215]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:17:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.523+0000 7efffa538700 1 -- 172.21.15.19:0/1307357448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff4069e60 msgr2=0x7efff406a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.523+0000 7efffa538700 1 --2- 172.21.15.19:0/1307357448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff4069e60 0x7efff406a240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7effe4009910 tx=0x7effe4009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.523+0000 7efffa538700 1 -- 172.21.15.19:0/1307357448 shutdown_connections 2024-04-23T18:17:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.523+0000 7efffa538700 1 --2- 172.21.15.19:0/1307357448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff4069e60 0x7efff406a240 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.523+0000 7efffa538700 1 -- 172.21.15.19:0/1307357448 >> 172.21.15.19:0/1307357448 conn(0x7efff4076ea0 msgr2=0x7efff40772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.523+0000 7efffa538700 1 -- 172.21.15.19:0/1307357448 shutdown_connections 2024-04-23T18:17:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.523+0000 7efffa538700 1 -- 172.21.15.19:0/1307357448 wait complete. 2024-04-23T18:17:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.524+0000 7efffa538700 1 Processor -- start 2024-04-23T18:17:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.524+0000 7efffa538700 1 -- start start 2024-04-23T18:17:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.524+0000 7efffa538700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff4069e60 0x7efff419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.524+0000 7efffa538700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efff410eb90 con 0x7efff4069e60 2024-04-23T18:17:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.524+0000 7efff3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff4069e60 0x7efff419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.524+0000 7efff3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff4069e60 0x7efff419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37654/0 (socket says 172.21.15.19:37654) 2024-04-23T18:17:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.524+0000 7efff3fff700 1 -- 172.21.15.19:0/400472345 learned_addr learned my addr 172.21.15.19:0/400472345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:17:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.525+0000 7efff3fff700 1 -- 172.21.15.19:0/400472345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff419a640 con 0x7efff4069e60 2024-04-23T18:17:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.525+0000 7efff3fff700 1 --2- 172.21.15.19:0/400472345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff4069e60 0x7efff419a100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7effe40056d0 tx=0x7effe40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.525+0000 7efff17fa700 1 -- 172.21.15.19:0/400472345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effe4026070 con 0x7efff4069e60 2024-04-23T18:17:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.525+0000 7efff17fa700 1 -- 172.21.15.19:0/400472345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe400bc00 con 0x7efff4069e60 2024-04-23T18:17:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.525+0000 7efff17fa700 1 -- 172.21.15.19:0/400472345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effe4003920 con 0x7efff4069e60 2024-04-23T18:17:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.525+0000 7efffa538700 1 -- 172.21.15.19:0/400472345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff419a8d0 con 0x7efff4069e60 2024-04-23T18:17:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.525+0000 7efffa538700 1 -- 172.21.15.19:0/400472345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff419ad30 con 0x7efff4069e60 2024-04-23T18:17:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.526+0000 7efff17fa700 1 -- 172.21.15.19:0/400472345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7effe4029070 con 0x7efff4069e60 2024-04-23T18:17:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.526+0000 7efff17fa700 1 --2- 172.21.15.19:0/400472345 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7effdc0396b0 0x7effdc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.526+0000 7efff17fa700 1 -- 172.21.15.19:0/400472345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7effe401e070 con 0x7efff4069e60 2024-04-23T18:17:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.526+0000 7efffa538700 1 -- 172.21.15.19:0/400472345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff4101970 con 0x7efff4069e60 2024-04-23T18:17:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.526+0000 7efff37fe700 1 --2- 172.21.15.19:0/400472345 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7effdc0396b0 0x7effdc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.527+0000 7efff37fe700 1 --2- 172.21.15.19:0/400472345 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7effdc0396b0 0x7effdc03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7effe8006fd0 tx=0x7effe8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:04.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.531+0000 7efff17fa700 1 -- 172.21.15.19:0/400472345 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effe4030070 con 0x7efff4069e60 2024-04-23T18:17:04.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:04.748+0000 7efffa538700 1 -- 172.21.15.19:0/400472345 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) -- 0x7efff419b460 con 0x7effdc0396b0 2024-04-23T18:17:05.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:04 smithi019 ceph-mon[29215]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:06.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:06 smithi019 ceph-mon[29215]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:06.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:06.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:06.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:06.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:06.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:06.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:06.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:06.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:06 smithi019 ceph-mon[29215]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-23T18:17:07.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:07 smithi019 ceph-mon[29215]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:07.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:07 smithi019 ceph-mon[29215]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:07.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:07 smithi019 ceph-mon[29215]: Deploying cephadm binary to smithi155 2024-04-23T18:17:07.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:07 smithi019 ceph-mon[29215]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:07.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:07 smithi019 ceph-mon[29215]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:17:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.009+0000 7efff17fa700 1 -- 172.21.15.19:0/400472345 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7efff419b460 con 0x7effdc0396b0 2024-04-23T18:17:09.004 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-04-23T18:17:09.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.012+0000 7efffa538700 1 -- 172.21.15.19:0/400472345 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7effdc0396b0 msgr2=0x7effdc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:09.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.012+0000 7efffa538700 1 --2- 172.21.15.19:0/400472345 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7effdc0396b0 0x7effdc03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7effe8006fd0 tx=0x7effe8006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:09.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.013+0000 7efffa538700 1 -- 172.21.15.19:0/400472345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff4069e60 msgr2=0x7efff419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.013+0000 7efffa538700 1 --2- 172.21.15.19:0/400472345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff4069e60 0x7efff419a100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7effe40056d0 tx=0x7effe40057b0 comp rx=0 tx=0).stop 2024-04-23T18:17:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.013+0000 7efffa538700 1 -- 172.21.15.19:0/400472345 shutdown_connections 2024-04-23T18:17:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.013+0000 7efffa538700 1 --2- 172.21.15.19:0/400472345 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7effdc0396b0 0x7effdc03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.013+0000 7efffa538700 1 --2- 172.21.15.19:0/400472345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff4069e60 0x7efff419a100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.013+0000 7efffa538700 1 -- 172.21.15.19:0/400472345 >> 172.21.15.19:0/400472345 conn(0x7efff4076ea0 msgr2=0x7efff41099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.013+0000 7efffa538700 1 -- 172.21.15.19:0/400472345 shutdown_connections 2024-04-23T18:17:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:09.013+0000 7efffa538700 1 -- 172.21.15.19:0/400472345 wait complete. 2024-04-23T18:17:09.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:09 smithi019 ceph-mon[29215]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:09.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:09.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:09.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:09.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:09.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T18:17:09.951 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:10.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:10 smithi019 ceph-mon[29215]: Added host smithi155 2024-04-23T18:17:10.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:10 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:10.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:10 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:11.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:11 smithi019 ceph-mon[29215]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:11.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:11.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.720+0000 7f6596239700 1 -- 172.21.15.19:0/4075931521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65901015f0 msgr2=0x7f65901019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:11.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.720+0000 7f6596239700 1 --2- 172.21.15.19:0/4075931521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65901015f0 0x7f65901019d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6580009910 tx=0x7f6580009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:11.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.720+0000 7f6596239700 1 -- 172.21.15.19:0/4075931521 shutdown_connections 2024-04-23T18:17:11.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.720+0000 7f6596239700 1 --2- 172.21.15.19:0/4075931521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65901015f0 0x7f65901019d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:11.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.720+0000 7f6596239700 1 -- 172.21.15.19:0/4075931521 >> 172.21.15.19:0/4075931521 conn(0x7f6590076ea0 msgr2=0x7f65900772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:11.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.720+0000 7f6596239700 1 -- 172.21.15.19:0/4075931521 shutdown_connections 2024-04-23T18:17:11.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.720+0000 7f6596239700 1 -- 172.21.15.19:0/4075931521 wait complete. 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.721+0000 7f6596239700 1 Processor -- start 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.721+0000 7f6596239700 1 -- start start 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.721+0000 7f6596239700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65901015f0 0x7f6590196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.721+0000 7f6596239700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f659010b820 con 0x7f65901015f0 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.721+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65901015f0 0x7f6590196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.721+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65901015f0 0x7f6590196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43274/0 (socket says 172.21.15.19:43274) 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.721+0000 7f658f7fe700 1 -- 172.21.15.19:0/4087264559 learned_addr learned my addr 172.21.15.19:0/4087264559 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.722+0000 7f658f7fe700 1 -- 172.21.15.19:0/4087264559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6590197340 con 0x7f65901015f0 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.722+0000 7f658f7fe700 1 --2- 172.21.15.19:0/4087264559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65901015f0 0x7f6590196e00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f65800056d0 tx=0x7f65800057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.722+0000 7f658cff9700 1 -- 172.21.15.19:0/4087264559 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6580026070 con 0x7f65901015f0 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.722+0000 7f658cff9700 1 -- 172.21.15.19:0/4087264559 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f658000bba0 con 0x7f65901015f0 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.722+0000 7f658cff9700 1 -- 172.21.15.19:0/4087264559 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6580003920 con 0x7f65901015f0 2024-04-23T18:17:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.722+0000 7f6596239700 1 -- 172.21.15.19:0/4087264559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6590197630 con 0x7f65901015f0 2024-04-23T18:17:11.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.722+0000 7f6596239700 1 -- 172.21.15.19:0/4087264559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659019b680 con 0x7f65901015f0 2024-04-23T18:17:11.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.723+0000 7f658cff9700 1 -- 172.21.15.19:0/4087264559 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6580029070 con 0x7f65901015f0 2024-04-23T18:17:11.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.723+0000 7f6596239700 1 -- 172.21.15.19:0/4087264559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6590102a80 con 0x7f65901015f0 2024-04-23T18:17:11.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.723+0000 7f658cff9700 1 --2- 172.21.15.19:0/4087264559 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6578039650 0x7f657803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:11.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.723+0000 7f658cff9700 1 -- 172.21.15.19:0/4087264559 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f658001e070 con 0x7f65901015f0 2024-04-23T18:17:11.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.724+0000 7f658effd700 1 --2- 172.21.15.19:0/4087264559 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6578039650 0x7f657803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:11.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.724+0000 7f658effd700 1 --2- 172.21.15.19:0/4087264559 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6578039650 0x7f657803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6584006fd0 tx=0x7f6584006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:11.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.728+0000 7f658cff9700 1 -- 172.21.15.19:0/4087264559 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6580013070 con 0x7f65901015f0 2024-04-23T18:17:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.943+0000 7f6596239700 1 -- 172.21.15.19:0/4087264559 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6590198110 con 0x7f6578039650 2024-04-23T18:17:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.944+0000 7f658cff9700 1 -- 172.21.15.19:0/4087264559 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f6590198110 con 0x7f6578039650 2024-04-23T18:17:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:17:11.938 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-04-23T18:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.946+0000 7f6596239700 1 -- 172.21.15.19:0/4087264559 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6578039650 msgr2=0x7f657803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.946+0000 7f6596239700 1 --2- 172.21.15.19:0/4087264559 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6578039650 0x7f657803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6584006fd0 tx=0x7f6584006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.946+0000 7f6596239700 1 -- 172.21.15.19:0/4087264559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65901015f0 msgr2=0x7f6590196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.946+0000 7f6596239700 1 --2- 172.21.15.19:0/4087264559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65901015f0 0x7f6590196e00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f65800056d0 tx=0x7f65800057b0 comp rx=0 tx=0).stop 2024-04-23T18:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.946+0000 7f6596239700 1 -- 172.21.15.19:0/4087264559 shutdown_connections 2024-04-23T18:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.946+0000 7f6596239700 1 --2- 172.21.15.19:0/4087264559 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6578039650 0x7f657803bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.946+0000 7f6596239700 1 --2- 172.21.15.19:0/4087264559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65901015f0 0x7f6590196e00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.946+0000 7f6596239700 1 -- 172.21.15.19:0/4087264559 >> 172.21.15.19:0/4087264559 conn(0x7f6590076ea0 msgr2=0x7f6590102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.946+0000 7f6596239700 1 -- 172.21.15.19:0/4087264559 shutdown_connections 2024-04-23T18:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:11.946+0000 7f6596239700 1 -- 172.21.15.19:0/4087264559 wait complete. 2024-04-23T18:17:12.646 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-23T18:17:12.646 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-23T18:17:12.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:12 smithi019 ceph-mon[29215]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:12.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:12 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:12.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:12 smithi019 ceph-mon[29215]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:17:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.457+0000 7fec4518f700 1 -- 172.21.15.19:0/2471252096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec401036f0 msgr2=0x7fec40103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.457+0000 7fec4518f700 1 --2- 172.21.15.19:0/2471252096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec401036f0 0x7fec40103ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec30009910 tx=0x7fec30009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.457+0000 7fec4518f700 1 -- 172.21.15.19:0/2471252096 shutdown_connections 2024-04-23T18:17:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.457+0000 7fec4518f700 1 --2- 172.21.15.19:0/2471252096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec401036f0 0x7fec40103ad0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.457+0000 7fec4518f700 1 -- 172.21.15.19:0/2471252096 >> 172.21.15.19:0/2471252096 conn(0x7fec400fcea0 msgr2=0x7fec400ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.457+0000 7fec4518f700 1 -- 172.21.15.19:0/2471252096 shutdown_connections 2024-04-23T18:17:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.457+0000 7fec4518f700 1 -- 172.21.15.19:0/2471252096 wait complete. 2024-04-23T18:17:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.458+0000 7fec4518f700 1 Processor -- start 2024-04-23T18:17:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.458+0000 7fec4518f700 1 -- start start 2024-04-23T18:17:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec4518f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec401036f0 0x7fec40101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec4518f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec4010c9f0 con 0x7fec401036f0 2024-04-23T18:17:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec3ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec401036f0 0x7fec40101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec3ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec401036f0 0x7fec40101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43290/0 (socket says 172.21.15.19:43290) 2024-04-23T18:17:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec3ed9d700 1 -- 172.21.15.19:0/4097556967 learned_addr learned my addr 172.21.15.19:0/4097556967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:17:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec3ed9d700 1 -- 172.21.15.19:0/4097556967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec40103240 con 0x7fec401036f0 2024-04-23T18:17:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec3ed9d700 1 --2- 172.21.15.19:0/4097556967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec401036f0 0x7fec40101350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec300056d0 tx=0x7fec300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec27fff700 1 -- 172.21.15.19:0/4097556967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec30026070 con 0x7fec401036f0 2024-04-23T18:17:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec4518f700 1 -- 172.21.15.19:0/4097556967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec40101890 con 0x7fec401036f0 2024-04-23T18:17:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec27fff700 1 -- 172.21.15.19:0/4097556967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec30004370 con 0x7fec401036f0 2024-04-23T18:17:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec27fff700 1 -- 172.21.15.19:0/4097556967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec300039b0 con 0x7fec401036f0 2024-04-23T18:17:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.459+0000 7fec4518f700 1 -- 172.21.15.19:0/4097556967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec40101d50 con 0x7fec401036f0 2024-04-23T18:17:14.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.460+0000 7fec27fff700 1 -- 172.21.15.19:0/4097556967 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fec30029070 con 0x7fec401036f0 2024-04-23T18:17:14.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.460+0000 7fec4518f700 1 -- 172.21.15.19:0/4097556967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4006b0f0 con 0x7fec401036f0 2024-04-23T18:17:14.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.460+0000 7fec27fff700 1 --2- 172.21.15.19:0/4097556967 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec28039650 0x7fec2803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:14.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.461+0000 7fec27fff700 1 -- 172.21.15.19:0/4097556967 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fec3001e070 con 0x7fec401036f0 2024-04-23T18:17:14.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.461+0000 7fec3e59c700 1 --2- 172.21.15.19:0/4097556967 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec28039650 0x7fec2803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:14.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.461+0000 7fec3e59c700 1 --2- 172.21.15.19:0/4097556967 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec28039650 0x7fec2803bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fec34006fd0 tx=0x7fec34006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.466+0000 7fec27fff700 1 -- 172.21.15.19:0/4097556967 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec30013070 con 0x7fec401036f0 2024-04-23T18:17:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:14.682+0000 7fec4518f700 1 -- 172.21.15.19:0/4097556967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fec40102880 con 0x7fec401036f0 2024-04-23T18:17:15.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:14 smithi019 ceph-mon[29215]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:15.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.680+0000 7fec27fff700 1 -- 172.21.15.19:0/4097556967 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fec3000e3e0 con 0x7fec401036f0 2024-04-23T18:17:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.683+0000 7fec4518f700 1 -- 172.21.15.19:0/4097556967 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec28039650 msgr2=0x7fec2803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.683+0000 7fec4518f700 1 --2- 172.21.15.19:0/4097556967 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec28039650 0x7fec2803bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fec34006fd0 tx=0x7fec34006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.683+0000 7fec4518f700 1 -- 172.21.15.19:0/4097556967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec401036f0 msgr2=0x7fec40101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.683+0000 7fec4518f700 1 --2- 172.21.15.19:0/4097556967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec401036f0 0x7fec40101350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec300056d0 tx=0x7fec300057b0 comp rx=0 tx=0).stop 2024-04-23T18:17:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.683+0000 7fec4518f700 1 -- 172.21.15.19:0/4097556967 shutdown_connections 2024-04-23T18:17:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.683+0000 7fec4518f700 1 --2- 172.21.15.19:0/4097556967 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec28039650 0x7fec2803bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.683+0000 7fec4518f700 1 --2- 172.21.15.19:0/4097556967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec401036f0 0x7fec40101350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.683+0000 7fec4518f700 1 -- 172.21.15.19:0/4097556967 >> 172.21.15.19:0/4097556967 conn(0x7fec400fcea0 msgr2=0x7fec401044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.683+0000 7fec4518f700 1 -- 172.21.15.19:0/4097556967 shutdown_connections 2024-04-23T18:17:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:15.683+0000 7fec4518f700 1 -- 172.21.15.19:0/4097556967 wait complete. 2024-04-23T18:17:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-04-23T18:17:16.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:15 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4097556967' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T18:17:16.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:16.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:16.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:16.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:16.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:16.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:16.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:16.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:15 smithi019 ceph-mon[29215]: Updating smithi155:/etc/ceph/ceph.conf 2024-04-23T18:17:16.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:15 smithi019 ceph-mon[29215]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:16.304 INFO:tasks.cephadm:Adding mon.a on smithi019 2024-04-23T18:17:16.304 INFO:tasks.cephadm:Adding mon.b on smithi044 2024-04-23T18:17:16.304 INFO:tasks.cephadm:Adding mon.c on smithi155 2024-04-23T18:17:16.304 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi019:172.21.15.19=a;smithi044:172.21.15.44=b;smithi155:172.21.15.155=c' 2024-04-23T18:17:16.540 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:17.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:16 smithi019 ceph-mon[29215]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:17.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:16 smithi019 ceph-mon[29215]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:17.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:16 smithi019 ceph-mon[29215]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:17:17.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:16 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4097556967' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T18:17:17.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:16 smithi019 ceph-mon[29215]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:17.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:16 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:17.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:16 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:17.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:16 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:18.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.135+0000 7fc3d8d18700 1 -- 172.21.15.155:0/2512159876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d4108430 msgr2=0x7fc3d4108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:18.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.135+0000 7fc3d8d18700 1 --2- 172.21.15.155:0/2512159876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d4108430 0x7fc3d4108810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009910 tx=0x7fc3c4009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:18.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.135+0000 7fc3d8d18700 1 -- 172.21.15.155:0/2512159876 shutdown_connections 2024-04-23T18:17:18.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.135+0000 7fc3d8d18700 1 --2- 172.21.15.155:0/2512159876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d4108430 0x7fc3d4108810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.135+0000 7fc3d8d18700 1 -- 172.21.15.155:0/2512159876 >> 172.21.15.155:0/2512159876 conn(0x7fc3d40788d0 msgr2=0x7fc3d4078ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.135+0000 7fc3d8d18700 1 -- 172.21.15.155:0/2512159876 shutdown_connections 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.135+0000 7fc3d8d18700 1 -- 172.21.15.155:0/2512159876 wait complete. 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.136+0000 7fc3d8d18700 1 Processor -- start 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.136+0000 7fc3d8d18700 1 -- start start 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.136+0000 7fc3d8d18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d4108430 0x7fc3d4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.136+0000 7fc3d8d18700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d410c920 con 0x7fc3d4108430 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.137+0000 7fc3d259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d4108430 0x7fc3d4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.137+0000 7fc3d259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d4108430 0x7fc3d4197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.155:37938/0 (socket says 172.21.15.155:37938) 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.137+0000 7fc3d259c700 1 -- 172.21.15.155:0/474974998 learned_addr learned my addr 172.21.15.155:0/474974998 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.137+0000 7fc3d259c700 1 -- 172.21.15.155:0/474974998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d41983d0 con 0x7fc3d4108430 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.137+0000 7fc3d259c700 1 --2- 172.21.15.155:0/474974998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d4108430 0x7fc3d4197e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40056d0 tx=0x7fc3c40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.137+0000 7fc3c37fe700 1 -- 172.21.15.155:0/474974998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3c4026070 con 0x7fc3d4108430 2024-04-23T18:17:18.133 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.138+0000 7fc3c37fe700 1 -- 172.21.15.155:0/474974998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3c400bba0 con 0x7fc3d4108430 2024-04-23T18:17:18.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.138+0000 7fc3d8d18700 1 -- 172.21.15.155:0/474974998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d41986c0 con 0x7fc3d4108430 2024-04-23T18:17:18.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.138+0000 7fc3c37fe700 1 -- 172.21.15.155:0/474974998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3c4003920 con 0x7fc3d4108430 2024-04-23T18:17:18.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.138+0000 7fc3d8d18700 1 -- 172.21.15.155:0/474974998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d419c760 con 0x7fc3d4108430 2024-04-23T18:17:18.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.138+0000 7fc3d8d18700 1 -- 172.21.15.155:0/474974998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d406a620 con 0x7fc3d4108430 2024-04-23T18:17:18.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.138+0000 7fc3c37fe700 1 -- 172.21.15.155:0/474974998 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc3c4029070 con 0x7fc3d4108430 2024-04-23T18:17:18.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.139+0000 7fc3c37fe700 1 --2- 172.21.15.155:0/474974998 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc3bc039650 0x7fc3bc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:18.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.139+0000 7fc3c37fe700 1 -- 172.21.15.155:0/474974998 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc3c401e070 con 0x7fc3d4108430 2024-04-23T18:17:18.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.139+0000 7fc3d1d9b700 1 --2- 172.21.15.155:0/474974998 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc3bc039650 0x7fc3bc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:18.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.139+0000 7fc3d1d9b700 1 --2- 172.21.15.155:0/474974998 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc3bc039650 0x7fc3bc03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8006fd0 tx=0x7fc3c8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:18.139 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.145+0000 7fc3c37fe700 1 -- 172.21.15.155:0/474974998 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3c4013070 con 0x7fc3d4108430 2024-04-23T18:17:18.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.351+0000 7fc3d8d18700 1 -- 172.21.15.155:0/474974998 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi044:172.21.15.44=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}) -- 0x7fc3d41991f0 con 0x7fc3bc039650 2024-04-23T18:17:18.353 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.358+0000 7fc3c37fe700 1 -- 172.21.15.155:0/474974998 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc3d41991f0 con 0x7fc3bc039650 2024-04-23T18:17:18.353 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mon update... 2024-04-23T18:17:18.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.360+0000 7fc3d8d18700 1 -- 172.21.15.155:0/474974998 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc3bc039650 msgr2=0x7fc3bc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:18.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.360+0000 7fc3d8d18700 1 --2- 172.21.15.155:0/474974998 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc3bc039650 0x7fc3bc03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8006fd0 tx=0x7fc3c8006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:18.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.360+0000 7fc3d8d18700 1 -- 172.21.15.155:0/474974998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d4108430 msgr2=0x7fc3d4197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:18.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.360+0000 7fc3d8d18700 1 --2- 172.21.15.155:0/474974998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d4108430 0x7fc3d4197e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40056d0 tx=0x7fc3c40057b0 comp rx=0 tx=0).stop 2024-04-23T18:17:18.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.360+0000 7fc3d8d18700 1 -- 172.21.15.155:0/474974998 shutdown_connections 2024-04-23T18:17:18.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.360+0000 7fc3d8d18700 1 --2- 172.21.15.155:0/474974998 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc3bc039650 0x7fc3bc03bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:18.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.360+0000 7fc3d8d18700 1 --2- 172.21.15.155:0/474974998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d4108430 0x7fc3d4197e90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:18.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.360+0000 7fc3d8d18700 1 -- 172.21.15.155:0/474974998 >> 172.21.15.155:0/474974998 conn(0x7fc3d40788d0 msgr2=0x7fc3d40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:18.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.360+0000 7fc3d8d18700 1 -- 172.21.15.155:0/474974998 shutdown_connections 2024-04-23T18:17:18.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:18.360+0000 7fc3d8d18700 1 -- 172.21.15.155:0/474974998 wait complete. 2024-04-23T18:17:18.932 DEBUG:teuthology.orchestra.run.smithi044:mon.b> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mon.b.service 2024-04-23T18:17:18.935 DEBUG:teuthology.orchestra.run.smithi155:mon.c> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mon.c.service 2024-04-23T18:17:18.937 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:17:18.937 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:17:19.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:18 smithi019 ceph-mon[29215]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:19.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:19.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:19.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:19.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:19.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:19.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:19.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:19.972 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:20.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:19 smithi019 ceph-mon[29215]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi044:172.21.15.44=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:20.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:19 smithi019 ceph-mon[29215]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi044:172.21.15.44=b;smithi155:172.21.15.155=c;count:3 2024-04-23T18:17:20.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:19 smithi019 ceph-mon[29215]: Deploying daemon mon.c on smithi155 2024-04-23T18:17:21.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:20 smithi019 ceph-mon[29215]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.764+0000 7fac645b7700 1 -- 172.21.15.155:0/2724761238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5c103670 msgr2=0x7fac5c103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.764+0000 7fac645b7700 1 --2- 172.21.15.155:0/2724761238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5c103670 0x7fac5c103a50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009910 tx=0x7fac4c009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.765+0000 7fac645b7700 1 -- 172.21.15.155:0/2724761238 shutdown_connections 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.765+0000 7fac645b7700 1 --2- 172.21.15.155:0/2724761238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5c103670 0x7fac5c103a50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.765+0000 7fac645b7700 1 -- 172.21.15.155:0/2724761238 >> 172.21.15.155:0/2724761238 conn(0x7fac5c0fce40 msgr2=0x7fac5c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.765+0000 7fac645b7700 1 -- 172.21.15.155:0/2724761238 shutdown_connections 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.765+0000 7fac645b7700 1 -- 172.21.15.155:0/2724761238 wait complete. 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.765+0000 7fac645b7700 1 Processor -- start 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.765+0000 7fac645b7700 1 -- start start 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.765+0000 7fac645b7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5c103670 0x7fac5c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:22.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.765+0000 7fac645b7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c10c980 con 0x7fac5c103670 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.766+0000 7fac62353700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5c103670 0x7fac5c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.766+0000 7fac62353700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5c103670 0x7fac5c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.155:32978/0 (socket says 172.21.15.155:32978) 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.766+0000 7fac62353700 1 -- 172.21.15.155:0/1213442309 learned_addr learned my addr 172.21.15.155:0/1213442309 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.766+0000 7fac62353700 1 -- 172.21.15.155:0/1213442309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5c198430 con 0x7fac5c103670 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.766+0000 7fac62353700 1 --2- 172.21.15.155:0/1213442309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5c103670 0x7fac5c197ef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0056d0 tx=0x7fac4c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.766+0000 7fac537fe700 1 -- 172.21.15.155:0/1213442309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac4c026070 con 0x7fac5c103670 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.767+0000 7fac645b7700 1 -- 172.21.15.155:0/1213442309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5c1986c0 con 0x7fac5c103670 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.767+0000 7fac537fe700 1 -- 172.21.15.155:0/1213442309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4c004370 con 0x7fac5c103670 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.767+0000 7fac537fe700 1 -- 172.21.15.155:0/1213442309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac4c0039b0 con 0x7fac5c103670 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.767+0000 7fac645b7700 1 -- 172.21.15.155:0/1213442309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5c198b20 con 0x7fac5c103670 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.767+0000 7fac537fe700 1 -- 172.21.15.155:0/1213442309 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fac4c029070 con 0x7fac5c103670 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.768+0000 7fac537fe700 1 --2- 172.21.15.155:0/1213442309 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fac48039650 0x7fac4803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.768+0000 7fac537fe700 1 -- 172.21.15.155:0/1213442309 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fac4c01e070 con 0x7fac5c103670 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.768+0000 7fac645b7700 1 -- 172.21.15.155:0/1213442309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac5c06af50 con 0x7fac5c103670 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.768+0000 7fac61b52700 1 --2- 172.21.15.155:0/1213442309 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fac48039650 0x7fac4803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:22.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.768+0000 7fac61b52700 1 --2- 172.21.15.155:0/1213442309 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fac48039650 0x7fac4803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac58006fd0 tx=0x7fac58006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:22.785 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:22.773+0000 7fac537fe700 1 -- 172.21.15.155:0/1213442309 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac4c030080 con 0x7fac5c103670 2024-04-23T18:17:22.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:22 smithi155 systemd[1]: Starting Ceph mon.c for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:17:23.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:22 smithi019 ceph-mon[29215]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:23.081 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.087+0000 7fac645b7700 1 -- 172.21.15.155:0/1213442309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fac5c199250 con 0x7fac5c103670 2024-04-23T18:17:23.081 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.087+0000 7fac537fe700 1 -- 172.21.15.155:0/1213442309 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fac4c013070 con 0x7fac5c103670 2024-04-23T18:17:23.081 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:17:23.081 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"46706838-019d-11ef-bc93-c7b262605968","modified":"2024-04-23T18:15:28.877188Z","created":"2024-04-23T18:15:28.877188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T18:17:23.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.089+0000 7fac645b7700 1 -- 172.21.15.155:0/1213442309 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fac48039650 msgr2=0x7fac4803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:23.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.089+0000 7fac645b7700 1 --2- 172.21.15.155:0/1213442309 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fac48039650 0x7fac4803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac58006fd0 tx=0x7fac58006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:23.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.089+0000 7fac645b7700 1 -- 172.21.15.155:0/1213442309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5c103670 msgr2=0x7fac5c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:23.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.089+0000 7fac645b7700 1 --2- 172.21.15.155:0/1213442309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5c103670 0x7fac5c197ef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0056d0 tx=0x7fac4c0057b0 comp rx=0 tx=0).stop 2024-04-23T18:17:23.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.089+0000 7fac645b7700 1 -- 172.21.15.155:0/1213442309 shutdown_connections 2024-04-23T18:17:23.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.089+0000 7fac645b7700 1 --2- 172.21.15.155:0/1213442309 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fac48039650 0x7fac4803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:23.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.089+0000 7fac645b7700 1 --2- 172.21.15.155:0/1213442309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5c103670 0x7fac5c197ef0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:23.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.089+0000 7fac645b7700 1 -- 172.21.15.155:0/1213442309 >> 172.21.15.155:0/1213442309 conn(0x7fac5c0fce40 msgr2=0x7fac5c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:23.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.089+0000 7fac645b7700 1 -- 172.21.15.155:0/1213442309 shutdown_connections 2024-04-23T18:17:23.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:23.089+0000 7fac645b7700 1 -- 172.21.15.155:0/1213442309 wait complete. 2024-04-23T18:17:23.084 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-04-23T18:17:23.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 podman[33433]: 2024-04-23 18:17:22.937469418 +0000 UTC m=+0.017987148 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:17:23.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 podman[33433]: 2024-04-23 18:17:23.04841863 +0000 UTC m=+0.128936346 container create 88528f6d2fdd42a3f10680ccfd54ef31e3d4d9e4ab4140087243bfcd5bc33f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:17:23.801 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 podman[33433]: 2024-04-23 18:17:23.780538751 +0000 UTC m=+0.861056467 container init 88528f6d2fdd42a3f10680ccfd54ef31e3d4d9e4ab4140087243bfcd5bc33f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-23T18:17:23.801 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 podman[33433]: 2024-04-23 18:17:23.783805191 +0000 UTC m=+0.864322908 container start 88528f6d2fdd42a3f10680ccfd54ef31e3d4d9e4ab4140087243bfcd5bc33f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-23T18:17:24.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:23 smithi019 ceph-mon[29215]: from='client.? 172.21.15.155:0/1213442309' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:17:24.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T18:17:24.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-23T18:17:24.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pidfile_write: ignore empty --pid-file 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: load: jerasure load: lrc 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: RocksDB version: 7.9.2 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Git sha 0 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: DB SUMMARY 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: DB Session ID: P9DRS2MBYMPO44POD8MI 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: CURRENT file: CURRENT 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-23T18:17:24.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.error_if_exists: 0 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.create_if_missing: 0 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.paranoid_checks: 1 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.env: 0x55de0a32dcc0 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.info_log: 0x55de0ac25be0 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.statistics: (nil) 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.use_fsync: 0 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_log_file_size: 0 2024-04-23T18:17:24.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.allow_fallocate: 1 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.use_direct_reads: 0 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.db_log_dir: 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.wal_dir: 2024-04-23T18:17:24.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T18:17:24.078 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T18:17:24.078 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T18:17:24.078 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T18:17:24.078 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T18:17:24.078 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T18:17:24.078 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T18:17:24.078 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T18:17:24.078 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.write_buffer_manager: 0x55de0abb90e0 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.unordered_write: 0 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.row_cache: None 2024-04-23T18:17:24.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.wal_filter: None 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.two_write_queues: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.wal_compression: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.atomic_flush: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.log_readahead_size: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T18:17:24.080 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_background_jobs: 2 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_background_compactions: -1 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_subcompactions: 1 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T18:17:24.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_open_files: -1 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_background_flushes: -1 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Compression algorithms supported: 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: kZSTD supported: 0 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: kXpressCompression supported: 0 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: kZlibCompression supported: 1 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: kSnappyCompression supported: 1 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: kLZ4Compression supported: 1 2024-04-23T18:17:24.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: kBZip2Compression supported: 0 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.merge_operator: 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_filter: None 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55de0ac25dc0) 2024-04-23T18:17:24.083 INFO:journalctl@ceph.mon.c.smithi155.stdout: cache_index_and_filter_blocks: 1 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: pin_top_level_index_and_filter: 1 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: index_type: 0 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: data_block_index_type: 0 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: index_shortening: 1 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: checksum: 4 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: no_block_cache: 0 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: block_cache: 0x55de0abeb1f0 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: block_cache_name: BinnedLRUCache 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: block_cache_options: 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: capacity : 536870912 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: num_shard_bits : 4 2024-04-23T18:17:24.084 INFO:journalctl@ceph.mon.c.smithi155.stdout: strict_capacity_limit : 0 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: high_pri_pool_ratio: 0.000 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: block_cache_compressed: (nil) 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: persistent_cache: (nil) 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: block_size: 4096 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: block_size_deviation: 10 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: block_restart_interval: 16 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: index_block_restart_interval: 1 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: metadata_block_size: 4096 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: partition_filters: 0 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: use_delta_encoding: 1 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: filter_policy: bloomfilter 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: whole_key_filtering: 1 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: verify_compression: 0 2024-04-23T18:17:24.085 INFO:journalctl@ceph.mon.c.smithi155.stdout: read_amp_bytes_per_bit: 0 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout: format_version: 5 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout: enable_index_compression: 1 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout: block_align: 0 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout: max_auto_readahead_size: 262144 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout: prepopulate_block_cache: 0 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout: initial_auto_readahead_size: 8192 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compression: NoCompression 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T18:17:24.086 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.num_levels: 7 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T18:17:24.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T18:17:24.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T18:17:24.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.inplace_update_support: 0 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T18:17:24.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.bloom_locality: 0 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.max_successive_merges: 0 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.ttl: 2592000 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.enable_blob_files: false 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.min_blob_size: 0 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T18:17:24.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e96ab583-ff9a-4a6e-aad0-cde46fbbf342 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896243815649, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896243816062, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713896243, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e96ab583-ff9a-4a6e-aad0-cde46fbbf342", "db_session_id": "P9DRS2MBYMPO44POD8MI", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896243816142, "job": 1, "event": "recovery_finished"} 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55de0ad00000 2024-04-23T18:17:24.092 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: DB pointer 0x55de0acea000 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: ** DB Stats ** 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: ** Compaction Stats [default] ** 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:17:24.093 INFO:journalctl@ceph.mon.c.smithi155.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: ** Compaction Stats [default] ** 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T18:17:24.094 INFO:journalctl@ceph.mon.c.smithi155.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout: Block cache BinnedLRUCache@0x55de0abeb1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: using public_addr v2:172.21.15.155:0/0 -> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: starting mon.c rank -1 at public addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] at bind addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(???) e0 preinit fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).mds e1 new map 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout: e1 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:17:24.095 INFO:journalctl@ceph.mon.c.smithi155.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout: legacy client fscid: -1 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout: No filesystems configured 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:24.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Deploying cephadm binary to smithi155 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Added host smithi155 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4097556967' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Updating smithi155:/etc/ceph/ceph.conf 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4097556967' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:24.099 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi044:172.21.15.44=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi044:172.21.15.44=b;smithi155:172.21.15.155=c;count:3 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: Deploying daemon mon.c on smithi155 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: from='client.? 172.21.15.155:0/1213442309' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 ceph-mon[33468]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 bash[33433]: 88528f6d2fdd42a3f10680ccfd54ef31e3d4d9e4ab4140087243bfcd5bc33f3c 2024-04-23T18:17:24.100 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:23 smithi155 systemd[1]: Started Ceph mon.c for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:17:24.690 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:17:24.691 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:17:24.938 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:17:29.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:29.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:17:29.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: mon.a calling monitor election 2024-04-23T18:17:29.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:29.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: mon.c calling monitor election 2024-04-23T18:17:29.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: monmap epoch 2 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: last_changed 2024-04-23T18:17:23.838266+0000 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: min_mon_release 19 (squid) 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: election_strategy: 1 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: fsmap 2024-04-23T18:17:29.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:29.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: mgrmap e13: x(active, since 53s) 2024-04-23T18:17:29.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: overall HEALTH_OK 2024-04-23T18:17:29.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:29.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:29.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:29.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:29.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:29.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: mon.a calling monitor election 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: mon.c calling monitor election 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: monmap epoch 2 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:29.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: last_changed 2024-04-23T18:17:23.838266+0000 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: min_mon_release 19 (squid) 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: election_strategy: 1 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: fsmap 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: mgrmap e13: x(active, since 53s) 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: overall HEALTH_OK 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:29.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:29.353 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.359+0000 7f6839697700 1 -- 172.21.15.155:0/1851712669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f681c0035c0 msgr2=0x7f681c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:29.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.359+0000 7f6839697700 1 --2- 172.21.15.155:0/1851712669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f681c0035c0 0x7f681c005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f68341013a0 tx=0x7f682c00feb0 comp rx=0 tx=0).stop 2024-04-23T18:17:29.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.359+0000 7f6839697700 1 -- 172.21.15.155:0/1851712669 shutdown_connections 2024-04-23T18:17:29.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.359+0000 7f6839697700 1 --2- 172.21.15.155:0/1851712669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f681c0035c0 0x7f681c005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:29.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.359+0000 7f6839697700 1 -- 172.21.15.155:0/1851712669 >> 172.21.15.155:0/1851712669 conn(0x7f68340fcf80 msgr2=0x7f68340ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:29.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.360+0000 7f6839697700 1 -- 172.21.15.155:0/1851712669 shutdown_connections 2024-04-23T18:17:29.354 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.360+0000 7f6839697700 1 -- 172.21.15.155:0/1851712669 wait complete. 2024-04-23T18:17:29.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.361+0000 7f6839697700 1 Processor -- start 2024-04-23T18:17:29.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.361+0000 7f6839697700 1 -- start start 2024-04-23T18:17:29.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.361+0000 7f6839697700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f681c0035c0 0x7f6834197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:29.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.361+0000 7f6839697700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68341984d0 0x7f683419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:29.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.361+0000 7f6839697700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68341093c0 con 0x7f68341984d0 2024-04-23T18:17:29.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.361+0000 7f6839697700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6834109240 con 0x7f681c0035c0 2024-04-23T18:17:29.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.361+0000 7f6832ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f681c0035c0 0x7f6834197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:29.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f6832ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f681c0035c0 0x7f6834197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:54424/0 (socket says 172.21.15.155:54424) 2024-04-23T18:17:29.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f6832ffd700 1 -- 172.21.15.155:0/1172403258 learned_addr learned my addr 172.21.15.155:0/1172403258 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f6832ffd700 1 -- 172.21.15.155:0/1172403258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f681c0035c0 msgr2=0x7f6834197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f6832ffd700 1 -- 172.21.15.155:0/1172403258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f681c0035c0 msgr2=0x7f6834197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f6832ffd700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f681c0035c0 0x7f6834197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f6832ffd700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f681c0035c0 0x7f6834197f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f68327fc700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68341984d0 0x7f683419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f68327fc700 1 -- 172.21.15.155:0/1172403258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f681c0035c0 msgr2=0x7f6834197f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f68327fc700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f681c0035c0 0x7f6834197f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f68327fc700 1 -- 172.21.15.155:0/1172403258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6834198b10 con 0x7f68341984d0 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.362+0000 7f68327fc700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68341984d0 0x7f683419c860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f682800b4d0 tx=0x7f682800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.363+0000 7f681bfff700 1 -- 172.21.15.155:0/1172403258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6828011630 con 0x7f68341984d0 2024-04-23T18:17:29.357 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.363+0000 7f681bfff700 1 -- 172.21.15.155:0/1172403258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6828011c80 con 0x7f68341984d0 2024-04-23T18:17:29.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.363+0000 7f6839697700 1 -- 172.21.15.155:0/1172403258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683419cda0 con 0x7f68341984d0 2024-04-23T18:17:29.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.363+0000 7f681bfff700 1 -- 172.21.15.155:0/1172403258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6828010da0 con 0x7f68341984d0 2024-04-23T18:17:29.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.363+0000 7f6839697700 1 -- 172.21.15.155:0/1172403258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683419d380 con 0x7f68341984d0 2024-04-23T18:17:29.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.364+0000 7f681bfff700 1 -- 172.21.15.155:0/1172403258 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6828018020 con 0x7f68341984d0 2024-04-23T18:17:29.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.364+0000 7f6839697700 1 -- 172.21.15.155:0/1172403258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683419d690 con 0x7f68341984d0 2024-04-23T18:17:29.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.364+0000 7f681bfff700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6820039440 0x7f682003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:29.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.364+0000 7f681bfff700 1 -- 172.21.15.155:0/1172403258 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f682804c180 con 0x7f68341984d0 2024-04-23T18:17:29.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.365+0000 7f6832ffd700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6820039440 0x7f682003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:29.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.365+0000 7f6832ffd700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6820039440 0x7f682003b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f682c00fa60 tx=0x7f682c005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:29.363 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.370+0000 7f681bfff700 1 -- 172.21.15.155:0/1172403258 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f683419d690 con 0x7f68341984d0 2024-04-23T18:17:29.645 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.652+0000 7f6839697700 1 -- 172.21.15.155:0/1172403258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6834199380 con 0x7f68341984d0 2024-04-23T18:17:29.645 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.652+0000 7f681bfff700 1 -- 172.21.15.155:0/1172403258 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f6834199380 con 0x7f68341984d0 2024-04-23T18:17:29.646 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:17:29.646 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":2,"fsid":"46706838-019d-11ef-bc93-c7b262605968","modified":"2024-04-23T18:17:23.838266Z","created":"2024-04-23T18:15:28.877188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T18:17:29.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.654+0000 7f6839697700 1 -- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6820039440 msgr2=0x7f682003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:29.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.654+0000 7f6839697700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6820039440 0x7f682003b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f682c00fa60 tx=0x7f682c005a90 comp rx=0 tx=0).stop 2024-04-23T18:17:29.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.655+0000 7f6839697700 1 -- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68341984d0 msgr2=0x7f683419c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:29.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.655+0000 7f6839697700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68341984d0 0x7f683419c860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f682800b4d0 tx=0x7f682800b890 comp rx=0 tx=0).stop 2024-04-23T18:17:29.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.655+0000 7f6839697700 1 -- 172.21.15.155:0/1172403258 shutdown_connections 2024-04-23T18:17:29.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.655+0000 7f6839697700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6820039440 0x7f682003b900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:29.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.655+0000 7f6839697700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f681c0035c0 0x7f6834197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:29.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.655+0000 7f6839697700 1 --2- 172.21.15.155:0/1172403258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68341984d0 0x7f683419c860 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:29.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.655+0000 7f6839697700 1 -- 172.21.15.155:0/1172403258 >> 172.21.15.155:0/1172403258 conn(0x7f68340fcf80 msgr2=0x7f683406b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:29.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.655+0000 7f6839697700 1 -- 172.21.15.155:0/1172403258 shutdown_connections 2024-04-23T18:17:29.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:29.655+0000 7f6839697700 1 -- 172.21.15.155:0/1172403258 wait complete. 2024-04-23T18:17:29.649 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 2 2024-04-23T18:17:30.214 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:29 smithi155 ceph-mon[33468]: Deploying daemon mon.b on smithi044 2024-04-23T18:17:30.214 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:29 smithi155 ceph-mon[33468]: from='client.? 172.21.15.155:0/1172403258' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:17:30.214 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:29 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:30.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:29 smithi019 ceph-mon[29215]: Deploying daemon mon.b on smithi044 2024-04-23T18:17:30.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:29 smithi019 ceph-mon[29215]: from='client.? 172.21.15.155:0/1172403258' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:17:30.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:29 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:31.241 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:17:31.241 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:17:31.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:30 smithi019 ceph-mon[29215]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:31.266 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:30 smithi155 ceph-mon[33468]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:31.482 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:17:32.538 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 podman[32439]: 2024-04-23 18:17:32.341114727 +0000 UTC m=+0.018814318 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:17:32.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 podman[32439]: 2024-04-23 18:17:32.642462986 +0000 UTC m=+0.320162563 container init 4edd044bd4d648fa83b8d50525e6550410a7457155b763ab3fed4499a2e117bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-23T18:17:32.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 podman[32439]: 2024-04-23 18:17:32.64558507 +0000 UTC m=+0.323284665 container start 4edd044bd4d648fa83b8d50525e6550410a7457155b763ab3fed4499a2e117bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T18:17:32.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T18:17:32.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-23T18:17:32.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pidfile_write: ignore empty --pid-file 2024-04-23T18:17:32.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: load: jerasure load: lrc 2024-04-23T18:17:32.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: RocksDB version: 7.9.2 2024-04-23T18:17:32.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Git sha 0 2024-04-23T18:17:32.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-23T18:17:32.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: DB SUMMARY 2024-04-23T18:17:32.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: DB Session ID: I3NZFOF6JIENU95FX0OF 2024-04-23T18:17:32.815 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: CURRENT file: CURRENT 2024-04-23T18:17:32.815 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T18:17:32.815 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-23T18:17:32.815 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-23T18:17:32.815 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-23T18:17:32.815 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.error_if_exists: 0 2024-04-23T18:17:32.815 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.create_if_missing: 0 2024-04-23T18:17:32.815 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.paranoid_checks: 1 2024-04-23T18:17:32.815 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T18:17:32.815 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.env: 0x561dca14bcc0 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.info_log: 0x561dcc00fbe0 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.statistics: (nil) 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.use_fsync: 0 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_log_file_size: 0 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T18:17:32.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.allow_fallocate: 1 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.use_direct_reads: 0 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.db_log_dir: 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.wal_dir: 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T18:17:32.817 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.write_buffer_manager: 0x561dcbfa30e0 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.unordered_write: 0 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T18:17:32.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T18:17:32.819 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T18:17:32.819 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.row_cache: None 2024-04-23T18:17:32.819 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.wal_filter: None 2024-04-23T18:17:32.820 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T18:17:32.820 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T18:17:32.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.two_write_queues: 0 2024-04-23T18:17:32.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T18:17:32.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.wal_compression: 0 2024-04-23T18:17:32.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.atomic_flush: 0 2024-04-23T18:17:32.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T18:17:32.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T18:17:32.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T18:17:32.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.log_readahead_size: 0 2024-04-23T18:17:32.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T18:17:32.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_background_jobs: 2 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_background_compactions: -1 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_subcompactions: 1 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T18:17:32.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_open_files: -1 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_background_flushes: -1 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Compression algorithms supported: 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: kZSTD supported: 0 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: kXpressCompression supported: 0 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: kZlibCompression supported: 1 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: kSnappyCompression supported: 1 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: kLZ4Compression supported: 1 2024-04-23T18:17:32.823 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: kBZip2Compression supported: 0 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.merge_operator: 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_filter: None 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561dcc00fdc0) 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout: cache_index_and_filter_blocks: 1 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T18:17:32.824 INFO:journalctl@ceph.mon.b.smithi044.stdout: pin_top_level_index_and_filter: 1 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: index_type: 0 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: data_block_index_type: 0 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: index_shortening: 1 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: checksum: 4 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: no_block_cache: 0 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_cache: 0x561dcbfd51f0 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_cache_name: BinnedLRUCache 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_cache_options: 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: capacity : 536870912 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: num_shard_bits : 4 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: strict_capacity_limit : 0 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: high_pri_pool_ratio: 0.000 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_cache_compressed: (nil) 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: persistent_cache: (nil) 2024-04-23T18:17:32.825 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_size: 4096 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_size_deviation: 10 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_restart_interval: 16 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: index_block_restart_interval: 1 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: metadata_block_size: 4096 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: partition_filters: 0 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: use_delta_encoding: 1 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: filter_policy: bloomfilter 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: whole_key_filtering: 1 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: verify_compression: 0 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: read_amp_bytes_per_bit: 0 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: format_version: 5 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: enable_index_compression: 1 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_align: 0 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: max_auto_readahead_size: 262144 2024-04-23T18:17:32.826 INFO:journalctl@ceph.mon.b.smithi044.stdout: prepopulate_block_cache: 0 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout: initial_auto_readahead_size: 8192 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compression: NoCompression 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.num_levels: 7 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T18:17:32.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:17:32.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T18:17:32.829 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T18:17:32.830 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.inplace_update_support: 0 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.bloom_locality: 0 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.max_successive_merges: 0 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.ttl: 2592000 2024-04-23T18:17:32.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.enable_blob_files: false 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.min_blob_size: 0 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-23T18:17:32.832 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 66e00dc7-f8fb-4359-b1ef-b51a9e3955fe 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896252675841, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896252676256, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713896252, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "66e00dc7-f8fb-4359-b1ef-b51a9e3955fe", "db_session_id": "I3NZFOF6JIENU95FX0OF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896252676338, "job": 1, "event": "recovery_finished"} 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561dcc0ea000 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: DB pointer 0x561dcc0d4000 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout: ** DB Stats ** 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:17:32.833 INFO:journalctl@ceph.mon.b.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: ** Compaction Stats [default] ** 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: ** Compaction Stats [default] ** 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T18:17:32.834 INFO:journalctl@ceph.mon.b.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: Block cache BinnedLRUCache@0x561dcbfd51f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-04-23T18:17:32.835 INFO:journalctl@ceph.mon.b.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: using public_addr v2:172.21.15.44:0/0 -> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: starting mon.b rank -1 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(???) e0 preinit fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).mds e1 new map 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout: e1 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout: legacy client fscid: -1 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout: No filesystems configured 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-23T18:17:32.836 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:32.837 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Deploying cephadm binary to smithi155 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Added host smithi155 2024-04-23T18:17:32.838 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4097556967' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:32.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Updating smithi155:/etc/ceph/ceph.conf 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4097556967' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:32.840 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi044:172.21.15.44=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi044:172.21.15.44=b;smithi155:172.21.15.155=c;count:3 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Deploying daemon mon.c on smithi155 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='client.? 172.21.15.155:0/1213442309' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.a calling monitor election 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:32.841 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.c calling monitor election 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: monmap epoch 2 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: last_changed 2024-04-23T18:17:23.838266+0000 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: min_mon_release 19 (squid) 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: election_strategy: 1 2024-04-23T18:17:32.842 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: fsmap 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mgrmap e13: x(active, since 53s) 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: overall HEALTH_OK 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: Deploying daemon mon.b on smithi044 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='client.? 172.21.15.155:0/1172403258' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 ceph-mon[32458]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 bash[32439]: 4edd044bd4d648fa83b8d50525e6550410a7457155b763ab3fed4499a2e117bd 2024-04-23T18:17:32.843 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:32 smithi044 systemd[1]: Started Ceph mon.b for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:17:38.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:17:38.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: mon.a calling monitor election 2024-04-23T18:17:38.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: mon.c calling monitor election 2024-04-23T18:17:38.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:38.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:38.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:38.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:17:38.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: monmap epoch 3 2024-04-23T18:17:38.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:38.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: last_changed 2024-04-23T18:17:32.700005+0000 2024-04-23T18:17:38.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:17:38.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: min_mon_release 19 (squid) 2024-04-23T18:17:38.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: election_strategy: 1 2024-04-23T18:17:38.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:17:38.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-04-23T18:17:38.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-04-23T18:17:38.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: fsmap 2024-04-23T18:17:38.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:38.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: mgrmap e13: x(active, since 62s) 2024-04-23T18:17:38.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: overall HEALTH_OK 2024-04-23T18:17:38.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:38.016 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:38.016 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: mon.a calling monitor election 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: mon.c calling monitor election 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: monmap epoch 3 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: last_changed 2024-04-23T18:17:32.700005+0000 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: min_mon_release 19 (squid) 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: election_strategy: 1 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: fsmap 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: mgrmap e13: x(active, since 62s) 2024-04-23T18:17:38.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: overall HEALTH_OK 2024-04-23T18:17:38.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:38.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:38.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:39.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.048+0000 7fcb6f748700 1 -- 172.21.15.155:0/2472742797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb54006240 msgr2=0x7fcb540086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:39.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.048+0000 7fcb6f748700 1 --2- 172.21.15.155:0/2472742797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb54006240 0x7fcb540086b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb68103620 tx=0x7fcb5800fe90 comp rx=0 tx=0).stop 2024-04-23T18:17:39.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.049+0000 7fcb6f748700 1 -- 172.21.15.155:0/2472742797 shutdown_connections 2024-04-23T18:17:39.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.049+0000 7fcb6f748700 1 --2- 172.21.15.155:0/2472742797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb54006240 0x7fcb540086b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:39.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.049+0000 7fcb6f748700 1 -- 172.21.15.155:0/2472742797 >> 172.21.15.155:0/2472742797 conn(0x7fcb680fe050 msgr2=0x7fcb68100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:39.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.049+0000 7fcb6f748700 1 -- 172.21.15.155:0/2472742797 shutdown_connections 2024-04-23T18:17:39.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.049+0000 7fcb6f748700 1 -- 172.21.15.155:0/2472742797 wait complete. 2024-04-23T18:17:39.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:38 smithi155 ceph-mon[33468]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:39.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:39.044 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:39.044 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:39.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.049+0000 7fcb6f748700 1 Processor -- start 2024-04-23T18:17:39.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.049+0000 7fcb6f748700 1 -- start start 2024-04-23T18:17:39.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6f748700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb54006240 0x7fcb68198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:39.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6f748700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb68198650 0x7fcb6819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:39.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6f748700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6819cf20 0x7fcb6819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:39.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6f748700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6810a460 con 0x7fcb54006240 2024-04-23T18:17:39.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6f748700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6810a2e0 con 0x7fcb6819cf20 2024-04-23T18:17:39.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6f748700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6810a5e0 con 0x7fcb68198650 2024-04-23T18:17:39.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6cce3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb68198650 0x7fcb6819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6cce3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb68198650 0x7fcb6819c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:54436/0 (socket says 172.21.15.155:54436) 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6cce3700 1 -- 172.21.15.155:0/642099134 learned_addr learned my addr 172.21.15.155:0/642099134 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6d4e4700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb54006240 0x7fcb68198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6dce5700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6819cf20 0x7fcb6819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6cce3700 1 -- 172.21.15.155:0/642099134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6819cf20 msgr2=0x7fcb6819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6cce3700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6819cf20 0x7fcb6819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6cce3700 1 -- 172.21.15.155:0/642099134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb54006240 msgr2=0x7fcb68198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6cce3700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb54006240 0x7fcb68198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6cce3700 1 -- 172.21.15.155:0/642099134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb681a4c90 con 0x7fcb68198650 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb6cce3700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb68198650 0x7fcb6819c9e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400b780 tx=0x7fcb6400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.050+0000 7fcb5e7fc700 1 -- 172.21.15.155:0/642099134 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64004230 con 0x7fcb68198650 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.051+0000 7fcb5e7fc700 1 -- 172.21.15.155:0/642099134 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb640043d0 con 0x7fcb68198650 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.051+0000 7fcb6f748700 1 -- 172.21.15.155:0/642099134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb681a4ec0 con 0x7fcb68198650 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.051+0000 7fcb5e7fc700 1 -- 172.21.15.155:0/642099134 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64011b20 con 0x7fcb68198650 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.051+0000 7fcb6f748700 1 -- 172.21.15.155:0/642099134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb681a5440 con 0x7fcb68198650 2024-04-23T18:17:39.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.051+0000 7fcb5e7fc700 1 -- 172.21.15.155:0/642099134 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcb64011cc0 con 0x7fcb68198650 2024-04-23T18:17:39.048 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.052+0000 7fcb5e7fc700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb4c039ae0 0x7fcb4c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:39.048 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.052+0000 7fcb5e7fc700 1 -- 172.21.15.155:0/642099134 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcb6404d2c0 con 0x7fcb68198650 2024-04-23T18:17:39.048 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.052+0000 7fcb6d4e4700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb4c039ae0 0x7fcb4c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:39.048 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.052+0000 7fcb6f748700 1 -- 172.21.15.155:0/642099134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb680640a0 con 0x7fcb68198650 2024-04-23T18:17:39.048 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.052+0000 7fcb6d4e4700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb4c039ae0 0x7fcb4c03bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800fa40 tx=0x7fcb58005ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:39.050 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.057+0000 7fcb5e7fc700 1 -- 172.21.15.155:0/642099134 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb6401fad0 con 0x7fcb68198650 2024-04-23T18:17:39.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:38 smithi019 ceph-mon[29215]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:39.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:39.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:39.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:39.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.347+0000 7fcb6f748700 1 -- 172.21.15.155:0/642099134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcb6802df80 con 0x7fcb68198650 2024-04-23T18:17:39.342 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.348+0000 7fcb5e7fc700 1 -- 172.21.15.155:0/642099134 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7fcb64018350 con 0x7fcb68198650 2024-04-23T18:17:39.342 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-23T18:17:39.342 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":3,"fsid":"46706838-019d-11ef-bc93-c7b262605968","modified":"2024-04-23T18:17:32.700005Z","created":"2024-04-23T18:15:28.877188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T18:17:39.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 -- 172.21.15.155:0/642099134 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb4c039ae0 msgr2=0x7fcb4c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:39.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb4c039ae0 0x7fcb4c03bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800fa40 tx=0x7fcb58005ad0 comp rx=0 tx=0).stop 2024-04-23T18:17:39.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 -- 172.21.15.155:0/642099134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb68198650 msgr2=0x7fcb6819c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:39.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb68198650 0x7fcb6819c9e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400b780 tx=0x7fcb6400ba90 comp rx=0 tx=0).stop 2024-04-23T18:17:39.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 -- 172.21.15.155:0/642099134 shutdown_connections 2024-04-23T18:17:39.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb4c039ae0 0x7fcb4c03bfa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:39.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6819cf20 0x7fcb6819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:39.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb54006240 0x7fcb68198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:39.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 --2- 172.21.15.155:0/642099134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb68198650 0x7fcb6819c9e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:39.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 -- 172.21.15.155:0/642099134 >> 172.21.15.155:0/642099134 conn(0x7fcb680fe050 msgr2=0x7fcb68107290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:39.345 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 -- 172.21.15.155:0/642099134 shutdown_connections 2024-04-23T18:17:39.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:39.350+0000 7fcb6f748700 1 -- 172.21.15.155:0/642099134 wait complete. 2024-04-23T18:17:39.346 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 3 2024-04-23T18:17:40.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: mon.a calling monitor election 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: mon.c calling monitor election 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:17:40.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: monmap epoch 3 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: last_changed 2024-04-23T18:17:32.700005+0000 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: min_mon_release 19 (squid) 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: election_strategy: 1 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: fsmap 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: mgrmap e13: x(active, since 62s) 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: overall HEALTH_OK 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:40.066 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.066 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:40.066 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:40.215 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: mon.b calling monitor election 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: Updating smithi155:/etc/ceph/ceph.conf 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: mon.b calling monitor election 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: mon.a calling monitor election 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: mon.c calling monitor election 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: monmap epoch 3 2024-04-23T18:17:40.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: last_changed 2024-04-23T18:17:32.700005+0000 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: min_mon_release 19 (squid) 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: election_strategy: 1 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: fsmap 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: mgrmap e13: x(active, since 64s) 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: overall HEALTH_OK 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:40.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:40 smithi155 ceph-mon[33468]: Reconfiguring daemon mon.a on smithi019 2024-04-23T18:17:40.242 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-23T18:17:40.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-23T18:17:40.473 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: mon.b calling monitor election 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: Updating smithi155:/etc/ceph/ceph.conf 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: mon.b calling monitor election 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: mon.a calling monitor election 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: mon.c calling monitor election 2024-04-23T18:17:40.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: monmap epoch 3 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: last_changed 2024-04-23T18:17:32.700005+0000 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: min_mon_release 19 (squid) 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: election_strategy: 1 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: fsmap 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: mgrmap e13: x(active, since 64s) 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: overall HEALTH_OK 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:40.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:40 smithi019 ceph-mon[29215]: Reconfiguring daemon mon.a on smithi019 2024-04-23T18:17:40.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: mon.b calling monitor election 2024-04-23T18:17:40.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-23T18:17:40.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-23T18:17:40.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: Updating smithi155:/etc/ceph/ceph.conf 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: mon.b calling monitor election 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: mon.a calling monitor election 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: mon.c calling monitor election 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: monmap epoch 3 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: last_changed 2024-04-23T18:17:32.700005+0000 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: created 2024-04-23T18:15:28.877188+0000 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: min_mon_release 19 (squid) 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: election_strategy: 1 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-23T18:17:40.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-04-23T18:17:40.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-04-23T18:17:40.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: fsmap 2024-04-23T18:17:40.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:17:40.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: mgrmap e13: x(active, since 64s) 2024-04-23T18:17:40.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: overall HEALTH_OK 2024-04-23T18:17:40.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:40.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:40 smithi044 ceph-mon[32458]: Reconfiguring daemon mon.a on smithi019 2024-04-23T18:17:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:41.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:41 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:41.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:41 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:41.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:41 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:41.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:41 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:41.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:41 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:41.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:41 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:17:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:42 smithi019 ceph-mon[29215]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:42 smithi019 ceph-mon[29215]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:17:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:17:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:42 smithi019 ceph-mon[29215]: Reconfiguring daemon mon.b on smithi044 2024-04-23T18:17:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:42 smithi044 ceph-mon[32458]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:42 smithi044 ceph-mon[32458]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:17:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:17:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:42 smithi044 ceph-mon[32458]: Reconfiguring daemon mon.b on smithi044 2024-04-23T18:17:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:42 smithi155 ceph-mon[33468]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:42 smithi155 ceph-mon[33468]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:17:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:17:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:42 smithi155 ceph-mon[33468]: Reconfiguring daemon mon.b on smithi044 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.141+0000 7eff4b6a4700 1 -- 172.21.15.19:0/693216200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4406afe0 msgr2=0x7eff4406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.141+0000 7eff4b6a4700 1 --2- 172.21.15.19:0/693216200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4406afe0 0x7eff4406b440 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7eff34009990 tx=0x7eff34009ca0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.141+0000 7eff4b6a4700 1 -- 172.21.15.19:0/693216200 shutdown_connections 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.141+0000 7eff4b6a4700 1 --2- 172.21.15.19:0/693216200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff4406b980 0x7eff4410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.141+0000 7eff4b6a4700 1 --2- 172.21.15.19:0/693216200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff44108a20 0x7eff44108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.141+0000 7eff4b6a4700 1 --2- 172.21.15.19:0/693216200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4406afe0 0x7eff4406b440 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.141+0000 7eff4b6a4700 1 -- 172.21.15.19:0/693216200 >> 172.21.15.19:0/693216200 conn(0x7eff440788b0 msgr2=0x7eff44078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.141+0000 7eff4b6a4700 1 -- 172.21.15.19:0/693216200 shutdown_connections 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.141+0000 7eff4b6a4700 1 -- 172.21.15.19:0/693216200 wait complete. 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff4b6a4700 1 Processor -- start 2024-04-23T18:17:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff4b6a4700 1 -- start start 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff4b6a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4406afe0 0x7eff44199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff4b6a4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff4406b980 0x7eff441996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff4b6a4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff44108a20 0x7eff4419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff4b6a4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff44112c10 con 0x7eff4406afe0 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff4b6a4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff44112a90 con 0x7eff4406b980 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff4b6a4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7eff44112d90 con 0x7eff44108a20 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff49440700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4406afe0 0x7eff44199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff49440700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4406afe0 0x7eff44199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49960/0 (socket says 172.21.15.19:49960) 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff49440700 1 -- 172.21.15.19:0/850857226 learned_addr learned my addr 172.21.15.19:0/850857226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff48c3f700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff4406b980 0x7eff441996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff49c41700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff44108a20 0x7eff4419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff49440700 1 -- 172.21.15.19:0/850857226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff44108a20 msgr2=0x7eff4419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:43.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff49440700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff44108a20 0x7eff4419da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff49440700 1 -- 172.21.15.19:0/850857226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff4406b980 msgr2=0x7eff441996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff49440700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff4406b980 0x7eff441996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.142+0000 7eff49440700 1 -- 172.21.15.19:0/850857226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4419e140 con 0x7eff4406afe0 2024-04-23T18:17:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.143+0000 7eff49440700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4406afe0 0x7eff44199180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff4000e9a0 tx=0x7eff4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.143+0000 7eff3a7fc700 1 -- 172.21.15.19:0/850857226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4000ca10 con 0x7eff4406afe0 2024-04-23T18:17:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.143+0000 7eff4b6a4700 1 -- 172.21.15.19:0/850857226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4419e430 con 0x7eff4406afe0 2024-04-23T18:17:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.143+0000 7eff3a7fc700 1 -- 172.21.15.19:0/850857226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff4000cbb0 con 0x7eff4406afe0 2024-04-23T18:17:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.143+0000 7eff3a7fc700 1 -- 172.21.15.19:0/850857226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40018680 con 0x7eff4406afe0 2024-04-23T18:17:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.143+0000 7eff4b6a4700 1 -- 172.21.15.19:0/850857226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff441a5cb0 con 0x7eff4406afe0 2024-04-23T18:17:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.144+0000 7eff3a7fc700 1 -- 172.21.15.19:0/850857226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7eff40018820 con 0x7eff4406afe0 2024-04-23T18:17:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.144+0000 7eff4b6a4700 1 -- 172.21.15.19:0/850857226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff44069730 con 0x7eff4406afe0 2024-04-23T18:17:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.144+0000 7eff3a7fc700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7eff30039890 0x7eff3003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.144+0000 7eff3a7fc700 1 -- 172.21.15.19:0/850857226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7eff4004cad0 con 0x7eff4406afe0 2024-04-23T18:17:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.144+0000 7eff48c3f700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7eff30039890 0x7eff3003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.145+0000 7eff48c3f700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7eff30039890 0x7eff3003bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff44078f20 tx=0x7eff34005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:43.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.150+0000 7eff3a7fc700 1 -- 172.21.15.19:0/850857226 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff4000fc70 con 0x7eff4406afe0 2024-04-23T18:17:43.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.363+0000 7eff4b6a4700 1 -- 172.21.15.19:0/850857226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7eff4402df80 con 0x7eff4406afe0 2024-04-23T18:17:43.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.364+0000 7eff3a7fc700 1 -- 172.21.15.19:0/850857226 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7eff4001fdd0 con 0x7eff4406afe0 2024-04-23T18:17:43.357 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:43.358 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-04-23T18:17:43.358 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:43.358 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2024-04-23T18:17:43.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 -- 172.21.15.19:0/850857226 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7eff30039890 msgr2=0x7eff3003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:43.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7eff30039890 0x7eff3003bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff44078f20 tx=0x7eff34005fd0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 -- 172.21.15.19:0/850857226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4406afe0 msgr2=0x7eff44199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:43.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4406afe0 0x7eff44199180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff4000e9a0 tx=0x7eff4000ecb0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 -- 172.21.15.19:0/850857226 shutdown_connections 2024-04-23T18:17:43.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7eff30039890 0x7eff3003bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff44108a20 0x7eff4419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4406afe0 0x7eff44199180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 --2- 172.21.15.19:0/850857226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff4406b980 0x7eff441996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:43.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 -- 172.21.15.19:0/850857226 >> 172.21.15.19:0/850857226 conn(0x7eff440788b0 msgr2=0x7eff440ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:43.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.366+0000 7eff4b6a4700 1 -- 172.21.15.19:0/850857226 shutdown_connections 2024-04-23T18:17:43.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:43.367+0000 7eff4b6a4700 1 -- 172.21.15.19:0/850857226 wait complete. 2024-04-23T18:17:44.077 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-23T18:17:44.078 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:17:44.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:17:44.105 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:17:44.105 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:44.171 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:17:44.171 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:17:44.204 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:17:44.204 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:44.273 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:17:44.274 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:17:44.300 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:17:44.300 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:44.361 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:44.361 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:44.361 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:44 smithi155 ceph-mon[33468]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:17:44.361 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:44.361 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:17:44.362 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:44.362 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:44 smithi155 ceph-mon[33468]: Reconfiguring daemon mon.c on smithi155 2024-04-23T18:17:44.362 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:44 smithi155 ceph-mon[33468]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:44.362 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:44 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/850857226' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:44.365 INFO:tasks.cephadm:Adding mgr.x on smithi019 2024-04-23T18:17:44.365 INFO:tasks.cephadm:Adding mgr.y on smithi044 2024-04-23T18:17:44.365 INFO:tasks.cephadm:Adding mgr.z on smithi155 2024-04-23T18:17:44.365 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi019=x;smithi044=y;smithi155=z' 2024-04-23T18:17:44.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:44.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:44.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:44 smithi019 ceph-mon[29215]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:17:44.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:44.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:17:44.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:44.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:44 smithi019 ceph-mon[29215]: Reconfiguring daemon mon.c on smithi155 2024-04-23T18:17:44.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:44 smithi019 ceph-mon[29215]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:44.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:44 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/850857226' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:44.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:44.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:44.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:44 smithi044 ceph-mon[32458]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:17:44.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:17:44.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:17:44.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:44.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:44 smithi044 ceph-mon[32458]: Reconfiguring daemon mon.c on smithi155 2024-04-23T18:17:44.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:44 smithi044 ceph-mon[32458]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:44.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:44 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/850857226' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:44.964 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:17:46.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:46.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:46.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:46.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:46.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:46.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:46.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:46.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:46.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:46.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:46.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:46.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:46.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.294+0000 7fcd65945700 1 -- 172.21.15.155:0/2400047506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd60101990 msgr2=0x7fcd60101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:46.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.294+0000 7fcd65945700 1 --2- 172.21.15.155:0/2400047506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd60101990 0x7fcd60101db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009990 tx=0x7fcd54009ca0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.294+0000 7fcd65945700 1 -- 172.21.15.155:0/2400047506 shutdown_connections 2024-04-23T18:17:46.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.294+0000 7fcd65945700 1 --2- 172.21.15.155:0/2400047506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60103530 0x7fcd60109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.294+0000 7fcd65945700 1 --2- 172.21.15.155:0/2400047506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd60101990 0x7fcd60101db0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.294+0000 7fcd65945700 1 --2- 172.21.15.155:0/2400047506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd60102b90 0x7fcd60102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.294+0000 7fcd65945700 1 -- 172.21.15.155:0/2400047506 >> 172.21.15.155:0/2400047506 conn(0x7fcd600fcf30 msgr2=0x7fcd600ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:46.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.294+0000 7fcd65945700 1 -- 172.21.15.155:0/2400047506 shutdown_connections 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.294+0000 7fcd65945700 1 -- 172.21.15.155:0/2400047506 wait complete. 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd65945700 1 Processor -- start 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd65945700 1 -- start start 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd65945700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd60101990 0x7fcd60198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd65945700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd60102b90 0x7fcd60199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd65945700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60103530 0x7fcd601a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd65945700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6010d420 con 0x7fcd60102b90 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd65945700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6010d2a0 con 0x7fcd60103530 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd65945700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6010d5a0 con 0x7fcd60101990 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd5effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd60101990 0x7fcd60198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd5effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd60101990 0x7fcd60198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51234/0 (socket says 172.21.15.155:51234) 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd5effd700 1 -- 172.21.15.155:0/1547124774 learned_addr learned my addr 172.21.15.155:0/1547124774 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-23T18:17:46.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.295+0000 7fcd5e7fc700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd60102b90 0x7fcd60199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:46.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd5f7fe700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60103530 0x7fcd601a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:46.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd5effd700 1 -- 172.21.15.155:0/1547124774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60103530 msgr2=0x7fcd601a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:46.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd5effd700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60103530 0x7fcd601a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd5effd700 1 -- 172.21.15.155:0/1547124774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd60102b90 msgr2=0x7fcd60199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:46.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd5effd700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd60102b90 0x7fcd60199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd5effd700 1 -- 172.21.15.155:0/1547124774 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd601a0840 con 0x7fcd60101990 2024-04-23T18:17:46.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd5effd700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd60101990 0x7fcd60198be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd540047b0 tx=0x7fcd54005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:46.292 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd64943700 1 -- 172.21.15.155:0/1547124774 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5400fc70 con 0x7fcd60101990 2024-04-23T18:17:46.292 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd64943700 1 -- 172.21.15.155:0/1547124774 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd5400fe10 con 0x7fcd60101990 2024-04-23T18:17:46.292 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd65945700 1 -- 172.21.15.155:0/1547124774 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd601a0ad0 con 0x7fcd60101990 2024-04-23T18:17:46.292 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd64943700 1 -- 172.21.15.155:0/1547124774 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54017750 con 0x7fcd60101990 2024-04-23T18:17:46.293 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.296+0000 7fcd65945700 1 -- 172.21.15.155:0/1547124774 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd601a0ec0 con 0x7fcd60101990 2024-04-23T18:17:46.293 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.297+0000 7fcd64943700 1 -- 172.21.15.155:0/1547124774 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcd54009d10 con 0x7fcd60101990 2024-04-23T18:17:46.293 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.297+0000 7fcd65945700 1 -- 172.21.15.155:0/1547124774 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd60069730 con 0x7fcd60101990 2024-04-23T18:17:46.293 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.297+0000 7fcd64943700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd48039890 0x7fcd4803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:46.293 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.297+0000 7fcd64943700 1 -- 172.21.15.155:0/1547124774 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcd5404db50 con 0x7fcd60101990 2024-04-23T18:17:46.293 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.298+0000 7fcd5e7fc700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd48039890 0x7fcd4803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:46.293 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.298+0000 7fcd5e7fc700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd48039890 0x7fcd4803bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009c60 tx=0x7fcd50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:46.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.303+0000 7fcd64943700 1 -- 172.21.15.155:0/1547124774 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd54013070 con 0x7fcd60101990 2024-04-23T18:17:46.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:46.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:46.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:46.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:46.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:46.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:46.516 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.523+0000 7fcd65945700 1 -- 172.21.15.155:0/1547124774 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi044=y;smithi155=z", "target": ["mon-mgr", ""]}) -- 0x7fcd6006b900 con 0x7fcd48039890 2024-04-23T18:17:46.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.530+0000 7fcd64943700 1 -- 172.21.15.155:0/1547124774 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcd6006b900 con 0x7fcd48039890 2024-04-23T18:17:46.523 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mgr update... 2024-04-23T18:17:46.525 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 -- 172.21.15.155:0/1547124774 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd48039890 msgr2=0x7fcd4803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:46.525 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd48039890 0x7fcd4803bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009c60 tx=0x7fcd50009260 comp rx=0 tx=0).stop 2024-04-23T18:17:46.525 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 -- 172.21.15.155:0/1547124774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd60101990 msgr2=0x7fcd60198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:46.526 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd60101990 0x7fcd60198be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd540047b0 tx=0x7fcd54005e80 comp rx=0 tx=0).stop 2024-04-23T18:17:46.526 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 -- 172.21.15.155:0/1547124774 shutdown_connections 2024-04-23T18:17:46.526 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcd48039890 0x7fcd4803bd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.526 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60103530 0x7fcd601a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.526 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd60101990 0x7fcd60198be0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.526 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 --2- 172.21.15.155:0/1547124774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd60102b90 0x7fcd60199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.526 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 -- 172.21.15.155:0/1547124774 >> 172.21.15.155:0/1547124774 conn(0x7fcd600fcf30 msgr2=0x7fcd6006b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:46.526 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 -- 172.21.15.155:0/1547124774 shutdown_connections 2024-04-23T18:17:46.526 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:17:46.532+0000 7fcd65945700 1 -- 172.21.15.155:0/1547124774 wait complete. 2024-04-23T18:17:47.127 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:46 smithi155 ceph-mon[33468]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:47.127 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:46 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:47.127 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:46 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:47.127 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:46 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:47.127 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:46 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:47.128 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:46 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:47.128 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:46 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:17:47.128 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:46 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:17:47.128 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:46 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:17:47.128 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:46 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:47.157 DEBUG:teuthology.orchestra.run.smithi044:mgr.y> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.y.service 2024-04-23T18:17:47.159 DEBUG:teuthology.orchestra.run.smithi155:mgr.z> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.z.service 2024-04-23T18:17:47.161 INFO:tasks.cephadm:Deploying OSDs... 2024-04-23T18:17:47.161 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:17:47.162 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:17:47.181 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:17:47.181 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-04-23T18:17:47.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:46 smithi044 ceph-mon[32458]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:47.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:46 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:47.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:46 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:47.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:46 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:47.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:46 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:47.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:46 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:47.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:46 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:17:47.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:46 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:17:47.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:46 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:17:47.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:46 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:17:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:47.240 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-23T18:17:47.240 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:47.240 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:47.240 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-23 18:16:49.566824955 +0000 2024-04-23T18:17:47.240 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-23 18:10:57.169641458 +0000 2024-04-23T18:17:47.240 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-23 18:10:57.169641458 +0000 2024-04-23T18:17:47.240 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-23 18:10:57.169641458 +0000 2024-04-23T18:17:47.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:17:47.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:46 smithi019 ceph-mon[29215]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:47.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:46 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:47.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:46 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:47.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:46 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:47.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:46 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:47.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:46 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:47.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:46 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:17:47.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:46 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:17:47.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:46 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:17:47.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:46 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-23T18:17:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-23T18:17:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000234247 s, 2.2 MB/s 2024-04-23T18:17:47.291 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:17:47.350 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-04-23T18:17:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:17:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:47.409 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-23T18:17:47.409 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:47.409 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:47.409 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-23 18:16:49.595824657 +0000 2024-04-23T18:17:47.409 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-23 18:10:57.553637641 +0000 2024-04-23T18:17:47.409 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-23 18:10:57.553637641 +0000 2024-04-23T18:17:47.409 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-23 18:10:57.553637641 +0000 2024-04-23T18:17:47.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:17:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-23T18:17:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-23T18:17:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000283193 s, 1.8 MB/s 2024-04-23T18:17:47.478 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:17:47.537 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-04-23T18:17:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:17:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:47.596 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-23T18:17:47.596 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:47.596 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:47.596 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-23 18:16:49.626824339 +0000 2024-04-23T18:17:47.596 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-23 18:10:57.901634183 +0000 2024-04-23T18:17:47.596 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-23 18:10:57.901634183 +0000 2024-04-23T18:17:47.596 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-23 18:10:57.901634183 +0000 2024-04-23T18:17:47.597 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:17:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-23T18:17:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-23T18:17:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.00024329 s, 2.1 MB/s 2024-04-23T18:17:47.665 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:17:47.724 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-04-23T18:17:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:17:47.782 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:47.782 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-23T18:17:47.782 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:47.782 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:47.782 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-23 18:16:49.652824072 +0000 2024-04-23T18:17:47.782 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-23 18:10:58.269630525 +0000 2024-04-23T18:17:47.783 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-23 18:10:58.269630525 +0000 2024-04-23T18:17:47.783 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-23 18:10:58.269630525 +0000 2024-04-23T18:17:47.783 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:17:47.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-23T18:17:47.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-23T18:17:47.848 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.00022902 s, 2.2 MB/s 2024-04-23T18:17:47.849 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:17:47.907 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-23T18:17:47.907 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:17:47.927 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:17:47.927 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-04-23T18:17:47.986 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:17:47.986 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:47.986 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-23T18:17:47.986 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:47.986 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:47.986 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-23 18:17:32.113048144 +0000 2024-04-23T18:17:47.986 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-23 18:11:20.747326770 +0000 2024-04-23T18:17:47.986 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-23 18:11:20.747326770 +0000 2024-04-23T18:17:47.986 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-23 18:11:20.747326770 +0000 2024-04-23T18:17:47.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:17:48.052 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-23T18:17:48.052 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-23T18:17:48.053 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000228045 s, 2.2 MB/s 2024-04-23T18:17:48.054 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:17:48.113 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-04-23T18:17:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:17:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:48.171 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-23T18:17:48.171 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:48.171 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:48.171 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-23 18:17:32.114048128 +0000 2024-04-23T18:17:48.171 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-23 18:11:21.094321285 +0000 2024-04-23T18:17:48.171 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-23 18:11:21.094321285 +0000 2024-04-23T18:17:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-23 18:11:21.094321285 +0000 2024-04-23T18:17:48.172 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:17:48.235 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:47 smithi044 ceph-mon[32458]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi044=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:48.235 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:47 smithi044 ceph-mon[32458]: Saving service mgr spec with placement smithi019=x;smithi044=y;smithi155=z;count:3 2024-04-23T18:17:48.235 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:47 smithi044 ceph-mon[32458]: Deploying daemon mgr.z on smithi155 2024-04-23T18:17:48.238 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-23T18:17:48.238 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-23T18:17:48.238 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.00022559 s, 2.3 MB/s 2024-04-23T18:17:48.239 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:17:48.242 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:47 smithi155 ceph-mon[33468]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi044=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:48.242 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:47 smithi155 ceph-mon[33468]: Saving service mgr spec with placement smithi019=x;smithi044=y;smithi155=z;count:3 2024-04-23T18:17:48.242 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:47 smithi155 ceph-mon[33468]: Deploying daemon mgr.z on smithi155 2024-04-23T18:17:48.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:47 smithi019 ceph-mon[29215]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi044=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:48.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:47 smithi019 ceph-mon[29215]: Saving service mgr spec with placement smithi019=x;smithi044=y;smithi155=z;count:3 2024-04-23T18:17:48.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:47 smithi019 ceph-mon[29215]: Deploying daemon mgr.z on smithi155 2024-04-23T18:17:48.298 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-04-23T18:17:48.355 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:17:48.355 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:48.355 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-23T18:17:48.356 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:48.356 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:48.356 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-23 18:17:32.114048128 +0000 2024-04-23T18:17:48.356 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-23 18:11:21.445315737 +0000 2024-04-23T18:17:48.356 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-23 18:11:21.445315737 +0000 2024-04-23T18:17:48.356 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-23 18:11:21.445315737 +0000 2024-04-23T18:17:48.356 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:17:48.421 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-23T18:17:48.421 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-23T18:17:48.421 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000218295 s, 2.3 MB/s 2024-04-23T18:17:48.422 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:17:48.479 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-04-23T18:17:48.536 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:17:48.537 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:48.537 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-23T18:17:48.537 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:48.537 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:48.537 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-23 18:17:32.114048128 +0000 2024-04-23T18:17:48.537 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-23 18:11:21.808309999 +0000 2024-04-23T18:17:48.537 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-23 18:11:21.808309999 +0000 2024-04-23T18:17:48.537 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-23 18:11:21.808309999 +0000 2024-04-23T18:17:48.537 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:17:48.602 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-23T18:17:48.602 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-23T18:17:48.602 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000244913 s, 2.1 MB/s 2024-04-23T18:17:48.603 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:17:48.661 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-23T18:17:48.661 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:17:48.677 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:17:48.678 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-04-23T18:17:48.734 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:17:48.734 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:48.734 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-23T18:17:48.734 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:48.735 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:48.735 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-23 18:17:48.638013651 +0000 2024-04-23T18:17:48.735 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-23 18:11:13.193397827 +0000 2024-04-23T18:17:48.735 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-23 18:11:13.193397827 +0000 2024-04-23T18:17:48.735 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-23 18:11:13.193397827 +0000 2024-04-23T18:17:48.735 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:17:48.798 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:48 smithi155 systemd[1]: Starting Ceph mgr.z for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:17:48.801 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-23T18:17:48.801 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-23T18:17:48.801 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000210968 s, 2.4 MB/s 2024-04-23T18:17:48.803 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:17:48.859 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-04-23T18:17:48.913 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:17:48.914 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:48.914 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-23T18:17:48.914 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:48.914 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:48.914 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-23 18:17:48.639013630 +0000 2024-04-23T18:17:48.914 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-23 18:11:13.563390267 +0000 2024-04-23T18:17:48.914 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-23 18:11:13.563390267 +0000 2024-04-23T18:17:48.914 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-23 18:11:13.563390267 +0000 2024-04-23T18:17:48.914 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:17:48.976 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-23T18:17:48.977 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-23T18:17:48.977 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000205955 s, 2.5 MB/s 2024-04-23T18:17:48.978 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:17:49.035 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-04-23T18:17:49.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:48 smithi155 ceph-mon[33468]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:49.073 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:48 smithi155 podman[35167]: 2024-04-23 18:17:48.821171412 +0000 UTC m=+0.017714971 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:17:49.073 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:48 smithi155 podman[35167]: 2024-04-23 18:17:48.926581343 +0000 UTC m=+0.123124901 container create ae6fd8769900ddb6f7943ec8553c398e5a3c5f3ea8e5e4e13ac82cdc27d5d1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-23T18:17:49.087 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:17:49.087 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:49.087 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-23T18:17:49.087 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:49.087 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:49.087 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-23 18:17:48.639013630 +0000 2024-04-23T18:17:49.087 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-23 18:11:13.910383178 +0000 2024-04-23T18:17:49.087 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-23 18:11:13.910383178 +0000 2024-04-23T18:17:49.087 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-23 18:11:13.910383178 +0000 2024-04-23T18:17:49.087 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:17:49.149 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-23T18:17:49.149 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-23T18:17:49.150 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000227479 s, 2.3 MB/s 2024-04-23T18:17:49.151 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:17:49.207 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-04-23T18:17:49.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:48 smithi019 ceph-mon[29215]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:49.263 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:17:49.263 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:17:49.263 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-23T18:17:49.263 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:17:49.263 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:17:49.263 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-23 18:17:48.639013630 +0000 2024-04-23T18:17:49.263 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-23 18:11:14.245376332 +0000 2024-04-23T18:17:49.263 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-23 18:11:14.245376332 +0000 2024-04-23T18:17:49.264 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-23 18:11:14.245376332 +0000 2024-04-23T18:17:49.264 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:17:49.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:48 smithi044 ceph-mon[32458]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:49.326 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:49 smithi155 podman[35167]: 2024-04-23 18:17:49.179950277 +0000 UTC m=+0.376493850 container init ae6fd8769900ddb6f7943ec8553c398e5a3c5f3ea8e5e4e13ac82cdc27d5d1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:17:49.326 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:49 smithi155 podman[35167]: 2024-04-23 18:17:49.182869569 +0000 UTC m=+0.379413129 container start ae6fd8769900ddb6f7943ec8553c398e5a3c5f3ea8e5e4e13ac82cdc27d5d1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-23T18:17:49.326 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:49 smithi155 ceph-mgr[35294]: -- 172.21.15.155:0/1704603194 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5637693605a0 con 0x563769334c00 2024-04-23T18:17:49.327 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:49 smithi155 bash[35167]: ae6fd8769900ddb6f7943ec8553c398e5a3c5f3ea8e5e4e13ac82cdc27d5d1ca 2024-04-23T18:17:49.327 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:49 smithi155 systemd[1]: Started Ceph mgr.z for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:17:49.329 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-23T18:17:49.329 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-23T18:17:49.329 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000181666 s, 2.8 MB/s 2024-04-23T18:17:49.330 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:17:49.386 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-04-23T18:17:49.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:17:49.620 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:49.338+0000 7f3221d0a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:17:49.620 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:49.525+0000 7f3221d0a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:17:49.620 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:49.625+0000 7f3221d0a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:17:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:50.323 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:49.933+0000 7f3221d0a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:17:50.323 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:50 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:50.017+0000 7f3221d0a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:17:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:50 smithi044 ceph-mon[32458]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:17:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:17:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:17:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:50.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:50 smithi044 ceph-mon[32458]: Deploying daemon mgr.y on smithi044 2024-04-23T18:17:50.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:50 smithi019 ceph-mon[29215]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:50.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:50.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:50.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:50.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:17:50.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:17:50.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:17:50.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:50.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:50 smithi019 ceph-mon[29215]: Deploying daemon mgr.y on smithi044 2024-04-23T18:17:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:50 smithi155 ceph-mon[33468]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:17:50.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:17:50.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:17:50.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:50.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:50 smithi155 ceph-mon[33468]: Deploying daemon mgr.y on smithi044 2024-04-23T18:17:51.536 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:51 smithi044 systemd[1]: Starting Ceph mgr.y for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:17:51.536 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:51 smithi044 podman[33875]: 2024-04-23 18:17:51.54293268 +0000 UTC m=+0.112267939 container create 853cb65869c8bbb93346c39e6a81a6f80486e5d09a3356223446f51290ffb2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-23T18:17:51.543 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:51.216+0000 7f3221d0a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:17:51.543 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:51.381+0000 7f3221d0a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:17:51.543 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:51.549+0000 7f3221d0a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:17:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:17:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:17:51.558 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-23T18:17:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-23T18:17:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111535 s, 940 MB/s 2024-04-23T18:17:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-23T18:17:51.813 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:51 smithi044 podman[33875]: 2024-04-23 18:17:51.449038623 +0000 UTC m=+0.018373824 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:17:51.813 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:51 smithi044 podman[33875]: 2024-04-23 18:17:51.747314626 +0000 UTC m=+0.316649827 container init 853cb65869c8bbb93346c39e6a81a6f80486e5d09a3356223446f51290ffb2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-23T18:17:51.813 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:51 smithi044 podman[33875]: 2024-04-23 18:17:51.750084965 +0000 UTC m=+0.319420169 container start 853cb65869c8bbb93346c39e6a81a6f80486e5d09a3356223446f51290ffb2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-23T18:17:51.813 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:51 smithi044 ceph-mgr[33894]: -- 172.21.15.44:0/2884842922 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614484065a0 con 0x5614483dac00 2024-04-23T18:17:52.100 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:51 smithi044 bash[33875]: 853cb65869c8bbb93346c39e6a81a6f80486e5d09a3356223446f51290ffb2cf 2024-04-23T18:17:52.100 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:51 smithi044 systemd[1]: Started Ceph mgr.y for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:17:52.100 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:51 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:51.914+0000 7fe59d43f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:17:52.100 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:52.106+0000 7fe59d43f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:17:52.118 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:51.830+0000 7f3221d0a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:17:52.118 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:51.921+0000 7f3221d0a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:17:52.118 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:52.124+0000 7f3221d0a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:17:52.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-04-23T18:17:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:52.539 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:52.214+0000 7fe59d43f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:17:52.539 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:52.546+0000 7fe59d43f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:17:52.573 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:52.324+0000 7f3221d0a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:17:52.813 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:52.641+0000 7fe59d43f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:17:53.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:52 smithi019 ceph-mon[29215]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:53.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:53.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:52 smithi044 ceph-mon[32458]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:53.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:53.323 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:52.952+0000 7f3221d0a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:17:53.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:52 smithi155 ceph-mon[33468]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:53.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:53.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:53.682 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:53.385+0000 7f3221d0a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:17:53.682 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:53.485+0000 7f3221d0a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:17:53.682 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:53.687+0000 7f3221d0a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:17:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.118+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/277332763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a64100d50 msgr2=0x7f2a64101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.118+0000 7f2a6c6d5700 1 --2- 172.21.15.19:0/277332763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a64100d50 0x7f2a64101170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400b0a0 tx=0x7f2a5400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:17:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.119+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/277332763 shutdown_connections 2024-04-23T18:17:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.119+0000 7f2a6c6d5700 1 --2- 172.21.15.19:0/277332763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a641046d0 0x7f2a6410af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.119+0000 7f2a6c6d5700 1 --2- 172.21.15.19:0/277332763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a64100d50 0x7f2a64101170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.119+0000 7f2a6c6d5700 1 --2- 172.21.15.19:0/277332763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a64103d30 0x7f2a64104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.119+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/277332763 >> 172.21.15.19:0/277332763 conn(0x7f2a640fc6d0 msgr2=0x7f2a640feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:54.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.119+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/277332763 shutdown_connections 2024-04-23T18:17:54.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.119+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/277332763 wait complete. 2024-04-23T18:17:54.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.119+0000 7f2a6c6d5700 1 Processor -- start 2024-04-23T18:17:54.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6c6d5700 1 -- start start 2024-04-23T18:17:54.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6c6d5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a64100d50 0x7f2a6419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:54.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6c6d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a64103d30 0x7f2a6419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6c6d5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a641046d0 0x7f2a641a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6c6d5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6410e540 con 0x7f2a641046d0 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6c6d5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6410e3c0 con 0x7f2a64100d50 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6c6d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6410e6c0 con 0x7f2a64103d30 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6a471700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a64100d50 0x7f2a6419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6a471700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a64100d50 0x7f2a6419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:49274/0 (socket says 172.21.15.19:49274) 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6a471700 1 -- 172.21.15.19:0/2642786037 learned_addr learned my addr 172.21.15.19:0/2642786037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a69c70700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a64103d30 0x7f2a6419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6ac72700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a641046d0 0x7f2a641a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6a471700 1 -- 172.21.15.19:0/2642786037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a64103d30 msgr2=0x7f2a6419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6a471700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a64103d30 0x7f2a6419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6a471700 1 -- 172.21.15.19:0/2642786037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a641046d0 msgr2=0x7f2a641a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6a471700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a641046d0 0x7f2a641a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.120+0000 7f2a6a471700 1 -- 172.21.15.19:0/2642786037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a641a2a50 con 0x7f2a64100d50 2024-04-23T18:17:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.121+0000 7f2a6a471700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a64100d50 0x7f2a6419adf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400d010 tx=0x7f2a54004d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.121+0000 7f2a637fe700 1 -- 172.21.15.19:0/2642786037 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54021070 con 0x7f2a64100d50 2024-04-23T18:17:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.121+0000 7f2a637fe700 1 -- 172.21.15.19:0/2642786037 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a54007c40 con 0x7f2a64100d50 2024-04-23T18:17:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.121+0000 7f2a637fe700 1 -- 172.21.15.19:0/2642786037 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a540052a0 con 0x7f2a64100d50 2024-04-23T18:17:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.121+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/2642786037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a641a2ce0 con 0x7f2a64100d50 2024-04-23T18:17:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.121+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/2642786037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a641a3170 con 0x7f2a64100d50 2024-04-23T18:17:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.122+0000 7f2a637fe700 1 -- 172.21.15.19:0/2642786037 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2a54024070 con 0x7f2a64100d50 2024-04-23T18:17:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.122+0000 7f2a637fe700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2a4c039890 0x7f2a4c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.122+0000 7f2a637fe700 1 -- 172.21.15.19:0/2642786037 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2a5400f020 con 0x7f2a64100d50 2024-04-23T18:17:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.122+0000 7f2a69c70700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2a4c039890 0x7f2a4c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.123+0000 7f2a69c70700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2a4c039890 0x7f2a4c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a58005950 tx=0x7f2a5800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.123+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/2642786037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a641a3340 con 0x7f2a64100d50 2024-04-23T18:17:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.128+0000 7f2a637fe700 1 -- 172.21.15.19:0/2642786037 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a5402b020 con 0x7f2a64100d50 2024-04-23T18:17:54.277 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:53 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:53.932+0000 7fe59d43f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:17:54.277 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:54.107+0000 7fe59d43f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:17:54.277 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:54.283+0000 7fe59d43f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:17:54.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:54.358+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/2642786037 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2a6410ac70 con 0x7f2a4c039890 2024-04-23T18:17:54.536 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:54.209+0000 7f3221d0a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:54.300+0000 7f3221d0a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:17:54.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:17:54.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:54.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: Reconfiguring daemon mgr.x on smithi019 2024-04-23T18:17:54.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:17:54.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:17:54.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:17:54.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:54.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: Reconfiguring daemon mgr.x on smithi019 2024-04-23T18:17:54.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:17:54.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:17:54.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: Reconfiguring daemon mgr.x on smithi019 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:17:54.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:17:54.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:54.823 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:54.542+0000 7f3221d0a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:17:54.823 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:54.674+0000 7f3221d0a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:17:54.823 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:54.768+0000 7f3221d0a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:17:54.878 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:54.581+0000 7fe59d43f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:17:54.878 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:54.676+0000 7fe59d43f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:17:54.878 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:54.885+0000 7fe59d43f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:17:55.178 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:54.882+0000 7f3221d0a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:17:55.178 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:55.184+0000 7f3221d0a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:17:55.313 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:55 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:55.083+0000 7fe59d43f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:17:55.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:55 smithi019 ceph-mon[29215]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:55.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:55 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:55 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:55 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:55.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:55 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:55.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:55 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:55.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:55 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:55 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:55 smithi155 ceph-mon[33468]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:55.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:55 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:55 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:55 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:55.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:55 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:55.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:55 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:55.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:55 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:55 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.701 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:55 smithi044 ceph-mon[32458]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:55.701 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:55 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.702 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:55 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.702 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:55 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:55.702 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:55 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:55.702 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:55 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:55.702 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:55 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:55.702 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:55 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:17:56.063 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:55 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:55.708+0000 7fe59d43f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:17:56.073 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:55.786+0000 7f3221d0a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:17:56.073 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:17:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:17:56.000+0000 7f3221d0a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:17:56.395 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:56.141+0000 7fe59d43f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:17:56.395 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:56.240+0000 7fe59d43f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:17:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:56.408+0000 7f2a637fe700 1 -- 172.21.15.19:0/2642786037 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f2a54012cd0 con 0x7f2a64100d50 2024-04-23T18:17:56.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:56 smithi019 ceph-mon[29215]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:56.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:56 smithi019 ceph-mon[29215]: Standby manager daemon z started 2024-04-23T18:17:56.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:56 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:17:56.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:56 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:17:56.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:56 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:17:56.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:56 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:17:56.813 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:56.449+0000 7fe59d43f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:17:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:56 smithi044 ceph-mon[32458]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:56 smithi044 ceph-mon[32458]: Standby manager daemon z started 2024-04-23T18:17:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:56 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:17:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:56 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:17:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:56 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:17:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:56 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:17:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:56 smithi155 ceph-mon[33468]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:56 smithi155 ceph-mon[33468]: Standby manager daemon z started 2024-04-23T18:17:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:56 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:17:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:56 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:17:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:56 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:17:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:56 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.155:0/82846101' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:17:57.313 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:57.001+0000 7fe59d43f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:17:57.313 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:57.095+0000 7fe59d43f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:17:57.697 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:57 smithi044 ceph-mon[32458]: mgrmap e14: x(active, since 81s), standbys: z 2024-04-23T18:17:57.697 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:57 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:17:57.697 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:57.349+0000 7fe59d43f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:17:57.698 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:57.487+0000 7fe59d43f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:17:57.698 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:57.584+0000 7fe59d43f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:17:57.698 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:57.703+0000 7fe59d43f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:17:57.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:57 smithi019 ceph-mon[29215]: mgrmap e14: x(active, since 81s), standbys: z 2024-04-23T18:17:57.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:57 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:17:57.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:57 smithi155 ceph-mon[33468]: mgrmap e14: x(active, since 81s), standbys: z 2024-04-23T18:17:57.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:57 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:17:58.313 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:58 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:58.014+0000 7fe59d43f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:17:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:58 smithi019 ceph-mon[29215]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:58 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3071117532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78ea8328-c293-4ef2-af0d-81bfe0dccc4c"}]: dispatch 2024-04-23T18:17:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:58 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3071117532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78ea8328-c293-4ef2-af0d-81bfe0dccc4c"}]': finished 2024-04-23T18:17:58.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:58 smithi019 ceph-mon[29215]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:17:58.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:17:58.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:58 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2460591322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:17:58.813 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:58 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:58.639+0000 7fe59d43f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:17:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:58 smithi044 ceph-mon[32458]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:58 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3071117532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78ea8328-c293-4ef2-af0d-81bfe0dccc4c"}]: dispatch 2024-04-23T18:17:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:58 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3071117532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78ea8328-c293-4ef2-af0d-81bfe0dccc4c"}]': finished 2024-04-23T18:17:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:58 smithi044 ceph-mon[32458]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:17:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:17:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:58 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2460591322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:17:58.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:58 smithi155 ceph-mon[33468]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:58.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:58 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3071117532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78ea8328-c293-4ef2-af0d-81bfe0dccc4c"}]: dispatch 2024-04-23T18:17:58.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:58 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3071117532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78ea8328-c293-4ef2-af0d-81bfe0dccc4c"}]': finished 2024-04-23T18:17:58.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:58 smithi155 ceph-mon[33468]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:17:58.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:17:58.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:58 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2460591322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:17:59.313 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:17:58 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:17:58.860+0000 7fe59d43f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:17:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:17:59.423+0000 7f2a637fe700 1 -- 172.21.15.19:0/2642786037 <== mon.2 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f2a540057c0 con 0x7f2a64100d50 2024-04-23T18:17:59.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:59 smithi019 ceph-mon[29215]: Standby manager daemon y started 2024-04-23T18:17:59.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:59 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:17:59.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:59 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:17:59.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:59 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:17:59.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:17:59 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:17:59.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:59 smithi044 ceph-mon[32458]: Standby manager daemon y started 2024-04-23T18:17:59.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:59 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:17:59.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:59 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:17:59.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:59 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:17:59.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:17:59 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:17:59.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:59 smithi155 ceph-mon[33468]: Standby manager daemon y started 2024-04-23T18:17:59.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:59 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:17:59.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:59 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:17:59.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:59 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:17:59.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:17:59 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.44:0/3705239405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:18:00.676 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:00 smithi044 ceph-mon[32458]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:00.676 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:00 smithi044 ceph-mon[32458]: mgrmap e15: x(active, since 84s), standbys: z, y 2024-04-23T18:18:00.676 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:00 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:18:00.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:00 smithi019 ceph-mon[29215]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:00.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:00 smithi019 ceph-mon[29215]: mgrmap e15: x(active, since 84s), standbys: z, y 2024-04-23T18:18:00.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:00 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:18:00.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:00 smithi155 ceph-mon[33468]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:00.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:00 smithi155 ceph-mon[33468]: mgrmap e15: x(active, since 84s), standbys: z, y 2024-04-23T18:18:00.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:00 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:18:02.757 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:02 smithi019 ceph-mon[29215]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:02.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:02 smithi044 ceph-mon[32458]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:02.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:02 smithi155 ceph-mon[33468]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:04.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:04 smithi019 ceph-mon[29215]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:04.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:04 smithi044 ceph-mon[32458]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:04.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:04 smithi155 ceph-mon[33468]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:05.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:05 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:18:05.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:05 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:05.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:05 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:18:05.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:05 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:05.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:05 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:18:05.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:05 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:06.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:06 smithi019 ceph-mon[29215]: Deploying daemon osd.0 on smithi019 2024-04-23T18:18:06.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:06 smithi019 ceph-mon[29215]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:06.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:06 smithi044 ceph-mon[32458]: Deploying daemon osd.0 on smithi019 2024-04-23T18:18:06.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:06 smithi044 ceph-mon[32458]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:06.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:06 smithi155 ceph-mon[33468]: Deploying daemon osd.0 on smithi019 2024-04-23T18:18:06.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:06 smithi155 ceph-mon[33468]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:08.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:08 smithi019 ceph-mon[29215]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:08.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:08 smithi044 ceph-mon[32458]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:08.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:08 smithi155 ceph-mon[33468]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:10.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:10 smithi019 ceph-mon[29215]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:10.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:10 smithi044 ceph-mon[32458]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:10.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:10 smithi155 ceph-mon[33468]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:13.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:12 smithi019 ceph-mon[29215]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:12 smithi044 ceph-mon[32458]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:12 smithi155 ceph-mon[33468]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:14.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:14 smithi019 ceph-mon[29215]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:14.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:14 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:14.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:14 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:14.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:14 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:14.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:14 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:14.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:14 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:14.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.815+0000 7f2a637fe700 1 -- 172.21.15.19:0/2642786037 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2a6410ac70 con 0x7f2a4c039890 2024-04-23T18:18:14.810 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-04-23T18:18:14.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:14 smithi044 ceph-mon[32458]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:14.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:14 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:14.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:14 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:14.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:14 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:14.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:14 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:14.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:14 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.818+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/2642786037 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2a4c039890 msgr2=0x7f2a4c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.818+0000 7f2a6c6d5700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2a4c039890 0x7f2a4c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a58005950 tx=0x7f2a5800b410 comp rx=0 tx=0).stop 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.818+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/2642786037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a64100d50 msgr2=0x7f2a6419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.818+0000 7f2a6c6d5700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a64100d50 0x7f2a6419adf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400d010 tx=0x7f2a54004d70 comp rx=0 tx=0).stop 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.818+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/2642786037 shutdown_connections 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.818+0000 7f2a6c6d5700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2a4c039890 0x7f2a4c03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.818+0000 7f2a6c6d5700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a641046d0 0x7f2a641a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.818+0000 7f2a6c6d5700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a64100d50 0x7f2a6419adf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.818+0000 7f2a6c6d5700 1 --2- 172.21.15.19:0/2642786037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a64103d30 0x7f2a6419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.818+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/2642786037 >> 172.21.15.19:0/2642786037 conn(0x7f2a640fc6d0 msgr2=0x7f2a64109550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.819+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/2642786037 shutdown_connections 2024-04-23T18:18:14.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:18:14.819+0000 7f2a6c6d5700 1 -- 172.21.15.19:0/2642786037 wait complete. 2024-04-23T18:18:14.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:14 smithi155 ceph-mon[33468]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:14.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:14 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:14.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:14 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:14.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:14 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:14.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:14 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:14.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:14 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:15.542 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.0.service 2024-04-23T18:18:15.544 INFO:tasks.cephadm:Deploying osd.1 on smithi044 with /dev/vg_nvme/lv_4... 2024-04-23T18:18:15.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:18:15.725 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:15 smithi155 ceph-mon[33468]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:18:15.725 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:15.725 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:15.725 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:15.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:15 smithi019 ceph-mon[29215]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:18:15.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:15.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:15.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:15.800 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:18:15.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:15 smithi044 ceph-mon[32458]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:18:15.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:15.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:15.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:16.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:16 smithi019 ceph-mon[29215]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:16.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:16 smithi019 ceph-mon[29215]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:18:16.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:16 smithi019 ceph-mon[29215]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:18:16.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:16 smithi019 ceph-mon[29215]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:18:16.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:16 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:16.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:16 smithi044 ceph-mon[32458]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:16.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:16 smithi044 ceph-mon[32458]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:18:16.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:16 smithi044 ceph-mon[32458]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:18:16.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:16 smithi044 ceph-mon[32458]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:18:16.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:16 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:16.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:16 smithi155 ceph-mon[33468]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:16.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:16 smithi155 ceph-mon[33468]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:18:16.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:16 smithi155 ceph-mon[33468]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:18:16.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:16 smithi155 ceph-mon[33468]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:18:16.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:16 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:17.039 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:18:16 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0[43670]: 2024-04-23T18:18:16.786+0000 7f41d9cc6700 -1 osd.0 0 waiting for initial osdmap 2024-04-23T18:18:17.039 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:18:16 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0[43670]: 2024-04-23T18:18:16.790+0000 7f41d1ab1700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:18:17.575 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:18:17.575 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:18:17.575 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-23T18:18:17.575 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-23T18:18:17.576 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109469 s, 958 MB/s 2024-04-23T18:18:17.576 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-23T18:18:17.599 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:17 smithi044 ceph-mon[32458]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:18:17.599 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:17 smithi044 ceph-mon[32458]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:18:17.600 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:17.600 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:17.600 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:17 smithi044 ceph-mon[32458]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' 2024-04-23T18:18:17.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:17 smithi019 ceph-mon[29215]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:18:17.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:17 smithi019 ceph-mon[29215]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:18:17.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:17.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:17.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:17 smithi019 ceph-mon[29215]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' 2024-04-23T18:18:17.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:17 smithi155 ceph-mon[33468]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:18:17.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:17 smithi155 ceph-mon[33468]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:18:17.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:17.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:17.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:17 smithi155 ceph-mon[33468]: from='osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748]' entity='osd.0' 2024-04-23T18:18:18.077 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_4 2024-04-23T18:18:18.327 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:18:18.619 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:18 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:18:18.619 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:18 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:18:18.619 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:18 smithi044 ceph-mon[32458]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:18.619 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:18 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:18.619 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:18 smithi044 ceph-mon[32458]: osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] boot 2024-04-23T18:18:18.620 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:18 smithi044 ceph-mon[32458]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:18:18.620 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:18 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:18.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:18 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:18:18.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:18 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:18:18.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:18 smithi019 ceph-mon[29215]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:18.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:18.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:18 smithi019 ceph-mon[29215]: osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] boot 2024-04-23T18:18:18.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:18 smithi019 ceph-mon[29215]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:18:18.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:18.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:18 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:18:18.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:18 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:18:18.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:18 smithi155 ceph-mon[33468]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:18.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:18 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:18.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:18 smithi155 ceph-mon[33468]: osd.0 [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] boot 2024-04-23T18:18:18.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:18 smithi155 ceph-mon[33468]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:18:18.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:18 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 -- 172.21.15.44:0/1211014110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948104660 msgr2=0x7fb94810af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 --2- 172.21.15.44:0/1211014110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948104660 0x7fb94810af10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb94000b0d0 tx=0x7fb94000b3e0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 -- 172.21.15.44:0/1211014110 shutdown_connections 2024-04-23T18:18:19.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 --2- 172.21.15.44:0/1211014110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948104660 0x7fb94810af10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 --2- 172.21.15.44:0/1211014110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948102ac0 0x7fb948102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 --2- 172.21.15.44:0/1211014110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb948103cc0 0x7fb948104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 -- 172.21.15.44:0/1211014110 >> 172.21.15.44:0/1211014110 conn(0x7fb9480fe060 msgr2=0x7fb9481004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 -- 172.21.15.44:0/1211014110 shutdown_connections 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 -- 172.21.15.44:0/1211014110 wait complete. 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 Processor -- start 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.848+0000 7fb94ffc4700 1 -- start start 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94ffc4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb948102ac0 0x7fb948198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94ffc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948103cc0 0x7fb948199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94ffc4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948104660 0x7fb9481a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94ffc4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810e540 con 0x7fb948103cc0 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94ffc4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810e3c0 con 0x7fb948104660 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94ffc4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810e6c0 con 0x7fb948102ac0 2024-04-23T18:18:19.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94e561700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948104660 0x7fb9481a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94e561700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948104660 0x7fb9481a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43282/0 (socket says 172.21.15.44:43282) 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94e561700 1 -- 172.21.15.44:0/3178370625 learned_addr learned my addr 172.21.15.44:0/3178370625 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94d55f700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948103cc0 0x7fb948199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94dd60700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb948102ac0 0x7fb948198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94e561700 1 -- 172.21.15.44:0/3178370625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb948102ac0 msgr2=0x7fb948198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94e561700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb948102ac0 0x7fb948198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94e561700 1 -- 172.21.15.44:0/3178370625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948103cc0 msgr2=0x7fb948199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94e561700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948103cc0 0x7fb948199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94e561700 1 -- 172.21.15.44:0/3178370625 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9481a07b0 con 0x7fb948104660 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.849+0000 7fb94e561700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948104660 0x7fb9481a0110 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb940009040 tx=0x7fb940004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.850+0000 7fb93effd700 1 -- 172.21.15.44:0/3178370625 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94001f030 con 0x7fb948104660 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.850+0000 7fb93effd700 1 -- 172.21.15.44:0/3178370625 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb940003a00 con 0x7fb948104660 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.850+0000 7fb94ffc4700 1 -- 172.21.15.44:0/3178370625 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9481a0a40 con 0x7fb948104660 2024-04-23T18:18:19.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.850+0000 7fb93effd700 1 -- 172.21.15.44:0/3178370625 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940004670 con 0x7fb948104660 2024-04-23T18:18:19.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.850+0000 7fb94ffc4700 1 -- 172.21.15.44:0/3178370625 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9481a0ef0 con 0x7fb948104660 2024-04-23T18:18:19.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.851+0000 7fb94ffc4700 1 -- 172.21.15.44:0/3178370625 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb948069730 con 0x7fb948104660 2024-04-23T18:18:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.852+0000 7fb93effd700 1 -- 172.21.15.44:0/3178370625 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fb940003bb0 con 0x7fb948104660 2024-04-23T18:18:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.852+0000 7fb93effd700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb9340a3c30 0x7fb9340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.853+0000 7fb93effd700 1 -- 172.21.15.44:0/3178370625 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fb94000f020 con 0x7fb948104660 2024-04-23T18:18:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.853+0000 7fb94dd60700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb9340a3c30 0x7fb9340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:19.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.853+0000 7fb94dd60700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb9340a3c30 0x7fb9340a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb948103b20 tx=0x7fb94400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:19.856+0000 7fb93effd700 1 -- 172.21.15.44:0/3178370625 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb940029020 con 0x7fb948104660 2024-04-23T18:18:20.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:19 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi019 2024-04-23T18:18:20.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:19 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:19 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:19 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:18:20.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:19 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi019 to 17093M 2024-04-23T18:18:20.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:19 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:19 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:20.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:19 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:20.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:19 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:19 smithi019 ceph-mon[29215]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:18:20.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:19 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi019 2024-04-23T18:18:20.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:19 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:19 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:19 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:18:20.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:19 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi019 to 17093M 2024-04-23T18:18:20.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:19 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:19 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:20.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:19 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:20.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:19 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:19 smithi044 ceph-mon[32458]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:18:20.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:20.075+0000 7fb94ffc4700 1 -- 172.21.15.44:0/3178370625 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb948062e80 con 0x7fb9340a3c30 2024-04-23T18:18:20.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:19 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi019 2024-04-23T18:18:20.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:19 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:19 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:19 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:18:20.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:19 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi019 to 17093M 2024-04-23T18:18:20.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:19 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:19 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:20.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:19 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:20.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:19 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:20.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:19 smithi155 ceph-mon[33468]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:18:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:20 smithi044 ceph-mon[32458]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:20 smithi044 ceph-mon[32458]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:18:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:18:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:21.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:20 smithi155 ceph-mon[33468]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:21.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:20 smithi155 ceph-mon[33468]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:21.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:18:21.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:18:21.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:21.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:20 smithi019 ceph-mon[29215]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:21.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:20 smithi019 ceph-mon[29215]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:21.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:18:21.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:18:21.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:23.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:22 smithi019 ceph-mon[29215]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:23.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:22 smithi044 ceph-mon[32458]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:23.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:22 smithi155 ceph-mon[33468]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:24.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:23 smithi044 ceph-mon[32458]: from='client.? 172.21.15.44:0/692313187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9e3631-24c8-42d1-84c6-48cf0cda9283"}]: dispatch 2024-04-23T18:18:24.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:23 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9e3631-24c8-42d1-84c6-48cf0cda9283"}]: dispatch 2024-04-23T18:18:24.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:23 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e9e3631-24c8-42d1-84c6-48cf0cda9283"}]': finished 2024-04-23T18:18:24.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:23 smithi044 ceph-mon[32458]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:18:24.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:24.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:23 smithi044 ceph-mon[32458]: from='client.? 172.21.15.44:0/1972409011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:18:24.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:23 smithi155 ceph-mon[33468]: from='client.? 172.21.15.44:0/692313187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9e3631-24c8-42d1-84c6-48cf0cda9283"}]: dispatch 2024-04-23T18:18:24.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:23 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9e3631-24c8-42d1-84c6-48cf0cda9283"}]: dispatch 2024-04-23T18:18:24.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:23 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e9e3631-24c8-42d1-84c6-48cf0cda9283"}]': finished 2024-04-23T18:18:24.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:23 smithi155 ceph-mon[33468]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:18:24.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:24.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:23 smithi155 ceph-mon[33468]: from='client.? 172.21.15.44:0/1972409011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:18:24.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:23 smithi019 ceph-mon[29215]: from='client.? 172.21.15.44:0/692313187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9e3631-24c8-42d1-84c6-48cf0cda9283"}]: dispatch 2024-04-23T18:18:24.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:23 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9e3631-24c8-42d1-84c6-48cf0cda9283"}]: dispatch 2024-04-23T18:18:24.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:23 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e9e3631-24c8-42d1-84c6-48cf0cda9283"}]': finished 2024-04-23T18:18:24.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:23 smithi019 ceph-mon[29215]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:18:24.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:24.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:23 smithi019 ceph-mon[29215]: from='client.? 172.21.15.44:0/1972409011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:18:25.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:24 smithi044 ceph-mon[32458]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:25.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:24 smithi155 ceph-mon[33468]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:25.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:24 smithi019 ceph-mon[29215]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:27.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:26 smithi044 ceph-mon[32458]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:27.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:26 smithi155 ceph-mon[33468]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:27.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:26 smithi019 ceph-mon[29215]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:28.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:28 smithi044 ceph-mon[32458]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:29.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:28 smithi155 ceph-mon[33468]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:29.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:28 smithi019 ceph-mon[29215]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:30.896 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:30 smithi044 ceph-mon[32458]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:30.896 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:30 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:18:30.896 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:30 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:30.896 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:30 smithi044 ceph-mon[32458]: Deploying daemon osd.1 on smithi044 2024-04-23T18:18:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:30 smithi155 ceph-mon[33468]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:30 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:18:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:30 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:30 smithi155 ceph-mon[33468]: Deploying daemon osd.1 on smithi044 2024-04-23T18:18:31.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:30 smithi019 ceph-mon[29215]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:31.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:30 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:18:31.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:30 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:31.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:30 smithi019 ceph-mon[29215]: Deploying daemon osd.1 on smithi044 2024-04-23T18:18:33.116 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:32 smithi019 ceph-mon[29215]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:33.197 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:32 smithi044 ceph-mon[32458]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:33.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:32 smithi155 ceph-mon[33468]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:35.077 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:34 smithi044 ceph-mon[32458]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:35.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:34 smithi019 ceph-mon[29215]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:35.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:34 smithi155 ceph-mon[33468]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:36.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:36.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:36.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:36.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:36 smithi019 ceph-mon[29215]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:36.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:36.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:36.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:36.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:36 smithi044 ceph-mon[32458]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:36.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:36.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:36.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:36.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:36 smithi155 ceph-mon[33468]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:38.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:38 smithi044 ceph-mon[32458]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:38.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.644+0000 7fb93effd700 1 -- 172.21.15.44:0/3178370625 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb948062e80 con 0x7fb9340a3c30 2024-04-23T18:18:38.639 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 1 on host 'smithi044' 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 -- 172.21.15.44:0/3178370625 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb9340a3c30 msgr2=0x7fb9340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb9340a3c30 0x7fb9340a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb948103b20 tx=0x7fb94400b410 comp rx=0 tx=0).stop 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 -- 172.21.15.44:0/3178370625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948104660 msgr2=0x7fb9481a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948104660 0x7fb9481a0110 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb940009040 tx=0x7fb940004d50 comp rx=0 tx=0).stop 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 -- 172.21.15.44:0/3178370625 shutdown_connections 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb9340a3c30 0x7fb9340a60f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948104660 0x7fb9481a0110 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb948102ac0 0x7fb948198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 --2- 172.21.15.44:0/3178370625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948103cc0 0x7fb948199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 -- 172.21.15.44:0/3178370625 >> 172.21.15.44:0/3178370625 conn(0x7fb9480fe060 msgr2=0x7fb948107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 -- 172.21.15.44:0/3178370625 shutdown_connections 2024-04-23T18:18:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:18:38.646+0000 7fb94ffc4700 1 -- 172.21.15.44:0/3178370625 wait complete. 2024-04-23T18:18:38.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:38 smithi019 ceph-mon[29215]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:38 smithi155 ceph-mon[33468]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:39.218 DEBUG:teuthology.orchestra.run.smithi044:osd.1> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.1.service 2024-04-23T18:18:39.220 INFO:tasks.cephadm:Deploying osd.2 on smithi155 with /dev/vg_nvme/lv_4... 2024-04-23T18:18:39.220 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:18:39.468 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:18:39.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:39.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:39.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:39.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:39.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:39 smithi044 ceph-mon[32458]: from='osd.1 [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:18:39.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:39 smithi044 ceph-mon[32458]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:18:39.850 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:39 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:39.851 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:39 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:39.851 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:39 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:39.851 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:39 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:39.851 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:39 smithi155 ceph-mon[33468]: from='osd.1 [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:18:39.851 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:39 smithi155 ceph-mon[33468]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:18:40.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:39 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:40.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:39 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:40.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:39 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:40.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:39 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:40.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:39 smithi019 ceph-mon[29215]: from='osd.1 [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:18:40.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:39 smithi019 ceph-mon[29215]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:18:40.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:40 smithi044 ceph-mon[32458]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:40.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:40 smithi044 ceph-mon[32458]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:18:40.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:40 smithi044 ceph-mon[32458]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:18:40.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:40 smithi044 ceph-mon[32458]: from='osd.1 [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:18:40.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:40.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:40 smithi044 ceph-mon[32458]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:18:41.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:40 smithi019 ceph-mon[29215]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:41.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:40 smithi019 ceph-mon[29215]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:18:41.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:40 smithi019 ceph-mon[29215]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:18:41.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:40 smithi019 ceph-mon[29215]: from='osd.1 [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:18:41.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:41.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:40 smithi019 ceph-mon[29215]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:18:41.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:40 smithi155 ceph-mon[33468]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:41.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:40 smithi155 ceph-mon[33468]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:18:41.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:40 smithi155 ceph-mon[33468]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:18:41.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:40 smithi155 ceph-mon[33468]: from='osd.1 [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:18:41.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:41.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:40 smithi155 ceph-mon[33468]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:18:41.388 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:18:41 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1[37100]: 2024-04-23T18:18:41.143+0000 7f3abe62a700 -1 osd.1 0 waiting for initial osdmap 2024-04-23T18:18:41.389 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:18:41 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1[37100]: 2024-04-23T18:18:41.146+0000 7f3ab6415700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:18:41.516 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:18:41.517 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:18:41.517 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-04-23T18:18:41.517 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-04-23T18:18:41.517 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115946 s, 904 MB/s 2024-04-23T18:18:41.517 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-04-23T18:18:42.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:41 smithi019 ceph-mon[29215]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:18:42.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:41 smithi019 ceph-mon[29215]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:18:42.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:41 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:42.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:41 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:42.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:41 smithi019 ceph-mon[29215]: from='osd.1 ' entity='osd.1' 2024-04-23T18:18:42.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:41 smithi044 ceph-mon[32458]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:18:42.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:41 smithi044 ceph-mon[32458]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:18:42.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:41 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:42.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:41 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:42.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:41 smithi044 ceph-mon[32458]: from='osd.1 ' entity='osd.1' 2024-04-23T18:18:42.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:41 smithi155 ceph-mon[33468]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:18:42.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:41 smithi155 ceph-mon[33468]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:18:42.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:41 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:42.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:41 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:42.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:41 smithi155 ceph-mon[33468]: from='osd.1 ' entity='osd.1' 2024-04-23T18:18:42.140 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-04-23T18:18:42.391 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:18:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:18:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:18:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: osd.1 [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] boot 2024-04-23T18:18:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi044 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:42.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: osd.1 [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] boot 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi044 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:43.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:43.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:43.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:43.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:43.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:18:43.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: osd.1 [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] boot 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi044 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:43.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:43.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:43.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:18:43.958 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.964+0000 7f81cfd5f700 1 -- 172.21.15.155:0/4068140875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81c8104640 msgr2=0x7f81c810aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:43.958 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.964+0000 7f81cfd5f700 1 --2- 172.21.15.155:0/4068140875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81c8104640 0x7f81c810aef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81c000b0a0 tx=0x7f81c000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.964+0000 7f81cfd5f700 1 -- 172.21.15.155:0/4068140875 shutdown_connections 2024-04-23T18:18:43.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.964+0000 7f81cfd5f700 1 --2- 172.21.15.155:0/4068140875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81c8104640 0x7f81c810aef0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.964+0000 7f81cfd5f700 1 --2- 172.21.15.155:0/4068140875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c8102aa0 0x7f81c8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.964+0000 7f81cfd5f700 1 --2- 172.21.15.155:0/4068140875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81c8103ca0 0x7f81c8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.964+0000 7f81cfd5f700 1 -- 172.21.15.155:0/4068140875 >> 172.21.15.155:0/4068140875 conn(0x7f81c80fe040 msgr2=0x7f81c8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:43.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.964+0000 7f81cfd5f700 1 -- 172.21.15.155:0/4068140875 shutdown_connections 2024-04-23T18:18:43.959 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.964+0000 7f81cfd5f700 1 -- 172.21.15.155:0/4068140875 wait complete. 2024-04-23T18:18:43.959 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:43 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi044 to 17093M 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cfd5f700 1 Processor -- start 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cfd5f700 1 -- start start 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cfd5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81c8102aa0 0x7f81c8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cfd5f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c8103ca0 0x7f81c8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cfd5f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81c8104640 0x7f81c81a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cfd5f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81c810e5b0 con 0x7f81c8102aa0 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cfd5f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81c810e430 con 0x7f81c8103ca0 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cfd5f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f81c810e730 con 0x7f81c8104640 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81ce2fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81c8104640 0x7f81c81a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cdafb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81c8102aa0 0x7f81c8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81ce2fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81c8104640 0x7f81c81a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:42740/0 (socket says 172.21.15.155:42740) 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81ce2fc700 1 -- 172.21.15.155:0/1217060823 learned_addr learned my addr 172.21.15.155:0/1217060823 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cd2fa700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c8103ca0 0x7f81c8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81ce2fc700 1 -- 172.21.15.155:0/1217060823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c8103ca0 msgr2=0x7f81c8199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:43.961 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81ce2fc700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c8103ca0 0x7f81c8199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81ce2fc700 1 -- 172.21.15.155:0/1217060823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81c8102aa0 msgr2=0x7f81c8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81ce2fc700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81c8102aa0 0x7f81c8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81ce2fc700 1 -- 172.21.15.155:0/1217060823 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c81a07a0 con 0x7f81c8104640 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.965+0000 7f81cd2fa700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c8103ca0 0x7f81c8199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.966+0000 7f81ce2fc700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81c8104640 0x7f81c81a0100 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f81c0006070 tx=0x7f81c0004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.966+0000 7f81beffd700 1 -- 172.21.15.155:0/1217060823 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0021070 con 0x7f81c8104640 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.966+0000 7f81beffd700 1 -- 172.21.15.155:0/1217060823 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c0007d00 con 0x7f81c8104640 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.966+0000 7f81cfd5f700 1 -- 172.21.15.155:0/1217060823 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c81a0a30 con 0x7f81c8104640 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.966+0000 7f81beffd700 1 -- 172.21.15.155:0/1217060823 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c001d450 con 0x7f81c8104640 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.966+0000 7f81cfd5f700 1 -- 172.21.15.155:0/1217060823 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c81a0f60 con 0x7f81c8104640 2024-04-23T18:18:43.962 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.967+0000 7f81cfd5f700 1 -- 172.21.15.155:0/1217060823 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c8069730 con 0x7f81c8104640 2024-04-23T18:18:43.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.969+0000 7f81beffd700 1 -- 172.21.15.155:0/1217060823 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f81c0004000 con 0x7f81c8104640 2024-04-23T18:18:43.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.969+0000 7f81beffd700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3bb0 0x7f81b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:43.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.969+0000 7f81beffd700 1 -- 172.21.15.155:0/1217060823 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f81c000f020 con 0x7f81c8104640 2024-04-23T18:18:43.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.969+0000 7f81cdafb700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3bb0 0x7f81b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:43.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.970+0000 7f81cdafb700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3bb0 0x7f81b40a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81c8103b00 tx=0x7f81c4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:43.967 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:43.973+0000 7f81beffd700 1 -- 172.21.15.155:0/1217060823 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81c002a020 con 0x7f81c8104640 2024-04-23T18:18:44.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:43 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi044 to 17093M 2024-04-23T18:18:44.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:43 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi044 to 17093M 2024-04-23T18:18:44.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:18:44.176+0000 7f81cfd5f700 1 -- 172.21.15.155:0/1217060823 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f81c8062e80 con 0x7f81b40a3bb0 2024-04-23T18:18:44.974 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:44 smithi155 ceph-mon[33468]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:44.974 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:44 smithi155 ceph-mon[33468]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:18:44.974 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:18:44.974 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:18:44.974 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:45.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:44 smithi019 ceph-mon[29215]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:45.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:44 smithi019 ceph-mon[29215]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:18:45.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:18:45.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:18:45.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:45.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:44 smithi044 ceph-mon[32458]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:45.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:44 smithi044 ceph-mon[32458]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:18:45.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:18:45.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:18:45.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:46.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:45 smithi019 ceph-mon[29215]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:46.053 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:45 smithi155 ceph-mon[33468]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:46.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:45 smithi044 ceph-mon[32458]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:47.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:46 smithi019 ceph-mon[29215]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:47.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:46 smithi044 ceph-mon[32458]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:47.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:46 smithi155 ceph-mon[33468]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:48.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:47 smithi019 ceph-mon[29215]: from='client.? 172.21.15.155:0/1780918394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680b8f94-cb10-444d-a1a4-22c12f9cbe1f"}]: dispatch 2024-04-23T18:18:48.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:47 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680b8f94-cb10-444d-a1a4-22c12f9cbe1f"}]: dispatch 2024-04-23T18:18:48.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:47 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "680b8f94-cb10-444d-a1a4-22c12f9cbe1f"}]': finished 2024-04-23T18:18:48.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:47 smithi019 ceph-mon[29215]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:18:48.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:18:48.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:47 smithi019 ceph-mon[29215]: from='client.? 172.21.15.155:0/309215283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:18:48.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:47 smithi044 ceph-mon[32458]: from='client.? 172.21.15.155:0/1780918394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680b8f94-cb10-444d-a1a4-22c12f9cbe1f"}]: dispatch 2024-04-23T18:18:48.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:47 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680b8f94-cb10-444d-a1a4-22c12f9cbe1f"}]: dispatch 2024-04-23T18:18:48.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:47 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "680b8f94-cb10-444d-a1a4-22c12f9cbe1f"}]': finished 2024-04-23T18:18:48.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:47 smithi044 ceph-mon[32458]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:18:48.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:18:48.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:47 smithi044 ceph-mon[32458]: from='client.? 172.21.15.155:0/309215283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:18:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:47 smithi155 ceph-mon[33468]: from='client.? 172.21.15.155:0/1780918394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680b8f94-cb10-444d-a1a4-22c12f9cbe1f"}]: dispatch 2024-04-23T18:18:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:47 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680b8f94-cb10-444d-a1a4-22c12f9cbe1f"}]: dispatch 2024-04-23T18:18:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:47 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "680b8f94-cb10-444d-a1a4-22c12f9cbe1f"}]': finished 2024-04-23T18:18:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:47 smithi155 ceph-mon[33468]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:18:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:18:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:47 smithi155 ceph-mon[33468]: from='client.? 172.21.15.155:0/309215283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:18:49.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:48 smithi019 ceph-mon[29215]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:49.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:48 smithi044 ceph-mon[32458]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:49.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:48 smithi155 ceph-mon[33468]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:51.103 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:50 smithi155 ceph-mon[33468]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:51.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:50 smithi019 ceph-mon[29215]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:51.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:50 smithi044 ceph-mon[32458]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:53.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:52 smithi019 ceph-mon[29215]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:53.312 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:52 smithi155 ceph-mon[33468]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:53.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:52 smithi044 ceph-mon[32458]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:54 smithi155 ceph-mon[33468]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:18:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:54 smithi155 ceph-mon[33468]: Deploying daemon osd.2 on smithi155 2024-04-23T18:18:55.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:54 smithi019 ceph-mon[29215]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:55.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:18:55.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:55.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:54 smithi019 ceph-mon[29215]: Deploying daemon osd.2 on smithi155 2024-04-23T18:18:55.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:54 smithi044 ceph-mon[32458]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:55.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:18:55.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:55.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:54 smithi044 ceph-mon[32458]: Deploying daemon osd.2 on smithi155 2024-04-23T18:18:57.192 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:56 smithi155 ceph-mon[33468]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:57.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:56 smithi019 ceph-mon[29215]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:57.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:56 smithi044 ceph-mon[32458]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:59.029 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:18:58 smithi155 ceph-mon[33468]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:59.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:18:58 smithi019 ceph-mon[29215]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:18:59.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:18:58 smithi044 ceph-mon[32458]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:00.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:00 smithi044 ceph-mon[32458]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:00.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:00 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:00.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:00 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:00.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:00 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:00 smithi155 ceph-mon[33468]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:00 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:00 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:00 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:00.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:00 smithi019 ceph-mon[29215]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:00.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:00 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:00.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:00 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:00.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:00 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:02.917 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:02 smithi155 ceph-mon[33468]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:02.918 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:02.918 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:02.918 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:02.918 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:02.918 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:02.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:02.998+0000 7f81beffd700 1 -- 172.21.15.155:0/1217060823 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f81c8062e80 con 0x7f81b40a3bb0 2024-04-23T18:19:02.992 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 2 on host 'smithi155' 2024-04-23T18:19:02.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 -- 172.21.15.155:0/1217060823 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3bb0 msgr2=0x7f81b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:02.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3bb0 0x7f81b40a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81c8103b00 tx=0x7f81c4009190 comp rx=0 tx=0).stop 2024-04-23T18:19:02.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 -- 172.21.15.155:0/1217060823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81c8104640 msgr2=0x7f81c81a0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:02.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81c8104640 0x7f81c81a0100 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f81c0006070 tx=0x7f81c0004e40 comp rx=0 tx=0).stop 2024-04-23T18:19:02.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 -- 172.21.15.155:0/1217060823 shutdown_connections 2024-04-23T18:19:02.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3bb0 0x7f81b40a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:02.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81c8104640 0x7f81c81a0100 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:02.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81c8102aa0 0x7f81c8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:02.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 --2- 172.21.15.155:0/1217060823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81c8103ca0 0x7f81c8199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:02.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 -- 172.21.15.155:0/1217060823 >> 172.21.15.155:0/1217060823 conn(0x7f81c80fe040 msgr2=0x7f81c8106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:02.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 -- 172.21.15.155:0/1217060823 shutdown_connections 2024-04-23T18:19:02.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:19:03.001+0000 7f81cfd5f700 1 -- 172.21.15.155:0/1217060823 wait complete. 2024-04-23T18:19:03.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:02 smithi019 ceph-mon[29215]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:03.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:03.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:03.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:03.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:03.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:03.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:02 smithi044 ceph-mon[32458]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:03.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:03.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:03.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:03.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:03.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:03.605 DEBUG:teuthology.orchestra.run.smithi155:osd.2> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.2.service 2024-04-23T18:19:03.607 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_3... 2024-04-23T18:19:03.608 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:19:03.864 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:04.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:03 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:04.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:03 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:04.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:03 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:04.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:03 smithi019 ceph-mon[29215]: from='osd.2 [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:04.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:03 smithi019 ceph-mon[29215]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:03 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:03 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:03 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:03 smithi044 ceph-mon[32458]: from='osd.2 [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:03 smithi044 ceph-mon[32458]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:03 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:03 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:03 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:03 smithi155 ceph-mon[33468]: from='osd.2 [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:03 smithi155 ceph-mon[33468]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:04 smithi155 ceph-mon[33468]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:04 smithi155 ceph-mon[33468]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:19:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:04 smithi155 ceph-mon[33468]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:19:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:04 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:04 smithi155 ceph-mon[33468]: from='osd.2 [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:19:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:04 smithi155 ceph-mon[33468]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:19:05.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:04 smithi019 ceph-mon[29215]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:05.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:04 smithi019 ceph-mon[29215]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:19:05.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:04 smithi019 ceph-mon[29215]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:19:05.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:04 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:05.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:04 smithi019 ceph-mon[29215]: from='osd.2 [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:19:05.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:04 smithi019 ceph-mon[29215]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:19:05.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:04 smithi044 ceph-mon[32458]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:05.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:04 smithi044 ceph-mon[32458]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:19:05.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:04 smithi044 ceph-mon[32458]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:19:05.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:04 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:05.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:04 smithi044 ceph-mon[32458]: from='osd.2 [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:19:05.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:04 smithi044 ceph-mon[32458]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:19:05.823 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:19:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2[38659]: 2024-04-23T18:19:05.446+0000 7fa283351700 -1 osd.2 0 waiting for initial osdmap 2024-04-23T18:19:05.823 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:19:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2[38659]: 2024-04-23T18:19:05.451+0000 7fa27b13c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:19:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:19:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:19:06.018 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-23T18:19:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-23T18:19:06.018 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113476 s, 924 MB/s 2024-04-23T18:19:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-23T18:19:06.252 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:05 smithi155 ceph-mon[33468]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:19:06.252 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:05 smithi155 ceph-mon[33468]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:19:06.252 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:05 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:06.252 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:05 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:06.252 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:05 smithi155 ceph-mon[33468]: from='osd.2 ' entity='osd.2' 2024-04-23T18:19:06.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:05 smithi019 ceph-mon[29215]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:19:06.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:05 smithi019 ceph-mon[29215]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:19:06.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:05 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:06.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:05 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:06.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:05 smithi019 ceph-mon[29215]: from='osd.2 ' entity='osd.2' 2024-04-23T18:19:06.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:05 smithi044 ceph-mon[32458]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:19:06.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:05 smithi044 ceph-mon[32458]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:19:06.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:05 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:06.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:05 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:06.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:05 smithi044 ceph-mon[32458]: from='osd.2 ' entity='osd.2' 2024-04-23T18:19:06.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-04-23T18:19:06.993 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:07.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:19:07.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:19:07.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: osd.2 [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] boot 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:07.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: osd.2 [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] boot 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:07.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:07.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: osd.2 [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] boot 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:07.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:07.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:07.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:07.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:08.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:07 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi155 2024-04-23T18:19:08.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:07 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi155 to 17088M 2024-04-23T18:19:08.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:07 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:08.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:07 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi155 2024-04-23T18:19:08.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:07 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi155 to 17088M 2024-04-23T18:19:08.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:07 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:08.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:07 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi155 2024-04-23T18:19:08.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:07 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi155 to 17088M 2024-04-23T18:19:08.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:07 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.768+0000 7f194fef2700 1 -- 172.21.15.19:0/2213101748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1948102f80 msgr2=0x7f19481033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.768+0000 7f194fef2700 1 --2- 172.21.15.19:0/2213101748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1948102f80 0x7f19481033a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f19440099c0 tx=0x7f1944009cd0 comp rx=0 tx=0).stop 2024-04-23T18:19:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.768+0000 7f194fef2700 1 -- 172.21.15.19:0/2213101748 shutdown_connections 2024-04-23T18:19:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.768+0000 7f194fef2700 1 --2- 172.21.15.19:0/2213101748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1948102660 0x7f1948109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.768+0000 7f194fef2700 1 --2- 172.21.15.19:0/2213101748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1948102f80 0x7f19481033a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.768+0000 7f194fef2700 1 --2- 172.21.15.19:0/2213101748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1948101cc0 0x7f1948102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.768+0000 7f194fef2700 1 -- 172.21.15.19:0/2213101748 >> 172.21.15.19:0/2213101748 conn(0x7f19480fc710 msgr2=0x7f19480feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.768+0000 7f194fef2700 1 -- 172.21.15.19:0/2213101748 shutdown_connections 2024-04-23T18:19:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.768+0000 7f194fef2700 1 -- 172.21.15.19:0/2213101748 wait complete. 2024-04-23T18:19:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.769+0000 7f194fef2700 1 Processor -- start 2024-04-23T18:19:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.769+0000 7f194fef2700 1 -- start start 2024-04-23T18:19:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.769+0000 7f194fef2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1948101cc0 0x7f1948101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.769+0000 7f194fef2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1948102660 0x7f194819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.769+0000 7f194fef2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1948102f80 0x7f194819ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.769+0000 7f194fef2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f194810d480 con 0x7f1948102f80 2024-04-23T18:19:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.769+0000 7f194fef2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f194810d300 con 0x7f1948102660 2024-04-23T18:19:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.769+0000 7f194fef2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f194810d600 con 0x7f1948101cc0 2024-04-23T18:19:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.769+0000 7f194e48f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1948102f80 0x7f194819ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194e48f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1948102f80 0x7f194819ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58290/0 (socket says 172.21.15.19:58290) 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194e48f700 1 -- 172.21.15.19:0/3799879383 learned_addr learned my addr 172.21.15.19:0/3799879383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194dc8e700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1948101cc0 0x7f1948101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194d48d700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1948102660 0x7f194819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194e48f700 1 -- 172.21.15.19:0/3799879383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1948101cc0 msgr2=0x7f1948101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194e48f700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1948101cc0 0x7f1948101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194e48f700 1 -- 172.21.15.19:0/3799879383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1948102660 msgr2=0x7f194819d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194e48f700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1948102660 0x7f194819d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194e48f700 1 -- 172.21.15.19:0/3799879383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19481a0800 con 0x7f1948102f80 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194d48d700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1948102660 0x7f194819d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194e48f700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1948102f80 0x7f194819ff30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f194000ada0 tx=0x7f194000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f193effd700 1 -- 172.21.15.19:0/3799879383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940017020 con 0x7f1948102f80 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f193effd700 1 -- 172.21.15.19:0/3799879383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f194000ce30 con 0x7f1948102f80 2024-04-23T18:19:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194fef2700 1 -- 172.21.15.19:0/3799879383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19481a0af0 con 0x7f1948102f80 2024-04-23T18:19:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f193effd700 1 -- 172.21.15.19:0/3799879383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940005350 con 0x7f1948102f80 2024-04-23T18:19:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.770+0000 7f194fef2700 1 -- 172.21.15.19:0/3799879383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19481a1080 con 0x7f1948102f80 2024-04-23T18:19:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.771+0000 7f194fef2700 1 -- 172.21.15.19:0/3799879383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1948069730 con 0x7f1948102f80 2024-04-23T18:19:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.773+0000 7f193effd700 1 -- 172.21.15.19:0/3799879383 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f194000c980 con 0x7f1948102f80 2024-04-23T18:19:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.773+0000 7f193effd700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f19340a3c30 0x7f19340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:08.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.773+0000 7f194dc8e700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f19340a3c30 0x7f19340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:08.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.773+0000 7f193effd700 1 -- 172.21.15.19:0/3799879383 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f19400ceb10 con 0x7f1948102f80 2024-04-23T18:19:08.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.774+0000 7f194dc8e700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f19340a3c30 0x7f19340a60f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f19440038c0 tx=0x7f194400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:08.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:08.777+0000 7f193effd700 1 -- 172.21.15.19:0/3799879383 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1940021020 con 0x7f1948102f80 2024-04-23T18:19:08.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:09.000+0000 7f194fef2700 1 -- 172.21.15.19:0/3799879383 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1948062e80 con 0x7f19340a3c30 2024-04-23T18:19:09.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:08 smithi019 ceph-mon[29215]: pgmap v84: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:09.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:08 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:09.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:08 smithi019 ceph-mon[29215]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:19:09.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:08 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:09.234 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:08 smithi155 ceph-mon[33468]: pgmap v84: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:09.234 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:08 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:09.234 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:08 smithi155 ceph-mon[33468]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:19:09.234 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:08 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:09.234 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42276]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:09.235 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42276]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.235 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42276]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.235 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42276]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:08 smithi044 ceph-mon[32458]: pgmap v84: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:08 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:08 smithi044 ceph-mon[32458]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:19:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:08 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:09.313 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41071]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:09.314 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41071]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.314 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41071]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.314 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41071]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.314 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41074]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:09.314 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41074]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.314 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41074]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.314 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41074]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.510 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48140]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:09.510 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48140]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.510 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48140]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.510 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48140]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.510 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48143]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:09.511 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48143]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.511 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48143]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.511 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48143]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48146]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:09.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48146]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48146]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48146]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48149]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:09.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48149]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48149]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 sudo[48149]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42282]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42282]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42282]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42282]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42285]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42285]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42285]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42285]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.574 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42279]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:09.574 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42279]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.574 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42279]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.574 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:19:09 smithi155 sudo[42279]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41077]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:09.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41077]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41077]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41077]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:09.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:09.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:09.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41080]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:09.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 sudo[41080]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:10.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:10.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:19:10.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:10.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:10.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:10.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:10.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:10.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:10.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:09 smithi019 ceph-mon[29215]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:10.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:10.289+0000 7f193effd700 1 -- 172.21.15.19:0/3799879383 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f19400989b0 con 0x7f1948102f80 2024-04-23T18:19:10.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:10.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:19:10.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:10.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:10.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:10.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:10.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:10.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:09 smithi044 ceph-mon[32458]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:10.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:10.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:19:10.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:10.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:10.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:10.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:10.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:10.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:10.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:10.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:09 smithi155 ceph-mon[33468]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:11.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:11 smithi019 ceph-mon[29215]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:11.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:11 smithi019 ceph-mon[29215]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:19:11.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:11 smithi019 ceph-mon[29215]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:19:11.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:11 smithi044 ceph-mon[32458]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:11.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:11 smithi044 ceph-mon[32458]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:19:11.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:11 smithi044 ceph-mon[32458]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:19:11.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:11 smithi155 ceph-mon[33468]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:11.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:11 smithi155 ceph-mon[33468]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:19:11.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:11 smithi155 ceph-mon[33468]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:19:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:13 smithi019 ceph-mon[29215]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:13 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2744994886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e55972-9f96-4714-b681-fa4af600b35b"}]: dispatch 2024-04-23T18:19:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:13 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2744994886' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31e55972-9f96-4714-b681-fa4af600b35b"}]': finished 2024-04-23T18:19:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:13 smithi019 ceph-mon[29215]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:19:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:13 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:13 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1944875465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:13 smithi044 ceph-mon[32458]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:13 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2744994886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e55972-9f96-4714-b681-fa4af600b35b"}]: dispatch 2024-04-23T18:19:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:13 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2744994886' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31e55972-9f96-4714-b681-fa4af600b35b"}]': finished 2024-04-23T18:19:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:13 smithi044 ceph-mon[32458]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:19:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:13 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:13 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1944875465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:13 smithi155 ceph-mon[33468]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:13 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2744994886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e55972-9f96-4714-b681-fa4af600b35b"}]: dispatch 2024-04-23T18:19:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:13 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2744994886' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31e55972-9f96-4714-b681-fa4af600b35b"}]': finished 2024-04-23T18:19:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:13 smithi155 ceph-mon[33468]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:19:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:13 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:13 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1944875465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:15.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:15 smithi019 ceph-mon[29215]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:15.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:15 smithi044 ceph-mon[32458]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:15 smithi155 ceph-mon[33468]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:17.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:17 smithi019 ceph-mon[29215]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:17.307 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:17 smithi155 ceph-mon[33468]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:17.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:17 smithi044 ceph-mon[32458]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:19.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:19 smithi019 ceph-mon[29215]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:19.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:19 smithi044 ceph-mon[32458]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:19 smithi155 ceph-mon[33468]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:20.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:19:20.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:20.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:19:20.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:20.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:19:20.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:21.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:21 smithi019 ceph-mon[29215]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:21.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:21 smithi019 ceph-mon[29215]: Deploying daemon osd.3 on smithi019 2024-04-23T18:19:21.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:21 smithi044 ceph-mon[32458]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:21.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:21 smithi044 ceph-mon[32458]: Deploying daemon osd.3 on smithi019 2024-04-23T18:19:21.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:21 smithi155 ceph-mon[33468]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:21.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:21 smithi155 ceph-mon[33468]: Deploying daemon osd.3 on smithi019 2024-04-23T18:19:23.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:23 smithi019 ceph-mon[29215]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:23.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:23 smithi044 ceph-mon[32458]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:23.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:23 smithi155 ceph-mon[33468]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:25 smithi019 ceph-mon[29215]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:25.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:25 smithi044 ceph-mon[32458]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:25.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:25 smithi155 ceph-mon[33468]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:26.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:26.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:26.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:26.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:26 smithi019 ceph-mon[29215]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:26.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:26.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:26.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:26.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:26 smithi044 ceph-mon[32458]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:26.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:26.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:26.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:26.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:26 smithi155 ceph-mon[33468]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:28.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:28 smithi019 ceph-mon[29215]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:28.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:28.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:28.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:28.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:28.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:28 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:28.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:28 smithi044 ceph-mon[32458]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:28.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:28 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:28.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:28 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:28.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:28 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:28.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:28 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:28.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:28 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:28.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:28 smithi155 ceph-mon[33468]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:28.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:28.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:28.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:28.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:28.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:28 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:29.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.166+0000 7f193effd700 1 -- 172.21.15.19:0/3799879383 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1948062e80 con 0x7f19340a3c30 2024-04-23T18:19:29.160 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-04-23T18:19:29.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 -- 172.21.15.19:0/3799879383 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f19340a3c30 msgr2=0x7f19340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f19340a3c30 0x7f19340a60f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f19440038c0 tx=0x7f194400b540 comp rx=0 tx=0).stop 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 -- 172.21.15.19:0/3799879383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1948102f80 msgr2=0x7f194819ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1948102f80 0x7f194819ff30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f194000ada0 tx=0x7f194000c330 comp rx=0 tx=0).stop 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 -- 172.21.15.19:0/3799879383 shutdown_connections 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f19340a3c30 0x7f19340a60f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1948102f80 0x7f194819ff30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1948101cc0 0x7f1948101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 --2- 172.21.15.19:0/3799879383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1948102660 0x7f194819d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 -- 172.21.15.19:0/3799879383 >> 172.21.15.19:0/3799879383 conn(0x7f19480fc710 msgr2=0x7f1948105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 -- 172.21.15.19:0/3799879383 shutdown_connections 2024-04-23T18:19:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:19:29.169+0000 7f194fef2700 1 -- 172.21.15.19:0/3799879383 wait complete. 2024-04-23T18:19:29.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:29 smithi019 ceph-mon[29215]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:19:29.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:29 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:29.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:29 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:29.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:29 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:29.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:29 smithi044 ceph-mon[32458]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:19:29.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:29 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:29.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:29 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:29.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:29 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:29.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:29 smithi155 ceph-mon[33468]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:19:29.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:29 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:29.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:29 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:29.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:29 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:29.852 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.3.service 2024-04-23T18:19:29.854 INFO:tasks.cephadm:Deploying osd.4 on smithi044 with /dev/vg_nvme/lv_3... 2024-04-23T18:19:29.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:19:30.109 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:19:30.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:30 smithi019 ceph-mon[29215]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:30.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:30 smithi019 ceph-mon[29215]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:19:30.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:30 smithi019 ceph-mon[29215]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:19:30.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:30 smithi019 ceph-mon[29215]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:19:30.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:30 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:30.486 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:30 smithi044 ceph-mon[32458]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:30.486 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:30 smithi044 ceph-mon[32458]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:19:30.486 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:30 smithi044 ceph-mon[32458]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:19:30.486 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:30 smithi044 ceph-mon[32458]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:19:30.486 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:30 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:30.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:30 smithi155 ceph-mon[33468]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:30.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:30 smithi155 ceph-mon[33468]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:19:30.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:30 smithi155 ceph-mon[33468]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:19:30.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:30 smithi155 ceph-mon[33468]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:19:30.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:30 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:31.169 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:19:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3[50358]: 2024-04-23T18:19:30.785+0000 7f3fe17f6700 -1 osd.3 0 waiting for initial osdmap 2024-04-23T18:19:31.169 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:19:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3[50358]: 2024-04-23T18:19:30.792+0000 7f3fd95e1700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:19:31.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:31 smithi019 ceph-mon[29215]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:19:31.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:31 smithi019 ceph-mon[29215]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:19:31.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:31 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:31.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:31 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:31.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:31 smithi019 ceph-mon[29215]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' 2024-04-23T18:19:31.515 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:31 smithi044 ceph-mon[32458]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:19:31.515 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:31 smithi044 ceph-mon[32458]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:19:31.515 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:31 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:31.516 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:31 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:31.516 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:31 smithi044 ceph-mon[32458]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' 2024-04-23T18:19:31.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:31 smithi155 ceph-mon[33468]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:19:31.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:31 smithi155 ceph-mon[33468]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:19:31.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:31 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:31.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:31 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:31.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:31 smithi155 ceph-mon[33468]: from='osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737]' entity='osd.3' 2024-04-23T18:19:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:19:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:19:32.093 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-23T18:19:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-23T18:19:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.010883 s, 964 MB/s 2024-04-23T18:19:32.093 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 2024-04-23T18:19:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-23T18:19:32.685 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:32 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:19:32.685 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:32 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:19:32.685 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:32 smithi044 ceph-mon[32458]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:32.685 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:32 smithi044 ceph-mon[32458]: osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] boot 2024-04-23T18:19:32.685 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:32 smithi044 ceph-mon[32458]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:19:32.685 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:32.685 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:32 smithi044 ceph-mon[32458]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:19:32.713 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_3 2024-04-23T18:19:32.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:32 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:19:32.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:32 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:19:32.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:32 smithi019 ceph-mon[29215]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:32.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:32 smithi019 ceph-mon[29215]: osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] boot 2024-04-23T18:19:32.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:32 smithi019 ceph-mon[29215]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:19:32.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:32.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:32 smithi019 ceph-mon[29215]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:19:32.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:32 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:19:32.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:32 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:19:32.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:32 smithi155 ceph-mon[33468]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:32.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:32 smithi155 ceph-mon[33468]: osd.3 [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] boot 2024-04-23T18:19:32.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:32 smithi155 ceph-mon[33468]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:19:32.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:32.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:32 smithi155 ceph-mon[33468]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:19:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:19:34.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi019 2024-04-23T18:19:34.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:34.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:34.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi019 to 8546M 2024-04-23T18:19:34.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:34.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:34.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:34 smithi044 ceph-mon[32458]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:19:34.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi019 2024-04-23T18:19:34.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:34.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:34.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi019 to 8546M 2024-04-23T18:19:34.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:34.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:34.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:34 smithi019 ceph-mon[29215]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:19:34.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi019 2024-04-23T18:19:34.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:34.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:34.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi019 to 8546M 2024-04-23T18:19:34.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:34.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:34.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:34.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:34 smithi155 ceph-mon[33468]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:19:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.662+0000 7f1175da1700 1 -- 172.21.15.44:0/4049075282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1170102ad0 msgr2=0x7f1170102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.662+0000 7f1175da1700 1 --2- 172.21.15.44:0/4049075282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1170102ad0 0x7f1170102f30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f115c0099c0 tx=0x7f115c009cd0 comp rx=0 tx=0).stop 2024-04-23T18:19:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.662+0000 7f1175da1700 1 -- 172.21.15.44:0/4049075282 shutdown_connections 2024-04-23T18:19:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.662+0000 7f1175da1700 1 --2- 172.21.15.44:0/4049075282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1170103470 0x7f1170109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.662+0000 7f1175da1700 1 --2- 172.21.15.44:0/4049075282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11701018d0 0x7f1170101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.662+0000 7f1175da1700 1 --2- 172.21.15.44:0/4049075282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1170102ad0 0x7f1170102f30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.662+0000 7f1175da1700 1 -- 172.21.15.44:0/4049075282 >> 172.21.15.44:0/4049075282 conn(0x7f11700fce50 msgr2=0x7f11700ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.662+0000 7f1175da1700 1 -- 172.21.15.44:0/4049075282 shutdown_connections 2024-04-23T18:19:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.662+0000 7f1175da1700 1 -- 172.21.15.44:0/4049075282 wait complete. 2024-04-23T18:19:34.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1175da1700 1 Processor -- start 2024-04-23T18:19:34.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1175da1700 1 -- start start 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1175da1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11701018d0 0x7f117006b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1175da1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1170102ad0 0x7f117019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1175da1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1170103470 0x7f117019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1175da1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f117010d340 con 0x7f1170103470 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1175da1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f117010d1c0 con 0x7f11701018d0 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1175da1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f117010d4c0 con 0x7f1170102ad0 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1174d9f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11701018d0 0x7f117006b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1174d9f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11701018d0 0x7f117006b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39006/0 (socket says 172.21.15.44:39006) 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.663+0000 7f1174d9f700 1 -- 172.21.15.44:0/4252642879 learned_addr learned my addr 172.21.15.44:0/4252642879 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f116ffff700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1170102ad0 0x7f117019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f11755a0700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1170103470 0x7f117019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f1174d9f700 1 -- 172.21.15.44:0/4252642879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1170102ad0 msgr2=0x7f117019d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f1174d9f700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1170102ad0 0x7f117019d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f1174d9f700 1 -- 172.21.15.44:0/4252642879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1170103470 msgr2=0x7f117019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f1174d9f700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1170103470 0x7f117019fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:34.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f1174d9f700 1 -- 172.21.15.44:0/4252642879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11701a06f0 con 0x7f11701018d0 2024-04-23T18:19:34.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f116ffff700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1170102ad0 0x7f117019d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:34.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f1174d9f700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11701018d0 0x7f117006b2a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f116400e970 tx=0x7f116400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:34.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f116dffb700 1 -- 172.21.15.44:0/4252642879 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116400cbd0 con 0x7f11701018d0 2024-04-23T18:19:34.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f116dffb700 1 -- 172.21.15.44:0/4252642879 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11640044d0 con 0x7f11701018d0 2024-04-23T18:19:34.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f1175da1700 1 -- 172.21.15.44:0/4252642879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11701a09e0 con 0x7f11701018d0 2024-04-23T18:19:34.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f116dffb700 1 -- 172.21.15.44:0/4252642879 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164003e40 con 0x7f11701018d0 2024-04-23T18:19:34.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.664+0000 7f1175da1700 1 -- 172.21.15.44:0/4252642879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11701a0f20 con 0x7f11701018d0 2024-04-23T18:19:34.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.665+0000 7f1175da1700 1 -- 172.21.15.44:0/4252642879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1170069730 con 0x7f11701018d0 2024-04-23T18:19:34.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.666+0000 7f116dffb700 1 -- 172.21.15.44:0/4252642879 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1164003fe0 con 0x7f11701018d0 2024-04-23T18:19:34.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.667+0000 7f116dffb700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f11600a3c40 0x7f11600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:34.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.667+0000 7f116dffb700 1 -- 172.21.15.44:0/4252642879 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f11640cf5c0 con 0x7f11701018d0 2024-04-23T18:19:34.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.667+0000 7f116ffff700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f11600a3c40 0x7f11600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:34.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.668+0000 7f116ffff700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f11600a3c40 0x7f11600a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f115c006010 tx=0x7f115c005c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:34.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.672+0000 7f116dffb700 1 -- 172.21.15.44:0/4252642879 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1164016330 con 0x7f11701018d0 2024-04-23T18:19:34.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:34.890+0000 7f1175da1700 1 -- 172.21.15.44:0/4252642879 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1170062e80 con 0x7f11600a3c40 2024-04-23T18:19:35.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:35 smithi019 ceph-mon[29215]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:35.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:35 smithi019 ceph-mon[29215]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:19:35.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:35.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:35.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:35.512 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:35 smithi044 ceph-mon[32458]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:35.512 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:35 smithi044 ceph-mon[32458]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:19:35.512 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:35.513 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:35.513 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:35.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:35 smithi155 ceph-mon[33468]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:35.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:35 smithi155 ceph-mon[33468]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:19:35.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:35.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:35.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:36.451 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:36 smithi044 ceph-mon[32458]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:36.451 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:36 smithi044 ceph-mon[32458]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:19:36.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:36 smithi019 ceph-mon[29215]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:36.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:36 smithi019 ceph-mon[29215]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:19:36.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:36 smithi155 ceph-mon[33468]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:36.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:36 smithi155 ceph-mon[33468]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:19:37.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:37 smithi019 ceph-mon[29215]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:37.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:37 smithi044 ceph-mon[32458]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:37.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:37 smithi155 ceph-mon[33468]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:38.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:38 smithi044 ceph-mon[32458]: from='client.? 172.21.15.44:0/3402895233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a48cc342-4cea-4410-8b19-0f049a183667"}]: dispatch 2024-04-23T18:19:38.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:38 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a48cc342-4cea-4410-8b19-0f049a183667"}]: dispatch 2024-04-23T18:19:38.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:38 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a48cc342-4cea-4410-8b19-0f049a183667"}]': finished 2024-04-23T18:19:38.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:38 smithi044 ceph-mon[32458]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:19:38.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:38.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:38 smithi019 ceph-mon[29215]: from='client.? 172.21.15.44:0/3402895233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a48cc342-4cea-4410-8b19-0f049a183667"}]: dispatch 2024-04-23T18:19:38.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:38 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a48cc342-4cea-4410-8b19-0f049a183667"}]: dispatch 2024-04-23T18:19:38.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:38 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a48cc342-4cea-4410-8b19-0f049a183667"}]': finished 2024-04-23T18:19:38.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:38 smithi019 ceph-mon[29215]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:19:38.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:38.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:38 smithi155 ceph-mon[33468]: from='client.? 172.21.15.44:0/3402895233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a48cc342-4cea-4410-8b19-0f049a183667"}]: dispatch 2024-04-23T18:19:38.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:38 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a48cc342-4cea-4410-8b19-0f049a183667"}]: dispatch 2024-04-23T18:19:38.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:38 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a48cc342-4cea-4410-8b19-0f049a183667"}]': finished 2024-04-23T18:19:38.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:38 smithi155 ceph-mon[33468]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:19:38.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:39.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:39 smithi019 ceph-mon[29215]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:39.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:39 smithi019 ceph-mon[29215]: from='client.? 172.21.15.44:0/4132944311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:39.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:39 smithi044 ceph-mon[32458]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:39.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:39 smithi044 ceph-mon[32458]: from='client.? 172.21.15.44:0/4132944311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:39.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:39 smithi155 ceph-mon[33468]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:39.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:39 smithi155 ceph-mon[33468]: from='client.? 172.21.15.44:0/4132944311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:41.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:41 smithi019 ceph-mon[29215]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T18:19:41.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:41 smithi044 ceph-mon[32458]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T18:19:41.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:41 smithi155 ceph-mon[33468]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T18:19:43.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:43 smithi044 ceph-mon[32458]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-23T18:19:43.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:43 smithi019 ceph-mon[29215]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-23T18:19:43.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:43 smithi155 ceph-mon[33468]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-23T18:19:45.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:45 smithi019 ceph-mon[29215]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:19:45.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:19:45.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:45.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:45 smithi044 ceph-mon[32458]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:19:45.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:19:45.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:45.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:45 smithi155 ceph-mon[33468]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:19:45.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:19:45.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:46.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:46 smithi044 ceph-mon[32458]: Deploying daemon osd.4 on smithi044 2024-04-23T18:19:46.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:46 smithi019 ceph-mon[29215]: Deploying daemon osd.4 on smithi044 2024-04-23T18:19:46.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:46 smithi155 ceph-mon[33468]: Deploying daemon osd.4 on smithi044 2024-04-23T18:19:47.489 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:47 smithi044 ceph-mon[32458]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:19:47.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:47 smithi019 ceph-mon[29215]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:19:47.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:47 smithi155 ceph-mon[33468]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:19:48.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:48 smithi019 ceph-mon[29215]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:19:48.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:48 smithi044 ceph-mon[32458]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:19:48.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:48 smithi155 ceph-mon[33468]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:19:50.451 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:50 smithi044 ceph-mon[32458]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-23T18:19:50.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:50 smithi019 ceph-mon[29215]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-23T18:19:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:50 smithi155 ceph-mon[33468]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-23T18:19:52.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:51 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:52.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:51 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:52.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:51 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:52.054 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:51 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:52.054 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:51 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:52.054 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:51 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:52.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:51 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:52.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:51 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:52.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:51 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:52.942 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:52 smithi044 ceph-mon[32458]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:53.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:52 smithi019 ceph-mon[29215]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:53.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:52 smithi155 ceph-mon[33468]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:54.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.509+0000 7f116dffb700 1 -- 172.21.15.44:0/4252642879 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1170062e80 con 0x7f11600a3c40 2024-04-23T18:19:54.504 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 4 on host 'smithi044' 2024-04-23T18:19:54.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.511+0000 7f1175da1700 1 -- 172.21.15.44:0/4252642879 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f11600a3c40 msgr2=0x7f11600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:54.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.511+0000 7f1175da1700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f11600a3c40 0x7f11600a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f115c006010 tx=0x7f115c005c20 comp rx=0 tx=0).stop 2024-04-23T18:19:54.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.512+0000 7f1175da1700 1 -- 172.21.15.44:0/4252642879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11701018d0 msgr2=0x7f117006b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:54.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.512+0000 7f1175da1700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11701018d0 0x7f117006b2a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f116400e970 tx=0x7f116400ed30 comp rx=0 tx=0).stop 2024-04-23T18:19:54.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.512+0000 7f1175da1700 1 -- 172.21.15.44:0/4252642879 shutdown_connections 2024-04-23T18:19:54.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.512+0000 7f1175da1700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f11600a3c40 0x7f11600a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.512+0000 7f1175da1700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1170103470 0x7f117019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.512+0000 7f1175da1700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11701018d0 0x7f117006b2a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.512+0000 7f1175da1700 1 --2- 172.21.15.44:0/4252642879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1170102ad0 0x7f117019d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.512+0000 7f1175da1700 1 -- 172.21.15.44:0/4252642879 >> 172.21.15.44:0/4252642879 conn(0x7f11700fce50 msgr2=0x7f1170105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:54.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.512+0000 7f1175da1700 1 -- 172.21.15.44:0/4252642879 shutdown_connections 2024-04-23T18:19:54.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:19:54.512+0000 7f1175da1700 1 -- 172.21.15.44:0/4252642879 wait complete. 2024-04-23T18:19:54.784 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:54 smithi044 ceph-mon[32458]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:54.784 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.784 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.784 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:54.784 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:54.785 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.785 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.785 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.785 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:54.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:54 smithi019 ceph-mon[29215]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:54.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:54.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:54.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:54 smithi155 ceph-mon[33468]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:54.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:55.112 DEBUG:teuthology.orchestra.run.smithi044:osd.4> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.4.service 2024-04-23T18:19:55.114 INFO:tasks.cephadm:Deploying osd.5 on smithi155 with /dev/vg_nvme/lv_3... 2024-04-23T18:19:55.114 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:19:55.357 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:19:55.779 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:55 smithi155 ceph-mon[33468]: from='osd.4 [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:19:55.779 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:55 smithi155 ceph-mon[33468]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:19:55.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:55 smithi044 ceph-mon[32458]: from='osd.4 [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:19:55.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:55 smithi044 ceph-mon[32458]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:19:56.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:55 smithi019 ceph-mon[29215]: from='osd.4 [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:19:56.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:55 smithi019 ceph-mon[29215]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:19:56.806 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:56 smithi155 ceph-mon[33468]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:56.806 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:56 smithi155 ceph-mon[33468]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:19:56.806 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:56 smithi155 ceph-mon[33468]: from='osd.4 [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:19:56.806 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:56 smithi155 ceph-mon[33468]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:19:56.806 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:56.806 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:56 smithi155 ceph-mon[33468]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:19:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:56 smithi044 ceph-mon[32458]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:56 smithi044 ceph-mon[32458]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:19:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:56 smithi044 ceph-mon[32458]: from='osd.4 [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:19:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:56 smithi044 ceph-mon[32458]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:19:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:56 smithi044 ceph-mon[32458]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:19:56.813 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:19:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4[43914]: 2024-04-23T18:19:56.610+0000 7f790f219700 -1 osd.4 0 waiting for initial osdmap 2024-04-23T18:19:56.813 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:19:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4[43914]: 2024-04-23T18:19:56.616+0000 7f790b00c700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:19:57.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:56 smithi019 ceph-mon[29215]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:57.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:56 smithi019 ceph-mon[29215]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:19:57.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:56 smithi019 ceph-mon[29215]: from='osd.4 [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:19:57.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:56 smithi019 ceph-mon[29215]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:19:57.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:57.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:56 smithi019 ceph-mon[29215]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:19:57.575 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:19:57.575 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:19:57.575 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-04-23T18:19:57.575 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-04-23T18:19:57.575 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129497 s, 810 MB/s 2024-04-23T18:19:57.575 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-04-23T18:19:57.889 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:57 smithi155 ceph-mon[33468]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:19:57.889 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:57 smithi155 ceph-mon[33468]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:19:57.889 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:57 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:57.889 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:57 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:57.944 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:57 smithi044 ceph-mon[32458]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:19:57.944 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:57 smithi044 ceph-mon[32458]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:19:57.944 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:57 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:57.944 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:57 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:58.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:57 smithi019 ceph-mon[29215]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:19:58.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:57 smithi019 ceph-mon[29215]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:19:58.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:57 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:58.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:57 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:58.212 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-04-23T18:19:58.446 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:19:58.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:19:58.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:19:58.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: osd.4 [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] boot 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi044 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi044 to 8546M 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:58.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:19:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:59.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: osd.4 [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] boot 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi044 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi044 to 8546M 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:59.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:59.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:59.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:19:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:59.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:19:59.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:19:59.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:19:59.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:59.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: osd.4 [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] boot 2024-04-23T18:19:59.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:19:59.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:19:59.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi044 2024-04-23T18:19:59.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:59.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:59.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:59.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:59.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi044 to 8546M 2024-04-23T18:19:59.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:19:59.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:59.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:59.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:19:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:00.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.214+0000 7f81d297a700 1 -- 172.21.15.155:0/2589881091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81cc1019a0 msgr2=0x7f81cc101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:00.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.214+0000 7f81d297a700 1 --2- 172.21.15.155:0/2589881091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81cc1019a0 0x7f81cc101dc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009990 tx=0x7f81bc009ca0 comp rx=0 tx=0).stop 2024-04-23T18:20:00.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.214+0000 7f81d297a700 1 -- 172.21.15.155:0/2589881091 shutdown_connections 2024-04-23T18:20:00.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.214+0000 7f81d297a700 1 --2- 172.21.15.155:0/2589881091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81cc103540 0x7f81cc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:00.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.214+0000 7f81d297a700 1 --2- 172.21.15.155:0/2589881091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81cc1019a0 0x7f81cc101dc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:00.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.214+0000 7f81d297a700 1 --2- 172.21.15.155:0/2589881091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81cc102ba0 0x7f81cc103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:00.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.214+0000 7f81d297a700 1 -- 172.21.15.155:0/2589881091 >> 172.21.15.155:0/2589881091 conn(0x7f81cc0fcf40 msgr2=0x7f81cc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:00.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.214+0000 7f81d297a700 1 -- 172.21.15.155:0/2589881091 shutdown_connections 2024-04-23T18:20:00.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.214+0000 7f81d297a700 1 -- 172.21.15.155:0/2589881091 wait complete. 2024-04-23T18:20:00.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.215+0000 7f81d297a700 1 Processor -- start 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.215+0000 7f81d297a700 1 -- start start 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.215+0000 7f81d297a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81cc1019a0 0x7f81cc06bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.215+0000 7f81d297a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81cc102ba0 0x7f81cc19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.215+0000 7f81d297a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81cc103540 0x7f81cc19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.215+0000 7f81d297a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc10d430 con 0x7f81cc103540 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.215+0000 7f81d297a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc10d2b0 con 0x7f81cc1019a0 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.215+0000 7f81d297a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc10d5b0 con 0x7f81cc102ba0 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.215+0000 7f81cb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81cc102ba0 0x7f81cc19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81cb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81cc102ba0 0x7f81cc19d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:33770/0 (socket says 172.21.15.155:33770) 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81cb7fe700 1 -- 172.21.15.155:0/3652157853 learned_addr learned my addr 172.21.15.155:0/3652157853 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81cbfff700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81cc1019a0 0x7f81cc06bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:00.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81d0f17700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81cc103540 0x7f81cc19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81cb7fe700 1 -- 172.21.15.155:0/3652157853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81cc1019a0 msgr2=0x7f81cc06bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81cb7fe700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81cc1019a0 0x7f81cc06bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81cb7fe700 1 -- 172.21.15.155:0/3652157853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81cc103540 msgr2=0x7f81cc19fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81cb7fe700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81cc103540 0x7f81cc19fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81cb7fe700 1 -- 172.21.15.155:0/3652157853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc1a07b0 con 0x7f81cc102ba0 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81cb7fe700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81cc102ba0 0x7f81cc19d910 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81c000c9b0 tx=0x7f81c000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81c97fa700 1 -- 172.21.15.155:0/3652157853 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0007b00 con 0x7f81cc102ba0 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81d297a700 1 -- 172.21.15.155:0/3652157853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc1a0aa0 con 0x7f81cc102ba0 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81c97fa700 1 -- 172.21.15.155:0/3652157853 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c0007ca0 con 0x7f81cc102ba0 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81c97fa700 1 -- 172.21.15.155:0/3652157853 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0018660 con 0x7f81cc102ba0 2024-04-23T18:20:00.212 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.216+0000 7f81d297a700 1 -- 172.21.15.155:0/3652157853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc1a0fe0 con 0x7f81cc102ba0 2024-04-23T18:20:00.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.217+0000 7f81d297a700 1 -- 172.21.15.155:0/3652157853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81cc069730 con 0x7f81cc102ba0 2024-04-23T18:20:00.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.218+0000 7f81c97fa700 1 -- 172.21.15.155:0/3652157853 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f81c000f450 con 0x7f81cc102ba0 2024-04-23T18:20:00.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.219+0000 7f81c97fa700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3c40 0x7f81b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:00.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.219+0000 7f81c97fa700 1 -- 172.21.15.155:0/3652157853 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f81c00cf390 con 0x7f81cc102ba0 2024-04-23T18:20:00.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.219+0000 7f81cbfff700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3c40 0x7f81b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:00.213 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.220+0000 7f81cbfff700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3c40 0x7f81b40a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0038a0 tx=0x7f81bc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:00.216 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.224+0000 7f81c97fa700 1 -- 172.21.15.155:0/3652157853 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81c0014030 con 0x7f81cc102ba0 2024-04-23T18:20:00.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:00 smithi044 ceph-mon[32458]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:20:00.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:00 smithi044 ceph-mon[32458]: overall HEALTH_OK 2024-04-23T18:20:00.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:00 smithi155 ceph-mon[33468]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:20:00.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:00 smithi155 ceph-mon[33468]: overall HEALTH_OK 2024-04-23T18:20:00.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:00.433+0000 7f81d297a700 1 -- 172.21.15.155:0/3652157853 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f81cc062e80 con 0x7f81b40a3c40 2024-04-23T18:20:00.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:00 smithi019 ceph-mon[29215]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:20:00.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:00 smithi019 ceph-mon[29215]: overall HEALTH_OK 2024-04-23T18:20:01.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:01 smithi044 ceph-mon[32458]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:01.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:01 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:01.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:01 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:01.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:01 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:01.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:01 smithi155 ceph-mon[33468]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:01.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:01 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:01.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:01 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:01.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:01 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:01.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:01 smithi019 ceph-mon[29215]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:01.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:01 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:01.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:01 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:01.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:01 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:02.277 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:02 smithi155 ceph-mon[33468]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:02.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:02 smithi044 ceph-mon[32458]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:02.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:02 smithi019 ceph-mon[29215]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:03.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:03 smithi044 ceph-mon[32458]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:03.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:03 smithi155 ceph-mon[33468]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:03 smithi019 ceph-mon[29215]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:04 smithi044 ceph-mon[32458]: from='client.? 172.21.15.155:0/455862227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777efb4b-82b9-4ce6-8187-80697bffc735"}]: dispatch 2024-04-23T18:20:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:04 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777efb4b-82b9-4ce6-8187-80697bffc735"}]: dispatch 2024-04-23T18:20:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:04 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777efb4b-82b9-4ce6-8187-80697bffc735"}]': finished 2024-04-23T18:20:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:04 smithi044 ceph-mon[32458]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:20:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:04 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:04.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:04 smithi044 ceph-mon[32458]: from='client.? 172.21.15.155:0/777291985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:04 smithi155 ceph-mon[33468]: from='client.? 172.21.15.155:0/455862227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777efb4b-82b9-4ce6-8187-80697bffc735"}]: dispatch 2024-04-23T18:20:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:04 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777efb4b-82b9-4ce6-8187-80697bffc735"}]: dispatch 2024-04-23T18:20:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:04 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777efb4b-82b9-4ce6-8187-80697bffc735"}]': finished 2024-04-23T18:20:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:04 smithi155 ceph-mon[33468]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:20:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:04 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:04.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:04 smithi155 ceph-mon[33468]: from='client.? 172.21.15.155:0/777291985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:04.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:04 smithi019 ceph-mon[29215]: from='client.? 172.21.15.155:0/455862227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777efb4b-82b9-4ce6-8187-80697bffc735"}]: dispatch 2024-04-23T18:20:04.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:04 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777efb4b-82b9-4ce6-8187-80697bffc735"}]: dispatch 2024-04-23T18:20:04.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:04 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777efb4b-82b9-4ce6-8187-80697bffc735"}]': finished 2024-04-23T18:20:04.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:04 smithi019 ceph-mon[29215]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:20:04.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:04 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:04.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:04 smithi019 ceph-mon[29215]: from='client.? 172.21.15.155:0/777291985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:05.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:05 smithi019 ceph-mon[29215]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:05.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:05 smithi044 ceph-mon[32458]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:05.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:05 smithi155 ceph-mon[33468]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:06.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:06 smithi019 ceph-mon[29215]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:06.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:06 smithi044 ceph-mon[32458]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:06.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:06 smithi155 ceph-mon[33468]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:08.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:08 smithi155 ceph-mon[33468]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:08.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:08 smithi019 ceph-mon[29215]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:08.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:08 smithi044 ceph-mon[32458]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:10.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:10 smithi155 ceph-mon[33468]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:10.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:10 smithi019 ceph-mon[29215]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:10.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:10 smithi044 ceph-mon[32458]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:11.562 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:20:11.562 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:11.562 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:11 smithi155 ceph-mon[33468]: Deploying daemon osd.5 on smithi155 2024-04-23T18:20:11.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:20:11.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:11.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:11 smithi019 ceph-mon[29215]: Deploying daemon osd.5 on smithi155 2024-04-23T18:20:11.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:20:11.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:11.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:11 smithi044 ceph-mon[32458]: Deploying daemon osd.5 on smithi155 2024-04-23T18:20:12.496 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:12 smithi155 ceph-mon[33468]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:12.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:12 smithi019 ceph-mon[29215]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:12.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:12 smithi044 ceph-mon[32458]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:14.585 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:14 smithi155 ceph-mon[33468]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:14.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:14 smithi019 ceph-mon[29215]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:14.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:14 smithi044 ceph-mon[32458]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:16.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:16 smithi155 ceph-mon[33468]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:16.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:16 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:16.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:16 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:16.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:16 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:16.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:16 smithi019 ceph-mon[29215]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:16.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:16 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:16.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:16 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:16.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:16 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:16.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:16 smithi044 ceph-mon[32458]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:16.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:16 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:16.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:16 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:16.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:16 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:18.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:18 smithi155 ceph-mon[33468]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:18.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:18 smithi019 ceph-mon[29215]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:18.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:18 smithi044 ceph-mon[32458]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:20.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.205+0000 7f81c97fa700 1 -- 172.21.15.155:0/3652157853 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f81cc062e80 con 0x7f81b40a3c40 2024-04-23T18:20:20.199 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 5 on host 'smithi155' 2024-04-23T18:20:20.201 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.207+0000 7f81d297a700 1 -- 172.21.15.155:0/3652157853 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3c40 msgr2=0x7f81b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.207+0000 7f81d297a700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3c40 0x7f81b40a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0038a0 tx=0x7f81bc005fb0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.207+0000 7f81d297a700 1 -- 172.21.15.155:0/3652157853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81cc102ba0 msgr2=0x7f81cc19d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.207+0000 7f81d297a700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81cc102ba0 0x7f81cc19d910 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81c000c9b0 tx=0x7f81c000ccc0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.207+0000 7f81d297a700 1 -- 172.21.15.155:0/3652157853 shutdown_connections 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.207+0000 7f81d297a700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f81b40a3c40 0x7f81b40a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.207+0000 7f81d297a700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81cc103540 0x7f81cc19fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.207+0000 7f81d297a700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81cc1019a0 0x7f81cc06bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.207+0000 7f81d297a700 1 --2- 172.21.15.155:0/3652157853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81cc102ba0 0x7f81cc19d910 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.207+0000 7f81d297a700 1 -- 172.21.15.155:0/3652157853 >> 172.21.15.155:0/3652157853 conn(0x7f81cc0fcf40 msgr2=0x7f81cc105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.208+0000 7f81d297a700 1 -- 172.21.15.155:0/3652157853 shutdown_connections 2024-04-23T18:20:20.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:20:20.208+0000 7f81d297a700 1 -- 172.21.15.155:0/3652157853 wait complete. 2024-04-23T18:20:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:20.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: from='osd.5 [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:20.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:20 smithi044 ceph-mon[32458]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:20.564 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: from='osd.5 [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:20 smithi155 ceph-mon[33468]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:20.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:20.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:20.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:20.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:20.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:20.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: from='osd.5 [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:20.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:20 smithi019 ceph-mon[29215]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:20.902 DEBUG:teuthology.orchestra.run.smithi155:osd.5> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.5.service 2024-04-23T18:20:20.904 INFO:tasks.cephadm:Deploying osd.6 on smithi019 with /dev/vg_nvme/lv_2... 2024-04-23T18:20:20.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:20:21.161 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:22.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:22 smithi155 ceph-mon[33468]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:20:22.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:22 smithi155 ceph-mon[33468]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:20:22.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:22 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:22.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:22 smithi155 ceph-mon[33468]: from='osd.5 [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:20:22.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:22 smithi155 ceph-mon[33468]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:20:22.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:22 smithi155 ceph-mon[33468]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:22.453 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:22 smithi019 ceph-mon[29215]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:20:22.453 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:22 smithi019 ceph-mon[29215]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:20:22.453 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:22 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:22.453 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:22 smithi019 ceph-mon[29215]: from='osd.5 [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:20:22.453 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:22 smithi019 ceph-mon[29215]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:20:22.454 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:22 smithi019 ceph-mon[29215]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:22.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:22 smithi044 ceph-mon[32458]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:20:22.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:22 smithi044 ceph-mon[32458]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:20:22.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:22 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:22.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:22 smithi044 ceph-mon[32458]: from='osd.5 [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:20:22.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:22 smithi044 ceph-mon[32458]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:20:22.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:22 smithi044 ceph-mon[32458]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:23.074 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:20:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:20:22.639+0000 7f55a7271700 -1 osd.5 0 waiting for initial osdmap 2024-04-23T18:20:23.074 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:20:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:20:22.647+0000 7f559f05c700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:20:23.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:23 smithi019 ceph-mon[29215]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:20:23.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:23 smithi019 ceph-mon[29215]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:20:23.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:23 smithi019 ceph-mon[29215]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T18:20:23.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:23 smithi019 ceph-mon[29215]: from='osd.5 ' entity='osd.5' 2024-04-23T18:20:23.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.483 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:20:23.483 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:20:23.483 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-23T18:20:23.483 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-23T18:20:23.483 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0118674 s, 884 MB/s 2024-04-23T18:20:23.483 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-23T18:20:23.555 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:23 smithi155 ceph-mon[33468]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:20:23.555 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:23 smithi155 ceph-mon[33468]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:20:23.555 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.555 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.555 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:23 smithi155 ceph-mon[33468]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T18:20:23.555 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.555 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:23 smithi155 ceph-mon[33468]: from='osd.5 ' entity='osd.5' 2024-04-23T18:20:23.555 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:23 smithi044 ceph-mon[32458]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:20:23.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:23 smithi044 ceph-mon[32458]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:20:23.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:23 smithi044 ceph-mon[32458]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T18:20:23.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:23.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:23 smithi044 ceph-mon[32458]: from='osd.5 ' entity='osd.5' 2024-04-23T18:20:23.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:24.292 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-04-23T18:20:24.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:20:24.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: osd.5 [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] boot 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi155 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi155 to 8544M 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:24.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:24.512 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:24 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.538 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:24.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:20:24.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:20:24.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:24.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: osd.5 [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] boot 2024-04-23T18:20:24.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:20:24.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:24.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi155 2024-04-23T18:20:24.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:24.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:24.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi155 to 8544M 2024-04-23T18:20:24.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:24.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:24.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:24 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: osd.5 [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] boot 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi155 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:24.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:24.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi155 to 8544M 2024-04-23T18:20:24.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:24.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:24.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:24.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:24 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:25.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:25 smithi019 ceph-mon[29215]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:20:26.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:25 smithi044 ceph-mon[32458]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:20:26.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:25 smithi155 ceph-mon[33468]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:20:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.322+0000 7f435d126700 1 -- 172.21.15.19:0/4141122279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4358103cc0 msgr2=0x7f4358104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.322+0000 7f435d126700 1 --2- 172.21.15.19:0/4141122279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4358103cc0 0x7f4358104120 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f434c0099c0 tx=0x7f434c009cd0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.322+0000 7f435d126700 1 -- 172.21.15.19:0/4141122279 shutdown_connections 2024-04-23T18:20:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.322+0000 7f435d126700 1 --2- 172.21.15.19:0/4141122279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4358104660 0x7f435810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.322+0000 7f435d126700 1 --2- 172.21.15.19:0/4141122279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4358102ac0 0x7f4358102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.322+0000 7f435d126700 1 --2- 172.21.15.19:0/4141122279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4358103cc0 0x7f4358104120 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.322+0000 7f435d126700 1 -- 172.21.15.19:0/4141122279 >> 172.21.15.19:0/4141122279 conn(0x7f43580fe060 msgr2=0x7f43581004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.322+0000 7f435d126700 1 -- 172.21.15.19:0/4141122279 shutdown_connections 2024-04-23T18:20:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.322+0000 7f435d126700 1 -- 172.21.15.19:0/4141122279 wait complete. 2024-04-23T18:20:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.323+0000 7f435d126700 1 Processor -- start 2024-04-23T18:20:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435d126700 1 -- start start 2024-04-23T18:20:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435d126700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4358102ac0 0x7f4358198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435d126700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4358103cc0 0x7f4358198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435d126700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4358104660 0x7f43581a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435d126700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f435810e530 con 0x7f4358103cc0 2024-04-23T18:20:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435d126700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f435810e3b0 con 0x7f4358104660 2024-04-23T18:20:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435d126700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f435810e6b0 con 0x7f4358102ac0 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4358103cc0 0x7f4358198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4358103cc0 0x7f4358198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52866/0 (socket says 172.21.15.19:52866) 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435659c700 1 -- 172.21.15.19:0/3973712324 learned_addr learned my addr 172.21.15.19:0/3973712324 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f4356d9d700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4358102ac0 0x7f4358198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435759e700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4358104660 0x7f43581a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435659c700 1 -- 172.21.15.19:0/3973712324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4358102ac0 msgr2=0x7f4358198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435659c700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4358102ac0 0x7f4358198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435659c700 1 -- 172.21.15.19:0/3973712324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4358104660 msgr2=0x7f43581a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435659c700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4358104660 0x7f43581a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435659c700 1 -- 172.21.15.19:0/3973712324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43581a0710 con 0x7f4358103cc0 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435759e700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4358104660 0x7f43581a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.324+0000 7f435659c700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4358103cc0 0x7f4358198ff0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f434c005910 tx=0x7f434c003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.325+0000 7f433ffff700 1 -- 172.21.15.19:0/3973712324 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c027070 con 0x7f4358103cc0 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.325+0000 7f433ffff700 1 -- 172.21.15.19:0/3973712324 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c022470 con 0x7f4358103cc0 2024-04-23T18:20:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.325+0000 7f435d126700 1 -- 172.21.15.19:0/3973712324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43581a09a0 con 0x7f4358103cc0 2024-04-23T18:20:26.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.325+0000 7f433ffff700 1 -- 172.21.15.19:0/3973712324 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c00f6c0 con 0x7f4358103cc0 2024-04-23T18:20:26.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.325+0000 7f435d126700 1 -- 172.21.15.19:0/3973712324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43581a0ed0 con 0x7f4358103cc0 2024-04-23T18:20:26.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.326+0000 7f435d126700 1 -- 172.21.15.19:0/3973712324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4358069730 con 0x7f4358103cc0 2024-04-23T18:20:26.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.327+0000 7f433ffff700 1 -- 172.21.15.19:0/3973712324 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f434c02a070 con 0x7f4358103cc0 2024-04-23T18:20:26.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.328+0000 7f433ffff700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f43400a3cc0 0x7f43400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:26.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.328+0000 7f4356d9d700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f43400a3cc0 0x7f43400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:26.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.328+0000 7f433ffff700 1 -- 172.21.15.19:0/3973712324 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f434c0d07e0 con 0x7f4358103cc0 2024-04-23T18:20:26.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.328+0000 7f4356d9d700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f43400a3cc0 0x7f43400a6180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4348005fd0 tx=0x7f4348005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.331+0000 7f433ffff700 1 -- 172.21.15.19:0/3973712324 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f434c031080 con 0x7f4358103cc0 2024-04-23T18:20:26.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:26.552+0000 7f435d126700 1 -- 172.21.15.19:0/3973712324 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4358062e80 con 0x7f43400a3cc0 2024-04-23T18:20:27.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:26 smithi019 ceph-mon[29215]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:27.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:26 smithi019 ceph-mon[29215]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:20:27.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:27.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:27.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:27.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:26 smithi044 ceph-mon[32458]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:27.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:26 smithi044 ceph-mon[32458]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:20:27.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:27.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:27.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:27.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:26 smithi155 ceph-mon[33468]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:27.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:26 smithi155 ceph-mon[33468]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:20:27.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:27.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:27.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:27.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:27 smithi155 ceph-mon[33468]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:27.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:27 smithi155 ceph-mon[33468]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:20:27.907 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:27 smithi019 ceph-mon[29215]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:27.907 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:27 smithi019 ceph-mon[29215]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:20:28.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:27 smithi044 ceph-mon[32458]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:28.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:27 smithi044 ceph-mon[32458]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:20:28.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:28 smithi019 ceph-mon[29215]: pgmap v148: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:28.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:28 smithi019 ceph-mon[29215]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T18:20:29.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:28 smithi044 ceph-mon[32458]: pgmap v148: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:29.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:28 smithi044 ceph-mon[32458]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T18:20:29.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:28 smithi155 ceph-mon[33468]: pgmap v148: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:29.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:28 smithi155 ceph-mon[33468]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T18:20:29.923 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:29 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3522003532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0be6429-6e18-42e7-b385-34826780f118"}]: dispatch 2024-04-23T18:20:29.923 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:29 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3522003532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0be6429-6e18-42e7-b385-34826780f118"}]': finished 2024-04-23T18:20:29.923 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:29 smithi019 ceph-mon[29215]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:20:29.923 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:29 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:30.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:29 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3522003532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0be6429-6e18-42e7-b385-34826780f118"}]: dispatch 2024-04-23T18:20:30.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:29 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3522003532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0be6429-6e18-42e7-b385-34826780f118"}]': finished 2024-04-23T18:20:30.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:29 smithi044 ceph-mon[32458]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:20:30.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:29 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:30.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:29 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3522003532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0be6429-6e18-42e7-b385-34826780f118"}]: dispatch 2024-04-23T18:20:30.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:29 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3522003532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0be6429-6e18-42e7-b385-34826780f118"}]': finished 2024-04-23T18:20:30.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:29 smithi155 ceph-mon[33468]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:20:30.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:29 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:31.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:30 smithi019 ceph-mon[29215]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-23T18:20:31.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:30 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1426727936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:31.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:30 smithi044 ceph-mon[32458]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-23T18:20:31.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:30 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1426727936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:30 smithi155 ceph-mon[33468]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-23T18:20:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:30 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1426727936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:32 smithi019 ceph-mon[29215]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-23T18:20:33.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:32 smithi044 ceph-mon[32458]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-23T18:20:33.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:32 smithi155 ceph-mon[33468]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-23T18:20:34.887 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:34 smithi019 ceph-mon[29215]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-23T18:20:35.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:34 smithi044 ceph-mon[32458]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-23T18:20:35.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:34 smithi155 ceph-mon[33468]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-23T18:20:36.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:36 smithi019 ceph-mon[29215]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:20:37.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:36 smithi044 ceph-mon[32458]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:20:37.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:36 smithi155 ceph-mon[33468]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:20:37.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:20:37.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:38.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:37 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:20:38.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:37 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:38.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:20:38.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:38.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:38 smithi019 ceph-mon[29215]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-23T18:20:38.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:38 smithi019 ceph-mon[29215]: Deploying daemon osd.6 on smithi019 2024-04-23T18:20:39.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:38 smithi044 ceph-mon[32458]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-23T18:20:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:38 smithi044 ceph-mon[32458]: Deploying daemon osd.6 on smithi019 2024-04-23T18:20:39.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:38 smithi155 ceph-mon[33468]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-23T18:20:39.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:38 smithi155 ceph-mon[33468]: Deploying daemon osd.6 on smithi019 2024-04-23T18:20:40.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:40 smithi019 ceph-mon[29215]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:41.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:40 smithi044 ceph-mon[32458]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:41.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:40 smithi155 ceph-mon[33468]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:42 smithi019 ceph-mon[29215]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:43.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:42 smithi044 ceph-mon[32458]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:43.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:42 smithi155 ceph-mon[33468]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:44.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:44 smithi019 ceph-mon[29215]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:44.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:44.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:44.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:44.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:44 smithi044 ceph-mon[32458]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:44.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:44.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:44.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:44.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:44 smithi155 ceph-mon[33468]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:44.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:44.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:44.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:46.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:46 smithi019 ceph-mon[29215]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:46.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:46 smithi044 ceph-mon[32458]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:46.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:46 smithi155 ceph-mon[33468]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.654+0000 7f433ffff700 1 -- 172.21.15.19:0/3973712324 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4358062e80 con 0x7f43400a3cc0 2024-04-23T18:20:47.648 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 6 on host 'smithi019' 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.656+0000 7f435d126700 1 -- 172.21.15.19:0/3973712324 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f43400a3cc0 msgr2=0x7f43400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.656+0000 7f435d126700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f43400a3cc0 0x7f43400a6180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4348005fd0 tx=0x7f4348005e50 comp rx=0 tx=0).stop 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.656+0000 7f435d126700 1 -- 172.21.15.19:0/3973712324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4358103cc0 msgr2=0x7f4358198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.656+0000 7f435d126700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4358103cc0 0x7f4358198ff0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f434c005910 tx=0x7f434c003960 comp rx=0 tx=0).stop 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.656+0000 7f435d126700 1 -- 172.21.15.19:0/3973712324 shutdown_connections 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.656+0000 7f435d126700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f43400a3cc0 0x7f43400a6180 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.656+0000 7f435d126700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4358104660 0x7f43581a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.656+0000 7f435d126700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4358102ac0 0x7f4358198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.656+0000 7f435d126700 1 --2- 172.21.15.19:0/3973712324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4358103cc0 0x7f4358198ff0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.656+0000 7f435d126700 1 -- 172.21.15.19:0/3973712324 >> 172.21.15.19:0/3973712324 conn(0x7f43580fe060 msgr2=0x7f4358106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.657+0000 7f435d126700 1 -- 172.21.15.19:0/3973712324 shutdown_connections 2024-04-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:20:47.657+0000 7f435d126700 1 -- 172.21.15.19:0/3973712324 wait complete. 2024-04-23T18:20:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:47 smithi155 ceph-mon[33468]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:20:48.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:48.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:48.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:48.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:47 smithi019 ceph-mon[29215]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:20:48.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:48.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:48.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:48.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:47 smithi044 ceph-mon[32458]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:20:48.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:48.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:48.475 DEBUG:teuthology.orchestra.run.smithi019:osd.6> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.6.service 2024-04-23T18:20:48.477 INFO:tasks.cephadm:Deploying osd.7 on smithi044 with /dev/vg_nvme/lv_2... 2024-04-23T18:20:48.478 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:20:48.735 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:20:48.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:48 smithi019 ceph-mon[29215]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:48.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:48 smithi019 ceph-mon[29215]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:20:48.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:48 smithi019 ceph-mon[29215]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:20:48.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:48 smithi019 ceph-mon[29215]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:20:48.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:48 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:49.097 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:48 smithi044 ceph-mon[32458]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:49.097 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:48 smithi044 ceph-mon[32458]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:20:49.097 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:48 smithi044 ceph-mon[32458]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:20:49.097 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:48 smithi044 ceph-mon[32458]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:20:49.097 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:48 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:49.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:48 smithi155 ceph-mon[33468]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:49.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:48 smithi155 ceph-mon[33468]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:20:49.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:48 smithi155 ceph-mon[33468]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:20:49.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:48 smithi155 ceph-mon[33468]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:20:49.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:48 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:49.760 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:20:49 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:20:49.283+0000 7fa7947d0700 -1 osd.6 0 waiting for initial osdmap 2024-04-23T18:20:49.760 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:20:49 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:20:49.291+0000 7fa78bdba700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:20:50.149 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:49 smithi044 ceph-mon[32458]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:20:50.149 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:49 smithi044 ceph-mon[32458]: osdmap e46: 7 total, 6 up, 7 in 2024-04-23T18:20:50.149 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:49 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:50.149 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:49 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:50.149 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:49 smithi044 ceph-mon[32458]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' 2024-04-23T18:20:50.185 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:49 smithi019 ceph-mon[29215]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:20:50.185 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:49 smithi019 ceph-mon[29215]: osdmap e46: 7 total, 6 up, 7 in 2024-04-23T18:20:50.185 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:49 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:50.185 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:49 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:50.185 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:49 smithi019 ceph-mon[29215]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' 2024-04-23T18:20:50.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:49 smithi155 ceph-mon[33468]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:20:50.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:49 smithi155 ceph-mon[33468]: osdmap e46: 7 total, 6 up, 7 in 2024-04-23T18:20:50.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:49 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:50.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:49 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:50.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:49 smithi155 ceph-mon[33468]: from='osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219]' entity='osd.6' 2024-04-23T18:20:50.834 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:20:50.834 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:20:50.834 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-23T18:20:50.834 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-23T18:20:50.834 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110142 s, 952 MB/s 2024-04-23T18:20:50.834 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-23T18:20:51.167 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:50 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:20:51.167 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:50 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:20:51.167 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:50 smithi044 ceph-mon[32458]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:51.167 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:51.167 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:50 smithi044 ceph-mon[32458]: osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] boot 2024-04-23T18:20:51.167 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:50 smithi044 ceph-mon[32458]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:20:51.167 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:51.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:50 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:20:51.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:50 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:20:51.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:50 smithi019 ceph-mon[29215]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:51.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:51.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:50 smithi019 ceph-mon[29215]: osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] boot 2024-04-23T18:20:51.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:50 smithi019 ceph-mon[29215]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:20:51.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:51.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:50 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:20:51.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:50 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:20:51.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:50 smithi155 ceph-mon[33468]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:20:51.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:51.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:50 smithi155 ceph-mon[33468]: osd.6 [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] boot 2024-04-23T18:20:51.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:50 smithi155 ceph-mon[33468]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:20:51.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:51.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_2 2024-04-23T18:20:51.745 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:20:52.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: osdmap e48: 7 total, 7 up, 7 in 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi019 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi019 to 5697M 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:52.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.558 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: osdmap e48: 7 total, 7 up, 7 in 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi019 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi019 to 5697M 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:52.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: osdmap e48: 7 total, 7 up, 7 in 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi019 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi019 to 5697M 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:52.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:20:53.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.325+0000 7f3632cc9700 1 -- 172.21.15.44:0/199013177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f362c105e80 msgr2=0x7f362c108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:53.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.325+0000 7f3632cc9700 1 --2- 172.21.15.44:0/199013177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f362c105e80 0x7f362c108270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3618009990 tx=0x7f3618009ca0 comp rx=0 tx=0).stop 2024-04-23T18:20:53.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 -- 172.21.15.44:0/199013177 shutdown_connections 2024-04-23T18:20:53.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 --2- 172.21.15.44:0/199013177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362c1087b0 0x7f362c10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:53.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 --2- 172.21.15.44:0/199013177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f362c103550 0x7f362c105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:53.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 --2- 172.21.15.44:0/199013177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f362c105e80 0x7f362c108270 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:53.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 -- 172.21.15.44:0/199013177 >> 172.21.15.44:0/199013177 conn(0x7f362c0fcf60 msgr2=0x7f362c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:53.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 -- 172.21.15.44:0/199013177 shutdown_connections 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 -- 172.21.15.44:0/199013177 wait complete. 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 Processor -- start 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 -- start start 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f362c103550 0x7f362c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362c105e80 0x7f362c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f362c1087b0 0x7f362c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f362c10e5c0 con 0x7f362c1087b0 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f362c10e440 con 0x7f362c103550 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3632cc9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f362c10e740 con 0x7f362c105e80 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.326+0000 7f3630a65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f362c103550 0x7f362c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3630a65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f362c103550 0x7f362c198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57114/0 (socket says 172.21.15.44:57114) 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3630a65700 1 -- 172.21.15.44:0/465856926 learned_addr learned my addr 172.21.15.44:0/465856926 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-23T18:20:53.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f362bfff700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362c105e80 0x7f362c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3631266700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f362c1087b0 0x7f362c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3630a65700 1 -- 172.21.15.44:0/465856926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362c105e80 msgr2=0x7f362c1990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3630a65700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362c105e80 0x7f362c1990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3630a65700 1 -- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f362c1087b0 msgr2=0x7f362c1a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3630a65700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f362c1087b0 0x7f362c1a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3630a65700 1 -- 172.21.15.44:0/465856926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c1a07d0 con 0x7f362c103550 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3631266700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f362c1087b0 0x7f362c1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3630a65700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f362c103550 0x7f362c198b70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f362000c9b0 tx=0x7f362000cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3629ffb700 1 -- 172.21.15.44:0/465856926 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3620007b00 con 0x7f362c103550 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3632cc9700 1 -- 172.21.15.44:0/465856926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362c1a0ac0 con 0x7f362c103550 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3629ffb700 1 -- 172.21.15.44:0/465856926 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3620007ca0 con 0x7f362c103550 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3629ffb700 1 -- 172.21.15.44:0/465856926 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36200186f0 con 0x7f362c103550 2024-04-23T18:20:53.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.327+0000 7f3632cc9700 1 -- 172.21.15.44:0/465856926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362c1a1050 con 0x7f362c103550 2024-04-23T18:20:53.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.328+0000 7f3632cc9700 1 -- 172.21.15.44:0/465856926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f362c069730 con 0x7f362c103550 2024-04-23T18:20:53.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.329+0000 7f3629ffb700 1 -- 172.21.15.44:0/465856926 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f362000f450 con 0x7f362c103550 2024-04-23T18:20:53.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.330+0000 7f3629ffb700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f361c0a3c40 0x7f361c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:53.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.330+0000 7f3629ffb700 1 -- 172.21.15.44:0/465856926 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f36200cf710 con 0x7f362c103550 2024-04-23T18:20:53.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.330+0000 7f362bfff700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f361c0a3c40 0x7f361c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:53.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.331+0000 7f362bfff700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f361c0a3c40 0x7f361c0a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f361800b5c0 tx=0x7f3618005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:53.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.335+0000 7f3629ffb700 1 -- 172.21.15.44:0/465856926 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3620014030 con 0x7f362c103550 2024-04-23T18:20:53.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:20:53.552+0000 7f3632cc9700 1 -- 172.21.15.44:0/465856926 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f362c062e80 con 0x7f361c0a3c40 2024-04-23T18:20:54.503 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:54 smithi044 ceph-mon[32458]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:54.503 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:54 smithi044 ceph-mon[32458]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:54.503 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:54.503 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:54.504 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:54 smithi019 ceph-mon[29215]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:54 smithi019 ceph-mon[29215]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:54 smithi155 ceph-mon[33468]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:54 smithi155 ceph-mon[33468]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:56.648 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:56 smithi044 ceph-mon[32458]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:56.648 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:56 smithi044 ceph-mon[32458]: from='client.? 172.21.15.44:0/1986037582' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71f568c1-298c-4d3a-a191-02ffd057f620"}]: dispatch 2024-04-23T18:20:56.648 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:56 smithi044 ceph-mon[32458]: from='client.? 172.21.15.44:0/1986037582' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71f568c1-298c-4d3a-a191-02ffd057f620"}]': finished 2024-04-23T18:20:56.648 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:56 smithi044 ceph-mon[32458]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:20:56.648 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:20:56.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:56 smithi019 ceph-mon[29215]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:56.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:56 smithi019 ceph-mon[29215]: from='client.? 172.21.15.44:0/1986037582' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71f568c1-298c-4d3a-a191-02ffd057f620"}]: dispatch 2024-04-23T18:20:56.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:56 smithi019 ceph-mon[29215]: from='client.? 172.21.15.44:0/1986037582' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71f568c1-298c-4d3a-a191-02ffd057f620"}]': finished 2024-04-23T18:20:56.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:56 smithi019 ceph-mon[29215]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:20:56.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:20:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:56 smithi155 ceph-mon[33468]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:56 smithi155 ceph-mon[33468]: from='client.? 172.21.15.44:0/1986037582' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71f568c1-298c-4d3a-a191-02ffd057f620"}]: dispatch 2024-04-23T18:20:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:56 smithi155 ceph-mon[33468]: from='client.? 172.21.15.44:0/1986037582' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71f568c1-298c-4d3a-a191-02ffd057f620"}]': finished 2024-04-23T18:20:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:56 smithi155 ceph-mon[33468]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:20:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:20:57.667 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:57 smithi155 ceph-mon[33468]: from='client.? 172.21.15.44:0/3780388752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:57.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:57 smithi019 ceph-mon[29215]: from='client.? 172.21.15.44:0/3780388752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:57.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:57 smithi044 ceph-mon[32458]: from='client.? 172.21.15.44:0/3780388752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:58.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:20:58 smithi155 ceph-mon[33468]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:20:58 smithi019 ceph-mon[29215]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:20:58.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:20:58 smithi044 ceph-mon[32458]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:00.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:00 smithi044 ceph-mon[32458]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:00.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:00 smithi019 ceph-mon[29215]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:00.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:00 smithi155 ceph-mon[33468]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:02 smithi044 ceph-mon[32458]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:02.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:02 smithi019 ceph-mon[29215]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:02.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:02 smithi155 ceph-mon[33468]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:04.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:04 smithi044 ceph-mon[32458]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:04.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:04 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:21:04.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:04 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:04.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:04 smithi044 ceph-mon[32458]: Deploying daemon osd.7 on smithi044 2024-04-23T18:21:04.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:04 smithi019 ceph-mon[29215]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:04.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:04 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:21:04.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:04 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:04.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:04 smithi019 ceph-mon[29215]: Deploying daemon osd.7 on smithi044 2024-04-23T18:21:04.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:04 smithi155 ceph-mon[33468]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:04.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:04 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:21:04.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:04 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:04.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:04 smithi155 ceph-mon[33468]: Deploying daemon osd.7 on smithi044 2024-04-23T18:21:06.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:06 smithi044 ceph-mon[32458]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:06.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:06 smithi019 ceph-mon[29215]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:06.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:06 smithi155 ceph-mon[33468]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:08.619 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:08 smithi044 ceph-mon[32458]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:08.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:08 smithi019 ceph-mon[29215]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:08.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:08 smithi155 ceph-mon[33468]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:09.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:09.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:09.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:09.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:09.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:09.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:09.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:09.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:09.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:10.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:10 smithi044 ceph-mon[32458]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:10.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:10 smithi019 ceph-mon[29215]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:10.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:10 smithi155 ceph-mon[33468]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:13.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.073+0000 7f3629ffb700 1 -- 172.21.15.44:0/465856926 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f362c062e80 con 0x7f361c0a3c40 2024-04-23T18:21:13.067 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 7 on host 'smithi044' 2024-04-23T18:21:13.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 -- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f361c0a3c40 msgr2=0x7f361c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:13.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f361c0a3c40 0x7f361c0a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f361800b5c0 tx=0x7f3618005fd0 comp rx=0 tx=0).stop 2024-04-23T18:21:13.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 -- 172.21.15.44:0/465856926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f362c103550 msgr2=0x7f362c198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:13.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f362c103550 0x7f362c198b70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f362000c9b0 tx=0x7f362000cd70 comp rx=0 tx=0).stop 2024-04-23T18:21:13.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 -- 172.21.15.44:0/465856926 shutdown_connections 2024-04-23T18:21:13.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f361c0a3c40 0x7f361c0a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:13.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f362c1087b0 0x7f362c1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:13.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f362c103550 0x7f362c198b70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 --2- 172.21.15.44:0/465856926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362c105e80 0x7f362c1990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 -- 172.21.15.44:0/465856926 >> 172.21.15.44:0/465856926 conn(0x7f362c0fcf60 msgr2=0x7f362c103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 -- 172.21.15.44:0/465856926 shutdown_connections 2024-04-23T18:21:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:21:13.076+0000 7f3632cc9700 1 -- 172.21.15.44:0/465856926 wait complete. 2024-04-23T18:21:13.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:13.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:13.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:13.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:13.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: from='osd.7 [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:13.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:13 smithi044 ceph-mon[32458]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:13.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:13.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:13.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:13.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:13.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: from='osd.7 [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:13.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:13 smithi019 ceph-mon[29215]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: from='osd.7 [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:13 smithi155 ceph-mon[33468]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:13.652 DEBUG:teuthology.orchestra.run.smithi044:osd.7> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.7.service 2024-04-23T18:21:13.654 INFO:tasks.cephadm:Deploying osd.8 on smithi155 with /dev/vg_nvme/lv_2... 2024-04-23T18:21:13.654 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:21:13.907 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:21:15.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:15 smithi044 ceph-mon[32458]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:15.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:15 smithi044 ceph-mon[32458]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:21:15.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:15 smithi044 ceph-mon[32458]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:21:15.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:15.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:15 smithi044 ceph-mon[32458]: from='osd.7 [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:21:15.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:15 smithi044 ceph-mon[32458]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:21:15.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:15 smithi155 ceph-mon[33468]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:15 smithi155 ceph-mon[33468]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:21:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:15 smithi155 ceph-mon[33468]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:21:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:15 smithi155 ceph-mon[33468]: from='osd.7 [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:21:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:15 smithi155 ceph-mon[33468]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:21:15.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:15 smithi019 ceph-mon[29215]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:15.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:15 smithi019 ceph-mon[29215]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:21:15.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:15 smithi019 ceph-mon[29215]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:21:15.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:15.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:15 smithi019 ceph-mon[29215]: from='osd.7 [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:21:15.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:15 smithi019 ceph-mon[29215]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:21:15.813 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:21:15 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:21:15.510+0000 7f0e34974700 -1 osd.7 0 waiting for initial osdmap 2024-04-23T18:21:15.813 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:21:15 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:21:15.521+0000 7f0e2c75f700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:21:15.905 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:21:15.905 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:21:15.905 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-04-23T18:21:15.905 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-04-23T18:21:15.905 INFO:teuthology.orchestra.run.smithi155.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113359 s, 925 MB/s 2024-04-23T18:21:15.905 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-04-23T18:21:16.224 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:16 smithi155 ceph-mon[33468]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:21:16.225 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:16 smithi155 ceph-mon[33468]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T18:21:16.225 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:16 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:16.225 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:16 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:16.225 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:16 smithi155 ceph-mon[33468]: from='osd.7 ' entity='osd.7' 2024-04-23T18:21:16.255 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:16 smithi044 ceph-mon[32458]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:21:16.255 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:16 smithi044 ceph-mon[32458]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T18:21:16.255 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:16 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:16.255 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:16 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:16.255 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:16 smithi044 ceph-mon[32458]: from='osd.7 ' entity='osd.7' 2024-04-23T18:21:16.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:16 smithi019 ceph-mon[29215]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:21:16.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:16 smithi019 ceph-mon[29215]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T18:21:16.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:16 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:16.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:16 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:16.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:16 smithi019 ceph-mon[29215]: from='osd.7 ' entity='osd.7' 2024-04-23T18:21:16.535 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-04-23T18:21:16.779 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:21:17.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:21:17.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:21:17.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: osd.7 [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] boot 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:17.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:17 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: osd.7 [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] boot 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:17.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:17 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:21:17.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:21:17.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:17.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:17.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: osd.7 [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] boot 2024-04-23T18:21:17.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T18:21:17.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:17.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:17.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:17.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:17.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:17.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:17.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:17.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:17 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:18.224 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:18 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi044 2024-04-23T18:21:18.224 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:18 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi044 to 5697M 2024-04-23T18:21:18.224 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:18 smithi155 ceph-mon[33468]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.461+0000 7f55bd18c700 1 -- 172.21.15.155:0/1719840072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b8102ba0 msgr2=0x7f55b8103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.461+0000 7f55bd18c700 1 --2- 172.21.15.155:0/1719840072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b8102ba0 0x7f55b8103000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009990 tx=0x7f55a8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.461+0000 7f55bd18c700 1 -- 172.21.15.155:0/1719840072 shutdown_connections 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.461+0000 7f55bd18c700 1 --2- 172.21.15.155:0/1719840072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b8103540 0x7f55b8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.461+0000 7f55bd18c700 1 --2- 172.21.15.155:0/1719840072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b81019a0 0x7f55b8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.461+0000 7f55bd18c700 1 --2- 172.21.15.155:0/1719840072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b8102ba0 0x7f55b8103000 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.461+0000 7f55bd18c700 1 -- 172.21.15.155:0/1719840072 >> 172.21.15.155:0/1719840072 conn(0x7f55b80fcf60 msgr2=0x7f55b80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.461+0000 7f55bd18c700 1 -- 172.21.15.155:0/1719840072 shutdown_connections 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.461+0000 7f55bd18c700 1 -- 172.21.15.155:0/1719840072 wait complete. 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55bd18c700 1 Processor -- start 2024-04-23T18:21:18.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55bd18c700 1 -- start start 2024-04-23T18:21:18.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55bd18c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b81019a0 0x7f55b806bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:18.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55bd18c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b8102ba0 0x7f55b819d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:18.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55bd18c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b8103540 0x7f55b81a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:18.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55bd18c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55b810d530 con 0x7f55b81019a0 2024-04-23T18:21:18.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55bd18c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55b810d3b0 con 0x7f55b8102ba0 2024-04-23T18:21:18.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55bd18c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55b810d6b0 con 0x7f55b8103540 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55b759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b8103540 0x7f55b81a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55b759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b8103540 0x7f55b81a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:50532/0 (socket says 172.21.15.155:50532) 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55b759e700 1 -- 172.21.15.155:0/2480947210 learned_addr learned my addr 172.21.15.155:0/2480947210 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.462+0000 7f55b6d9d700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b81019a0 0x7f55b806bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f55b659c700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b8102ba0 0x7f55b819d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f55b759e700 1 -- 172.21.15.155:0/2480947210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b8102ba0 msgr2=0x7f55b819d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f55b759e700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b8102ba0 0x7f55b819d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f55b759e700 1 -- 172.21.15.155:0/2480947210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b81019a0 msgr2=0x7f55b806bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f55b759e700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b81019a0 0x7f55b806bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f55b759e700 1 -- 172.21.15.155:0/2480947210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b81a0910 con 0x7f55b8103540 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f55b659c700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b8102ba0 0x7f55b819d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f55b759e700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b8103540 0x7f55b81a0040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f55b000b970 tx=0x7f55b000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f559ffff700 1 -- 172.21.15.155:0/2480947210 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0019070 con 0x7f55b8103540 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f55bd18c700 1 -- 172.21.15.155:0/2480947210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b81a0c00 con 0x7f55b8103540 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f559ffff700 1 -- 172.21.15.155:0/2480947210 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55b000cde0 con 0x7f55b8103540 2024-04-23T18:21:18.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f559ffff700 1 -- 172.21.15.155:0/2480947210 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0007370 con 0x7f55b8103540 2024-04-23T18:21:18.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.463+0000 7f55bd18c700 1 -- 172.21.15.155:0/2480947210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b81a1210 con 0x7f55b8103540 2024-04-23T18:21:18.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.464+0000 7f55bd18c700 1 -- 172.21.15.155:0/2480947210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b8069730 con 0x7f55b8103540 2024-04-23T18:21:18.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.465+0000 7f559ffff700 1 -- 172.21.15.155:0/2480947210 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f55b001c070 con 0x7f55b8103540 2024-04-23T18:21:18.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.466+0000 7f559ffff700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f55a00a3c40 0x7f55a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:18.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.466+0000 7f559ffff700 1 -- 172.21.15.155:0/2480947210 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f55b00cf480 con 0x7f55b8103540 2024-04-23T18:21:18.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.466+0000 7f55b6d9d700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f55a00a3c40 0x7f55a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:18.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.467+0000 7f55b6d9d700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f55a00a3c40 0x7f55a00a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f55b8102a00 tx=0x7f55ac009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:18.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.470+0000 7f559ffff700 1 -- 172.21.15.155:0/2480947210 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55b0023070 con 0x7f55b8103540 2024-04-23T18:21:18.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:18 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi044 2024-04-23T18:21:18.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:18 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi044 to 5697M 2024-04-23T18:21:18.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:18 smithi019 ceph-mon[29215]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:21:18.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:18 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi044 2024-04-23T18:21:18.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:18 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi044 to 5697M 2024-04-23T18:21:18.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:18 smithi044 ceph-mon[32458]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:21:18.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:18.676+0000 7f55bd18c700 1 -- 172.21.15.155:0/2480947210 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f55b8062e80 con 0x7f55a00a3c40 2024-04-23T18:21:19.395 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:19 smithi155 ceph-mon[33468]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:19.395 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:19 smithi155 ceph-mon[33468]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:21:19.395 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:19 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:19.395 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:19 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:19.395 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:19 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:19.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:19 smithi019 ceph-mon[29215]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:19.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:19 smithi019 ceph-mon[29215]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:21:19.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:19 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:19.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:19 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:19.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:19 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:19.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:19 smithi044 ceph-mon[32458]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:19.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:19 smithi044 ceph-mon[32458]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:21:19.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:19 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:19.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:19 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:19.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:19 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:20.269 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:20 smithi155 ceph-mon[33468]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:20.270 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:20 smithi155 ceph-mon[33468]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:21:20.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:20 smithi019 ceph-mon[29215]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:20.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:20 smithi019 ceph-mon[29215]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:21:20.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:20 smithi044 ceph-mon[32458]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:20 smithi044 ceph-mon[32458]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:21:21.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:21 smithi019 ceph-mon[29215]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:21.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:21 smithi019 ceph-mon[29215]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T18:21:21.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:21 smithi044 ceph-mon[32458]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:21.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:21 smithi044 ceph-mon[32458]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T18:21:21.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:21 smithi155 ceph-mon[33468]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:21.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:21 smithi155 ceph-mon[33468]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T18:21:22.464 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:22 smithi044 ceph-mon[32458]: from='client.? 172.21.15.155:0/2514384579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49dae33b-4d26-4145-a4eb-3ffd2dc50e46"}]: dispatch 2024-04-23T18:21:22.464 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:22 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49dae33b-4d26-4145-a4eb-3ffd2dc50e46"}]: dispatch 2024-04-23T18:21:22.464 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:22 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49dae33b-4d26-4145-a4eb-3ffd2dc50e46"}]': finished 2024-04-23T18:21:22.464 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:22 smithi044 ceph-mon[32458]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T18:21:22.464 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:22 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:22.464 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:22 smithi044 ceph-mon[32458]: from='client.? 172.21.15.155:0/3435692004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:22.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:22 smithi019 ceph-mon[29215]: from='client.? 172.21.15.155:0/2514384579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49dae33b-4d26-4145-a4eb-3ffd2dc50e46"}]: dispatch 2024-04-23T18:21:22.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:22 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49dae33b-4d26-4145-a4eb-3ffd2dc50e46"}]: dispatch 2024-04-23T18:21:22.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:22 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49dae33b-4d26-4145-a4eb-3ffd2dc50e46"}]': finished 2024-04-23T18:21:22.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:22 smithi019 ceph-mon[29215]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T18:21:22.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:22 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:22.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:22 smithi019 ceph-mon[29215]: from='client.? 172.21.15.155:0/3435692004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:22.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:22 smithi155 ceph-mon[33468]: from='client.? 172.21.15.155:0/2514384579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49dae33b-4d26-4145-a4eb-3ffd2dc50e46"}]: dispatch 2024-04-23T18:21:22.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:22 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49dae33b-4d26-4145-a4eb-3ffd2dc50e46"}]: dispatch 2024-04-23T18:21:22.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:22 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49dae33b-4d26-4145-a4eb-3ffd2dc50e46"}]': finished 2024-04-23T18:21:22.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:22 smithi155 ceph-mon[33468]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T18:21:22.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:22 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:22.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:22 smithi155 ceph-mon[33468]: from='client.? 172.21.15.155:0/3435692004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:23.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:23 smithi019 ceph-mon[29215]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:23.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:23 smithi044 ceph-mon[32458]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:23.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:23 smithi155 ceph-mon[33468]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:25.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:25 smithi155 ceph-mon[33468]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:25.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:25 smithi019 ceph-mon[29215]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:25.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:25 smithi044 ceph-mon[32458]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:27.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:27 smithi155 ceph-mon[33468]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:27.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:27 smithi019 ceph-mon[29215]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:27.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:27 smithi044 ceph-mon[32458]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:29.467 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:29 smithi155 ceph-mon[33468]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:29.467 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:29 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:21:29.467 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:29 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:29.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:29 smithi019 ceph-mon[29215]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:29.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:29 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:21:29.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:29 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:29.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:29 smithi044 ceph-mon[32458]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:29.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:29 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:21:29.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:29 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:30.499 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:30 smithi155 ceph-mon[33468]: Deploying daemon osd.8 on smithi155 2024-04-23T18:21:30.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:30 smithi019 ceph-mon[29215]: Deploying daemon osd.8 on smithi155 2024-04-23T18:21:30.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:30 smithi044 ceph-mon[32458]: Deploying daemon osd.8 on smithi155 2024-04-23T18:21:31.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:31 smithi019 ceph-mon[29215]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:31.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:31 smithi044 ceph-mon[32458]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:31.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:31 smithi155 ceph-mon[33468]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:33.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:33 smithi155 ceph-mon[33468]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:33.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:33 smithi019 ceph-mon[29215]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:33.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:33 smithi044 ceph-mon[32458]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:35.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:35 smithi155 ceph-mon[33468]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:35.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:35.306 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:35.306 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:35.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:35 smithi019 ceph-mon[29215]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:35.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:35.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:35.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:35.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:35 smithi044 ceph-mon[32458]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:35.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:35.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:35.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:37.459 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:37 smithi155 ceph-mon[33468]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:37.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:37 smithi019 ceph-mon[29215]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:37.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:37 smithi044 ceph-mon[32458]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:38.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.550+0000 7f559ffff700 1 -- 172.21.15.155:0/2480947210 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f55b8062e80 con 0x7f55a00a3c40 2024-04-23T18:21:38.543 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 8 on host 'smithi155' 2024-04-23T18:21:38.546 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 -- 172.21.15.155:0/2480947210 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f55a00a3c40 msgr2=0x7f55a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.546 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f55a00a3c40 0x7f55a00a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f55b8102a00 tx=0x7f55ac009190 comp rx=0 tx=0).stop 2024-04-23T18:21:38.546 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 -- 172.21.15.155:0/2480947210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b8103540 msgr2=0x7f55b81a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.546 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b8103540 0x7f55b81a0040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f55b000b970 tx=0x7f55b000bc80 comp rx=0 tx=0).stop 2024-04-23T18:21:38.546 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 -- 172.21.15.155:0/2480947210 shutdown_connections 2024-04-23T18:21:38.546 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f55a00a3c40 0x7f55a00a6100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b8103540 0x7f55b81a0040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b81019a0 0x7f55b806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 --2- 172.21.15.155:0/2480947210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b8102ba0 0x7f55b819d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 -- 172.21.15.155:0/2480947210 >> 172.21.15.155:0/2480947210 conn(0x7f55b80fcf60 msgr2=0x7f55b8105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:38.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 -- 172.21.15.155:0/2480947210 shutdown_connections 2024-04-23T18:21:38.547 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:21:38.553+0000 7f55bd18c700 1 -- 172.21.15.155:0/2480947210 wait complete. 2024-04-23T18:21:38.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: from='osd.8 [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:21:38.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:38 smithi155 ceph-mon[33468]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:21:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:39.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:39.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:39.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:39.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: from='osd.8 [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:21:39.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:38 smithi019 ceph-mon[29215]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: from='osd.8 [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:21:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:38 smithi044 ceph-mon[32458]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:21:39.197 DEBUG:teuthology.orchestra.run.smithi155:osd.8> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.8.service 2024-04-23T18:21:39.199 INFO:tasks.cephadm:Deploying osd.9 on smithi019 with /dev/vg_nvme/lv_1... 2024-04-23T18:21:39.199 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:21:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:21:40.671 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:40 smithi155 ceph-mon[33468]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:40.671 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:40 smithi155 ceph-mon[33468]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:21:40.671 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:40 smithi155 ceph-mon[33468]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:21:40.671 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:40.672 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:40 smithi155 ceph-mon[33468]: from='osd.8 [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:21:40.672 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:40 smithi155 ceph-mon[33468]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:21:40.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:40 smithi019 ceph-mon[29215]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:40.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:40 smithi019 ceph-mon[29215]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:21:40.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:40 smithi019 ceph-mon[29215]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:21:40.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:40.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:40 smithi019 ceph-mon[29215]: from='osd.8 [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:21:40.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:40 smithi019 ceph-mon[29215]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:21:40.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:40 smithi044 ceph-mon[32458]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:40.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:40 smithi044 ceph-mon[32458]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:21:40.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:40 smithi044 ceph-mon[32458]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:21:40.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:40.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:40 smithi044 ceph-mon[32458]: from='osd.8 [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:21:40.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:40 smithi044 ceph-mon[32458]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:21:40.925 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:21:40 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:21:40.922+0000 7f92e8b1b700 -1 osd.8 0 waiting for initial osdmap 2024-04-23T18:21:40.925 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:21:40 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:21:40.932+0000 7f92e0906700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:21:41.648 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:21:41.648 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:21:41.649 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-23T18:21:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-23T18:21:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114179 s, 918 MB/s 2024-04-23T18:21:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-23T18:21:41.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:41 smithi019 ceph-mon[29215]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:21:41.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:41 smithi019 ceph-mon[29215]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T18:21:41.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:41 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:41.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:41 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:41.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:41 smithi019 ceph-mon[29215]: from='osd.8 ' entity='osd.8' 2024-04-23T18:21:41.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:41 smithi019 ceph-mon[29215]: osd.8 [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] boot 2024-04-23T18:21:41.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:41 smithi019 ceph-mon[29215]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T18:21:41.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:41 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:41.782 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:41 smithi155 ceph-mon[33468]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:21:41.782 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:41 smithi155 ceph-mon[33468]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T18:21:41.782 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:41 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:41.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:41 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:41.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:41 smithi155 ceph-mon[33468]: from='osd.8 ' entity='osd.8' 2024-04-23T18:21:41.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:41 smithi155 ceph-mon[33468]: osd.8 [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] boot 2024-04-23T18:21:41.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:41 smithi155 ceph-mon[33468]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T18:21:41.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:41 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:41.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:41 smithi044 ceph-mon[32458]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:21:41.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:41 smithi044 ceph-mon[32458]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T18:21:41.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:41 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:41.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:41 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:41.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:41 smithi044 ceph-mon[32458]: from='osd.8 ' entity='osd.8' 2024-04-23T18:21:41.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:41 smithi044 ceph-mon[32458]: osd.8 [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] boot 2024-04-23T18:21:41.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:41 smithi044 ceph-mon[32458]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T18:21:41.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:41 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:42.469 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-04-23T18:21:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:42.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:21:43.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:43 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi155 2024-04-23T18:21:43.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:43 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi155 to 5696M 2024-04-23T18:21:43.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:43 smithi044 ceph-mon[32458]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:21:43.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:43 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi155 2024-04-23T18:21:43.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:43 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi155 to 5696M 2024-04-23T18:21:43.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:43 smithi155 ceph-mon[33468]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:21:44.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:43 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi155 2024-04-23T18:21:44.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:43 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi155 to 5696M 2024-04-23T18:21:44.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:43 smithi019 ceph-mon[29215]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:21:44.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.591+0000 7f7fcdcec700 1 -- 172.21.15.19:0/3864320420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fc81112c0 msgr2=0x7f7fc81136b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:44.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.591+0000 7f7fcdcec700 1 --2- 172.21.15.19:0/3864320420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fc81112c0 0x7f7fc81136b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000b0a0 tx=0x7f7fc000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:21:44.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.591+0000 7f7fcdcec700 1 -- 172.21.15.19:0/3864320420 shutdown_connections 2024-04-23T18:21:44.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.591+0000 7f7fcdcec700 1 --2- 172.21.15.19:0/3864320420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fc81112c0 0x7f7fc81136b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:44.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.591+0000 7f7fcdcec700 1 --2- 172.21.15.19:0/3864320420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc8109a40 0x7f7fc8109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:44.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.591+0000 7f7fcdcec700 1 --2- 172.21.15.19:0/3864320420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fc810a3a0 0x7f7fc8110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:44.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.591+0000 7f7fcdcec700 1 -- 172.21.15.19:0/3864320420 >> 172.21.15.19:0/3864320420 conn(0x7f7fc806fcc0 msgr2=0x7f7fc80720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:44.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.591+0000 7f7fcdcec700 1 -- 172.21.15.19:0/3864320420 shutdown_connections 2024-04-23T18:21:44.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.591+0000 7f7fcdcec700 1 -- 172.21.15.19:0/3864320420 wait complete. 2024-04-23T18:21:44.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.592+0000 7f7fcdcec700 1 Processor -- start 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.592+0000 7f7fcdcec700 1 -- start start 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.592+0000 7f7fcdcec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fc8109a40 0x7f7fc81a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.592+0000 7f7fcdcec700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fc810a3a0 0x7f7fc81a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.592+0000 7f7fcdcec700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc81112c0 0x7f7fc81a8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.592+0000 7f7fcdcec700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8116d40 con 0x7f7fc8109a40 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.592+0000 7f7fcdcec700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8116bc0 con 0x7f7fc81112c0 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.592+0000 7f7fcdcec700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8116ec0 con 0x7f7fc810a3a0 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fc8109a40 0x7f7fc81a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fc8109a40 0x7f7fc81a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48722/0 (socket says 172.21.15.19:48722) 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc77fe700 1 -- 172.21.15.19:0/1567037870 learned_addr learned my addr 172.21.15.19:0/1567037870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc6ffd700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fc810a3a0 0x7f7fc81a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc7fff700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc81112c0 0x7f7fc81a8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc77fe700 1 -- 172.21.15.19:0/1567037870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fc810a3a0 msgr2=0x7f7fc81a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc77fe700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fc810a3a0 0x7f7fc81a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc77fe700 1 -- 172.21.15.19:0/1567037870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc81112c0 msgr2=0x7f7fc81a8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc77fe700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc81112c0 0x7f7fc81a8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc77fe700 1 -- 172.21.15.19:0/1567037870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc81a9000 con 0x7f7fc8109a40 2024-04-23T18:21:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc7fff700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc81112c0 0x7f7fc81a8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc77fe700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fc8109a40 0x7f7fc81a13a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800d9b0 tx=0x7f7fb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc4ff9700 1 -- 172.21.15.19:0/1567037870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8016070 con 0x7f7fc8109a40 2024-04-23T18:21:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fcdcec700 1 -- 172.21.15.19:0/1567037870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc81a92f0 con 0x7f7fc8109a40 2024-04-23T18:21:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc4ff9700 1 -- 172.21.15.19:0/1567037870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fb800be00 con 0x7f7fc8109a40 2024-04-23T18:21:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fc4ff9700 1 -- 172.21.15.19:0/1567037870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb800f3c0 con 0x7f7fc8109a40 2024-04-23T18:21:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.593+0000 7f7fcdcec700 1 -- 172.21.15.19:0/1567037870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc81a9880 con 0x7f7fc8109a40 2024-04-23T18:21:44.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.594+0000 7f7fcdcec700 1 -- 172.21.15.19:0/1567037870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc8069730 con 0x7f7fc8109a40 2024-04-23T18:21:44.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.595+0000 7f7fc4ff9700 1 -- 172.21.15.19:0/1567037870 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7fb8019070 con 0x7f7fc8109a40 2024-04-23T18:21:44.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.596+0000 7f7fc4ff9700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7fb00a3cc0 0x7f7fb00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:44.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.596+0000 7f7fc6ffd700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7fb00a3cc0 0x7f7fb00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:44.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.596+0000 7f7fc4ff9700 1 -- 172.21.15.19:0/1567037870 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f7fb80cf9c0 con 0x7f7fc8109a40 2024-04-23T18:21:44.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.597+0000 7f7fc6ffd700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7fb00a3cc0 0x7f7fb00a6180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8074690 tx=0x7f7fbc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:44.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.601+0000 7f7fc4ff9700 1 -- 172.21.15.19:0/1567037870 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fb8020020 con 0x7f7fc8109a40 2024-04-23T18:21:44.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:44 smithi019 ceph-mon[29215]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:21:44.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:44 smithi019 ceph-mon[29215]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:21:44.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:44 smithi044 ceph-mon[32458]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:21:44.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:44 smithi044 ceph-mon[32458]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:21:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:21:44.819+0000 7f7fcdcec700 1 -- 172.21.15.19:0/1567037870 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7fc8062e80 con 0x7f7fb00a3cc0 2024-04-23T18:21:44.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:44 smithi155 ceph-mon[33468]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:21:44.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:44 smithi155 ceph-mon[33468]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:21:45.725 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:45 smithi019 ceph-mon[29215]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:21:45.725 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:45 smithi019 ceph-mon[29215]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:45.725 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:45.726 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:45.726 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:45.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:45 smithi044 ceph-mon[32458]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:21:45.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:45 smithi044 ceph-mon[32458]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:45.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:45.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:45.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:45.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:45 smithi155 ceph-mon[33468]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:21:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:45 smithi155 ceph-mon[33468]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:46.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:46 smithi044 ceph-mon[32458]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:21:46.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:46 smithi044 ceph-mon[32458]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T18:21:46.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:46 smithi155 ceph-mon[33468]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:21:46.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:46 smithi155 ceph-mon[33468]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T18:21:46.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:46 smithi019 ceph-mon[29215]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:21:46.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:46 smithi019 ceph-mon[29215]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T18:21:48.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:48 smithi019 ceph-mon[29215]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-23T18:21:48.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:48 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1416485563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7715f388-45b3-47c4-82a2-d67a2de63b07"}]: dispatch 2024-04-23T18:21:48.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:48 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1416485563' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7715f388-45b3-47c4-82a2-d67a2de63b07"}]': finished 2024-04-23T18:21:48.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:48 smithi019 ceph-mon[29215]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T18:21:48.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:48 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:21:48.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:48 smithi044 ceph-mon[32458]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-23T18:21:48.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:48 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1416485563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7715f388-45b3-47c4-82a2-d67a2de63b07"}]: dispatch 2024-04-23T18:21:48.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:48 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1416485563' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7715f388-45b3-47c4-82a2-d67a2de63b07"}]': finished 2024-04-23T18:21:48.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:48 smithi044 ceph-mon[32458]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T18:21:48.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:48 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:21:48.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:48 smithi155 ceph-mon[33468]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-23T18:21:48.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:48 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1416485563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7715f388-45b3-47c4-82a2-d67a2de63b07"}]: dispatch 2024-04-23T18:21:48.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:48 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1416485563' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7715f388-45b3-47c4-82a2-d67a2de63b07"}]': finished 2024-04-23T18:21:48.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:48 smithi155 ceph-mon[33468]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T18:21:48.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:48 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:21:49.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:49 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2564911717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:49.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:49 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2564911717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:50.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:49 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2564911717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:50.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:50 smithi044 ceph-mon[32458]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-23T18:21:50.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:50 smithi155 ceph-mon[33468]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-23T18:21:51.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:50 smithi019 ceph-mon[29215]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-23T18:21:52.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:52 smithi019 ceph-mon[29215]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-23T18:21:52.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:52 smithi044 ceph-mon[32458]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-23T18:21:52.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:52 smithi155 ceph-mon[33468]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-23T18:21:54.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:54 smithi019 ceph-mon[29215]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:21:54.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:54 smithi044 ceph-mon[32458]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:21:54.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:54 smithi155 ceph-mon[33468]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:21:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:56 smithi044 ceph-mon[32458]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-23T18:21:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:21:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:56.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:56 smithi044 ceph-mon[32458]: Deploying daemon osd.9 on smithi019 2024-04-23T18:21:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:56 smithi155 ceph-mon[33468]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-23T18:21:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:21:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:56.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:56 smithi155 ceph-mon[33468]: Deploying daemon osd.9 on smithi019 2024-04-23T18:21:56.852 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:56 smithi019 ceph-mon[29215]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-23T18:21:56.852 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:21:56.852 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:56.852 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:56 smithi019 ceph-mon[29215]: Deploying daemon osd.9 on smithi019 2024-04-23T18:21:58.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:21:58 smithi019 ceph-mon[29215]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:21:58.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:21:58 smithi044 ceph-mon[32458]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:21:58.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:21:58 smithi155 ceph-mon[33468]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:00.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:00 smithi019 ceph-mon[29215]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:00.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:00 smithi044 ceph-mon[32458]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:00.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:00 smithi155 ceph-mon[33468]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:03 smithi019 ceph-mon[29215]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:03 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:03 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:03 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:03.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:03 smithi044 ceph-mon[32458]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:03.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:03 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:03.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:03 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:03.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:03 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:03.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:03 smithi155 ceph-mon[33468]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:03.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:03 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:03.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:03 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:03.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:03 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:05.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:05 smithi019 ceph-mon[29215]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:05.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:05 smithi044 ceph-mon[32458]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:05.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:05 smithi155 ceph-mon[33468]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:06.105 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 9 on host 'smithi019' 2024-04-23T18:22:06.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.111+0000 7f7fc4ff9700 1 -- 172.21.15.19:0/1567037870 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7fc8062e80 con 0x7f7fb00a3cc0 2024-04-23T18:22:06.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.113+0000 7f7fcdcec700 1 -- 172.21.15.19:0/1567037870 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7fb00a3cc0 msgr2=0x7f7fb00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:06.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.113+0000 7f7fcdcec700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7fb00a3cc0 0x7f7fb00a6180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8074690 tx=0x7f7fbc00b410 comp rx=0 tx=0).stop 2024-04-23T18:22:06.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.114+0000 7f7fcdcec700 1 -- 172.21.15.19:0/1567037870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fc8109a40 msgr2=0x7f7fc81a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:06.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.114+0000 7f7fcdcec700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fc8109a40 0x7f7fc81a13a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800d9b0 tx=0x7f7fb800dcc0 comp rx=0 tx=0).stop 2024-04-23T18:22:06.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.114+0000 7f7fcdcec700 1 -- 172.21.15.19:0/1567037870 shutdown_connections 2024-04-23T18:22:06.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.114+0000 7f7fcdcec700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7fb00a3cc0 0x7f7fb00a6180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:06.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.114+0000 7f7fcdcec700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fc81112c0 0x7f7fc81a8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:06.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.114+0000 7f7fcdcec700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fc8109a40 0x7f7fc81a13a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:06.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.114+0000 7f7fcdcec700 1 --2- 172.21.15.19:0/1567037870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fc810a3a0 0x7f7fc81a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:06.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.114+0000 7f7fcdcec700 1 -- 172.21.15.19:0/1567037870 >> 172.21.15.19:0/1567037870 conn(0x7f7fc806fcc0 msgr2=0x7f7fc8070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:06.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.114+0000 7f7fcdcec700 1 -- 172.21.15.19:0/1567037870 shutdown_connections 2024-04-23T18:22:06.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:22:06.114+0000 7f7fcdcec700 1 -- 172.21.15.19:0/1567037870 wait complete. 2024-04-23T18:22:06.484 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:06 smithi019 ceph-mon[29215]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:22:06.484 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:06.484 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:06 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:06.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:06 smithi044 ceph-mon[32458]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:22:06.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:06.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:06 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:06.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:06 smithi155 ceph-mon[33468]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:22:06.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:06.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:06 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:06.856 DEBUG:teuthology.orchestra.run.smithi019:osd.9> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.9.service 2024-04-23T18:22:06.859 INFO:tasks.cephadm:Deploying osd.10 on smithi044 with /dev/vg_nvme/lv_1... 2024-04-23T18:22:06.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:22:07.118 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:22:07.449 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:07 smithi044 ceph-mon[32458]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:07.450 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:07 smithi044 ceph-mon[32458]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:22:07.450 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:07 smithi044 ceph-mon[32458]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:22:07.450 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:07 smithi044 ceph-mon[32458]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:22:07.450 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:07 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:07.450 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:07 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:07.450 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:07 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:07.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:07 smithi019 ceph-mon[29215]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:07.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:07 smithi019 ceph-mon[29215]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:22:07.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:07 smithi019 ceph-mon[29215]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:22:07.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:07 smithi019 ceph-mon[29215]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:22:07.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:07 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:07.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:07 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:07.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:07 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:07.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:07 smithi155 ceph-mon[33468]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:07.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:07 smithi155 ceph-mon[33468]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:22:07.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:07 smithi155 ceph-mon[33468]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:22:07.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:07 smithi155 ceph-mon[33468]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-23T18:22:07.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:07 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:07.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:07 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:07.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:07 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:07.989 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:22:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:22:07.712+0000 7fe1a4e32700 -1 osd.9 0 waiting for initial osdmap 2024-04-23T18:22:07.989 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:22:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:22:07.726+0000 7fe19cc1d700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:22:08.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:08 smithi019 ceph-mon[29215]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:22:08.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:08 smithi019 ceph-mon[29215]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T18:22:08.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:08 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:08.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:08 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:08.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:08 smithi019 ceph-mon[29215]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' 2024-04-23T18:22:08.494 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:08 smithi044 ceph-mon[32458]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:22:08.494 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:08 smithi044 ceph-mon[32458]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T18:22:08.494 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:08 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:08.494 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:08 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:08.494 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:08 smithi044 ceph-mon[32458]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' 2024-04-23T18:22:08.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:08 smithi155 ceph-mon[33468]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-23T18:22:08.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:08 smithi155 ceph-mon[33468]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T18:22:08.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:08 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:08.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:08 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:08.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:08 smithi155 ceph-mon[33468]: from='osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237]' entity='osd.9' 2024-04-23T18:22:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:22:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:22:09.290 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-23T18:22:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-23T18:22:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110192 s, 952 MB/s 2024-04-23T18:22:09.290 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 2024-04-23T18:22:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-23T18:22:09.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:09 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:22:09.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:09 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:22:09.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:09 smithi019 ceph-mon[29215]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:09.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:09.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:09 smithi019 ceph-mon[29215]: osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] boot 2024-04-23T18:22:09.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:09 smithi019 ceph-mon[29215]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T18:22:09.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:09 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:09.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:09 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:22:09.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:09 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:22:09.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:09 smithi044 ceph-mon[32458]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:09.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:09.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:09 smithi044 ceph-mon[32458]: osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] boot 2024-04-23T18:22:09.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:09 smithi044 ceph-mon[32458]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T18:22:09.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:09 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:09 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:22:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:09 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:22:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:09 smithi155 ceph-mon[33468]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:09 smithi155 ceph-mon[33468]: osd.9 [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] boot 2024-04-23T18:22:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:09 smithi155 ceph-mon[33468]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T18:22:09.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:09 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:09.933 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_1 2024-04-23T18:22:10.190 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:22:10.996 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:10 smithi044 ceph-mon[32458]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:10.996 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:10 smithi044 ceph-mon[32458]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T18:22:11.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:10 smithi019 ceph-mon[29215]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:11.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:10 smithi019 ceph-mon[29215]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T18:22:11.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:10 smithi155 ceph-mon[33468]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:11.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:10 smithi155 ceph-mon[33468]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T18:22:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4088175573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72f0103560 msgr2=0x7f72f0109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 --2- 172.21.15.44:0/4088175573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72f0103560 0x7f72f0109e10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f72e800b0d0 tx=0x7f72e800b3e0 comp rx=0 tx=0).stop 2024-04-23T18:22:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4088175573 shutdown_connections 2024-04-23T18:22:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 --2- 172.21.15.44:0/4088175573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72f0103560 0x7f72f0109e10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 --2- 172.21.15.44:0/4088175573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72f01019c0 0x7f72f0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 --2- 172.21.15.44:0/4088175573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f0102bc0 0x7f72f0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4088175573 >> 172.21.15.44:0/4088175573 conn(0x7f72f00fcf60 msgr2=0x7f72f00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4088175573 shutdown_connections 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4088175573 wait complete. 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 Processor -- start 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 -- start start 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72f01019c0 0x7f72f010c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f0102bc0 0x7f72f010a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72f0103560 0x7f72f010af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72f010e590 con 0x7f72f0103560 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72f010e410 con 0x7f72f01019c0 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.791+0000 7f72f6dc3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72f010e710 con 0x7f72f0102bc0 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f4b5f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72f01019c0 0x7f72f010c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f4b5f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72f01019c0 0x7f72f010c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53198/0 (socket says 172.21.15.44:53198) 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f4b5f700 1 -- 172.21.15.44:0/4255640445 learned_addr learned my addr 172.21.15.44:0/4255640445 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-23T18:22:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72effff700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f0102bc0 0x7f72f010a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f4b5f700 1 -- 172.21.15.44:0/4255640445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f0102bc0 msgr2=0x7f72f010a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f4b5f700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f0102bc0 0x7f72f010a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f4b5f700 1 -- 172.21.15.44:0/4255640445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72f0103560 msgr2=0x7f72f010af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f5360700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72f0103560 0x7f72f010af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f4b5f700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72f0103560 0x7f72f010af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f4b5f700 1 -- 172.21.15.44:0/4255640445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f010b7d0 con 0x7f72f01019c0 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f4b5f700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72f01019c0 0x7f72f010c390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f72e400c990 tx=0x7f72e400cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72edffb700 1 -- 172.21.15.44:0/4255640445 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4007b00 con 0x7f72f01019c0 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72edffb700 1 -- 172.21.15.44:0/4255640445 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e4007ca0 con 0x7f72f01019c0 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4255640445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f01a2c60 con 0x7f72f01019c0 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72edffb700 1 -- 172.21.15.44:0/4255640445 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4018660 con 0x7f72f01019c0 2024-04-23T18:22:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.792+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4255640445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f01a31a0 con 0x7f72f01019c0 2024-04-23T18:22:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.793+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4255640445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72f0069730 con 0x7f72f01019c0 2024-04-23T18:22:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.795+0000 7f72edffb700 1 -- 172.21.15.44:0/4255640445 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f72e4004020 con 0x7f72f01019c0 2024-04-23T18:22:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.796+0000 7f72edffb700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f72e00a3cc0 0x7f72e00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.796+0000 7f72edffb700 1 -- 172.21.15.44:0/4255640445 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f72e40cfc40 con 0x7f72f01019c0 2024-04-23T18:22:11.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.796+0000 7f72effff700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f72e00a3cc0 0x7f72e00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:11.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.796+0000 7f72effff700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f72e00a3cc0 0x7f72e00a6180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f72dc005950 tx=0x7f72dc00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:11.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:11.800+0000 7f72edffb700 1 -- 172.21.15.44:0/4255640445 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72e4014030 con 0x7f72f01019c0 2024-04-23T18:22:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:12.018+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4255640445 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f72f0062e80 con 0x7f72e00a3cc0 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi019 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:11 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi019 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:12.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:12.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:11 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi019 2024-04-23T18:22:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:12.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-23T18:22:12.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:12.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:12.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:12.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:12.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:12.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:12.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:11 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:13.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:12 smithi019 ceph-mon[29215]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:13.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:12 smithi019 ceph-mon[29215]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:13.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:12 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:13.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:12 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:13.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:12 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:13.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:12 smithi044 ceph-mon[32458]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:12 smithi044 ceph-mon[32458]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:12 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:12 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:13.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:12 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:13.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:12 smithi155 ceph-mon[33468]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:12 smithi155 ceph-mon[33468]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:12 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:12 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:13.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:12 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:15.212 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:14 smithi044 ceph-mon[32458]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:15.212 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:14 smithi044 ceph-mon[32458]: from='client.? 172.21.15.44:0/1105352314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393911c2-6938-4f7a-93d6-6c187f236b60"}]: dispatch 2024-04-23T18:22:15.212 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:14 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393911c2-6938-4f7a-93d6-6c187f236b60"}]: dispatch 2024-04-23T18:22:15.212 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:14 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "393911c2-6938-4f7a-93d6-6c187f236b60"}]': finished 2024-04-23T18:22:15.212 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:14 smithi044 ceph-mon[32458]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T18:22:15.212 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:14 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:15.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:14 smithi019 ceph-mon[29215]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:15.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:14 smithi019 ceph-mon[29215]: from='client.? 172.21.15.44:0/1105352314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393911c2-6938-4f7a-93d6-6c187f236b60"}]: dispatch 2024-04-23T18:22:15.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:14 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393911c2-6938-4f7a-93d6-6c187f236b60"}]: dispatch 2024-04-23T18:22:15.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:14 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "393911c2-6938-4f7a-93d6-6c187f236b60"}]': finished 2024-04-23T18:22:15.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:14 smithi019 ceph-mon[29215]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T18:22:15.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:14 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:15.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:14 smithi155 ceph-mon[33468]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:14 smithi155 ceph-mon[33468]: from='client.? 172.21.15.44:0/1105352314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393911c2-6938-4f7a-93d6-6c187f236b60"}]: dispatch 2024-04-23T18:22:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:14 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "393911c2-6938-4f7a-93d6-6c187f236b60"}]: dispatch 2024-04-23T18:22:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:14 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "393911c2-6938-4f7a-93d6-6c187f236b60"}]': finished 2024-04-23T18:22:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:14 smithi155 ceph-mon[33468]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T18:22:15.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:14 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:16.224 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:15 smithi155 ceph-mon[33468]: from='client.? 172.21.15.44:0/800108736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:16.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:15 smithi019 ceph-mon[29215]: from='client.? 172.21.15.44:0/800108736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:16.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:15 smithi044 ceph-mon[32458]: from='client.? 172.21.15.44:0/800108736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:17.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:16 smithi019 ceph-mon[29215]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:17.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:16 smithi044 ceph-mon[32458]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:17.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:16 smithi155 ceph-mon[33468]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:19.178 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:18 smithi044 ceph-mon[32458]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:19.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:18 smithi019 ceph-mon[29215]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:19.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:18 smithi155 ceph-mon[33468]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:21.257 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:20 smithi044 ceph-mon[32458]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:21.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:20 smithi019 ceph-mon[29215]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:21.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:20 smithi155 ceph-mon[33468]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:23.257 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:22 smithi044 ceph-mon[32458]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:23.257 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:22 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:22:23.257 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:22 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:23.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:22 smithi019 ceph-mon[29215]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:23.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:22 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:22:23.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:22 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:23.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:22 smithi155 ceph-mon[33468]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:23.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:22 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:22:23.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:22 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:24.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:23 smithi019 ceph-mon[29215]: Deploying daemon osd.10 on smithi044 2024-04-23T18:22:24.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:23 smithi044 ceph-mon[32458]: Deploying daemon osd.10 on smithi044 2024-04-23T18:22:24.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:23 smithi155 ceph-mon[33468]: Deploying daemon osd.10 on smithi044 2024-04-23T18:22:25.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:24 smithi019 ceph-mon[29215]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:25.294 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:24 smithi044 ceph-mon[32458]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:25.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:24 smithi155 ceph-mon[33468]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:27.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:26 smithi044 ceph-mon[32458]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:27.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:26 smithi019 ceph-mon[29215]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:27.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:26 smithi155 ceph-mon[33468]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:29.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:29 smithi019 ceph-mon[29215]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:29.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:29 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:29.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:29 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:29.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:29 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:29.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:29 smithi044 ceph-mon[32458]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:29.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:29 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:29.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:29 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:29.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:29 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:29.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:29 smithi155 ceph-mon[33468]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:29.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:29 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:29.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:29 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:29.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:29 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:31.204 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:31 smithi044 ceph-mon[32458]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:31.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:31 smithi019 ceph-mon[29215]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:31.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:31 smithi155 ceph-mon[33468]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:31.683 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 10 on host 'smithi044' 2024-04-23T18:22:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.689+0000 7f72edffb700 1 -- 172.21.15.44:0/4255640445 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f72f0062e80 con 0x7f72e00a3cc0 2024-04-23T18:22:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4255640445 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f72e00a3cc0 msgr2=0x7f72e00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f72e00a3cc0 0x7f72e00a6180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f72dc005950 tx=0x7f72dc00a300 comp rx=0 tx=0).stop 2024-04-23T18:22:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4255640445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72f01019c0 msgr2=0x7f72f010c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72f01019c0 0x7f72f010c390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f72e400c990 tx=0x7f72e400cca0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4255640445 shutdown_connections 2024-04-23T18:22:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f72e00a3cc0 0x7f72e00a6180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72f0103560 0x7f72f010af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72f01019c0 0x7f72f010c390 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 --2- 172.21.15.44:0/4255640445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f0102bc0 0x7f72f010a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4255640445 >> 172.21.15.44:0/4255640445 conn(0x7f72f00fcf60 msgr2=0x7f72f0105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4255640445 shutdown_connections 2024-04-23T18:22:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-23T18:22:31.691+0000 7f72f6dc3700 1 -- 172.21.15.44:0/4255640445 wait complete. 2024-04-23T18:22:32.438 DEBUG:teuthology.orchestra.run.smithi044:osd.10> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.10.service 2024-04-23T18:22:32.440 INFO:tasks.cephadm:Deploying osd.11 on smithi155 with /dev/vg_nvme/lv_1... 2024-04-23T18:22:32.441 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:22:32.688 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:22:32.810 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:32 smithi155 ceph-mon[33468]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:32.810 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:32.811 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:32.811 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:32 smithi155 ceph-mon[33468]: from='osd.10 [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:22:32.811 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:32 smithi155 ceph-mon[33468]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:22:32.811 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:32.811 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:32.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:32 smithi044 ceph-mon[32458]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:32.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:32.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:32.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:32 smithi044 ceph-mon[32458]: from='osd.10 [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:22:32.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:32 smithi044 ceph-mon[32458]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:22:32.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:32.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:32 smithi019 ceph-mon[29215]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:33.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:32 smithi019 ceph-mon[29215]: from='osd.10 [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:22:33.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:32 smithi019 ceph-mon[29215]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:22:33.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:33.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:33.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:33 smithi044 ceph-mon[32458]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:22:33.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:33 smithi044 ceph-mon[32458]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:22:33.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:33 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:33.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:33 smithi044 ceph-mon[32458]: from='osd.10 [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:22:33.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:33 smithi044 ceph-mon[32458]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:22:34.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:33 smithi019 ceph-mon[29215]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:22:34.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:33 smithi019 ceph-mon[29215]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:22:34.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:33 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:34.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:33 smithi019 ceph-mon[29215]: from='osd.10 [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:22:34.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:33 smithi019 ceph-mon[29215]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:22:34.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:33 smithi155 ceph-mon[33468]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:22:34.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:33 smithi155 ceph-mon[33468]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:22:34.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:33 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:34.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:33 smithi155 ceph-mon[33468]: from='osd.10 [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:22:34.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:33 smithi155 ceph-mon[33468]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-23T18:22:34.234 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:22:34 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:22:34.087+0000 7f701b967700 -1 osd.10 0 waiting for initial osdmap 2024-04-23T18:22:34.235 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:22:34 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:22:34.100+0000 7f7013752700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:22:34.807 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:22:34.807 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:22:34.807 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-04-23T18:22:34.807 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-04-23T18:22:34.807 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112587 s, 931 MB/s 2024-04-23T18:22:34.807 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-04-23T18:22:34.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:34 smithi044 ceph-mon[32458]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:34.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:34 smithi044 ceph-mon[32458]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:22:34.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:34 smithi044 ceph-mon[32458]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T18:22:34.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:34 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:34.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:34 smithi044 ceph-mon[32458]: from='osd.10 ' entity='osd.10' 2024-04-23T18:22:35.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:34 smithi019 ceph-mon[29215]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:35.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:34 smithi019 ceph-mon[29215]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:22:35.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:34 smithi019 ceph-mon[29215]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T18:22:35.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:34 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:35.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:34 smithi019 ceph-mon[29215]: from='osd.10 ' entity='osd.10' 2024-04-23T18:22:35.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:34 smithi155 ceph-mon[33468]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:35.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:34 smithi155 ceph-mon[33468]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-23T18:22:35.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:34 smithi155 ceph-mon[33468]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T18:22:35.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:34 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:35.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:34 smithi155 ceph-mon[33468]: from='osd.10 ' entity='osd.10' 2024-04-23T18:22:35.552 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-04-23T18:22:35.796 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:22:35.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:35 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:22:35.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:35 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:22:35.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:35 smithi155 ceph-mon[33468]: osd.10 [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] boot 2024-04-23T18:22:35.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:35 smithi155 ceph-mon[33468]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T18:22:35.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:36.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:35 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:22:36.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:35 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:22:36.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:35 smithi019 ceph-mon[29215]: osd.10 [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] boot 2024-04-23T18:22:36.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:35 smithi019 ceph-mon[29215]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T18:22:36.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:36.052 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:35 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:22:36.052 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:35 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:22:36.052 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:35 smithi044 ceph-mon[32458]: osd.10 [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] boot 2024-04-23T18:22:36.052 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:35 smithi044 ceph-mon[32458]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T18:22:36.052 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi044 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi044 to 4273M 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:36.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:36.940 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:36.940 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:36.940 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:37.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:37.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T18:22:37.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi044 2024-04-23T18:22:37.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:37.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:37.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:37.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:37.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:37.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:37.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi044 to 4273M 2024-04-23T18:22:37.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:37.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:37.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:37.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:37.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T18:22:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi044 2024-04-23T18:22:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:37.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:37.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:37.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi044 to 4273M 2024-04-23T18:22:37.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:37.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:37.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:37.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:37.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:37.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.485+0000 7fa33420d700 1 -- 172.21.15.155:0/2263083279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa32c102ad0 msgr2=0x7fa32c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:37.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.485+0000 7fa33420d700 1 --2- 172.21.15.155:0/2263083279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa32c102ad0 0x7fa32c102ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa3280098f0 tx=0x7fa328009c00 comp rx=0 tx=0).stop 2024-04-23T18:22:37.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.485+0000 7fa33420d700 1 -- 172.21.15.155:0/2263083279 shutdown_connections 2024-04-23T18:22:37.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.485+0000 7fa33420d700 1 --2- 172.21.15.155:0/2263083279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa32c104670 0x7fa32c10af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:37.479 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.485+0000 7fa33420d700 1 --2- 172.21.15.155:0/2263083279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa32c102ad0 0x7fa32c102ef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:37.479 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.485+0000 7fa33420d700 1 --2- 172.21.15.155:0/2263083279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa32c103cd0 0x7fa32c104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:37.479 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.485+0000 7fa33420d700 1 -- 172.21.15.155:0/2263083279 >> 172.21.15.155:0/2263083279 conn(0x7fa32c0fe090 msgr2=0x7fa32c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:37.479 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.485+0000 7fa33420d700 1 -- 172.21.15.155:0/2263083279 shutdown_connections 2024-04-23T18:22:37.479 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.485+0000 7fa33420d700 1 -- 172.21.15.155:0/2263083279 wait complete. 2024-04-23T18:22:37.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.486+0000 7fa33420d700 1 Processor -- start 2024-04-23T18:22:37.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.486+0000 7fa33420d700 1 -- start start 2024-04-23T18:22:37.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.486+0000 7fa33420d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa32c102ad0 0x7fa32c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:37.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.486+0000 7fa33420d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa32c103cd0 0x7fa32c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:37.481 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.486+0000 7fa33420d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa32c104670 0x7fa32c1a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.486+0000 7fa33420d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c10e560 con 0x7fa32c103cd0 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.486+0000 7fa33420d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c10e3e0 con 0x7fa32c104670 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.486+0000 7fa33420d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c10e6e0 con 0x7fa32c102ad0 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.486+0000 7fa331fa9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa32c102ad0 0x7fa32c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.486+0000 7fa331fa9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa32c102ad0 0x7fa32c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:47450/0 (socket says 172.21.15.155:47450) 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa331fa9700 1 -- 172.21.15.155:0/1394479988 learned_addr learned my addr 172.21.15.155:0/1394479988 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa3317a8700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa32c103cd0 0x7fa32c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa3327aa700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa32c104670 0x7fa32c1a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa331fa9700 1 -- 172.21.15.155:0/1394479988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa32c104670 msgr2=0x7fa32c1a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa331fa9700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa32c104670 0x7fa32c1a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa331fa9700 1 -- 172.21.15.155:0/1394479988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa32c103cd0 msgr2=0x7fa32c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:37.482 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa331fa9700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa32c103cd0 0x7fa32c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:37.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa331fa9700 1 -- 172.21.15.155:0/1394479988 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32c1a08f0 con 0x7fa32c102ad0 2024-04-23T18:22:37.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa331fa9700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa32c102ad0 0x7fa32c198c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa328009810 tx=0x7fa328005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:37.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa322ffd700 1 -- 172.21.15.155:0/1394479988 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328005460 con 0x7fa32c102ad0 2024-04-23T18:22:37.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa322ffd700 1 -- 172.21.15.155:0/1394479988 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa328005600 con 0x7fa32c102ad0 2024-04-23T18:22:37.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa33420d700 1 -- 172.21.15.155:0/1394479988 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32c1a0b80 con 0x7fa32c102ad0 2024-04-23T18:22:37.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa322ffd700 1 -- 172.21.15.155:0/1394479988 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328003b80 con 0x7fa32c102ad0 2024-04-23T18:22:37.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.487+0000 7fa33420d700 1 -- 172.21.15.155:0/1394479988 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32c1a1030 con 0x7fa32c102ad0 2024-04-23T18:22:37.483 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.488+0000 7fa33420d700 1 -- 172.21.15.155:0/1394479988 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa32c069730 con 0x7fa32c102ad0 2024-04-23T18:22:37.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.489+0000 7fa322ffd700 1 -- 172.21.15.155:0/1394479988 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa32800b560 con 0x7fa32c102ad0 2024-04-23T18:22:37.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.490+0000 7fa322ffd700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3180a3cc0 0x7fa3180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:37.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.490+0000 7fa3317a8700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3180a3cc0 0x7fa3180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:37.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.490+0000 7fa322ffd700 1 -- 172.21.15.155:0/1394479988 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7fa3280d0d90 con 0x7fa32c102ad0 2024-04-23T18:22:37.484 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.491+0000 7fa3317a8700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3180a3cc0 0x7fa3180a6180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa32c19a180 tx=0x7fa31c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:37.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.495+0000 7fa322ffd700 1 -- 172.21.15.155:0/1394479988 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa328013030 con 0x7fa32c102ad0 2024-04-23T18:22:37.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:37.706+0000 7fa33420d700 1 -- 172.21.15.155:0/1394479988 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa32c062e80 con 0x7fa3180a3cc0 2024-04-23T18:22:37.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:37 smithi019 ceph-mon[29215]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:22:37.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:37.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:37.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:37 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:37.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:37 smithi019 ceph-mon[29215]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:22:37.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:37 smithi155 ceph-mon[33468]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:22:37.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:37.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:37.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:37 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:37.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:37 smithi155 ceph-mon[33468]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:22:38.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:37 smithi044 ceph-mon[32458]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:22:38.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:37 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:38.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:37 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:38.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:37 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:38.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:37 smithi044 ceph-mon[32458]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:22:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:38 smithi019 ceph-mon[29215]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:22:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:38 smithi019 ceph-mon[29215]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:38 smithi019 ceph-mon[29215]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T18:22:39.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:38 smithi044 ceph-mon[32458]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:22:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:38 smithi044 ceph-mon[32458]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:39.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:38 smithi044 ceph-mon[32458]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T18:22:39.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:38 smithi155 ceph-mon[33468]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 365 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:22:39.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:38 smithi155 ceph-mon[33468]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:39.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:39.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:39.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:39.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:38 smithi155 ceph-mon[33468]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T18:22:41.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:40 smithi019 ceph-mon[29215]: pgmap v249: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 366 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:22:41.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:40 smithi019 ceph-mon[29215]: from='client.? 172.21.15.155:0/3638486907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d577a0-4fd5-4e4d-a402-71434ca33c95"}]: dispatch 2024-04-23T18:22:41.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:40 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d577a0-4fd5-4e4d-a402-71434ca33c95"}]: dispatch 2024-04-23T18:22:41.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:40 smithi019 ceph-mon[29215]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35d577a0-4fd5-4e4d-a402-71434ca33c95"}]': finished 2024-04-23T18:22:41.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:40 smithi019 ceph-mon[29215]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T18:22:41.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:40 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:22:41.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:40 smithi044 ceph-mon[32458]: pgmap v249: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 366 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:22:41.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:40 smithi044 ceph-mon[32458]: from='client.? 172.21.15.155:0/3638486907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d577a0-4fd5-4e4d-a402-71434ca33c95"}]: dispatch 2024-04-23T18:22:41.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:40 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d577a0-4fd5-4e4d-a402-71434ca33c95"}]: dispatch 2024-04-23T18:22:41.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:40 smithi044 ceph-mon[32458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35d577a0-4fd5-4e4d-a402-71434ca33c95"}]': finished 2024-04-23T18:22:41.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:40 smithi044 ceph-mon[32458]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T18:22:41.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:40 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:22:41.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:40 smithi155 ceph-mon[33468]: pgmap v249: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 366 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:22:41.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:40 smithi155 ceph-mon[33468]: from='client.? 172.21.15.155:0/3638486907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d577a0-4fd5-4e4d-a402-71434ca33c95"}]: dispatch 2024-04-23T18:22:41.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:40 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d577a0-4fd5-4e4d-a402-71434ca33c95"}]: dispatch 2024-04-23T18:22:41.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:40 smithi155 ceph-mon[33468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35d577a0-4fd5-4e4d-a402-71434ca33c95"}]': finished 2024-04-23T18:22:41.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:40 smithi155 ceph-mon[33468]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T18:22:41.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:40 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:22:42.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:41 smithi019 ceph-mon[29215]: from='client.? 172.21.15.155:0/910142338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:42.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:41 smithi044 ceph-mon[32458]: from='client.? 172.21.15.155:0/910142338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:42.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:41 smithi155 ceph-mon[33468]: from='client.? 172.21.15.155:0/910142338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:43.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:42 smithi019 ceph-mon[29215]: pgmap v251: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-23T18:22:43.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:42 smithi044 ceph-mon[32458]: pgmap v251: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-23T18:22:43.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:42 smithi155 ceph-mon[33468]: pgmap v251: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-23T18:22:45.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:44 smithi019 ceph-mon[29215]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:45.050 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:44 smithi155 ceph-mon[33468]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:45.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:44 smithi044 ceph-mon[32458]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:46.980 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:46 smithi155 ceph-mon[33468]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:47.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:46 smithi019 ceph-mon[29215]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:47.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:46 smithi044 ceph-mon[32458]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:49.007 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:48 smithi155 ceph-mon[33468]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:49.008 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:48 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:22:49.008 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:48 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:49.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:48 smithi019 ceph-mon[29215]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:49.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:48 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:22:49.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:48 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:49.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:48 smithi044 ceph-mon[32458]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:49.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:48 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:22:49.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:48 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:50.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:49 smithi019 ceph-mon[29215]: Deploying daemon osd.11 on smithi155 2024-04-23T18:22:50.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:49 smithi155 ceph-mon[33468]: Deploying daemon osd.11 on smithi155 2024-04-23T18:22:50.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:49 smithi044 ceph-mon[32458]: Deploying daemon osd.11 on smithi155 2024-04-23T18:22:50.852 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:50 smithi155 ceph-mon[33468]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:51.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:50 smithi019 ceph-mon[29215]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:51.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:50 smithi044 ceph-mon[32458]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:52.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:52 smithi155 ceph-mon[33468]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:53.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:52 smithi019 ceph-mon[29215]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:53.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:52 smithi044 ceph-mon[32458]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:54.972 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:54 smithi155 ceph-mon[33468]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:54.972 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:54.972 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:54.972 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:54 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:55.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:54 smithi019 ceph-mon[29215]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:55.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:55.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:55.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:54 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:55.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:54 smithi044 ceph-mon[32458]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:55.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:55.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:55.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:54 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:57.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:56 smithi019 ceph-mon[29215]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:57.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:56 smithi155 ceph-mon[33468]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:57.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:56 smithi044 ceph-mon[32458]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:58.172 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.178+0000 7fa322ffd700 1 -- 172.21.15.155:0/1394479988 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa32c062e80 con 0x7fa3180a3cc0 2024-04-23T18:22:58.172 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 11 on host 'smithi155' 2024-04-23T18:22:58.174 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 -- 172.21.15.155:0/1394479988 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3180a3cc0 msgr2=0x7fa3180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:58.174 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3180a3cc0 0x7fa3180a6180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa32c19a180 tx=0x7fa31c009260 comp rx=0 tx=0).stop 2024-04-23T18:22:58.174 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 -- 172.21.15.155:0/1394479988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa32c102ad0 msgr2=0x7fa32c198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:58.174 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa32c102ad0 0x7fa32c198c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa328009810 tx=0x7fa328005270 comp rx=0 tx=0).stop 2024-04-23T18:22:58.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 -- 172.21.15.155:0/1394479988 shutdown_connections 2024-04-23T18:22:58.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3180a3cc0 0x7fa3180a6180 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:58.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa32c104670 0x7fa32c1a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:58.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa32c102ad0 0x7fa32c198c70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:58.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 --2- 172.21.15.155:0/1394479988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa32c103cd0 0x7fa32c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:58.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 -- 172.21.15.155:0/1394479988 >> 172.21.15.155:0/1394479988 conn(0x7fa32c0fe090 msgr2=0x7fa32c107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:58.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 -- 172.21.15.155:0/1394479988 shutdown_connections 2024-04-23T18:22:58.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:22:58.180+0000 7fa33420d700 1 -- 172.21.15.155:0/1394479988 wait complete. 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: from='osd.11 [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:22:58.410 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:22:58 smithi155 ceph-mon[33468]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:22:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:58.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: from='osd.11 [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:22:58.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:22:58 smithi019 ceph-mon[29215]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: from='osd.11 [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:22:58.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:22:58 smithi044 ceph-mon[32458]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:22:58.867 DEBUG:teuthology.orchestra.run.smithi155:osd.11> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.11.service 2024-04-23T18:22:58.870 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-23T18:22:58.870 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-23T18:22:59.130 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:23:00.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:00 smithi019 ceph-mon[29215]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:23:00.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:00 smithi019 ceph-mon[29215]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T18:23:00.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:00 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:00.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:00 smithi019 ceph-mon[29215]: from='osd.11 [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:23:00.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:00 smithi019 ceph-mon[29215]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:23:00.455 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:00 smithi155 ceph-mon[33468]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:23:00.455 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:00 smithi155 ceph-mon[33468]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T18:23:00.455 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:00 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:00.455 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:00 smithi155 ceph-mon[33468]: from='osd.11 [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:23:00.455 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:00 smithi155 ceph-mon[33468]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:23:00.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:00 smithi044 ceph-mon[32458]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:23:00.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:00 smithi044 ceph-mon[32458]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T18:23:00.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:00 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:00.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:00 smithi044 ceph-mon[32458]: from='osd.11 [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:23:00.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:00 smithi044 ceph-mon[32458]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-04-23T18:23:00.823 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:23:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:23:00.618+0000 7fa8ed82e700 -1 osd.11 0 waiting for initial osdmap 2024-04-23T18:23:00.823 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:23:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:23:00.638+0000 7fa8e4e18700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:23:00.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 -- 172.21.15.19:0/1897294358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc1048f0 msgr2=0x7f59dc104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:00.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 --2- 172.21.15.19:0/1897294358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc1048f0 0x7f59dc104d50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009990 tx=0x7f59cc009ca0 comp rx=0 tx=0).stop 2024-04-23T18:23:00.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 -- 172.21.15.19:0/1897294358 shutdown_connections 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 --2- 172.21.15.19:0/1897294358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc105290 0x7f59dc10d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 --2- 172.21.15.19:0/1897294358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc1063b0 0x7f59dc1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 --2- 172.21.15.19:0/1897294358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc1048f0 0x7f59dc104d50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 -- 172.21.15.19:0/1897294358 >> 172.21.15.19:0/1897294358 conn(0x7f59dc100290 msgr2=0x7f59dc1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 -- 172.21.15.19:0/1897294358 shutdown_connections 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 -- 172.21.15.19:0/1897294358 wait complete. 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 Processor -- start 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.977+0000 7f59e37da700 1 -- start start 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e37da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc1048f0 0x7f59dc19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e37da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc105290 0x7f59dc19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e37da700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc1063b0 0x7f59dc1a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e37da700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc110840 con 0x7f59dc105290 2024-04-23T18:23:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e37da700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc1106c0 con 0x7f59dc1048f0 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e37da700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc1109c0 con 0x7f59dc1063b0 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e0d75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc105290 0x7f59dc19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e0d75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc105290 0x7f59dc19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38474/0 (socket says 172.21.15.19:38474) 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e0d75700 1 -- 172.21.15.19:0/2403797624 learned_addr learned my addr 172.21.15.19:0/2403797624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e1d77700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc1063b0 0x7f59dc1a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e1576700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc1048f0 0x7f59dc19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e0d75700 1 -- 172.21.15.19:0/2403797624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc1063b0 msgr2=0x7f59dc1a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e0d75700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc1063b0 0x7f59dc1a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e0d75700 1 -- 172.21.15.19:0/2403797624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc1048f0 msgr2=0x7f59dc19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e0d75700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc1048f0 0x7f59dc19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e0d75700 1 -- 172.21.15.19:0/2403797624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc1a2af0 con 0x7f59dc105290 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e0d75700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc105290 0x7f59dc19b3d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0098e0 tx=0x7f59cc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59d27fc700 1 -- 172.21.15.19:0/2403797624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc027070 con 0x7f59dc105290 2024-04-23T18:23:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e37da700 1 -- 172.21.15.19:0/2403797624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc1a2d80 con 0x7f59dc105290 2024-04-23T18:23:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59d27fc700 1 -- 172.21.15.19:0/2403797624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59cc00bd90 con 0x7f59dc105290 2024-04-23T18:23:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59d27fc700 1 -- 172.21.15.19:0/2403797624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc022dd0 con 0x7f59dc105290 2024-04-23T18:23:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.978+0000 7f59e37da700 1 -- 172.21.15.19:0/2403797624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc1a31d0 con 0x7f59dc105290 2024-04-23T18:23:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.979+0000 7f59e37da700 1 -- 172.21.15.19:0/2403797624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59dc069730 con 0x7f59dc105290 2024-04-23T18:23:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.981+0000 7f59d27fc700 1 -- 172.21.15.19:0/2403797624 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f59cc02a070 con 0x7f59dc105290 2024-04-23T18:23:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.981+0000 7f59d27fc700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3c40 0x7f59c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.981+0000 7f59e1576700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3c40 0x7f59c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.981+0000 7f59d27fc700 1 -- 172.21.15.19:0/2403797624 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f59cc0d13d0 con 0x7f59dc105290 2024-04-23T18:23:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.982+0000 7f59e1576700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3c40 0x7f59c80a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f59dc105d30 tx=0x7f59d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:00.985+0000 7f59d27fc700 1 -- 172.21.15.19:0/2403797624 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59cc031020 con 0x7f59dc105290 2024-04-23T18:23:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.200+0000 7f59e37da700 1 -- 172.21.15.19:0/2403797624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f59dc02df80 con 0x7f59dc105290 2024-04-23T18:23:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.200+0000 7f59d27fc700 1 -- 172.21.15.19:0/2403797624 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f59cc013070 con 0x7f59dc105290 2024-04-23T18:23:01.194 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 -- 172.21.15.19:0/2403797624 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3c40 msgr2=0x7f59c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3c40 0x7f59c80a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f59dc105d30 tx=0x7f59d800b410 comp rx=0 tx=0).stop 2024-04-23T18:23:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 -- 172.21.15.19:0/2403797624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc105290 msgr2=0x7f59dc19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc105290 0x7f59dc19b3d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0098e0 tx=0x7f59cc009580 comp rx=0 tx=0).stop 2024-04-23T18:23:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 -- 172.21.15.19:0/2403797624 shutdown_connections 2024-04-23T18:23:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3c40 0x7f59c80a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc1063b0 0x7f59dc1a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc1048f0 0x7f59dc19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:01.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 --2- 172.21.15.19:0/2403797624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc105290 0x7f59dc19b3d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:01.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 -- 172.21.15.19:0/2403797624 >> 172.21.15.19:0/2403797624 conn(0x7f59dc100290 msgr2=0x7f59dc1091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:01.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 -- 172.21.15.19:0/2403797624 shutdown_connections 2024-04-23T18:23:01.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:01.203+0000 7f59e37da700 1 -- 172.21.15.19:0/2403797624 wait complete. 2024-04-23T18:23:01.287 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:01 smithi019 ceph-mon[29215]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:01.287 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:01 smithi019 ceph-mon[29215]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:23:01.287 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:01 smithi019 ceph-mon[29215]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T18:23:01.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:01 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:01.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:01 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:01.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:01 smithi019 ceph-mon[29215]: from='osd.11 ' entity='osd.11' 2024-04-23T18:23:01.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:01 smithi044 ceph-mon[32458]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:01.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:01 smithi044 ceph-mon[32458]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:23:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:01 smithi044 ceph-mon[32458]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T18:23:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:01 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:01 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:01 smithi044 ceph-mon[32458]: from='osd.11 ' entity='osd.11' 2024-04-23T18:23:01.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:01 smithi155 ceph-mon[33468]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:01.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:01 smithi155 ceph-mon[33468]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-04-23T18:23:01.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:01 smithi155 ceph-mon[33468]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T18:23:01.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:01 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:01.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:01 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:01.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:01 smithi155 ceph-mon[33468]: from='osd.11 ' entity='osd.11' 2024-04-23T18:23:02.016 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1713896554,"num_in_osds":12,"osd_in_since":1713896560,"num_remapped_pgs":0} 2024-04-23T18:23:02.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: purged_snaps scrub starts 2024-04-23T18:23:02.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: purged_snaps scrub ok 2024-04-23T18:23:02.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:02.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2403797624' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:02.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: osd.11 [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] boot 2024-04-23T18:23:02.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T18:23:02.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:02.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:02.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:02.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:02 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: purged_snaps scrub starts 2024-04-23T18:23:02.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: purged_snaps scrub ok 2024-04-23T18:23:02.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:02.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2403797624' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:02.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: osd.11 [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] boot 2024-04-23T18:23:02.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T18:23:02.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:02.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:02.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:02.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:02 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: purged_snaps scrub starts 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: purged_snaps scrub ok 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2403797624' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: osd.11 [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] boot 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:02.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:02.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:02.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:02.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:02 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:03.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-23T18:23:03.278 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:23:03.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:03 smithi019 ceph-mon[29215]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:03.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:03 smithi019 ceph-mon[29215]: Detected new or changed devices on smithi155 2024-04-23T18:23:03.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:03 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi155 to 4272M 2024-04-23T18:23:03.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:03 smithi044 ceph-mon[32458]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:03.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:03 smithi044 ceph-mon[32458]: Detected new or changed devices on smithi155 2024-04-23T18:23:03.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:03 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi155 to 4272M 2024-04-23T18:23:03.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:03 smithi155 ceph-mon[33468]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:03.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:03 smithi155 ceph-mon[33468]: Detected new or changed devices on smithi155 2024-04-23T18:23:03.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:03 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi155 to 4272M 2024-04-23T18:23:04.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:04 smithi019 ceph-mon[29215]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T18:23:04.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:04 smithi044 ceph-mon[32458]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T18:23:04.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:04 smithi155 ceph-mon[33468]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T18:23:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.105+0000 7f71a8426700 1 -- 172.21.15.19:0/1670620004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a0102b90 msgr2=0x7f71a0102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.105+0000 7f71a8426700 1 --2- 172.21.15.19:0/1670620004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a0102b90 0x7f71a0102ff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f71900099c0 tx=0x7f7190009cd0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.105+0000 7f71a8426700 1 -- 172.21.15.19:0/1670620004 shutdown_connections 2024-04-23T18:23:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.105+0000 7f71a8426700 1 --2- 172.21.15.19:0/1670620004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a0103530 0x7f71a0109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.105+0000 7f71a8426700 1 --2- 172.21.15.19:0/1670620004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a0101990 0x7f71a0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.105+0000 7f71a8426700 1 --2- 172.21.15.19:0/1670620004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a0102b90 0x7f71a0102ff0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.105+0000 7f71a8426700 1 -- 172.21.15.19:0/1670620004 >> 172.21.15.19:0/1670620004 conn(0x7f71a00fcf30 msgr2=0x7f71a00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.105+0000 7f71a8426700 1 -- 172.21.15.19:0/1670620004 shutdown_connections 2024-04-23T18:23:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.105+0000 7f71a8426700 1 -- 172.21.15.19:0/1670620004 wait complete. 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a8426700 1 Processor -- start 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a8426700 1 -- start start 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a8426700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a0101990 0x7f71a0105920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a8426700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a0102b90 0x7f71a0103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a8426700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a0103530 0x7f71a01044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a8426700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f71a010d430 con 0x7f71a0101990 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a8426700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71a010d2b0 con 0x7f71a0102b90 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a8426700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71a010d5b0 con 0x7f71a0103530 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a61c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a0101990 0x7f71a0105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a61c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a0101990 0x7f71a0105920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38500/0 (socket says 172.21.15.19:38500) 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a61c2700 1 -- 172.21.15.19:0/1786350950 learned_addr learned my addr 172.21.15.19:0/1786350950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a59c1700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a0102b90 0x7f71a0103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a69c3700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a0103530 0x7f71a01044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a61c2700 1 -- 172.21.15.19:0/1786350950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a0103530 msgr2=0x7f71a01044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a61c2700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a0103530 0x7f71a01044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a61c2700 1 -- 172.21.15.19:0/1786350950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a0102b90 msgr2=0x7f71a0103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a61c2700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a0102b90 0x7f71a0103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a61c2700 1 -- 172.21.15.19:0/1786350950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a0104d60 con 0x7f71a0101990 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.106+0000 7f71a59c1700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a0102b90 0x7f71a0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.107+0000 7f71a61c2700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a0101990 0x7f71a0105920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f719c00d980 tx=0x7f719c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.107+0000 7f71977fe700 1 -- 172.21.15.19:0/1786350950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c017030 con 0x7f71a0101990 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.107+0000 7f71977fe700 1 -- 172.21.15.19:0/1786350950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f719c0044d0 con 0x7f71a0101990 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.107+0000 7f71a8426700 1 -- 172.21.15.19:0/1786350950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a01a2d80 con 0x7f71a0101990 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.107+0000 7f71977fe700 1 -- 172.21.15.19:0/1786350950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c003aa0 con 0x7f71a0101990 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.107+0000 7f71a8426700 1 -- 172.21.15.19:0/1786350950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a01a3200 con 0x7f71a0101990 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.108+0000 7f71a8426700 1 -- 172.21.15.19:0/1786350950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a0069730 con 0x7f71a0101990 2024-04-23T18:23:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.109+0000 7f71977fe700 1 -- 172.21.15.19:0/1786350950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f719c01a070 con 0x7f71a0101990 2024-04-23T18:23:05.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.109+0000 7f71977fe700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f718c0a3cc0 0x7f718c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:05.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.110+0000 7f71a59c1700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f718c0a3cc0 0x7f718c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:05.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.110+0000 7f71977fe700 1 -- 172.21.15.19:0/1786350950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f719c0cf8b0 con 0x7f71a0101990 2024-04-23T18:23:05.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.110+0000 7f71a59c1700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f718c0a3cc0 0x7f718c0a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7190000c00 tx=0x7f7190005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:05.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.115+0000 7f71977fe700 1 -- 172.21.15.19:0/1786350950 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f719c09c730 con 0x7f71a0101990 2024-04-23T18:23:05.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.321+0000 7f71a8426700 1 -- 172.21.15.19:0/1786350950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f71a01a37f0 con 0x7f71a0101990 2024-04-23T18:23:05.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.321+0000 7f71977fe700 1 -- 172.21.15.19:0/1786350950 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f719c09c030 con 0x7f71a0101990 2024-04-23T18:23:05.316 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 -- 172.21.15.19:0/1786350950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f718c0a3cc0 msgr2=0x7f718c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f718c0a3cc0 0x7f718c0a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7190000c00 tx=0x7f7190005fd0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 -- 172.21.15.19:0/1786350950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a0101990 msgr2=0x7f71a0105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a0101990 0x7f71a0105920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f719c00d980 tx=0x7f719c00dd40 comp rx=0 tx=0).stop 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 -- 172.21.15.19:0/1786350950 shutdown_connections 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f718c0a3cc0 0x7f718c0a6180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a0103530 0x7f71a01044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a0101990 0x7f71a0105920 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 --2- 172.21.15.19:0/1786350950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a0102b90 0x7f71a0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 -- 172.21.15.19:0/1786350950 >> 172.21.15.19:0/1786350950 conn(0x7f71a00fcf30 msgr2=0x7f71a00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.323+0000 7f71a8426700 1 -- 172.21.15.19:0/1786350950 shutdown_connections 2024-04-23T18:23:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:05.324+0000 7f71a8426700 1 -- 172.21.15.19:0/1786350950 wait complete. 2024-04-23T18:23:05.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:05 smithi019 ceph-mon[29215]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:05.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:05 smithi044 ceph-mon[32458]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:05.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:05 smithi155 ceph-mon[33468]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:06.014 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1713896581,"num_in_osds":12,"osd_in_since":1713896560,"num_remapped_pgs":0} 2024-04-23T18:23:06.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:23:06.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:06 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1786350950' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:06.264 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:23:06.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:06 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1786350950' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:06.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:06 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1786350950' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:07.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:07 smithi019 ceph-mon[29215]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:07.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:07 smithi044 ceph-mon[32458]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:07.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:07 smithi155 ceph-mon[33468]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.139+0000 7f07e755c700 1 -- 172.21.15.19:0/1787374893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e006a750 msgr2=0x7f07e006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.139+0000 7f07e755c700 1 --2- 172.21.15.19:0/1787374893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e006a750 0x7f07e006abb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009990 tx=0x7f07d0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.139+0000 7f07e755c700 1 -- 172.21.15.19:0/1787374893 shutdown_connections 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.139+0000 7f07e755c700 1 --2- 172.21.15.19:0/1787374893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07e006b0f0 0x7f07e010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.139+0000 7f07e755c700 1 --2- 172.21.15.19:0/1787374893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e0108a30 0x7f07e0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.139+0000 7f07e755c700 1 --2- 172.21.15.19:0/1787374893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e006a750 0x7f07e006abb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.139+0000 7f07e755c700 1 -- 172.21.15.19:0/1787374893 >> 172.21.15.19:0/1787374893 conn(0x7f07e00788f0 msgr2=0x7f07e0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.139+0000 7f07e755c700 1 -- 172.21.15.19:0/1787374893 shutdown_connections 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.139+0000 7f07e755c700 1 -- 172.21.15.19:0/1787374893 wait complete. 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e755c700 1 Processor -- start 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e755c700 1 -- start start 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e755c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07e006a750 0x7f07e0101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e755c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e006b0f0 0x7f07e0101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e755c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e0108a30 0x7f07e0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e755c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0112c70 con 0x7f07e006b0f0 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e755c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0112af0 con 0x7f07e0108a30 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e755c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0112df0 con 0x7f07e006a750 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e4af7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e006b0f0 0x7f07e0101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e4af7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e006b0f0 0x7f07e0101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38520/0 (socket says 172.21.15.19:38520) 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e4af7700 1 -- 172.21.15.19:0/1416501789 learned_addr learned my addr 172.21.15.19:0/1416501789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.140+0000 7f07e52f8700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07e006a750 0x7f07e0101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07e5af9700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e0108a30 0x7f07e0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07e4af7700 1 -- 172.21.15.19:0/1416501789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07e006a750 msgr2=0x7f07e0101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07e4af7700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07e006a750 0x7f07e0101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07e4af7700 1 -- 172.21.15.19:0/1416501789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e0108a30 msgr2=0x7f07e0102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07e4af7700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e0108a30 0x7f07e0102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07e4af7700 1 -- 172.21.15.19:0/1416501789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e01a76a0 con 0x7f07e006b0f0 2024-04-23T18:23:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07e52f8700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07e006a750 0x7f07e0101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07e4af7700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e006b0f0 0x7f07e0101f50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f07d00057c0 tx=0x7f07d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07d67fc700 1 -- 172.21.15.19:0/1416501789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0027070 con 0x7f07e006b0f0 2024-04-23T18:23:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07e755c700 1 -- 172.21.15.19:0/1416501789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e01a7930 con 0x7f07e006b0f0 2024-04-23T18:23:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07d67fc700 1 -- 172.21.15.19:0/1416501789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d0004020 con 0x7f07e006b0f0 2024-04-23T18:23:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07d67fc700 1 -- 172.21.15.19:0/1416501789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0022bf0 con 0x7f07e006b0f0 2024-04-23T18:23:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.141+0000 7f07e755c700 1 -- 172.21.15.19:0/1416501789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e01a7e10 con 0x7f07e006b0f0 2024-04-23T18:23:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.142+0000 7f07e755c700 1 -- 172.21.15.19:0/1416501789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e0069730 con 0x7f07e006b0f0 2024-04-23T18:23:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.143+0000 7f07d67fc700 1 -- 172.21.15.19:0/1416501789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f07d002a070 con 0x7f07e006b0f0 2024-04-23T18:23:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.144+0000 7f07d67fc700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f07cc0a3c40 0x7f07cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.144+0000 7f07e52f8700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f07cc0a3c40 0x7f07cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.144+0000 7f07d67fc700 1 -- 172.21.15.19:0/1416501789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f07d00d16f0 con 0x7f07e006b0f0 2024-04-23T18:23:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.144+0000 7f07e52f8700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f07cc0a3c40 0x7f07cc0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009dd0 tx=0x7f07dc009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.148+0000 7f07d67fc700 1 -- 172.21.15.19:0/1416501789 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07d0031020 con 0x7f07e006b0f0 2024-04-23T18:23:08.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.356+0000 7f07e755c700 1 -- 172.21.15.19:0/1416501789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07e002df80 con 0x7f07e006b0f0 2024-04-23T18:23:08.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.357+0000 7f07d67fc700 1 -- 172.21.15.19:0/1416501789 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18446 (secure 0 0 0) 0x7f07d0013070 con 0x7f07e006b0f0 2024-04-23T18:23:08.352 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:08.352 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:23:03.173026+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:23:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 -- 172.21.15.19:0/1416501789 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f07cc0a3c40 msgr2=0x7f07cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f07cc0a3c40 0x7f07cc0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009dd0 tx=0x7f07dc009040 comp rx=0 tx=0).stop 2024-04-23T18:23:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 -- 172.21.15.19:0/1416501789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e006b0f0 msgr2=0x7f07e0101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e006b0f0 0x7f07e0101f50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f07d00057c0 tx=0x7f07d00057f0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 -- 172.21.15.19:0/1416501789 shutdown_connections 2024-04-23T18:23:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f07cc0a3c40 0x7f07cc0a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e0108a30 0x7f07e0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07e006a750 0x7f07e0101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 --2- 172.21.15.19:0/1416501789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e006b0f0 0x7f07e0101f50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 -- 172.21.15.19:0/1416501789 >> 172.21.15.19:0/1416501789 conn(0x7f07e00788f0 msgr2=0x7f07e00ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 -- 172.21.15.19:0/1416501789 shutdown_connections 2024-04-23T18:23:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:08.360+0000 7f07e755c700 1 -- 172.21.15.19:0/1416501789 wait complete. 2024-04-23T18:23:09.101 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:23:09.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-23T18:23:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:23:09.471 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:09 smithi019 ceph-mon[29215]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:09.472 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:09 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1416501789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:09.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:09 smithi044 ceph-mon[32458]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:09.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:09 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1416501789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:09.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:09 smithi155 ceph-mon[33468]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:09.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:09 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1416501789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.206+0000 7f0d28704700 1 -- 172.21.15.19:0/3768331038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201022f0 msgr2=0x7f0d2010e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.206+0000 7f0d28704700 1 --2- 172.21.15.19:0/3768331038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201022f0 0x7f0d2010e7e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800b0a0 tx=0x7f0d1800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.206+0000 7f0d28704700 1 -- 172.21.15.19:0/3768331038 shutdown_connections 2024-04-23T18:23:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.206+0000 7f0d28704700 1 --2- 172.21.15.19:0/3768331038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201022f0 0x7f0d2010e7e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.206+0000 7f0d28704700 1 --2- 172.21.15.19:0/3768331038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d20107980 0x7f0d20107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.206+0000 7f0d28704700 1 --2- 172.21.15.19:0/3768331038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d20101930 0x7f0d20101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.206+0000 7f0d28704700 1 -- 172.21.15.19:0/3768331038 >> 172.21.15.19:0/3768331038 conn(0x7f0d200788b0 msgr2=0x7f0d20078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.206+0000 7f0d28704700 1 -- 172.21.15.19:0/3768331038 shutdown_connections 2024-04-23T18:23:11.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.206+0000 7f0d28704700 1 -- 172.21.15.19:0/3768331038 wait complete. 2024-04-23T18:23:11.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:11 smithi019 ceph-mon[29215]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:11.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d28704700 1 Processor -- start 2024-04-23T18:23:11.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d28704700 1 -- start start 2024-04-23T18:23:11.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d28704700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d20101930 0x7f0d20199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:11.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d28704700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d201022f0 0x7f0d2019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:11.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d28704700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d20107980 0x7f0d201a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:11.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d28704700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20111ad0 con 0x7f0d20101930 2024-04-23T18:23:11.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d28704700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20111950 con 0x7f0d201022f0 2024-04-23T18:23:11.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d28704700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20111c50 con 0x7f0d20107980 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d264a0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d20101930 0x7f0d20199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d264a0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d20101930 0x7f0d20199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39176/0 (socket says 172.21.15.19:39176) 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d264a0700 1 -- 172.21.15.19:0/3279335282 learned_addr learned my addr 172.21.15.19:0/3279335282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.207+0000 7f0d25c9f700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d201022f0 0x7f0d2019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d26ca1700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d20107980 0x7f0d201a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d264a0700 1 -- 172.21.15.19:0/3279335282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d20107980 msgr2=0x7f0d201a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d264a0700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d20107980 0x7f0d201a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d264a0700 1 -- 172.21.15.19:0/3279335282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d201022f0 msgr2=0x7f0d2019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d264a0700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d201022f0 0x7f0d2019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d264a0700 1 -- 172.21.15.19:0/3279335282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d201a54d0 con 0x7f0d20101930 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d26ca1700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d20107980 0x7f0d201a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d264a0700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d20101930 0x7f0d20199f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00e9a0 tx=0x7f0d1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d177fe700 1 -- 172.21.15.19:0/3279335282 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00cbd0 con 0x7f0d20101930 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d28704700 1 -- 172.21.15.19:0/3279335282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d201a56a0 con 0x7f0d20101930 2024-04-23T18:23:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d177fe700 1 -- 172.21.15.19:0/3279335282 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d1c00cd70 con 0x7f0d20101930 2024-04-23T18:23:11.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d28704700 1 -- 172.21.15.19:0/3279335282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d201a5c00 con 0x7f0d20101930 2024-04-23T18:23:11.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.208+0000 7f0d177fe700 1 -- 172.21.15.19:0/3279335282 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c018680 con 0x7f0d20101930 2024-04-23T18:23:11.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.209+0000 7f0d157fa700 1 -- 172.21.15.19:0/3279335282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d08005330 con 0x7f0d20101930 2024-04-23T18:23:11.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.211+0000 7f0d177fe700 1 -- 172.21.15.19:0/3279335282 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0d1c010430 con 0x7f0d20101930 2024-04-23T18:23:11.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.211+0000 7f0d177fe700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d0c0a3cc0 0x7f0d0c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:11.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.211+0000 7f0d25c9f700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d0c0a3cc0 0x7f0d0c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:11.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.211+0000 7f0d177fe700 1 -- 172.21.15.19:0/3279335282 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0d1c0d0300 con 0x7f0d20101930 2024-04-23T18:23:11.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.212+0000 7f0d25c9f700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d0c0a3cc0 0x7f0d0c0a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d20078f00 tx=0x7f0d100073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.215+0000 7f0d177fe700 1 -- 172.21.15.19:0/3279335282 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d1c09d180 con 0x7f0d20101930 2024-04-23T18:23:11.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.420+0000 7f0d157fa700 1 -- 172.21.15.19:0/3279335282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d080058f0 con 0x7f0d20101930 2024-04-23T18:23:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.421+0000 7f0d177fe700 1 -- 172.21.15.19:0/3279335282 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f0d1c0bd0c0 con 0x7f0d20101930 2024-04-23T18:23:11.419 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:23:11.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 -- 172.21.15.19:0/3279335282 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d0c0a3cc0 msgr2=0x7f0d0c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:11.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d0c0a3cc0 0x7f0d0c0a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d20078f00 tx=0x7f0d100073d0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 -- 172.21.15.19:0/3279335282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d20101930 msgr2=0x7f0d20199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:11.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d20101930 0x7f0d20199f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00e9a0 tx=0x7f0d1c00ecb0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 -- 172.21.15.19:0/3279335282 shutdown_connections 2024-04-23T18:23:11.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d0c0a3cc0 0x7f0d0c0a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d20107980 0x7f0d201a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d20101930 0x7f0d20199f10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 --2- 172.21.15.19:0/3279335282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d201022f0 0x7f0d2019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:11.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 -- 172.21.15.19:0/3279335282 >> 172.21.15.19:0/3279335282 conn(0x7f0d200788b0 msgr2=0x7f0d200ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:11.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 -- 172.21.15.19:0/3279335282 shutdown_connections 2024-04-23T18:23:11.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:11.423+0000 7f0d157fa700 1 -- 172.21.15.19:0/3279335282 wait complete. 2024-04-23T18:23:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:11 smithi044 ceph-mon[32458]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:11 smithi155 ceph-mon[33468]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.a on smithi019 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.d on smithi019 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.g on smithi019 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.j on smithi019 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.b on smithi044 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.e on smithi044 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.h on smithi044 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.k on smithi044 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.c on smithi155 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.f on smithi155 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.i on smithi155 2024-04-23T18:23:12.226 INFO:tasks.cephadm:Adding mds.l on smithi155 2024-04-23T18:23:12.227 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l' 2024-04-23T18:23:12.476 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:23:12.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:12 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3279335282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:12.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:12 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3279335282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:12.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:12 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3279335282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:13.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:13 smithi019 ceph-mon[29215]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:13.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:13 smithi044 ceph-mon[32458]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:13.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:13 smithi155 ceph-mon[33468]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:14.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.184+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/1523393265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4105ed0 msgr2=0x7f2fb41082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:14.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.184+0000 7f2fb8e3b700 1 --2- 172.21.15.155:0/1523393265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4105ed0 0x7f2fb41082c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009990 tx=0x7f2fa8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.184+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/1523393265 shutdown_connections 2024-04-23T18:23:14.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.184+0000 7f2fb8e3b700 1 --2- 172.21.15.155:0/1523393265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb4108800 0x7f2fb410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.184+0000 7f2fb8e3b700 1 --2- 172.21.15.155:0/1523393265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb41035a0 0x7f2fb4105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.184+0000 7f2fb8e3b700 1 --2- 172.21.15.155:0/1523393265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4105ed0 0x7f2fb41082c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.184+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/1523393265 >> 172.21.15.155:0/1523393265 conn(0x7f2fb40fcf70 msgr2=0x7f2fb40ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:14.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.184+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/1523393265 shutdown_connections 2024-04-23T18:23:14.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.184+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/1523393265 wait complete. 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb8e3b700 1 Processor -- start 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb8e3b700 1 -- start start 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb8e3b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb41035a0 0x7f2fb4102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb8e3b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4105ed0 0x7f2fb4101550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb8e3b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb4108800 0x7f2fb4101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb8e3b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410e5c0 con 0x7f2fb41035a0 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb8e3b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410e440 con 0x7f2fb4108800 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb8e3b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410e740 con 0x7f2fb4105ed0 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4105ed0 0x7f2fb4101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4105ed0 0x7f2fb4101550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:42462/0 (socket says 172.21.15.155:42462) 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.185+0000 7f2fb1d9b700 1 -- 172.21.15.155:0/2784076623 learned_addr learned my addr 172.21.15.155:0/2784076623 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-23T18:23:14.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fb259c700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb41035a0 0x7f2fb4102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fb1d9b700 1 -- 172.21.15.155:0/2784076623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb4108800 msgr2=0x7f2fb4101a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fb1d9b700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb4108800 0x7f2fb4101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fb1d9b700 1 -- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb41035a0 msgr2=0x7f2fb4102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fb1d9b700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb41035a0 0x7f2fb4102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fb1d9b700 1 -- 172.21.15.155:0/2784076623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb4102340 con 0x7f2fb4105ed0 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fb259c700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb41035a0 0x7f2fb4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fb1d9b700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4105ed0 0x7f2fb4101550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8006010 tx=0x7f2fa80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fa37fe700 1 -- 172.21.15.155:0/2784076623 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8027070 con 0x7f2fb4105ed0 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fa37fe700 1 -- 172.21.15.155:0/2784076623 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa8022470 con 0x7f2fb4105ed0 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fa37fe700 1 -- 172.21.15.155:0/2784076623 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800f6c0 con 0x7f2fb4105ed0 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/2784076623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb41a2da0 con 0x7f2fb4105ed0 2024-04-23T18:23:14.181 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.186+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/2784076623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb41a3220 con 0x7f2fb4105ed0 2024-04-23T18:23:14.182 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.187+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/2784076623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb4069730 con 0x7f2fb4105ed0 2024-04-23T18:23:14.182 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.188+0000 7f2fa37fe700 1 -- 172.21.15.155:0/2784076623 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2fa802a070 con 0x7f2fb4105ed0 2024-04-23T18:23:14.182 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.189+0000 7f2fa37fe700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2f9c0a3cc0 0x7f2f9c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:14.183 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.189+0000 7f2fa37fe700 1 -- 172.21.15.155:0/2784076623 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2fa80d1260 con 0x7f2fb4105ed0 2024-04-23T18:23:14.183 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.189+0000 7f2fb259c700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2f9c0a3cc0 0x7f2f9c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:14.183 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.190+0000 7f2fb259c700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2f9c0a3cc0 0x7f2f9c0a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4007900 tx=0x7f2fa4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:14.185 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.192+0000 7f2fa37fe700 1 -- 172.21.15.155:0/2784076623 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2fa8013070 con 0x7f2fb4105ed0 2024-04-23T18:23:14.404 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.411+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/2784076623 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}) -- 0x7f2fb4062f80 con 0x7f2f9c0a3cc0 2024-04-23T18:23:14.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.419+0000 7f2fa37fe700 1 -- 172.21.15.155:0/2784076623 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f2fb4062f80 con 0x7f2f9c0a3cc0 2024-04-23T18:23:14.412 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mds.all update... 2024-04-23T18:23:14.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2f9c0a3cc0 msgr2=0x7f2f9c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:14.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2f9c0a3cc0 0x7f2f9c0a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4007900 tx=0x7f2fa4008040 comp rx=0 tx=0).stop 2024-04-23T18:23:14.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/2784076623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4105ed0 msgr2=0x7f2fb4101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:14.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4105ed0 0x7f2fb4101550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8006010 tx=0x7f2fa80047b0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/2784076623 shutdown_connections 2024-04-23T18:23:14.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2f9c0a3cc0 0x7f2f9c0a6180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb4108800 0x7f2fb4101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb41035a0 0x7f2fb4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 --2- 172.21.15.155:0/2784076623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4105ed0 0x7f2fb4101550 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:14.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/2784076623 >> 172.21.15.155:0/2784076623 conn(0x7f2fb40fcf70 msgr2=0x7f2fb4103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:14.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.421+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/2784076623 shutdown_connections 2024-04-23T18:23:14.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-23T18:23:14.422+0000 7f2fb8e3b700 1 -- 172.21.15.155:0/2784076623 wait complete. 2024-04-23T18:23:14.457 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:14 smithi155 ceph-mon[33468]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:14.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:14 smithi019 ceph-mon[29215]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:14.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:14 smithi044 ceph-mon[32458]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:15.067 DEBUG:teuthology.orchestra.run.smithi019:mds.a> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.a.service 2024-04-23T18:23:15.069 DEBUG:teuthology.orchestra.run.smithi019:mds.d> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.d.service 2024-04-23T18:23:15.113 DEBUG:teuthology.orchestra.run.smithi019:mds.g> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.g.service 2024-04-23T18:23:15.156 DEBUG:teuthology.orchestra.run.smithi019:mds.j> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.j.service 2024-04-23T18:23:15.198 DEBUG:teuthology.orchestra.run.smithi044:mds.b> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.b.service 2024-04-23T18:23:15.201 DEBUG:teuthology.orchestra.run.smithi044:mds.e> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e.service 2024-04-23T18:23:15.244 DEBUG:teuthology.orchestra.run.smithi044:mds.h> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.h.service 2024-04-23T18:23:15.286 DEBUG:teuthology.orchestra.run.smithi044:mds.k> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.k.service 2024-04-23T18:23:15.329 DEBUG:teuthology.orchestra.run.smithi155:mds.c> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.c.service 2024-04-23T18:23:15.332 DEBUG:teuthology.orchestra.run.smithi155:mds.f> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.f.service 2024-04-23T18:23:15.376 DEBUG:teuthology.orchestra.run.smithi155:mds.i> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.i.service 2024-04-23T18:23:15.418 DEBUG:teuthology.orchestra.run.smithi155:mds.l> sudo journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.l.service 2024-04-23T18:23:15.462 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-23T18:23:15.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:23:15.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:15 smithi044 ceph-mon[32458]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:23:15.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:15 smithi044 ceph-mon[32458]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-04-23T18:23:15.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:15.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:15.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:15.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:15.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:15.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:15.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:15.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:15 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:15.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.581+0000 7f1732ff7640 1 -- 172.21.15.19:0/1583640291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c103b70 msgr2=0x7f172c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:15.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.581+0000 7f1732ff7640 1 --2- 172.21.15.19:0/1583640291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c103b70 0x7f172c10a400 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1714009900 tx=0x7f171402a410 comp rx=0 tx=0).stop 2024-04-23T18:23:15.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.582+0000 7f1732ff7640 1 -- 172.21.15.19:0/1583640291 shutdown_connections 2024-04-23T18:23:15.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.582+0000 7f1732ff7640 1 --2- 172.21.15.19:0/1583640291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c103b70 0x7f172c10a400 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.582+0000 7f1732ff7640 1 --2- 172.21.15.19:0/1583640291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c1031b0 0x7f172c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.582+0000 7f1732ff7640 1 --2- 172.21.15.19:0/1583640291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f172c101fc0 0x7f172c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.582+0000 7f1732ff7640 1 -- 172.21.15.19:0/1583640291 >> 172.21.15.19:0/1583640291 conn(0x7f172c0fd550 msgr2=0x7f172c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.582+0000 7f1732ff7640 1 -- 172.21.15.19:0/1583640291 shutdown_connections 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.582+0000 7f1732ff7640 1 -- 172.21.15.19:0/1583640291 wait complete. 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.582+0000 7f1732ff7640 1 Processor -- start 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.582+0000 7f1732ff7640 1 -- start start 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1732ff7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f172c101fc0 0x7f172c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1732ff7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c1031b0 0x7f172c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1732ff7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c103b70 0x7f172c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1732ff7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10db00 con 0x7f172c1031b0 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1732ff7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10d980 con 0x7f172c101fc0 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1732ff7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10dc80 con 0x7f172c103b70 2024-04-23T18:23:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f172bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c1031b0 0x7f172c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f172bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c1031b0 0x7f172c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39190/0 (socket says 172.21.15.19:39190) 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f172bfff640 1 -- 172.21.15.19:0/2121738215 learned_addr learned my addr 172.21.15.19:0/2121738215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1731594640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c103b70 0x7f172c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1730d93640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f172c101fc0 0x7f172c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f172bfff640 1 -- 172.21.15.19:0/2121738215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c103b70 msgr2=0x7f172c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f172bfff640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c103b70 0x7f172c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f172bfff640 1 -- 172.21.15.19:0/2121738215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f172c101fc0 msgr2=0x7f172c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f172bfff640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f172c101fc0 0x7f172c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f172bfff640 1 -- 172.21.15.19:0/2121738215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c19fdb0 con 0x7f172c1031b0 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f172bfff640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c1031b0 0x7f172c1986d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f171c00d770 tx=0x7f171c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1729ffb640 1 -- 172.21.15.19:0/2121738215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c017070 con 0x7f172c1031b0 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1729ffb640 1 -- 172.21.15.19:0/2121738215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f171c0054d0 con 0x7f172c1031b0 2024-04-23T18:23:15.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1732ff7640 1 -- 172.21.15.19:0/2121738215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c1a00a0 con 0x7f172c1031b0 2024-04-23T18:23:15.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.583+0000 7f1729ffb640 1 -- 172.21.15.19:0/2121738215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c004c30 con 0x7f172c1031b0 2024-04-23T18:23:15.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.584+0000 7f1732ff7640 1 -- 172.21.15.19:0/2121738215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c1a05a0 con 0x7f172c1031b0 2024-04-23T18:23:15.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.585+0000 7f1732ff7640 1 -- 172.21.15.19:0/2121738215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172c1092e0 con 0x7f172c1031b0 2024-04-23T18:23:15.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.585+0000 7f1729ffb640 1 -- 172.21.15.19:0/2121738215 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f171c004e10 con 0x7f172c1031b0 2024-04-23T18:23:15.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.586+0000 7f1729ffb640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f17180a3bb0 0x7f17180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:15.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.586+0000 7f1730d93640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f17180a3bb0 0x7f17180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:15.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.586+0000 7f1729ffb640 1 -- 172.21.15.19:0/2121738215 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f171c0cf970 con 0x7f172c1031b0 2024-04-23T18:23:15.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.586+0000 7f1730d93640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f17180a3bb0 0x7f17180a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f172c103010 tx=0x7f172400a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:15.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.591+0000 7f1729ffb640 1 -- 172.21.15.19:0/2121738215 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f171c09c1a0 con 0x7f172c1031b0 2024-04-23T18:23:15.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:15 smithi019 ceph-mon[29215]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:23:15.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:15 smithi019 ceph-mon[29215]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-04-23T18:23:15.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:15.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:15.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:15.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:15.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:15.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:15.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:15.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:15 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:15.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.778+0000 7f1732ff7640 1 -- 172.21.15.19:0/2121738215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f172c02d770 con 0x7f172c1031b0 2024-04-23T18:23:15.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.779+0000 7f1729ffb640 1 -- 172.21.15.19:0/2121738215 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18446 (secure 0 0 0) 0x7f171c0a04a0 con 0x7f172c1031b0 2024-04-23T18:23:15.772 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:15.772 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:23:03.173026+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:23:15.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 -- 172.21.15.19:0/2121738215 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f17180a3bb0 msgr2=0x7f17180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:15.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f17180a3bb0 0x7f17180a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f172c103010 tx=0x7f172400a590 comp rx=0 tx=0).stop 2024-04-23T18:23:15.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 -- 172.21.15.19:0/2121738215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c1031b0 msgr2=0x7f172c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:15.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c1031b0 0x7f172c1986d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f171c00d770 tx=0x7f171c00dc30 comp rx=0 tx=0).stop 2024-04-23T18:23:15.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 -- 172.21.15.19:0/2121738215 shutdown_connections 2024-04-23T18:23:15.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f17180a3bb0 0x7f17180a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c103b70 0x7f172c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c1031b0 0x7f172c1986d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 --2- 172.21.15.19:0/2121738215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f172c101fc0 0x7f172c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 -- 172.21.15.19:0/2121738215 >> 172.21.15.19:0/2121738215 conn(0x7f172c0fd550 msgr2=0x7f172c1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:15.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 -- 172.21.15.19:0/2121738215 shutdown_connections 2024-04-23T18:23:15.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.781+0000 7f1732ff7640 1 -- 172.21.15.19:0/2121738215 wait complete. 2024-04-23T18:23:15.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:23:15.786 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:23:15.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:15 smithi155 ceph-mon[33468]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:23:15.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:15 smithi155 ceph-mon[33468]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-04-23T18:23:15.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:15.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:15.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:15.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:15.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:15.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:15.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:15.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:15 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.938+0000 7f3ec66fd640 1 -- 172.21.15.19:0/497623770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec0102a00 msgr2=0x7f3ec0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.938+0000 7f3ec66fd640 1 --2- 172.21.15.19:0/497623770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec0102a00 0x7f3ec0109320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009ca0 tx=0x7f3eb802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.938+0000 7f3ec66fd640 1 -- 172.21.15.19:0/497623770 shutdown_connections 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.938+0000 7f3ec66fd640 1 --2- 172.21.15.19:0/497623770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec0102a00 0x7f3ec0109320 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.938+0000 7f3ec66fd640 1 --2- 172.21.15.19:0/497623770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ec0102040 0x7f3ec01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.938+0000 7f3ec66fd640 1 --2- 172.21.15.19:0/497623770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec0100e60 0x7f3ec0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.938+0000 7f3ec66fd640 1 -- 172.21.15.19:0/497623770 >> 172.21.15.19:0/497623770 conn(0x7f3ec00fc440 msgr2=0x7f3ec00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.938+0000 7f3ec66fd640 1 -- 172.21.15.19:0/497623770 shutdown_connections 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.938+0000 7f3ec66fd640 1 -- 172.21.15.19:0/497623770 wait complete. 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec66fd640 1 Processor -- start 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec66fd640 1 -- start start 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec66fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec0100e60 0x7f3ec0198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec66fd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ec0102040 0x7f3ec0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec66fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec0102a00 0x7f3ec019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec66fd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec010ca00 con 0x7f3ec0102a00 2024-04-23T18:23:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec66fd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec010c880 con 0x7f3ec0100e60 2024-04-23T18:23:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec66fd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec010cb80 con 0x7f3ec0102040 2024-04-23T18:23:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec4c9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec0102a00 0x7f3ec019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ebf7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ec0102040 0x7f3ec0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec4c9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec0102a00 0x7f3ec019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39204/0 (socket says 172.21.15.19:39204) 2024-04-23T18:23:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ec4c9a640 1 -- 172.21.15.19:0/1136287671 learned_addr learned my addr 172.21.15.19:0/1136287671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.939+0000 7f3ebffff640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec0100e60 0x7f3ec0198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ec4c9a640 1 -- 172.21.15.19:0/1136287671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ec0102040 msgr2=0x7f3ec0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ec4c9a640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ec0102040 0x7f3ec0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ec4c9a640 1 -- 172.21.15.19:0/1136287671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec0100e60 msgr2=0x7f3ec0198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ec4c9a640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec0100e60 0x7f3ec0198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ec4c9a640 1 -- 172.21.15.19:0/1136287671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec019fdd0 con 0x7f3ec0102a00 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ebf7fe640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ec0102040 0x7f3ec0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ec4c9a640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec0102a00 0x7f3ec019f760 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80097b0 tx=0x7f3eb8031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ebd7fa640 1 -- 172.21.15.19:0/1136287671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8003bf0 con 0x7f3ec0102a00 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ebd7fa640 1 -- 172.21.15.19:0/1136287671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb8003d90 con 0x7f3ec0102a00 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ec66fd640 1 -- 172.21.15.19:0/1136287671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec01a0060 con 0x7f3ec0102a00 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ebd7fa640 1 -- 172.21.15.19:0/1136287671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb802fc70 con 0x7f3ec0102a00 2024-04-23T18:23:15.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.940+0000 7f3ec66fd640 1 -- 172.21.15.19:0/1136287671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec01a05b0 con 0x7f3ec0102a00 2024-04-23T18:23:15.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.941+0000 7f3ec66fd640 1 -- 172.21.15.19:0/1136287671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec01a0810 con 0x7f3ec0102a00 2024-04-23T18:23:15.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.942+0000 7f3ebd7fa640 1 -- 172.21.15.19:0/1136287671 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3eb8004040 con 0x7f3ec0102a00 2024-04-23T18:23:15.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.943+0000 7f3ebd7fa640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3eac0a3bb0 0x7f3eac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:15.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.943+0000 7f3ebffff640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3eac0a3bb0 0x7f3eac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.943+0000 7f3ebd7fa640 1 -- 172.21.15.19:0/1136287671 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3eb80f1590 con 0x7f3ec0102a00 2024-04-23T18:23:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.943+0000 7f3ebffff640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3eac0a3bb0 0x7f3eac0a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8005fd0 tx=0x7f3ea800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:15.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:15.948+0000 7f3ebd7fa640 1 -- 172.21.15.19:0/1136287671 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3eb80be370 con 0x7f3ec0102a00 2024-04-23T18:23:16.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.134+0000 7f3ec66fd640 1 -- 172.21.15.19:0/1136287671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ec01a0ba0 con 0x7f3ec0102a00 2024-04-23T18:23:16.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.134+0000 7f3ebd7fa640 1 -- 172.21.15.19:0/1136287671 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f3eb80bdc70 con 0x7f3ec0102a00 2024-04-23T18:23:16.127 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:23:16.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 -- 172.21.15.19:0/1136287671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3eac0a3bb0 msgr2=0x7f3eac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:16.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3eac0a3bb0 0x7f3eac0a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8005fd0 tx=0x7f3ea800b410 comp rx=0 tx=0).stop 2024-04-23T18:23:16.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 -- 172.21.15.19:0/1136287671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec0102a00 msgr2=0x7f3ec019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:16.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec0102a00 0x7f3ec019f760 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80097b0 tx=0x7f3eb8031840 comp rx=0 tx=0).stop 2024-04-23T18:23:16.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 -- 172.21.15.19:0/1136287671 shutdown_connections 2024-04-23T18:23:16.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3eac0a3bb0 0x7f3eac0a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec0102a00 0x7f3ec019f760 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ec0102040 0x7f3ec0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 --2- 172.21.15.19:0/1136287671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec0100e60 0x7f3ec0198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 -- 172.21.15.19:0/1136287671 >> 172.21.15.19:0/1136287671 conn(0x7f3ec00fc440 msgr2=0x7f3ec00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:16.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 -- 172.21.15.19:0/1136287671 shutdown_connections 2024-04-23T18:23:16.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.136+0000 7f3ec66fd640 1 -- 172.21.15.19:0/1136287671 wait complete. 2024-04-23T18:23:16.140 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 -- 172.21.15.19:0/2230208210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b8103b50 msgr2=0x7fa9b810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 --2- 172.21.15.19:0/2230208210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b8103b50 0x7fa9b810a3e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009ca0 tx=0x7fa9b002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 -- 172.21.15.19:0/2230208210 shutdown_connections 2024-04-23T18:23:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 --2- 172.21.15.19:0/2230208210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b8103190 0x7fa9b8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 --2- 172.21.15.19:0/2230208210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b8103b50 0x7fa9b810a3e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 --2- 172.21.15.19:0/2230208210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b8101fa0 0x7fa9b81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 -- 172.21.15.19:0/2230208210 >> 172.21.15.19:0/2230208210 conn(0x7fa9b80fd530 msgr2=0x7fa9b80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 -- 172.21.15.19:0/2230208210 shutdown_connections 2024-04-23T18:23:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 -- 172.21.15.19:0/2230208210 wait complete. 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 Processor -- start 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 -- start start 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b8101fa0 0x7fa9b8198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b8103190 0x7fa9b8198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b8103b50 0x7fa9b819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b810dae0 con 0x7fa9b8103b50 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b810d960 con 0x7fa9b8101fa0 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.250+0000 7fa9c0613640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b810dc60 con 0x7fa9b8103190 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9bebb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b8103b50 0x7fa9b819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9bebb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b8103b50 0x7fa9b819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39214/0 (socket says 172.21.15.19:39214) 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9bebb0640 1 -- 172.21.15.19:0/3318777918 learned_addr learned my addr 172.21.15.19:0/3318777918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9be3af640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b8101fa0 0x7fa9b8198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9bdbae640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b8103190 0x7fa9b8198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9bebb0640 1 -- 172.21.15.19:0/3318777918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b8103190 msgr2=0x7fa9b8198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9bebb0640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b8103190 0x7fa9b8198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9bebb0640 1 -- 172.21.15.19:0/3318777918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b8101fa0 msgr2=0x7fa9b8198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9bebb0640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b8101fa0 0x7fa9b8198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9bebb0640 1 -- 172.21.15.19:0/3318777918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b819fd60 con 0x7fa9b8103b50 2024-04-23T18:23:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9bebb0640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b8103b50 0x7fa9b819f6f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00097b0 tx=0x7fa9b0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9ab7fe640 1 -- 172.21.15.19:0/3318777918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0004240 con 0x7fa9b8103b50 2024-04-23T18:23:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9ab7fe640 1 -- 172.21.15.19:0/3318777918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b00317c0 con 0x7fa9b8103b50 2024-04-23T18:23:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9c0613640 1 -- 172.21.15.19:0/3318777918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b819fff0 con 0x7fa9b8103b50 2024-04-23T18:23:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9ab7fe640 1 -- 172.21.15.19:0/3318777918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b002fd80 con 0x7fa9b8103b50 2024-04-23T18:23:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.251+0000 7fa9c0613640 1 -- 172.21.15.19:0/3318777918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b81a04b0 con 0x7fa9b8103b50 2024-04-23T18:23:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.252+0000 7fa9c0613640 1 -- 172.21.15.19:0/3318777918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b8109340 con 0x7fa9b8103b50 2024-04-23T18:23:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.254+0000 7fa9ab7fe640 1 -- 172.21.15.19:0/3318777918 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa9b00043e0 con 0x7fa9b8103b50 2024-04-23T18:23:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.255+0000 7fa9ab7fe640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa9a40a3bb0 0x7fa9a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.255+0000 7fa9be3af640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa9a40a3bb0 0x7fa9a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.255+0000 7fa9ab7fe640 1 -- 172.21.15.19:0/3318777918 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa9b00f1b60 con 0x7fa9b8103b50 2024-04-23T18:23:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.255+0000 7fa9be3af640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa9a40a3bb0 0x7fa9a40a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005fd0 tx=0x7fa9b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.258+0000 7fa9ab7fe640 1 -- 172.21.15.19:0/3318777918 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9b00bea60 con 0x7fa9b8103b50 2024-04-23T18:23:16.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.485+0000 7fa9c0613640 1 -- 172.21.15.19:0/3318777918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9b802d5c0 con 0x7fa9b8103b50 2024-04-23T18:23:16.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.485+0000 7fa9ab7fe640 1 -- 172.21.15.19:0/3318777918 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fa9b00be360 con 0x7fa9b8103b50 2024-04-23T18:23:16.479 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:16.479 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-23T18:23:16.479 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 1 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 -- 172.21.15.19:0/3318777918 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa9a40a3bb0 msgr2=0x7fa9a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa9a40a3bb0 0x7fa9a40a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005fd0 tx=0x7fa9b400b390 comp rx=0 tx=0).stop 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 -- 172.21.15.19:0/3318777918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b8103b50 msgr2=0x7fa9b819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b8103b50 0x7fa9b819f6f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00097b0 tx=0x7fa9b0003c20 comp rx=0 tx=0).stop 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 -- 172.21.15.19:0/3318777918 shutdown_connections 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa9a40a3bb0 0x7fa9a40a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b8103b50 0x7fa9b819f6f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b8103190 0x7fa9b8198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 --2- 172.21.15.19:0/3318777918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b8101fa0 0x7fa9b8198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 -- 172.21.15.19:0/3318777918 >> 172.21.15.19:0/3318777918 conn(0x7fa9b80fd530 msgr2=0x7fa9b8106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 -- 172.21.15.19:0/3318777918 shutdown_connections 2024-04-23T18:23:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:16.487+0000 7fa9c0613640 1 -- 172.21.15.19:0/3318777918 wait complete. 2024-04-23T18:23:16.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:16 smithi019 ceph-mon[29215]: Deploying daemon mds.l on smithi155 2024-04-23T18:23:16.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:16 smithi019 ceph-mon[29215]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:16.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:16 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2121738215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:16.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:16 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1136287671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:16.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:16 smithi044 ceph-mon[32458]: Deploying daemon mds.l on smithi155 2024-04-23T18:23:16.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:16 smithi044 ceph-mon[32458]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:16.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:16 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2121738215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:16.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:16 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1136287671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:16.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:16 smithi155 ceph-mon[33468]: Deploying daemon mds.l on smithi155 2024-04-23T18:23:16.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:16 smithi155 ceph-mon[33468]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:16.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:16 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2121738215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:16.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:16 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1136287671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:17.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:17 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3318777918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:17.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:17 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3318777918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:17.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:17 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3318777918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:17.855 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:23:17 smithi155 podman[62985]: 2024-04-23 18:23:17.839453867 +0000 UTC m=+0.364676953 container start c234d349ecda48453c437a2c1a9fbdd1ab54dfb5367500597b2ed31b8ece234d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True) 2024-04-23T18:23:17.855 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:23:17 smithi155 ceph-mds[63004]: -- 172.21.15.155:0/3852372088 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b7bee7a40 con 0x558b7bebac00 2024-04-23T18:23:18.323 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:23:17 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l[63000]: starting mds.l at 2024-04-23T18:23:18.323 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:23:17 smithi155 bash[62985]: c234d349ecda48453c437a2c1a9fbdd1ab54dfb5367500597b2ed31b8ece234d 2024-04-23T18:23:18.323 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:23:17 smithi155 systemd[1]: Started Ceph mds.l for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:18.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:18.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.613+0000 7fdf143ed640 1 -- 172.21.15.19:0/4228891381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c1053c0 msgr2=0x7fdf0c1077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:18.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.613+0000 7fdf143ed640 1 --2- 172.21.15.19:0/4228891381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c1053c0 0x7fdf0c1077b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009900 tx=0x7fdefc02a410 comp rx=0 tx=0).stop 2024-04-23T18:23:18.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.613+0000 7fdf143ed640 1 -- 172.21.15.19:0/4228891381 shutdown_connections 2024-04-23T18:23:18.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.613+0000 7fdf143ed640 1 --2- 172.21.15.19:0/4228891381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf0c107cf0 0x7fdf0c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:18.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.613+0000 7fdf143ed640 1 --2- 172.21.15.19:0/4228891381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c1053c0 0x7fdf0c1077b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:18.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.613+0000 7fdf143ed640 1 --2- 172.21.15.19:0/4228891381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf0c102a90 0x7fdf0c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:18.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.613+0000 7fdf143ed640 1 -- 172.21.15.19:0/4228891381 >> 172.21.15.19:0/4228891381 conn(0x7fdf0c0fc440 msgr2=0x7fdf0c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.613+0000 7fdf143ed640 1 -- 172.21.15.19:0/4228891381 shutdown_connections 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.613+0000 7fdf143ed640 1 -- 172.21.15.19:0/4228891381 wait complete. 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf143ed640 1 Processor -- start 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf143ed640 1 -- start start 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf143ed640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf0c102a90 0x7fdf0c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf143ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c1053c0 0x7fdf0c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf143ed640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf0c107cf0 0x7fdf0c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf143ed640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10daf0 con 0x7fdf0c1053c0 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf143ed640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10d970 con 0x7fdf0c107cf0 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf143ed640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10dc70 con 0x7fdf0c102a90 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf11988640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c1053c0 0x7fdf0c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf11988640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c1053c0 0x7fdf0c198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39232/0 (socket says 172.21.15.19:39232) 2024-04-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf11988640 1 -- 172.21.15.19:0/2960561464 learned_addr learned my addr 172.21.15.19:0/2960561464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf12189640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf0c102a90 0x7fdf0c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.614+0000 7fdf1298a640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf0c107cf0 0x7fdf0c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf11988640 1 -- 172.21.15.19:0/2960561464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf0c102a90 msgr2=0x7fdf0c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf11988640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf0c102a90 0x7fdf0c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf11988640 1 -- 172.21.15.19:0/2960561464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf0c107cf0 msgr2=0x7fdf0c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf11988640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf0c107cf0 0x7fdf0c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf11988640 1 -- 172.21.15.19:0/2960561464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c19fe60 con 0x7fdf0c1053c0 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf1298a640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf0c107cf0 0x7fdf0c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf12189640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf0c102a90 0x7fdf0c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf11988640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c1053c0 0x7fdf0c198780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdefc006010 tx=0x7fdefc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf037fe640 1 -- 172.21.15.19:0/2960561464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc03f030 con 0x7fdf0c1053c0 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf143ed640 1 -- 172.21.15.19:0/2960561464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c1a00f0 con 0x7fdf0c1053c0 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf037fe640 1 -- 172.21.15.19:0/2960561464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdefc031910 con 0x7fdf0c1053c0 2024-04-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf037fe640 1 -- 172.21.15.19:0/2960561464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc035070 con 0x7fdf0c1053c0 2024-04-23T18:23:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.615+0000 7fdf143ed640 1 -- 172.21.15.19:0/2960561464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c1a05e0 con 0x7fdf0c1053c0 2024-04-23T18:23:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.616+0000 7fdf143ed640 1 -- 172.21.15.19:0/2960561464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c0fdf30 con 0x7fdf0c1053c0 2024-04-23T18:23:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.618+0000 7fdf037fe640 1 -- 172.21.15.19:0/2960561464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdefc042030 con 0x7fdf0c1053c0 2024-04-23T18:23:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.618+0000 7fdf037fe640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdef80a3bb0 0x7fdef80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.619+0000 7fdf12189640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdef80a3bb0 0x7fdef80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.619+0000 7fdf037fe640 1 -- 172.21.15.19:0/2960561464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdefc0f0840 con 0x7fdf0c1053c0 2024-04-23T18:23:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.619+0000 7fdf12189640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdef80a3bb0 0x7fdef80a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdf080095f0 tx=0x7fdf08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:18.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.623+0000 7fdf037fe640 1 -- 172.21.15.19:0/2960561464 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdefc0bd070 con 0x7fdf0c1053c0 2024-04-23T18:23:18.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.848+0000 7fdf143ed640 1 -- 172.21.15.19:0/2960561464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf0c02d5c0 con 0x7fdf0c1053c0 2024-04-23T18:23:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.849+0000 7fdf037fe640 1 -- 172.21.15.19:0/2960561464 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fdefc0c1370 con 0x7fdf0c1053c0 2024-04-23T18:23:18.842 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:18.842 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T18:23:18.843 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 2 2024-04-23T18:23:18.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 -- 172.21.15.19:0/2960561464 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdef80a3bb0 msgr2=0x7fdef80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdef80a3bb0 0x7fdef80a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdf080095f0 tx=0x7fdf08005f90 comp rx=0 tx=0).stop 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 -- 172.21.15.19:0/2960561464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c1053c0 msgr2=0x7fdf0c198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c1053c0 0x7fdf0c198780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdefc006010 tx=0x7fdefc0042f0 comp rx=0 tx=0).stop 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 -- 172.21.15.19:0/2960561464 shutdown_connections 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdef80a3bb0 0x7fdef80a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf0c107cf0 0x7fdf0c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c1053c0 0x7fdf0c198780 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 --2- 172.21.15.19:0/2960561464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf0c102a90 0x7fdf0c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 -- 172.21.15.19:0/2960561464 >> 172.21.15.19:0/2960561464 conn(0x7fdf0c0fc440 msgr2=0x7fdf0c103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 -- 172.21.15.19:0/2960561464 shutdown_connections 2024-04-23T18:23:18.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:18.851+0000 7fdf143ed640 1 -- 172.21.15.19:0/2960561464 wait complete. 2024-04-23T18:23:18.971 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:18.971 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6832/3128313210,v1:172.21.15.155:6833/3128313210] up:boot 2024-04-23T18:23:18.971 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: fsmap 1 up:standby 2024-04-23T18:23:18.971 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:23:18.972 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:18.972 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:18.972 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:18.972 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:18.972 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:18.972 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:18.972 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: Deploying daemon mds.k on smithi044 2024-04-23T18:23:18.972 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:18 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2960561464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:19.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:19.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6832/3128313210,v1:172.21.15.155:6833/3128313210] up:boot 2024-04-23T18:23:19.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: fsmap 1 up:standby 2024-04-23T18:23:19.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:23:19.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:19.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:19.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:19.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:19.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:19.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:19.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: Deploying daemon mds.k on smithi044 2024-04-23T18:23:19.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:18 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2960561464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6832/3128313210,v1:172.21.15.155:6833/3128313210] up:boot 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: fsmap 1 up:standby 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: Deploying daemon mds.k on smithi044 2024-04-23T18:23:19.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:18 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2960561464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:20.313 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:23:20 smithi044 systemd[1]: Starting Ceph mds.k for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:20.634 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:23:20 smithi044 podman[61867]: 2024-04-23 18:23:20.265811933 +0000 UTC m=+0.018221173 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:20.634 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:23:20 smithi044 podman[61867]: 2024-04-23 18:23:20.371641776 +0000 UTC m=+0.124051036 container create 3fbb3b37d5decb19357278bceb1f7ddc170f4a7168427d0f5ba5368686427265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T18:23:20.634 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:23:20 smithi044 podman[61867]: 2024-04-23 18:23:20.64133024 +0000 UTC m=+0.393739455 container init 3fbb3b37d5decb19357278bceb1f7ddc170f4a7168427d0f5ba5368686427265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-23T18:23:20.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.978+0000 7fcb242cb640 1 -- 172.21.15.19:0/2180626972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c100860 msgr2=0x7fcb1c100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.978+0000 7fcb242cb640 1 --2- 172.21.15.19:0/2180626972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c100860 0x7fcb1c100c80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009ca0 tx=0x7fcb1802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.978+0000 7fcb242cb640 1 -- 172.21.15.19:0/2180626972 shutdown_connections 2024-04-23T18:23:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.978+0000 7fcb242cb640 1 --2- 172.21.15.19:0/2180626972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1c101b60 0x7fcb1c101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.978+0000 7fcb242cb640 1 --2- 172.21.15.19:0/2180626972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb1c1011c0 0x7fcb1c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.978+0000 7fcb242cb640 1 --2- 172.21.15.19:0/2180626972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c100860 0x7fcb1c100c80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.978+0000 7fcb242cb640 1 -- 172.21.15.19:0/2180626972 >> 172.21.15.19:0/2180626972 conn(0x7fcb1c0fc400 msgr2=0x7fcb1c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.978+0000 7fcb242cb640 1 -- 172.21.15.19:0/2180626972 shutdown_connections 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.978+0000 7fcb242cb640 1 -- 172.21.15.19:0/2180626972 wait complete. 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.978+0000 7fcb242cb640 1 Processor -- start 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb242cb640 1 -- start start 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb242cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1c100860 0x7fcb1c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb242cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c1011c0 0x7fcb1c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb242cb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb1c101b60 0x7fcb1c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb242cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c10fcb0 con 0x7fcb1c1011c0 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb242cb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c10fb30 con 0x7fcb1c100860 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb242cb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c10fe30 con 0x7fcb1c101b60 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb21866640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c1011c0 0x7fcb1c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb21866640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c1011c0 0x7fcb1c19a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44382/0 (socket says 172.21.15.19:44382) 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb21866640 1 -- 172.21.15.19:0/2548558847 learned_addr learned my addr 172.21.15.19:0/2548558847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb22067640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1c100860 0x7fcb1c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb22868640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb1c101b60 0x7fcb1c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb21866640 1 -- 172.21.15.19:0/2548558847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb1c101b60 msgr2=0x7fcb1c1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb21866640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb1c101b60 0x7fcb1c1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb21866640 1 -- 172.21.15.19:0/2548558847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1c100860 msgr2=0x7fcb1c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb21866640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1c100860 0x7fcb1c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb21866640 1 -- 172.21.15.19:0/2548558847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c1a1f90 con 0x7fcb1c1011c0 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb22067640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1c100860 0x7fcb1c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.979+0000 7fcb21866640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c1011c0 0x7fcb1c19a8b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00c920 tx=0x7fcb0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.980+0000 7fcb137fe640 1 -- 172.21.15.19:0/2548558847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c007c70 con 0x7fcb1c1011c0 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.980+0000 7fcb242cb640 1 -- 172.21.15.19:0/2548558847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c1a2280 con 0x7fcb1c1011c0 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.980+0000 7fcb137fe640 1 -- 172.21.15.19:0/2548558847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb0c007e10 con 0x7fcb1c1011c0 2024-04-23T18:23:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.980+0000 7fcb137fe640 1 -- 172.21.15.19:0/2548558847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c018820 con 0x7fcb1c1011c0 2024-04-23T18:23:20.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.980+0000 7fcb242cb640 1 -- 172.21.15.19:0/2548558847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c1a2820 con 0x7fcb1c1011c0 2024-04-23T18:23:20.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.981+0000 7fcb242cb640 1 -- 172.21.15.19:0/2548558847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1c10b7e0 con 0x7fcb1c1011c0 2024-04-23T18:23:20.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.982+0000 7fcb137fe640 1 -- 172.21.15.19:0/2548558847 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcb0c00f450 con 0x7fcb1c1011c0 2024-04-23T18:23:20.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.982+0000 7fcb137fe640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb080a3bb0 0x7fcb080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:20.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.983+0000 7fcb22067640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb080a3bb0 0x7fcb080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:20.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.983+0000 7fcb137fe640 1 -- 172.21.15.19:0/2548558847 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcb0c0d0110 con 0x7fcb1c1011c0 2024-04-23T18:23:20.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.983+0000 7fcb22067640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb080a3bb0 0x7fcb080a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb18033040 tx=0x7fcb18032fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:20.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:20.987+0000 7fcb137fe640 1 -- 172.21.15.19:0/2548558847 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb0c014030 con 0x7fcb1c1011c0 2024-04-23T18:23:21.062 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:23:20 smithi044 podman[61867]: 2024-04-23 18:23:20.644296842 +0000 UTC m=+0.396706066 container start 3fbb3b37d5decb19357278bceb1f7ddc170f4a7168427d0f5ba5368686427265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-23T18:23:21.063 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:23:20 smithi044 ceph-mds[61886]: -- 172.21.15.44:0/174698594 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55910e305a40 con 0x55910e2d8800 2024-04-23T18:23:21.063 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:23:20 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k[61882]: starting mds.k at 2024-04-23T18:23:21.063 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:23:20 smithi044 bash[61867]: 3fbb3b37d5decb19357278bceb1f7ddc170f4a7168427d0f5ba5368686427265 2024-04-23T18:23:21.063 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:23:20 smithi044 systemd[1]: Started Ceph mds.k for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:20 smithi044 ceph-mon[32458]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:21.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:21.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:21.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.216+0000 7fcb242cb640 1 -- 172.21.15.19:0/2548558847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb1c02d5c0 con 0x7fcb1c1011c0 2024-04-23T18:23:21.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.216+0000 7fcb137fe640 1 -- 172.21.15.19:0/2548558847 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fcb0c09cef0 con 0x7fcb1c1011c0 2024-04-23T18:23:21.209 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:21.210 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:23:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-04-23T18:23:21.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 -- 172.21.15.19:0/2548558847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb080a3bb0 msgr2=0x7fcb080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:21.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb080a3bb0 0x7fcb080a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb18033040 tx=0x7fcb18032fb0 comp rx=0 tx=0).stop 2024-04-23T18:23:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 -- 172.21.15.19:0/2548558847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c1011c0 msgr2=0x7fcb1c19a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c1011c0 0x7fcb1c19a8b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00c920 tx=0x7fcb0c00cde0 comp rx=0 tx=0).stop 2024-04-23T18:23:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 -- 172.21.15.19:0/2548558847 shutdown_connections 2024-04-23T18:23:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb080a3bb0 0x7fcb080a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb1c101b60 0x7fcb1c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c1011c0 0x7fcb1c19a8b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 --2- 172.21.15.19:0/2548558847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1c100860 0x7fcb1c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 -- 172.21.15.19:0/2548558847 >> 172.21.15.19:0/2548558847 conn(0x7fcb1c0fc400 msgr2=0x7fcb1c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 -- 172.21.15.19:0/2548558847 shutdown_connections 2024-04-23T18:23:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:21.218+0000 7fcb242cb640 1 -- 172.21.15.19:0/2548558847 wait complete. 2024-04-23T18:23:21.221 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:20 smithi019 ceph-mon[29215]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:21.221 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:21.221 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:21.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:21.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:21.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:21.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:21.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:20 smithi155 ceph-mon[33468]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:21.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:21.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:21.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:21.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:21.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:21.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:22.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:21 smithi019 ceph-mon[29215]: Deploying daemon mds.j on smithi019 2024-04-23T18:23:22.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:21 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6832/3545031610,v1:172.21.15.44:6833/3545031610] up:boot 2024-04-23T18:23:22.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:21 smithi019 ceph-mon[29215]: fsmap 2 up:standby 2024-04-23T18:23:22.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:21 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:23:22.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:21 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2548558847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:22.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:21 smithi044 ceph-mon[32458]: Deploying daemon mds.j on smithi019 2024-04-23T18:23:22.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:21 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6832/3545031610,v1:172.21.15.44:6833/3545031610] up:boot 2024-04-23T18:23:22.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:21 smithi044 ceph-mon[32458]: fsmap 2 up:standby 2024-04-23T18:23:22.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:21 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:23:22.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:21 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2548558847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:22.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:21 smithi155 ceph-mon[33468]: Deploying daemon mds.j on smithi019 2024-04-23T18:23:22.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:21 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6832/3545031610,v1:172.21.15.44:6833/3545031610] up:boot 2024-04-23T18:23:22.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:21 smithi155 ceph-mon[33468]: fsmap 2 up:standby 2024-04-23T18:23:22.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:21 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:23:22.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:21 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2548558847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:23.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:22 smithi044 ceph-mon[32458]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:23.140 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:22 smithi019 ceph-mon[29215]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:23.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:23.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:22 smithi155 ceph-mon[33468]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.344+0000 7f0ef71ad640 1 -- 172.21.15.19:0/2673097761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0103190 msgr2=0x7f0ef0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.344+0000 7f0ef71ad640 1 --2- 172.21.15.19:0/2673097761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0103190 0x7f0ef0103610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009900 tx=0x7f0edc02a410 comp rx=0 tx=0).stop 2024-04-23T18:23:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.344+0000 7f0ef71ad640 1 -- 172.21.15.19:0/2673097761 shutdown_connections 2024-04-23T18:23:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.344+0000 7f0ef71ad640 1 --2- 172.21.15.19:0/2673097761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103b50 0x7f0ef010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.344+0000 7f0ef71ad640 1 --2- 172.21.15.19:0/2673097761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0103190 0x7f0ef0103610 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.344+0000 7f0ef71ad640 1 --2- 172.21.15.19:0/2673097761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef0101fa0 0x7f0ef01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.344+0000 7f0ef71ad640 1 -- 172.21.15.19:0/2673097761 >> 172.21.15.19:0/2673097761 conn(0x7f0ef00fd530 msgr2=0x7f0ef00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.344+0000 7f0ef71ad640 1 -- 172.21.15.19:0/2673097761 shutdown_connections 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.344+0000 7f0ef71ad640 1 -- 172.21.15.19:0/2673097761 wait complete. 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef71ad640 1 Processor -- start 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef71ad640 1 -- start start 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef71ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0101fa0 0x7f0ef0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef71ad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103190 0x7f0ef0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef71ad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef0103b50 0x7f0ef0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef71ad640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010dae0 con 0x7f0ef0101fa0 2024-04-23T18:23:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef71ad640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010d960 con 0x7f0ef0103b50 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef71ad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010dc60 con 0x7f0ef0103190 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef4f49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0101fa0 0x7f0ef0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef4f49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0101fa0 0x7f0ef0075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44388/0 (socket says 172.21.15.19:44388) 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef4f49640 1 -- 172.21.15.19:0/3634424864 learned_addr learned my addr 172.21.15.19:0/3634424864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef574a640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef0103b50 0x7f0ef0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0eeffff640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103190 0x7f0ef0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef4f49640 1 -- 172.21.15.19:0/3634424864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103190 msgr2=0x7f0ef0074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef4f49640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103190 0x7f0ef0074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef4f49640 1 -- 172.21.15.19:0/3634424864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef0103b50 msgr2=0x7f0ef0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef4f49640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef0103b50 0x7f0ef0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0ef4f49640 1 -- 172.21.15.19:0/3634424864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef0075110 con 0x7f0ef0101fa0 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.345+0000 7f0eeffff640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103190 0x7f0ef0074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.346+0000 7f0ef4f49640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0101fa0 0x7f0ef0075d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e9c0 tx=0x7f0ee400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.346+0000 7f0eedffb640 1 -- 172.21.15.19:0/3634424864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400cd80 con 0x7f0ef0101fa0 2024-04-23T18:23:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.346+0000 7f0eedffb640 1 -- 172.21.15.19:0/3634424864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee40035f0 con 0x7f0ef0101fa0 2024-04-23T18:23:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.346+0000 7f0eedffb640 1 -- 172.21.15.19:0/3634424864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4010690 con 0x7f0ef0101fa0 2024-04-23T18:23:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.346+0000 7f0ef71ad640 1 -- 172.21.15.19:0/3634424864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef01a22b0 con 0x7f0ef0101fa0 2024-04-23T18:23:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.346+0000 7f0ef71ad640 1 -- 172.21.15.19:0/3634424864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01a27b0 con 0x7f0ef0101fa0 2024-04-23T18:23:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.347+0000 7f0ef71ad640 1 -- 172.21.15.19:0/3634424864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef0109380 con 0x7f0ef0101fa0 2024-04-23T18:23:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.348+0000 7f0eedffb640 1 -- 172.21.15.19:0/3634424864 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ee40037a0 con 0x7f0ef0101fa0 2024-04-23T18:23:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.348+0000 7f0eedffb640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0ee00a3bb0 0x7f0ee00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.349+0000 7f0eeffff640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0ee00a3bb0 0x7f0ee00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.349+0000 7f0eedffb640 1 -- 172.21.15.19:0/3634424864 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0ee40d03d0 con 0x7f0ef0101fa0 2024-04-23T18:23:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.349+0000 7f0eeffff640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0ee00a3bb0 0x7f0ee00a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0edc006010 tx=0x7f0edc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.353+0000 7f0eedffb640 1 -- 172.21.15.19:0/3634424864 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ee409d250 con 0x7f0ef0101fa0 2024-04-23T18:23:23.392 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:23:23 smithi019 podman[69359]: 2024-04-23 18:23:23.27525757 +0000 UTC m=+0.018992363 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:23.393 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:23:23 smithi019 podman[69359]: 2024-04-23 18:23:23.400216029 +0000 UTC m=+0.143950816 container create ae775ed8da7e113a304a7e9b2bd72bc5d6d759b57c0941b120ac121910330e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:23:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.582+0000 7f0ef71ad640 1 -- 172.21.15.19:0/3634424864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ef002d870 con 0x7f0ef0101fa0 2024-04-23T18:23:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.583+0000 7f0eedffb640 1 -- 172.21.15.19:0/3634424864 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f0ee409cb50 con 0x7f0ef0101fa0 2024-04-23T18:23:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:23.576 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:23:23.577 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-04-23T18:23:23.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 -- 172.21.15.19:0/3634424864 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0ee00a3bb0 msgr2=0x7f0ee00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0ee00a3bb0 0x7f0ee00a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0edc006010 tx=0x7f0edc000bc0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 -- 172.21.15.19:0/3634424864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0101fa0 msgr2=0x7f0ef0075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0101fa0 0x7f0ef0075d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e9c0 tx=0x7f0ee400ee80 comp rx=0 tx=0).stop 2024-04-23T18:23:23.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 -- 172.21.15.19:0/3634424864 shutdown_connections 2024-04-23T18:23:23.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0ee00a3bb0 0x7f0ee00a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef0103b50 0x7f0ef0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103190 0x7f0ef0074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 --2- 172.21.15.19:0/3634424864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0101fa0 0x7f0ef0075d00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 -- 172.21.15.19:0/3634424864 >> 172.21.15.19:0/3634424864 conn(0x7f0ef00fd530 msgr2=0x7f0ef0106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:23.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 -- 172.21.15.19:0/3634424864 shutdown_connections 2024-04-23T18:23:23.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:23.585+0000 7f0ef71ad640 1 -- 172.21.15.19:0/3634424864 wait complete. 2024-04-23T18:23:23.656 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:23:23 smithi019 podman[69359]: 2024-04-23 18:23:23.636313273 +0000 UTC m=+0.380048054 container init ae775ed8da7e113a304a7e9b2bd72bc5d6d759b57c0941b120ac121910330e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-23T18:23:23.656 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:23:23 smithi019 podman[69359]: 2024-04-23 18:23:23.639862154 +0000 UTC m=+0.383596935 container start ae775ed8da7e113a304a7e9b2bd72bc5d6d759b57c0941b120ac121910330e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-23T18:23:24.010 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:23:23 smithi019 ceph-mds[69403]: -- 172.21.15.19:0/4049949843 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559222675a40 con 0x559222648c00 2024-04-23T18:23:24.010 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:23:23 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-j[69397]: starting mds.j at 2024-04-23T18:23:24.010 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:23:23 smithi019 bash[69359]: ae775ed8da7e113a304a7e9b2bd72bc5d6d759b57c0941b120ac121910330e5b 2024-04-23T18:23:24.010 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:23:23 smithi019 systemd[1]: Started Ceph mds.j for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:24.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:23 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3634424864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:24.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:24.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:24.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:24.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:24.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:24.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:23 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:24.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:23 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3634424864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:24.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:24.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:24.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:24.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:24.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:24.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:23 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:24.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:23 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3634424864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:24.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:24.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:24.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:24.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:24.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:24.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:23 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:25.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:24 smithi019 ceph-mon[29215]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:25.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:24 smithi019 ceph-mon[29215]: Deploying daemon mds.d on smithi019 2024-04-23T18:23:25.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:24 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.19:6834/2967585130,v1:172.21.15.19:6835/2967585130] up:boot 2024-04-23T18:23:25.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:24 smithi019 ceph-mon[29215]: fsmap 3 up:standby 2024-04-23T18:23:25.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:24 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:23:25.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:24 smithi044 ceph-mon[32458]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:25.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:24 smithi044 ceph-mon[32458]: Deploying daemon mds.d on smithi019 2024-04-23T18:23:25.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:24 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.19:6834/2967585130,v1:172.21.15.19:6835/2967585130] up:boot 2024-04-23T18:23:25.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:24 smithi044 ceph-mon[32458]: fsmap 3 up:standby 2024-04-23T18:23:25.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:24 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:23:25.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:24 smithi155 ceph-mon[33468]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:25.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:24 smithi155 ceph-mon[33468]: Deploying daemon mds.d on smithi019 2024-04-23T18:23:25.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:24 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.19:6834/2967585130,v1:172.21.15.19:6835/2967585130] up:boot 2024-04-23T18:23:25.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:24 smithi155 ceph-mon[33468]: fsmap 3 up:standby 2024-04-23T18:23:25.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:24 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:23:25.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:25.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.714+0000 7fafa4e13640 1 -- 172.21.15.19:0/2602732566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafa0103b70 msgr2=0x7fafa010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:25.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.714+0000 7fafa4e13640 1 --2- 172.21.15.19:0/2602732566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafa0103b70 0x7fafa010a400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf98009ca0 tx=0x7faf9802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:25.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.715+0000 7fafa4e13640 1 -- 172.21.15.19:0/2602732566 shutdown_connections 2024-04-23T18:23:25.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.715+0000 7fafa4e13640 1 --2- 172.21.15.19:0/2602732566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafa01031b0 0x7fafa0103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:25.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.715+0000 7fafa4e13640 1 --2- 172.21.15.19:0/2602732566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafa0103b70 0x7fafa010a400 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:25.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.715+0000 7fafa4e13640 1 --2- 172.21.15.19:0/2602732566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa0101fc0 0x7fafa01023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.715+0000 7fafa4e13640 1 -- 172.21.15.19:0/2602732566 >> 172.21.15.19:0/2602732566 conn(0x7fafa00fd550 msgr2=0x7fafa00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.715+0000 7fafa4e13640 1 -- 172.21.15.19:0/2602732566 shutdown_connections 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.715+0000 7fafa4e13640 1 -- 172.21.15.19:0/2602732566 wait complete. 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.715+0000 7fafa4e13640 1 Processor -- start 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.715+0000 7fafa4e13640 1 -- start start 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7fafa4e13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafa0101fc0 0x7fafa0198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7fafa4e13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafa01031b0 0x7fafa01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7fafa4e13640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa0103b70 0x7fafa019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7fafa4e13640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafa010db00 con 0x7fafa01031b0 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7fafa4e13640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fafa010d980 con 0x7fafa0101fc0 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7fafa4e13640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafa010dc80 con 0x7fafa0103b70 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafa01031b0 0x7fafa01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafa01031b0 0x7fafa01986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44438/0 (socket says 172.21.15.19:44438) 2024-04-23T18:23:25.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9dd9b640 1 -- 172.21.15.19:0/3310262450 learned_addr learned my addr 172.21.15.19:0/3310262450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9e59c640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafa0101fc0 0x7fafa0198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9dd9b640 1 -- 172.21.15.19:0/3310262450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa0103b70 msgr2=0x7fafa019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9ed9d640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa0103b70 0x7fafa019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9dd9b640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa0103b70 0x7fafa019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9dd9b640 1 -- 172.21.15.19:0/3310262450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafa0101fc0 msgr2=0x7fafa0198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9dd9b640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafa0101fc0 0x7fafa0198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9dd9b640 1 -- 172.21.15.19:0/3310262450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa019fd90 con 0x7fafa01031b0 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf9dd9b640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafa01031b0 0x7fafa01986b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf9000d950 tx=0x7faf9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf977fe640 1 -- 172.21.15.19:0/3310262450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf90017030 con 0x7fafa01031b0 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf977fe640 1 -- 172.21.15.19:0/3310262450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf90004020 con 0x7fafa01031b0 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7faf977fe640 1 -- 172.21.15.19:0/3310262450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf900047a0 con 0x7fafa01031b0 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.716+0000 7fafa4e13640 1 -- 172.21.15.19:0/3310262450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa01a0080 con 0x7fafa01031b0 2024-04-23T18:23:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.717+0000 7fafa4e13640 1 -- 172.21.15.19:0/3310262450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa01a0620 con 0x7fafa01031b0 2024-04-23T18:23:25.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.718+0000 7fafa4e13640 1 -- 172.21.15.19:0/3310262450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa0109380 con 0x7fafa01031b0 2024-04-23T18:23:25.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.719+0000 7faf977fe640 1 -- 172.21.15.19:0/3310262450 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faf90009a30 con 0x7fafa01031b0 2024-04-23T18:23:25.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.720+0000 7faf977fe640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7faf8c0a3c30 0x7faf8c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:25.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.720+0000 7faf9e59c640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7faf8c0a3c30 0x7faf8c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:25.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.720+0000 7faf977fe640 1 -- 172.21.15.19:0/3310262450 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7faf900cf940 con 0x7fafa01031b0 2024-04-23T18:23:25.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.720+0000 7faf9e59c640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7faf8c0a3c30 0x7faf8c0a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fafa0103010 tx=0x7faf88005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:25.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.725+0000 7faf977fe640 1 -- 172.21.15.19:0/3310262450 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf9009c7c0 con 0x7fafa01031b0 2024-04-23T18:23:25.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.974+0000 7fafa4e13640 1 -- 172.21.15.19:0/3310262450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafa002d5c0 con 0x7fafa01031b0 2024-04-23T18:23:25.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.975+0000 7faf977fe640 1 -- 172.21.15.19:0/3310262450 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7faf9009c0c0 con 0x7fafa01031b0 2024-04-23T18:23:25.968 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:25.968 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:23:25.969 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 4 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.978+0000 7fafa4e13640 1 -- 172.21.15.19:0/3310262450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7faf8c0a3c30 msgr2=0x7faf8c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.978+0000 7fafa4e13640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7faf8c0a3c30 0x7faf8c0a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fafa0103010 tx=0x7faf88005f90 comp rx=0 tx=0).stop 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.978+0000 7fafa4e13640 1 -- 172.21.15.19:0/3310262450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafa01031b0 msgr2=0x7fafa01986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.978+0000 7fafa4e13640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafa01031b0 0x7fafa01986b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf9000d950 tx=0x7faf9000de10 comp rx=0 tx=0).stop 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.978+0000 7fafa4e13640 1 -- 172.21.15.19:0/3310262450 shutdown_connections 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.978+0000 7fafa4e13640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7faf8c0a3c30 0x7faf8c0a60f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.978+0000 7fafa4e13640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa0103b70 0x7fafa019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.978+0000 7fafa4e13640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafa01031b0 0x7fafa01986b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.978+0000 7fafa4e13640 1 --2- 172.21.15.19:0/3310262450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafa0101fc0 0x7fafa0198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.978+0000 7fafa4e13640 1 -- 172.21.15.19:0/3310262450 >> 172.21.15.19:0/3310262450 conn(0x7fafa00fd550 msgr2=0x7fafa0106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.979+0000 7fafa4e13640 1 -- 172.21.15.19:0/3310262450 shutdown_connections 2024-04-23T18:23:25.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:25.979+0000 7fafa4e13640 1 -- 172.21.15.19:0/3310262450 wait complete. 2024-04-23T18:23:26.260 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:23:26 smithi019 systemd[1]: Starting Ceph mds.d for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:26.516 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:23:26 smithi019 podman[69736]: 2024-04-23 18:23:26.23008522 +0000 UTC m=+0.018479044 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:26.516 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:23:26 smithi019 podman[69736]: 2024-04-23 18:23:26.365006003 +0000 UTC m=+0.153399807 container create 6ed95b660d4ec6bf9a0c1ed544866481b7e5e7948185e0778761f6cd838bf5b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-23T18:23:26.893 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:23:26 smithi019 podman[69736]: 2024-04-23 18:23:26.64290695 +0000 UTC m=+0.431300755 container init 6ed95b660d4ec6bf9a0c1ed544866481b7e5e7948185e0778761f6cd838bf5b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-04-23T18:23:26.893 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:23:26 smithi019 podman[69736]: 2024-04-23 18:23:26.645845172 +0000 UTC m=+0.434238977 container start 6ed95b660d4ec6bf9a0c1ed544866481b7e5e7948185e0778761f6cd838bf5b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:23:26.893 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:23:26 smithi019 ceph-mds[69755]: -- 172.21.15.19:0/355374339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627474d3a40 con 0x5627474a7000 2024-04-23T18:23:26.894 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:23:26 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-d[69751]: starting mds.d at 2024-04-23T18:23:26.894 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:23:26 smithi019 bash[69736]: 6ed95b660d4ec6bf9a0c1ed544866481b7e5e7948185e0778761f6cd838bf5b6 2024-04-23T18:23:26.894 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:23:26 smithi019 systemd[1]: Started Ceph mds.d for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:27.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:26 smithi019 ceph-mon[29215]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:27.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:26 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3310262450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:27.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:27.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:27.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:27.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:27.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:27.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:26 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:27.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:26 smithi044 ceph-mon[32458]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:27.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:26 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3310262450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:27.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:27.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:27.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:27.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:27.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:27.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:26 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:27.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:26 smithi155 ceph-mon[33468]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:27.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:26 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3310262450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:27.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:27.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:27.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:27.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:27.326 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:27.326 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:26 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:27.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:28.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:27 smithi044 ceph-mon[32458]: Deploying daemon mds.e on smithi044 2024-04-23T18:23:28.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:27 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.19:6836/781445125,v1:172.21.15.19:6837/781445125] up:boot 2024-04-23T18:23:28.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:27 smithi044 ceph-mon[32458]: fsmap 4 up:standby 2024-04-23T18:23:28.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:27 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:23:28.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.111+0000 7fe2661e2640 1 -- 172.21.15.19:0/752911483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260100e90 msgr2=0x7fe2601012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:28.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.111+0000 7fe2661e2640 1 --2- 172.21.15.19:0/752911483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260100e90 0x7fe2601012b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe248009900 tx=0x7fe24802a410 comp rx=0 tx=0).stop 2024-04-23T18:23:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.111+0000 7fe2661e2640 1 -- 172.21.15.19:0/752911483 shutdown_connections 2024-04-23T18:23:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.111+0000 7fe2661e2640 1 --2- 172.21.15.19:0/752911483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2601029a0 0x7fe2601092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.111+0000 7fe2661e2640 1 --2- 172.21.15.19:0/752911483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe260101fe0 0x7fe260102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.111+0000 7fe2661e2640 1 --2- 172.21.15.19:0/752911483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260100e90 0x7fe2601012b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.111+0000 7fe2661e2640 1 -- 172.21.15.19:0/752911483 >> 172.21.15.19:0/752911483 conn(0x7fe2600fc400 msgr2=0x7fe2600fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.111+0000 7fe2661e2640 1 -- 172.21.15.19:0/752911483 shutdown_connections 2024-04-23T18:23:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.111+0000 7fe2661e2640 1 -- 172.21.15.19:0/752911483 wait complete. 2024-04-23T18:23:28.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.112+0000 7fe2661e2640 1 Processor -- start 2024-04-23T18:23:28.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.112+0000 7fe2661e2640 1 -- start start 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.112+0000 7fe2661e2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe260100e90 0x7fe2601981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.112+0000 7fe2661e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260101fe0 0x7fe260198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.112+0000 7fe2661e2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2601029a0 0x7fe26019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.112+0000 7fe2661e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe26010ca10 con 0x7fe260101fe0 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.112+0000 7fe2661e2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe26010c890 con 0x7fe2601029a0 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.112+0000 7fe2661e2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe26010cb90 con 0x7fe260100e90 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.112+0000 7fe25effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260101fe0 0x7fe260198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260101fe0 0x7fe260198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44458/0 (socket says 172.21.15.19:44458) 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25effd640 1 -- 172.21.15.19:0/1924375952 learned_addr learned my addr 172.21.15.19:0/1924375952 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25ffff640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2601029a0 0x7fe26019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25f7fe640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe260100e90 0x7fe2601981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25effd640 1 -- 172.21.15.19:0/1924375952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe260100e90 msgr2=0x7fe2601981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25effd640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe260100e90 0x7fe2601981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25effd640 1 -- 172.21.15.19:0/1924375952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2601029a0 msgr2=0x7fe26019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25effd640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2601029a0 0x7fe26019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25effd640 1 -- 172.21.15.19:0/1924375952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26019fdf0 con 0x7fe260101fe0 2024-04-23T18:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25ffff640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2601029a0 0x7fe26019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25effd640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260101fe0 0x7fe260198710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe25000d950 tx=0x7fe25000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25cff9640 1 -- 172.21.15.19:0/1924375952 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250017020 con 0x7fe260101fe0 2024-04-23T18:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25cff9640 1 -- 172.21.15.19:0/1924375952 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe250009c30 con 0x7fe260101fe0 2024-04-23T18:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe2661e2640 1 -- 172.21.15.19:0/1924375952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2601a00e0 con 0x7fe260101fe0 2024-04-23T18:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe25cff9640 1 -- 172.21.15.19:0/1924375952 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25000f690 con 0x7fe260101fe0 2024-04-23T18:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.113+0000 7fe2661e2640 1 -- 172.21.15.19:0/1924375952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2601a0680 con 0x7fe260101fe0 2024-04-23T18:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.115+0000 7fe2661e2640 1 -- 172.21.15.19:0/1924375952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2601a0940 con 0x7fe260101fe0 2024-04-23T18:23:28.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.116+0000 7fe25cff9640 1 -- 172.21.15.19:0/1924375952 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe25001a020 con 0x7fe260101fe0 2024-04-23T18:23:28.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.116+0000 7fe25cff9640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe24c0a3c30 0x7fe24c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:28.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.116+0000 7fe25f7fe640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe24c0a3c30 0x7fe24c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:28.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.117+0000 7fe25cff9640 1 -- 172.21.15.19:0/1924375952 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe2500cfee0 con 0x7fe260101fe0 2024-04-23T18:23:28.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.117+0000 7fe25f7fe640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe24c0a3c30 0x7fe24c0a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe248000c00 tx=0x7fe2480314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:28.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.121+0000 7fe25cff9640 1 -- 172.21.15.19:0/1924375952 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe250021020 con 0x7fe260101fe0 2024-04-23T18:23:28.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:27 smithi019 ceph-mon[29215]: Deploying daemon mds.e on smithi044 2024-04-23T18:23:28.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:27 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.19:6836/781445125,v1:172.21.15.19:6837/781445125] up:boot 2024-04-23T18:23:28.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:27 smithi019 ceph-mon[29215]: fsmap 4 up:standby 2024-04-23T18:23:28.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:27 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:23:28.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:27 smithi155 ceph-mon[33468]: Deploying daemon mds.e on smithi044 2024-04-23T18:23:28.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:27 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.19:6836/781445125,v1:172.21.15.19:6837/781445125] up:boot 2024-04-23T18:23:28.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:27 smithi155 ceph-mon[33468]: fsmap 4 up:standby 2024-04-23T18:23:28.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:27 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:23:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.351+0000 7fe2661e2640 1 -- 172.21.15.19:0/1924375952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2601a0b10 con 0x7fe260101fe0 2024-04-23T18:23:28.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.352+0000 7fe25cff9640 1 -- 172.21.15.19:0/1924375952 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fe250009de0 con 0x7fe260101fe0 2024-04-23T18:23:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:28.346 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:23:28.347 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 5 2024-04-23T18:23:28.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.354+0000 7fe2661e2640 1 -- 172.21.15.19:0/1924375952 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe24c0a3c30 msgr2=0x7fe24c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.354+0000 7fe2661e2640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe24c0a3c30 0x7fe24c0a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe248000c00 tx=0x7fe2480314a0 comp rx=0 tx=0).stop 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.354+0000 7fe2661e2640 1 -- 172.21.15.19:0/1924375952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260101fe0 msgr2=0x7fe260198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.354+0000 7fe2661e2640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260101fe0 0x7fe260198710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe25000d950 tx=0x7fe25000de10 comp rx=0 tx=0).stop 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.354+0000 7fe2661e2640 1 -- 172.21.15.19:0/1924375952 shutdown_connections 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.354+0000 7fe2661e2640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe24c0a3c30 0x7fe24c0a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.354+0000 7fe2661e2640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2601029a0 0x7fe26019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.354+0000 7fe2661e2640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe260101fe0 0x7fe260198710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.354+0000 7fe2661e2640 1 --2- 172.21.15.19:0/1924375952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe260100e90 0x7fe2601981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.354+0000 7fe2661e2640 1 -- 172.21.15.19:0/1924375952 >> 172.21.15.19:0/1924375952 conn(0x7fe2600fc400 msgr2=0x7fe2600fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.355+0000 7fe2661e2640 1 -- 172.21.15.19:0/1924375952 shutdown_connections 2024-04-23T18:23:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:28.355+0000 7fe2661e2640 1 -- 172.21.15.19:0/1924375952 wait complete. 2024-04-23T18:23:29.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:28 smithi044 ceph-mon[32458]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:29.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:28 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1924375952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:29.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:28 smithi019 ceph-mon[29215]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:29.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:28 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1924375952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:29.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:28 smithi155 ceph-mon[33468]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:29.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:28 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1924375952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:29.563 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:23:29 smithi044 podman[62186]: 2024-04-23 18:23:29.453452632 +0000 UTC m=+0.124769317 container create 17343ea90f62c8fe4d22203ccac4e11fe324f24ba3a227ce13700048d9305ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:23:30.062 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:23:29 smithi044 podman[62186]: 2024-04-23 18:23:29.706688625 +0000 UTC m=+0.378005273 container init 17343ea90f62c8fe4d22203ccac4e11fe324f24ba3a227ce13700048d9305ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-23T18:23:30.063 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:23:29 smithi044 podman[62186]: 2024-04-23 18:23:29.709753469 +0000 UTC m=+0.381070113 container start 17343ea90f62c8fe4d22203ccac4e11fe324f24ba3a227ce13700048d9305ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-23T18:23:30.063 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:23:29 smithi044 ceph-mds[62205]: -- 172.21.15.44:0/2300283082 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b92971ba40 con 0x55b9296ee800 2024-04-23T18:23:30.063 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:23:29 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: starting mds.e at 2024-04-23T18:23:30.063 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:23:29 smithi044 bash[62186]: 17343ea90f62c8fe4d22203ccac4e11fe324f24ba3a227ce13700048d9305ad3 2024-04-23T18:23:30.063 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:23:29 smithi044 systemd[1]: Started Ceph mds.e for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:30.359 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:30.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.483+0000 7f2c679a2640 1 -- 172.21.15.19:0/153226147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c60101fc0 msgr2=0x7f2c601023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:30.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.483+0000 7f2c679a2640 1 --2- 172.21.15.19:0/153226147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c60101fc0 0x7f2c601023e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009900 tx=0x7f2c5c02a410 comp rx=0 tx=0).stop 2024-04-23T18:23:30.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.483+0000 7f2c679a2640 1 -- 172.21.15.19:0/153226147 shutdown_connections 2024-04-23T18:23:30.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.483+0000 7f2c679a2640 1 --2- 172.21.15.19:0/153226147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c601031b0 0x7f2c60103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:30.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.483+0000 7f2c679a2640 1 --2- 172.21.15.19:0/153226147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c60103b70 0x7f2c6010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:30.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.483+0000 7f2c679a2640 1 --2- 172.21.15.19:0/153226147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c60101fc0 0x7f2c601023e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:30.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.483+0000 7f2c679a2640 1 -- 172.21.15.19:0/153226147 >> 172.21.15.19:0/153226147 conn(0x7f2c600fd550 msgr2=0x7f2c600ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:30.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.483+0000 7f2c679a2640 1 -- 172.21.15.19:0/153226147 shutdown_connections 2024-04-23T18:23:30.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.483+0000 7f2c679a2640 1 -- 172.21.15.19:0/153226147 wait complete. 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c679a2640 1 Processor -- start 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c679a2640 1 -- start start 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c679a2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c60101fc0 0x7f2c601981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c679a2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c601031b0 0x7f2c60198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c679a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c60103b70 0x7f2c6019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c679a2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6010dac0 con 0x7f2c60103b70 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c679a2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6010d940 con 0x7f2c601031b0 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c679a2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6010dc40 con 0x7f2c60101fc0 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c65f3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c60103b70 0x7f2c6019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c65f3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c60103b70 0x7f2c6019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54172/0 (socket says 172.21.15.19:54172) 2024-04-23T18:23:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c65f3f640 1 -- 172.21.15.19:0/3996424081 learned_addr learned my addr 172.21.15.19:0/3996424081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c64f3d640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c601031b0 0x7f2c60198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c6573e640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c60101fc0 0x7f2c601981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c65f3f640 1 -- 172.21.15.19:0/3996424081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c60101fc0 msgr2=0x7f2c601981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c65f3f640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c60101fc0 0x7f2c601981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c65f3f640 1 -- 172.21.15.19:0/3996424081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c601031b0 msgr2=0x7f2c60198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c65f3f640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c601031b0 0x7f2c60198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.484+0000 7f2c65f3f640 1 -- 172.21.15.19:0/3996424081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6019fe10 con 0x7f2c60103b70 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.485+0000 7f2c65f3f640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c60103b70 0x7f2c6019f7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800eaa0 tx=0x7f2c5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.485+0000 7f2c567fc640 1 -- 172.21.15.19:0/3996424081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800cdb0 con 0x7f2c60103b70 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.485+0000 7f2c567fc640 1 -- 172.21.15.19:0/3996424081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c58005020 con 0x7f2c60103b70 2024-04-23T18:23:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.485+0000 7f2c679a2640 1 -- 172.21.15.19:0/3996424081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c601a0100 con 0x7f2c60103b70 2024-04-23T18:23:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.485+0000 7f2c567fc640 1 -- 172.21.15.19:0/3996424081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580116d0 con 0x7f2c60103b70 2024-04-23T18:23:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.485+0000 7f2c679a2640 1 -- 172.21.15.19:0/3996424081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c601a0600 con 0x7f2c60103b70 2024-04-23T18:23:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.486+0000 7f2c679a2640 1 -- 172.21.15.19:0/3996424081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c60109360 con 0x7f2c60103b70 2024-04-23T18:23:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.487+0000 7f2c567fc640 1 -- 172.21.15.19:0/3996424081 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2c580051d0 con 0x7f2c60103b70 2024-04-23T18:23:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.488+0000 7f2c567fc640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c4c0a3bb0 0x7f2c4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.488+0000 7f2c6573e640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c4c0a3bb0 0x7f2c4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.488+0000 7f2c567fc640 1 -- 172.21.15.19:0/3996424081 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2c580d04c0 con 0x7f2c60103b70 2024-04-23T18:23:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.489+0000 7f2c6573e640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c4c0a3bb0 0x7f2c4c0a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009230 tx=0x7f2c5c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:30.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.492+0000 7f2c567fc640 1 -- 172.21.15.19:0/3996424081 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c58016030 con 0x7f2c60103b70 2024-04-23T18:23:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.719+0000 7f2c679a2640 1 -- 172.21.15.19:0/3996424081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c6002d5c0 con 0x7f2c60103b70 2024-04-23T18:23:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.719+0000 7f2c567fc640 1 -- 172.21.15.19:0/3996424081 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f2c5809ccd0 con 0x7f2c60103b70 2024-04-23T18:23:30.713 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T18:23:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 6 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.721+0000 7f2c679a2640 1 -- 172.21.15.19:0/3996424081 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c4c0a3bb0 msgr2=0x7f2c4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.721+0000 7f2c679a2640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c4c0a3bb0 0x7f2c4c0a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009230 tx=0x7f2c5c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.721+0000 7f2c679a2640 1 -- 172.21.15.19:0/3996424081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c60103b70 msgr2=0x7f2c6019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.721+0000 7f2c679a2640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c60103b70 0x7f2c6019f7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800eaa0 tx=0x7f2c5800ef60 comp rx=0 tx=0).stop 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.722+0000 7f2c679a2640 1 -- 172.21.15.19:0/3996424081 shutdown_connections 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.722+0000 7f2c679a2640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c4c0a3bb0 0x7f2c4c0a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.722+0000 7f2c679a2640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c601031b0 0x7f2c60198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.722+0000 7f2c679a2640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c60103b70 0x7f2c6019f7a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.722+0000 7f2c679a2640 1 --2- 172.21.15.19:0/3996424081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c60101fc0 0x7f2c601981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.722+0000 7f2c679a2640 1 -- 172.21.15.19:0/3996424081 >> 172.21.15.19:0/3996424081 conn(0x7f2c600fd550 msgr2=0x7f2c60106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.722+0000 7f2c679a2640 1 -- 172.21.15.19:0/3996424081 shutdown_connections 2024-04-23T18:23:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:30.722+0000 7f2c679a2640 1 -- 172.21.15.19:0/3996424081 wait complete. 2024-04-23T18:23:31.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:31.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:31.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:31.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:31.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:31.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:31.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:31.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: Deploying daemon mds.f on smithi155 2024-04-23T18:23:31.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6834/659103876,v1:172.21.15.44:6835/659103876] up:boot 2024-04-23T18:23:31.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: fsmap 5 up:standby 2024-04-23T18:23:31.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:23:31.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:30 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3996424081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:31.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: Deploying daemon mds.f on smithi155 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6834/659103876,v1:172.21.15.44:6835/659103876] up:boot 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: fsmap 5 up:standby 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:23:31.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:30 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3996424081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:31.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: Deploying daemon mds.f on smithi155 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6834/659103876,v1:172.21.15.44:6835/659103876] up:boot 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: fsmap 5 up:standby 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:23:31.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:30 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3996424081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:32.223 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:23:32 smithi155 systemd[1]: Starting Ceph mds.f for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:32.500 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:23:32 smithi155 podman[63293]: 2024-04-23 18:23:32.132952461 +0000 UTC m=+0.018529224 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:32.501 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:23:32 smithi155 podman[63293]: 2024-04-23 18:23:32.263836382 +0000 UTC m=+0.149413126 container create c9f647b7340718d0eafea3939530866e2278232007073c9a985c6dc1d0d97991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-23T18:23:32.501 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:23:32 smithi155 podman[63293]: 2024-04-23 18:23:32.508219825 +0000 UTC m=+0.393796568 container init c9f647b7340718d0eafea3939530866e2278232007073c9a985c6dc1d0d97991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T18:23:32.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:32.822 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:23:32 smithi155 podman[63293]: 2024-04-23 18:23:32.510982564 +0000 UTC m=+0.396559311 container start c9f647b7340718d0eafea3939530866e2278232007073c9a985c6dc1d0d97991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:23:32.823 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:23:32 smithi155 ceph-mds[63312]: -- 172.21.15.155:0/499138316 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c43b17ba40 con 0x55c43b14f000 2024-04-23T18:23:32.823 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:23:32 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f[63308]: starting mds.f at 2024-04-23T18:23:32.823 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:23:32 smithi155 bash[63293]: c9f647b7340718d0eafea3939530866e2278232007073c9a985c6dc1d0d97991 2024-04-23T18:23:32.823 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:23:32 smithi155 systemd[1]: Started Ceph mds.f for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:32.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.848+0000 7fcb608a7640 1 -- 172.21.15.19:0/4255497730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5c107c80 msgr2=0x7fcb5c10a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:32.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.848+0000 7fcb608a7640 1 --2- 172.21.15.19:0/4255497730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5c107c80 0x7fcb5c10a0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009be0 tx=0x7fcb5402a6c0 comp rx=0 tx=0).stop 2024-04-23T18:23:32.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.848+0000 7fcb608a7640 1 -- 172.21.15.19:0/4255497730 shutdown_connections 2024-04-23T18:23:32.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.848+0000 7fcb608a7640 1 --2- 172.21.15.19:0/4255497730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5c107c80 0x7fcb5c10a0d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:32.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.848+0000 7fcb608a7640 1 --2- 172.21.15.19:0/4255497730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb5c105350 0x7fcb5c107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:32.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.848+0000 7fcb608a7640 1 --2- 172.21.15.19:0/4255497730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5c102a20 0x7fcb5c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:32.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.848+0000 7fcb608a7640 1 -- 172.21.15.19:0/4255497730 >> 172.21.15.19:0/4255497730 conn(0x7fcb5c0fc430 msgr2=0x7fcb5c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:32.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.848+0000 7fcb608a7640 1 -- 172.21.15.19:0/4255497730 shutdown_connections 2024-04-23T18:23:32.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.848+0000 7fcb608a7640 1 -- 172.21.15.19:0/4255497730 wait complete. 2024-04-23T18:23:32.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb608a7640 1 Processor -- start 2024-04-23T18:23:32.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb608a7640 1 -- start start 2024-04-23T18:23:32.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb608a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5c102a20 0x7fcb5c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:32.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb608a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb5c105350 0x7fcb5c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:32.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb608a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5c107c80 0x7fcb5c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb608a7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c10dae0 con 0x7fcb5c105350 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb608a7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c10d960 con 0x7fcb5c102a20 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb608a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c10dc60 con 0x7fcb5c107c80 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb5c105350 0x7fcb5c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb5c105350 0x7fcb5c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54186/0 (socket says 172.21.15.19:54186) 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb5ad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5c107c80 0x7fcb5c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb59d9b640 1 -- 172.21.15.19:0/2764235289 learned_addr learned my addr 172.21.15.19:0/2764235289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb5a59c640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5c102a20 0x7fcb5c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb59d9b640 1 -- 172.21.15.19:0/2764235289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5c107c80 msgr2=0x7fcb5c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb59d9b640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5c107c80 0x7fcb5c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb59d9b640 1 -- 172.21.15.19:0/2764235289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5c102a20 msgr2=0x7fcb5c198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb59d9b640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5c102a20 0x7fcb5c198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb59d9b640 1 -- 172.21.15.19:0/2764235289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c19fd50 con 0x7fcb5c105350 2024-04-23T18:23:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.849+0000 7fcb5a59c640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5c102a20 0x7fcb5c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.850+0000 7fcb59d9b640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb5c105350 0x7fcb5c198670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000e9a0 tx=0x7fcb5000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.850+0000 7fcb4b7fe640 1 -- 172.21.15.19:0/2764235289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000cd80 con 0x7fcb5c105350 2024-04-23T18:23:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.850+0000 7fcb4b7fe640 1 -- 172.21.15.19:0/2764235289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb500035f0 con 0x7fcb5c105350 2024-04-23T18:23:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.850+0000 7fcb608a7640 1 -- 172.21.15.19:0/2764235289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c19ffe0 con 0x7fcb5c105350 2024-04-23T18:23:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.850+0000 7fcb4b7fe640 1 -- 172.21.15.19:0/2764235289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50010690 con 0x7fcb5c105350 2024-04-23T18:23:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.850+0000 7fcb608a7640 1 -- 172.21.15.19:0/2764235289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c1a0550 con 0x7fcb5c105350 2024-04-23T18:23:32.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.852+0000 7fcb608a7640 1 -- 172.21.15.19:0/2764235289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5c0fdf20 con 0x7fcb5c105350 2024-04-23T18:23:32.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.853+0000 7fcb4b7fe640 1 -- 172.21.15.19:0/2764235289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcb50004000 con 0x7fcb5c105350 2024-04-23T18:23:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.853+0000 7fcb4b7fe640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb440a3c30 0x7fcb440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.853+0000 7fcb5a59c640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb440a3c30 0x7fcb440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.853+0000 7fcb4b7fe640 1 -- 172.21.15.19:0/2764235289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcb500d01c0 con 0x7fcb5c105350 2024-04-23T18:23:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.854+0000 7fcb5a59c640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb440a3c30 0x7fcb440a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00baa0 tx=0x7fcb4c005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:32.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:32.858+0000 7fcb4b7fe640 1 -- 172.21.15.19:0/2764235289 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb500042f0 con 0x7fcb5c105350 2024-04-23T18:23:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:32 smithi019 ceph-mon[29215]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:32 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.083+0000 7fcb608a7640 1 -- 172.21.15.19:0/2764235289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb5c02d810 con 0x7fcb5c105350 2024-04-23T18:23:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.084+0000 7fcb4b7fe640 1 -- 172.21.15.19:0/2764235289 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7fcb5009d040 con 0x7fcb5c105350 2024-04-23T18:23:33.077 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:33.077 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-23T18:23:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 7 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.085+0000 7fcb608a7640 1 -- 172.21.15.19:0/2764235289 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb440a3c30 msgr2=0x7fcb440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.085+0000 7fcb608a7640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb440a3c30 0x7fcb440a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00baa0 tx=0x7fcb4c005d90 comp rx=0 tx=0).stop 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.085+0000 7fcb608a7640 1 -- 172.21.15.19:0/2764235289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb5c105350 msgr2=0x7fcb5c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.085+0000 7fcb608a7640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb5c105350 0x7fcb5c198670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000e9a0 tx=0x7fcb5000ee60 comp rx=0 tx=0).stop 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.086+0000 7fcb608a7640 1 -- 172.21.15.19:0/2764235289 shutdown_connections 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.086+0000 7fcb608a7640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb440a3c30 0x7fcb440a60f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.086+0000 7fcb608a7640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5c107c80 0x7fcb5c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.086+0000 7fcb608a7640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb5c105350 0x7fcb5c198670 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.086+0000 7fcb608a7640 1 --2- 172.21.15.19:0/2764235289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5c102a20 0x7fcb5c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.086+0000 7fcb608a7640 1 -- 172.21.15.19:0/2764235289 >> 172.21.15.19:0/2764235289 conn(0x7fcb5c0fc430 msgr2=0x7fcb5c103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.086+0000 7fcb608a7640 1 -- 172.21.15.19:0/2764235289 shutdown_connections 2024-04-23T18:23:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:33.086+0000 7fcb608a7640 1 -- 172.21.15.19:0/2764235289 wait complete. 2024-04-23T18:23:33.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:32 smithi155 ceph-mon[33468]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:33.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:33.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:33.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:33.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:33.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:33.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:32 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:33.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:32 smithi044 ceph-mon[32458]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:33.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:33.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:33.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:33.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:33.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:33.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:32 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:34.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:33 smithi155 ceph-mon[33468]: Deploying daemon mds.c on smithi155 2024-04-23T18:23:34.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:33 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] up:boot 2024-04-23T18:23:34.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:33 smithi155 ceph-mon[33468]: fsmap 6 up:standby 2024-04-23T18:23:34.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:33 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:23:34.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:33 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2764235289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:34.125 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:33 smithi044 ceph-mon[32458]: Deploying daemon mds.c on smithi155 2024-04-23T18:23:34.125 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:33 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] up:boot 2024-04-23T18:23:34.125 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:33 smithi044 ceph-mon[32458]: fsmap 6 up:standby 2024-04-23T18:23:34.125 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:33 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:23:34.125 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:33 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2764235289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:34.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:33 smithi019 ceph-mon[29215]: Deploying daemon mds.c on smithi155 2024-04-23T18:23:34.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:33 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] up:boot 2024-04-23T18:23:34.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:33 smithi019 ceph-mon[29215]: fsmap 6 up:standby 2024-04-23T18:23:34.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:33 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:23:34.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:33 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2764235289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:35.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:34 smithi155 ceph-mon[33468]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:35.073 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:23:34 smithi155 systemd[1]: Starting Ceph mds.c for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:35.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:35.116 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:34 smithi019 ceph-mon[29215]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:35.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.211+0000 7f63a3f0f640 1 -- 172.21.15.19:0/3080915889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f639c107c90 msgr2=0x7f639c10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.211+0000 7f63a3f0f640 1 --2- 172.21.15.19:0/3080915889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f639c107c90 0x7f639c10a0e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6394009ca0 tx=0x7f639402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.211+0000 7f63a3f0f640 1 -- 172.21.15.19:0/3080915889 shutdown_connections 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.211+0000 7f63a3f0f640 1 --2- 172.21.15.19:0/3080915889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f639c107c90 0x7f639c10a0e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.211+0000 7f63a3f0f640 1 --2- 172.21.15.19:0/3080915889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f639c105360 0x7f639c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.211+0000 7f63a3f0f640 1 --2- 172.21.15.19:0/3080915889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f639c102a30 0x7f639c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.211+0000 7f63a3f0f640 1 -- 172.21.15.19:0/3080915889 >> 172.21.15.19:0/3080915889 conn(0x7f639c0fc440 msgr2=0x7f639c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.212+0000 7f63a3f0f640 1 -- 172.21.15.19:0/3080915889 shutdown_connections 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.212+0000 7f63a3f0f640 1 -- 172.21.15.19:0/3080915889 wait complete. 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.212+0000 7f63a3f0f640 1 Processor -- start 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.212+0000 7f63a3f0f640 1 -- start start 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.212+0000 7f63a3f0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f639c102a30 0x7f639c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.212+0000 7f63a3f0f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f639c105360 0x7f639c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.212+0000 7f63a3f0f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f639c107c90 0x7f639c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.212+0000 7f63a3f0f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f639c10db40 con 0x7f639c102a30 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.212+0000 7f63a3f0f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f639c10d9c0 con 0x7f639c105360 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.212+0000 7f63a3f0f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f639c10dcc0 con 0x7f639c107c90 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a1cab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f639c102a30 0x7f639c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a1cab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f639c102a30 0x7f639c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54218/0 (socket says 172.21.15.19:54218) 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a1cab640 1 -- 172.21.15.19:0/4168740234 learned_addr learned my addr 172.21.15.19:0/4168740234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a24ac640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f639c107c90 0x7f639c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a14aa640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f639c105360 0x7f639c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a1cab640 1 -- 172.21.15.19:0/4168740234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f639c107c90 msgr2=0x7f639c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a1cab640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f639c107c90 0x7f639c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a1cab640 1 -- 172.21.15.19:0/4168740234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f639c105360 msgr2=0x7f639c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a1cab640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f639c105360 0x7f639c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a1cab640 1 -- 172.21.15.19:0/4168740234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c19fdb0 con 0x7f639c102a30 2024-04-23T18:23:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a14aa640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f639c105360 0x7f639c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a1cab640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f639c102a30 0x7f639c198190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f639800e9f0 tx=0x7f639800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f6392ffd640 1 -- 172.21.15.19:0/4168740234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639800cd80 con 0x7f639c102a30 2024-04-23T18:23:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f6392ffd640 1 -- 172.21.15.19:0/4168740234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63980108e0 con 0x7f639c102a30 2024-04-23T18:23:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a3f0f640 1 -- 172.21.15.19:0/4168740234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c1a00a0 con 0x7f639c102a30 2024-04-23T18:23:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f6392ffd640 1 -- 172.21.15.19:0/4168740234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398018980 con 0x7f639c102a30 2024-04-23T18:23:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.213+0000 7f63a3f0f640 1 -- 172.21.15.19:0/4168740234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c1a0640 con 0x7f639c102a30 2024-04-23T18:23:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.214+0000 7f63a3f0f640 1 -- 172.21.15.19:0/4168740234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639c0fdf30 con 0x7f639c102a30 2024-04-23T18:23:35.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.216+0000 7f6392ffd640 1 -- 172.21.15.19:0/4168740234 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6398010430 con 0x7f639c102a30 2024-04-23T18:23:35.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.217+0000 7f6392ffd640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63880a3c30 0x7f63880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:35.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.217+0000 7f63a14aa640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63880a3c30 0x7f63880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:35.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.217+0000 7f6392ffd640 1 -- 172.21.15.19:0/4168740234 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f63980d07f0 con 0x7f639c102a30 2024-04-23T18:23:35.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.217+0000 7f63a14aa640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63880a3c30 0x7f63880a60f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f639c1996a0 tx=0x7f638c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:35.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.220+0000 7f6392ffd640 1 -- 172.21.15.19:0/4168740234 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f639809d5d0 con 0x7f639c102a30 2024-04-23T18:23:35.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:34 smithi044 ceph-mon[32458]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:35.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.444+0000 7f63a3f0f640 1 -- 172.21.15.19:0/4168740234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f639c02d5c0 con 0x7f639c102a30 2024-04-23T18:23:35.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.445+0000 7f6392ffd640 1 -- 172.21.15.19:0/4168740234 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7f639809ced0 con 0x7f639c102a30 2024-04-23T18:23:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:35.438 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-23T18:23:35.439 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 7 2024-04-23T18:23:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 -- 172.21.15.19:0/4168740234 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63880a3c30 msgr2=0x7f63880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63880a3c30 0x7f63880a60f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f639c1996a0 tx=0x7f638c005f90 comp rx=0 tx=0).stop 2024-04-23T18:23:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 -- 172.21.15.19:0/4168740234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f639c102a30 msgr2=0x7f639c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f639c102a30 0x7f639c198190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f639800e9f0 tx=0x7f639800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 -- 172.21.15.19:0/4168740234 shutdown_connections 2024-04-23T18:23:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63880a3c30 0x7f63880a60f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f639c105360 0x7f639c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f639c107c90 0x7f639c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 --2- 172.21.15.19:0/4168740234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f639c102a30 0x7f639c198190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 -- 172.21.15.19:0/4168740234 >> 172.21.15.19:0/4168740234 conn(0x7f639c0fc440 msgr2=0x7f639c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 -- 172.21.15.19:0/4168740234 shutdown_connections 2024-04-23T18:23:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:35.447+0000 7f63a3f0f640 1 -- 172.21.15.19:0/4168740234 wait complete. 2024-04-23T18:23:35.446 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:23:35 smithi155 podman[63597]: 2024-04-23 18:23:35.081626669 +0000 UTC m=+0.018363197 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:35.446 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:23:35 smithi155 podman[63597]: 2024-04-23 18:23:35.191393423 +0000 UTC m=+0.128129936 container create ef0ea28b882bd0f028b235298ec42605b44b4938a1349aba55ca23a471d05623 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T18:23:35.822 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:23:35 smithi155 podman[63597]: 2024-04-23 18:23:35.454067065 +0000 UTC m=+0.390803577 container init ef0ea28b882bd0f028b235298ec42605b44b4938a1349aba55ca23a471d05623 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-23T18:23:35.823 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:23:35 smithi155 podman[63597]: 2024-04-23 18:23:35.457187643 +0000 UTC m=+0.393924166 container start ef0ea28b882bd0f028b235298ec42605b44b4938a1349aba55ca23a471d05623 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-23T18:23:35.823 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:23:35 smithi155 ceph-mds[63617]: -- 172.21.15.155:0/1187605714 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a8bdcda40 con 0x557a8bda1000 2024-04-23T18:23:35.823 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:23:35 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c[63613]: starting mds.c at 2024-04-23T18:23:35.823 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:23:35 smithi155 bash[63597]: ef0ea28b882bd0f028b235298ec42605b44b4938a1349aba55ca23a471d05623 2024-04-23T18:23:35.823 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:23:35 smithi155 systemd[1]: Started Ceph mds.c for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:36.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:35 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4168740234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:36.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:36.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:36.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:36.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:36.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:36.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:36.305 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:35 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4168740234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:36.305 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:36.305 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:36.306 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:36.306 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:36.306 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:36.306 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:36.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:35 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4168740234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:36.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:36.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:36.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:36.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:36.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:36.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:37.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:36 smithi019 ceph-mon[29215]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:37.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:36 smithi019 ceph-mon[29215]: Deploying daemon mds.h on smithi044 2024-04-23T18:23:37.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:36 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] up:boot 2024-04-23T18:23:37.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:36 smithi019 ceph-mon[29215]: fsmap 7 up:standby 2024-04-23T18:23:37.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:23:37.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:36 smithi044 ceph-mon[32458]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:37.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:36 smithi044 ceph-mon[32458]: Deploying daemon mds.h on smithi044 2024-04-23T18:23:37.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:36 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] up:boot 2024-04-23T18:23:37.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:36 smithi044 ceph-mon[32458]: fsmap 7 up:standby 2024-04-23T18:23:37.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:23:37.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:36 smithi155 ceph-mon[33468]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:37.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:36 smithi155 ceph-mon[33468]: Deploying daemon mds.h on smithi044 2024-04-23T18:23:37.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:36 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] up:boot 2024-04-23T18:23:37.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:36 smithi155 ceph-mon[33468]: fsmap 7 up:standby 2024-04-23T18:23:37.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:23:37.452 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.573+0000 7f48da467640 1 -- 172.21.15.19:0/582066441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4100e60 msgr2=0x7f48d4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.573+0000 7f48da467640 1 --2- 172.21.15.19:0/582066441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4100e60 0x7f48d4101280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009900 tx=0x7f48c402a410 comp rx=0 tx=0).stop 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.573+0000 7f48da467640 1 -- 172.21.15.19:0/582066441 shutdown_connections 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.573+0000 7f48da467640 1 --2- 172.21.15.19:0/582066441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d4102a00 0x7f48d4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.573+0000 7f48da467640 1 --2- 172.21.15.19:0/582066441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4102040 0x7f48d41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.573+0000 7f48da467640 1 --2- 172.21.15.19:0/582066441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4100e60 0x7f48d4101280 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.573+0000 7f48da467640 1 -- 172.21.15.19:0/582066441 >> 172.21.15.19:0/582066441 conn(0x7f48d40fc440 msgr2=0x7f48d40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.573+0000 7f48da467640 1 -- 172.21.15.19:0/582066441 shutdown_connections 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.573+0000 7f48da467640 1 -- 172.21.15.19:0/582066441 wait complete. 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48da467640 1 Processor -- start 2024-04-23T18:23:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48da467640 1 -- start start 2024-04-23T18:23:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48da467640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d4100e60 0x7f48d4104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48da467640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4102040 0x7f48d41034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48da467640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4102a00 0x7f48d41039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48da467640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f48d410c9f0 con 0x7f48d4102040 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48da467640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48d410c870 con 0x7f48d4100e60 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48da467640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48d410cb70 con 0x7f48d4102a00 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48d37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4102040 0x7f48d41034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48d37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4102040 0x7f48d41034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54242/0 (socket says 172.21.15.19:54242) 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48d37fe640 1 -- 172.21.15.19:0/2798519229 learned_addr learned my addr 172.21.15.19:0/2798519229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48d3fff640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d4100e60 0x7f48d4104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.574+0000 7f48d8a04640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4102a00 0x7f48d41039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48d37fe640 1 -- 172.21.15.19:0/2798519229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4102a00 msgr2=0x7f48d41039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48d37fe640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4102a00 0x7f48d41039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48d37fe640 1 -- 172.21.15.19:0/2798519229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d4100e60 msgr2=0x7f48d4104e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48d37fe640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d4100e60 0x7f48d4104e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48d37fe640 1 -- 172.21.15.19:0/2798519229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d4104280 con 0x7f48d4102040 2024-04-23T18:23:37.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48d8a04640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4102a00 0x7f48d41039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48d37fe640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4102040 0x7f48d41034b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f48c800d770 tx=0x7f48c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48d17fa640 1 -- 172.21.15.19:0/2798519229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8017070 con 0x7f48d4102040 2024-04-23T18:23:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48d17fa640 1 -- 172.21.15.19:0/2798519229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48c80054d0 con 0x7f48d4102040 2024-04-23T18:23:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48da467640 1 -- 172.21.15.19:0/2798519229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d41a2390 con 0x7f48d4102040 2024-04-23T18:23:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48d17fa640 1 -- 172.21.15.19:0/2798519229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8004c30 con 0x7f48d4102040 2024-04-23T18:23:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.575+0000 7f48da467640 1 -- 172.21.15.19:0/2798519229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d41a2890 con 0x7f48d4102040 2024-04-23T18:23:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.576+0000 7f48da467640 1 -- 172.21.15.19:0/2798519229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d41a2b50 con 0x7f48d4102040 2024-04-23T18:23:37.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.577+0000 7f48d17fa640 1 -- 172.21.15.19:0/2798519229 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f48c8004e50 con 0x7f48d4102040 2024-04-23T18:23:37.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.578+0000 7f48d17fa640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48bc0a3bb0 0x7f48bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.578+0000 7f48d17fa640 1 -- 172.21.15.19:0/2798519229 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f48c80cfa40 con 0x7f48d4102040 2024-04-23T18:23:37.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.578+0000 7f48d3fff640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48bc0a3bb0 0x7f48bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.578+0000 7f48d3fff640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48bc0a3bb0 0x7f48bc0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009230 tx=0x7f48c40314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:37.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.582+0000 7f48d17fa640 1 -- 172.21.15.19:0/2798519229 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48c809c8c0 con 0x7f48d4102040 2024-04-23T18:23:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.809+0000 7f48da467640 1 -- 172.21.15.19:0/2798519229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48d41a2ee0 con 0x7f48d4102040 2024-04-23T18:23:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.809+0000 7f48d17fa640 1 -- 172.21.15.19:0/2798519229 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6340 (secure 0 0 0) 0x7f48c809c1c0 con 0x7f48d4102040 2024-04-23T18:23:37.803 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:37.803 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-23T18:23:37.803 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-23T18:23:37.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 -- 172.21.15.19:0/2798519229 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48bc0a3bb0 msgr2=0x7f48bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:37.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48bc0a3bb0 0x7f48bc0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009230 tx=0x7f48c40314a0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 -- 172.21.15.19:0/2798519229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4102040 msgr2=0x7f48d41034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4102040 0x7f48d41034b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f48c800d770 tx=0x7f48c800dc30 comp rx=0 tx=0).stop 2024-04-23T18:23:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 -- 172.21.15.19:0/2798519229 shutdown_connections 2024-04-23T18:23:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48bc0a3bb0 0x7f48bc0a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48d4102a00 0x7f48d41039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4102040 0x7f48d41034b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 --2- 172.21.15.19:0/2798519229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d4100e60 0x7f48d4104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 -- 172.21.15.19:0/2798519229 >> 172.21.15.19:0/2798519229 conn(0x7f48d40fc440 msgr2=0x7f48d406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 -- 172.21.15.19:0/2798519229 shutdown_connections 2024-04-23T18:23:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:37.811+0000 7f48da467640 1 -- 172.21.15.19:0/2798519229 wait complete. 2024-04-23T18:23:38.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:37 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2798519229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:38.063 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:23:37 smithi044 systemd[1]: Starting Ceph mds.h for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:38.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:37 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2798519229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:38.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:37 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2798519229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:38.396 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:23:38 smithi044 podman[62486]: 2024-04-23 18:23:38.013918362 +0000 UTC m=+0.018502742 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:38.396 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:23:38 smithi044 podman[62486]: 2024-04-23 18:23:38.143174528 +0000 UTC m=+0.147758936 container create f624123f2dae4482930e460de7bcc647675fdb61ea7a892ddab2f940c9561f50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:23:38.813 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:23:38 smithi044 podman[62486]: 2024-04-23 18:23:38.404123944 +0000 UTC m=+0.408708310 container init f624123f2dae4482930e460de7bcc647675fdb61ea7a892ddab2f940c9561f50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-23T18:23:38.813 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:23:38 smithi044 podman[62486]: 2024-04-23 18:23:38.406837244 +0000 UTC m=+0.411421612 container start f624123f2dae4482930e460de7bcc647675fdb61ea7a892ddab2f940c9561f50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-23T18:23:38.813 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:23:38 smithi044 ceph-mds[62505]: -- 172.21.15.44:0/1268328049 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad1a915a40 con 0x55ad1a8e9000 2024-04-23T18:23:38.813 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:23:38 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h[62501]: starting mds.h at 2024-04-23T18:23:38.813 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:23:38 smithi044 bash[62486]: f624123f2dae4482930e460de7bcc647675fdb61ea7a892ddab2f940c9561f50 2024-04-23T18:23:38.813 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:23:38 smithi044 systemd[1]: Started Ceph mds.h for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:38 smithi019 ceph-mon[29215]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:39.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:38 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:39.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:38 smithi155 ceph-mon[33468]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:39.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:39.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:39.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:39.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:39.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:39.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:38 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:39.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:38 smithi044 ceph-mon[32458]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:39.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:39.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:39.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:39.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:39.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:39.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:38 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:39.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:39.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 -- 172.21.15.19:0/576818974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b541011e0 msgr2=0x7f5b54101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 --2- 172.21.15.19:0/576818974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b541011e0 0x7f5b54101640 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009860 tx=0x7f5b4802a3d0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 -- 172.21.15.19:0/576818974 shutdown_connections 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 --2- 172.21.15.19:0/576818974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b54101b80 0x7f5b54102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 --2- 172.21.15.19:0/576818974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b541011e0 0x7f5b54101640 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 --2- 172.21.15.19:0/576818974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b54100880 0x7f5b54100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 -- 172.21.15.19:0/576818974 >> 172.21.15.19:0/576818974 conn(0x7f5b540fc400 msgr2=0x7f5b540fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 -- 172.21.15.19:0/576818974 shutdown_connections 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 -- 172.21.15.19:0/576818974 wait complete. 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 Processor -- start 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.939+0000 7f5b59ee1640 1 -- start start 2024-04-23T18:23:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b59ee1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b54100880 0x7f5b5419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b59ee1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b541011e0 0x7f5b5419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b59ee1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b54101b80 0x7f5b541a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b59ee1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5410fd20 con 0x7f5b54101b80 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b59ee1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5410fba0 con 0x7f5b54100880 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b59ee1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5410fea0 con 0x7f5b541011e0 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b54101b80 0x7f5b541a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b54101b80 0x7f5b541a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59430/0 (socket says 172.21.15.19:59430) 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b53fff640 1 -- 172.21.15.19:0/4073842194 learned_addr learned my addr 172.21.15.19:0/4073842194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b537fe640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b54100880 0x7f5b5419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b52ffd640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b541011e0 0x7f5b5419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b53fff640 1 -- 172.21.15.19:0/4073842194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b541011e0 msgr2=0x7f5b5419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b53fff640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b541011e0 0x7f5b5419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b53fff640 1 -- 172.21.15.19:0/4073842194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b54100880 msgr2=0x7f5b5419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b53fff640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b54100880 0x7f5b5419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b53fff640 1 -- 172.21.15.19:0/4073842194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b541a1fc0 con 0x7f5b54101b80 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.940+0000 7f5b53fff640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b54101b80 0x7f5b541a1950 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00eb10 tx=0x7f5b4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.941+0000 7f5b50ff9640 1 -- 172.21.15.19:0/4073842194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c004b20 con 0x7f5b54101b80 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.941+0000 7f5b50ff9640 1 -- 172.21.15.19:0/4073842194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b4c004cc0 con 0x7f5b54101b80 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.941+0000 7f5b59ee1640 1 -- 172.21.15.19:0/4073842194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b541a22b0 con 0x7f5b54101b80 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.941+0000 7f5b50ff9640 1 -- 172.21.15.19:0/4073842194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c010690 con 0x7f5b54101b80 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.941+0000 7f5b59ee1640 1 -- 172.21.15.19:0/4073842194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b541a27d0 con 0x7f5b54101b80 2024-04-23T18:23:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.942+0000 7f5b59ee1640 1 -- 172.21.15.19:0/4073842194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5410b7c0 con 0x7f5b54101b80 2024-04-23T18:23:39.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.943+0000 7f5b50ff9640 1 -- 172.21.15.19:0/4073842194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b4c005020 con 0x7f5b54101b80 2024-04-23T18:23:39.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.943+0000 7f5b50ff9640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5b3c0a3bb0 0x7f5b3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:39.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.944+0000 7f5b537fe640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5b3c0a3bb0 0x7f5b3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:39.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.944+0000 7f5b50ff9640 1 -- 172.21.15.19:0/4073842194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5b4c0d04a0 con 0x7f5b54101b80 2024-04-23T18:23:39.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.944+0000 7f5b537fe640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5b3c0a3bb0 0x7f5b3c0a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009530 tx=0x7f5b44009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:39.948+0000 7f5b50ff9640 1 -- 172.21.15.19:0/4073842194 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b4c09d320 con 0x7f5b54101b80 2024-04-23T18:23:40.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.173+0000 7f5b59ee1640 1 -- 172.21.15.19:0/4073842194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b5402d5c0 con 0x7f5b54101b80 2024-04-23T18:23:40.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.174+0000 7f5b50ff9640 1 -- 172.21.15.19:0/4073842194 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7162 (secure 0 0 0) 0x7f5b4c087090 con 0x7f5b54101b80 2024-04-23T18:23:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:40.167 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:23:40.168 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-23T18:23:40.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 -- 172.21.15.19:0/4073842194 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5b3c0a3bb0 msgr2=0x7f5b3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:40.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5b3c0a3bb0 0x7f5b3c0a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009530 tx=0x7f5b44009260 comp rx=0 tx=0).stop 2024-04-23T18:23:40.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 -- 172.21.15.19:0/4073842194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b54101b80 msgr2=0x7f5b541a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:40.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b54101b80 0x7f5b541a1950 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00eb10 tx=0x7f5b4c00c2d0 comp rx=0 tx=0).stop 2024-04-23T18:23:40.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 -- 172.21.15.19:0/4073842194 shutdown_connections 2024-04-23T18:23:40.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5b3c0a3bb0 0x7f5b3c0a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:40.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b54101b80 0x7f5b541a1950 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:40.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b541011e0 0x7f5b5419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:40.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 --2- 172.21.15.19:0/4073842194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b54100880 0x7f5b5419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:40.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 -- 172.21.15.19:0/4073842194 >> 172.21.15.19:0/4073842194 conn(0x7f5b540fc400 msgr2=0x7f5b540fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:40.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 -- 172.21.15.19:0/4073842194 shutdown_connections 2024-04-23T18:23:40.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:40.176+0000 7f5b59ee1640 1 -- 172.21.15.19:0/4073842194 wait complete. 2024-04-23T18:23:40.179 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:39 smithi019 ceph-mon[29215]: Deploying daemon mds.i on smithi155 2024-04-23T18:23:40.179 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:39 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] up:boot 2024-04-23T18:23:40.179 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:39 smithi019 ceph-mon[29215]: fsmap 8 up:standby 2024-04-23T18:23:40.179 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:39 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:23:40.254 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:39 smithi155 ceph-mon[33468]: Deploying daemon mds.i on smithi155 2024-04-23T18:23:40.254 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:39 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] up:boot 2024-04-23T18:23:40.254 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:39 smithi155 ceph-mon[33468]: fsmap 8 up:standby 2024-04-23T18:23:40.254 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:39 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:23:40.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:39 smithi044 ceph-mon[32458]: Deploying daemon mds.i on smithi155 2024-04-23T18:23:40.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:39 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] up:boot 2024-04-23T18:23:40.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:39 smithi044 ceph-mon[32458]: fsmap 8 up:standby 2024-04-23T18:23:40.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:39 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:23:41.057 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:40 smithi155 ceph-mon[33468]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:41.057 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:40 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4073842194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:41.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:40 smithi019 ceph-mon[29215]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:41.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:40 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4073842194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:41.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:40 smithi044 ceph-mon[32458]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:41.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:40 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4073842194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:41.822 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:23:41 smithi155 systemd[1]: Started Ceph mds.i for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:42.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:42.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.292+0000 7fc7cfc91640 1 -- 172.21.15.19:0/3790392848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c8102a10 msgr2=0x7fc7c8104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:42.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.292+0000 7fc7cfc91640 1 --2- 172.21.15.19:0/3790392848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c8102a10 0x7fc7c8104e00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009900 tx=0x7fc7b802a410 comp rx=0 tx=0).stop 2024-04-23T18:23:42.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.292+0000 7fc7cfc91640 1 -- 172.21.15.19:0/3790392848 shutdown_connections 2024-04-23T18:23:42.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.292+0000 7fc7cfc91640 1 --2- 172.21.15.19:0/3790392848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7c8105340 0x7fc7c8107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.292+0000 7fc7cfc91640 1 --2- 172.21.15.19:0/3790392848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8107c70 0x7fc7c810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.292+0000 7fc7cfc91640 1 --2- 172.21.15.19:0/3790392848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c8102a10 0x7fc7c8104e00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.292+0000 7fc7cfc91640 1 -- 172.21.15.19:0/3790392848 >> 172.21.15.19:0/3790392848 conn(0x7fc7c80fc400 msgr2=0x7fc7c80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:42.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cfc91640 1 -- 172.21.15.19:0/3790392848 shutdown_connections 2024-04-23T18:23:42.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cfc91640 1 -- 172.21.15.19:0/3790392848 wait complete. 2024-04-23T18:23:42.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cfc91640 1 Processor -- start 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cfc91640 1 -- start start 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cfc91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c8102a10 0x7fc7c8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cfc91640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8105340 0x7fc7c8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cfc91640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7c8107c70 0x7fc7c819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cfc91640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c810dae0 con 0x7fc7c8102a10 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cfc91640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c810d960 con 0x7fc7c8107c70 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cfc91640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c810dc60 con 0x7fc7c8105340 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cda2d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c8102a10 0x7fc7c8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cda2d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c8102a10 0x7fc7c8198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59454/0 (socket says 172.21.15.19:59454) 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cda2d640 1 -- 172.21.15.19:0/439665698 learned_addr learned my addr 172.21.15.19:0/439665698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.293+0000 7fc7cd22c640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8105340 0x7fc7c8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:42.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7ce22e640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7c8107c70 0x7fc7c819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7cda2d640 1 -- 172.21.15.19:0/439665698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8105340 msgr2=0x7fc7c8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7cda2d640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8105340 0x7fc7c8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7cda2d640 1 -- 172.21.15.19:0/439665698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7c8107c70 msgr2=0x7fc7c819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7cda2d640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7c8107c70 0x7fc7c819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7cda2d640 1 -- 172.21.15.19:0/439665698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c819fe30 con 0x7fc7c8102a10 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7cda2d640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c8102a10 0x7fc7c8198210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005940 tx=0x7fc7b8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7beffd640 1 -- 172.21.15.19:0/439665698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b803f030 con 0x7fc7c8102a10 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7beffd640 1 -- 172.21.15.19:0/439665698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b8003aa0 con 0x7fc7c8102a10 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7cfc91640 1 -- 172.21.15.19:0/439665698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c81a00c0 con 0x7fc7c8102a10 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7beffd640 1 -- 172.21.15.19:0/439665698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8035070 con 0x7fc7c8102a10 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.294+0000 7fc7cfc91640 1 -- 172.21.15.19:0/439665698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c81a05b0 con 0x7fc7c8102a10 2024-04-23T18:23:42.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.295+0000 7fc7cfc91640 1 -- 172.21.15.19:0/439665698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c80fdef0 con 0x7fc7c8102a10 2024-04-23T18:23:42.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.296+0000 7fc7beffd640 1 -- 172.21.15.19:0/439665698 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc7b8042030 con 0x7fc7c8102a10 2024-04-23T18:23:42.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.297+0000 7fc7beffd640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc7b40a3bb0 0x7fc7b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:42.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.297+0000 7fc7beffd640 1 -- 172.21.15.19:0/439665698 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc7b80f0c70 con 0x7fc7c8102a10 2024-04-23T18:23:42.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.297+0000 7fc7cd22c640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc7b40a3bb0 0x7fc7b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:42.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.297+0000 7fc7cd22c640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc7b40a3bb0 0x7fc7b40a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8199720 tx=0x7fc7c400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:42.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.301+0000 7fc7beffd640 1 -- 172.21.15.19:0/439665698 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7b80bda50 con 0x7fc7c8102a10 2024-04-23T18:23:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.529+0000 7fc7cfc91640 1 -- 172.21.15.19:0/439665698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7c802d5c0 con 0x7fc7c8102a10 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.530+0000 7fc7beffd640 1 -- 172.21.15.19:0/439665698 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7992 (secure 0 0 0) 0x7fc7b80bd350 con 0x7fc7c8102a10 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 10 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 -- 172.21.15.19:0/439665698 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc7b40a3bb0 msgr2=0x7fc7b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc7b40a3bb0 0x7fc7b40a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8199720 tx=0x7fc7c400b410 comp rx=0 tx=0).stop 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 -- 172.21.15.19:0/439665698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c8102a10 msgr2=0x7fc7c8198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c8102a10 0x7fc7c8198210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005940 tx=0x7fc7b8031a50 comp rx=0 tx=0).stop 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 -- 172.21.15.19:0/439665698 shutdown_connections 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc7b40a3bb0 0x7fc7b40a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7c8107c70 0x7fc7c819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8105340 0x7fc7c8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 --2- 172.21.15.19:0/439665698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c8102a10 0x7fc7c8198210 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 -- 172.21.15.19:0/439665698 >> 172.21.15.19:0/439665698 conn(0x7fc7c80fc400 msgr2=0x7fc7c81035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 -- 172.21.15.19:0/439665698 shutdown_connections 2024-04-23T18:23:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:42.532+0000 7fc7cfc91640 1 -- 172.21.15.19:0/439665698 wait complete. 2024-04-23T18:23:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: Deploying daemon mds.g on smithi019 2024-04-23T18:23:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] up:boot 2024-04-23T18:23:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: fsmap 9 up:standby 2024-04-23T18:23:42.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:42 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: Deploying daemon mds.g on smithi019 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] up:boot 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: fsmap 9 up:standby 2024-04-23T18:23:42.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:42 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:23:42.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:42.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: Deploying daemon mds.g on smithi019 2024-04-23T18:23:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] up:boot 2024-04-23T18:23:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: fsmap 9 up:standby 2024-04-23T18:23:42.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:42 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:23:43.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:43 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/439665698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:43.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:43 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/439665698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:43.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:43 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/439665698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:44.055 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:23:44 smithi019 podman[70344]: 2024-04-23 18:23:43.934484674 +0000 UTC m=+0.018833694 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:44.056 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:23:44 smithi019 podman[70344]: 2024-04-23 18:23:44.062871049 +0000 UTC m=+0.147220056 container create dbbf0ce9b4e9357d1eb0a0c800178742557a4c2024f6435e803c51567989f678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True) 2024-04-23T18:23:44.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:44.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:44 smithi019 ceph-mon[29215]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:44.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:44.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:44 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:44.559 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:23:44 smithi019 podman[70344]: 2024-04-23 18:23:44.315618907 +0000 UTC m=+0.399967913 container init dbbf0ce9b4e9357d1eb0a0c800178742557a4c2024f6435e803c51567989f678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-23T18:23:44.559 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:23:44 smithi019 podman[70344]: 2024-04-23 18:23:44.318844711 +0000 UTC m=+0.403193722 container start dbbf0ce9b4e9357d1eb0a0c800178742557a4c2024f6435e803c51567989f678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:23:44.559 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:23:44 smithi019 ceph-mds[70363]: -- 172.21.15.19:0/3311745516 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607218e9a40 con 0x5607218bc800 2024-04-23T18:23:44.560 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:23:44 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g[70359]: starting mds.g at 2024-04-23T18:23:44.560 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:23:44 smithi019 bash[70344]: dbbf0ce9b4e9357d1eb0a0c800178742557a4c2024f6435e803c51567989f678 2024-04-23T18:23:44.560 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:23:44 smithi019 systemd[1]: Started Ceph mds.g for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.650+0000 7fb7f6283640 1 -- 172.21.15.19:0/1933786265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f0101bc0 msgr2=0x7fb7f0102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.650+0000 7fb7f6283640 1 --2- 172.21.15.19:0/1933786265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f0101bc0 0x7fb7f0102040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009ca0 tx=0x7fb7e802a950 comp rx=0 tx=0).stop 2024-04-23T18:23:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.650+0000 7fb7f6283640 1 -- 172.21.15.19:0/1933786265 shutdown_connections 2024-04-23T18:23:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.650+0000 7fb7f6283640 1 --2- 172.21.15.19:0/1933786265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f0101bc0 0x7fb7f0102040 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.650+0000 7fb7f6283640 1 --2- 172.21.15.19:0/1933786265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f0101220 0x7fb7f0101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.650+0000 7fb7f6283640 1 --2- 172.21.15.19:0/1933786265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7f01008c0 0x7fb7f0100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.650+0000 7fb7f6283640 1 -- 172.21.15.19:0/1933786265 >> 172.21.15.19:0/1933786265 conn(0x7fb7f00fc440 msgr2=0x7fb7f00fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.650+0000 7fb7f6283640 1 -- 172.21.15.19:0/1933786265 shutdown_connections 2024-04-23T18:23:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.651+0000 7fb7f6283640 1 -- 172.21.15.19:0/1933786265 wait complete. 2024-04-23T18:23:44.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.651+0000 7fb7f6283640 1 Processor -- start 2024-04-23T18:23:44.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.651+0000 7fb7f6283640 1 -- start start 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.651+0000 7fb7f6283640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01008c0 0x7fb7f019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.651+0000 7fb7f6283640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7f0101220 0x7fb7f019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.651+0000 7fb7f6283640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.651+0000 7fb7f6283640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f010fd10 con 0x7fb7f0101bc0 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.651+0000 7fb7f6283640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f010fb90 con 0x7fb7f0101220 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.651+0000 7fb7f6283640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f010fe90 con 0x7fb7f01008c0 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f4820640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f4820640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59494/0 (socket says 172.21.15.19:59494) 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f4820640 1 -- 172.21.15.19:0/2193025353 learned_addr learned my addr 172.21.15.19:0/2193025353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7effff640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01008c0 0x7fb7f019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7ef7fe640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7f0101220 0x7fb7f019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f4820640 1 -- 172.21.15.19:0/2193025353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01008c0 msgr2=0x7fb7f019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f4820640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01008c0 0x7fb7f019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f4820640 1 -- 172.21.15.19:0/2193025353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7f0101220 msgr2=0x7fb7f019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f4820640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7f0101220 0x7fb7f019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f4820640 1 -- 172.21.15.19:0/2193025353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f01a2020 con 0x7fb7f0101bc0 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7ef7fe640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7f0101220 0x7fb7f019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7effff640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01008c0 0x7fb7f019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f4820640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01a19b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8006010 tx=0x7fb7e8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7ed7fa640 1 -- 172.21.15.19:0/2193025353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8034040 con 0x7fb7f0101bc0 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7ed7fa640 1 -- 172.21.15.19:0/2193025353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e80308f0 con 0x7fb7f0101bc0 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f6283640 1 -- 172.21.15.19:0/2193025353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f01a22b0 con 0x7fb7f0101bc0 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7ed7fa640 1 -- 172.21.15.19:0/2193025353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8045480 con 0x7fb7f0101bc0 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.652+0000 7fb7f6283640 1 -- 172.21.15.19:0/2193025353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f01a27a0 con 0x7fb7f0101bc0 2024-04-23T18:23:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.654+0000 7fb7f6283640 1 -- 172.21.15.19:0/2193025353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f010b850 con 0x7fb7f0101bc0 2024-04-23T18:23:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.655+0000 7fb7ed7fa640 1 -- 172.21.15.19:0/2193025353 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb7e8037020 con 0x7fb7f0101bc0 2024-04-23T18:23:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.656+0000 7fb7ed7fa640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb7d80a3bb0 0x7fb7d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.656+0000 7fb7effff640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb7d80a3bb0 0x7fb7d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.657+0000 7fb7ed7fa640 1 -- 172.21.15.19:0/2193025353 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb7e80f1400 con 0x7fb7f0101bc0 2024-04-23T18:23:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.657+0000 7fb7effff640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb7d80a3bb0 0x7fb7d80a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40058c0 tx=0x7fb7e4005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.660+0000 7fb7ed7fa640 1 -- 172.21.15.19:0/2193025353 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7e8080ff0 con 0x7fb7f0101bc0 2024-04-23T18:23:44.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:44 smithi044 ceph-mon[32458]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:44.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:44.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:44 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:44.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:44 smithi155 ceph-mon[33468]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:44.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:44.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:44 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:44.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.886+0000 7fb7f6283640 1 -- 172.21.15.19:0/2193025353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7f002d5c0 con 0x7fb7f0101bc0 2024-04-23T18:23:44.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.887+0000 7fb7ed7fa640 1 -- 172.21.15.19:0/2193025353 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8815 (secure 0 0 0) 0x7fb7e80be310 con 0x7fb7f0101bc0 2024-04-23T18:23:44.880 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:44.880 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T18:23:44.880 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-04-23T18:23:44.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 -- 172.21.15.19:0/2193025353 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb7d80a3bb0 msgr2=0x7fb7d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:44.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb7d80a3bb0 0x7fb7d80a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40058c0 tx=0x7fb7e4005ec0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 -- 172.21.15.19:0/2193025353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f0101bc0 msgr2=0x7fb7f01a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:44.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01a19b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8006010 tx=0x7fb7e8031ee0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 -- 172.21.15.19:0/2193025353 shutdown_connections 2024-04-23T18:23:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb7d80a3bb0 0x7fb7d80a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7f0101220 0x7fb7f019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01a19b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 --2- 172.21.15.19:0/2193025353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01008c0 0x7fb7f019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 -- 172.21.15.19:0/2193025353 >> 172.21.15.19:0/2193025353 conn(0x7fb7f00fc440 msgr2=0x7fb7f00fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 -- 172.21.15.19:0/2193025353 shutdown_connections 2024-04-23T18:23:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:44.889+0000 7fb7f6283640 1 -- 172.21.15.19:0/2193025353 wait complete. 2024-04-23T18:23:45.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:45.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:45.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:45 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.19:6838/716138605,v1:172.21.15.19:6839/716138605] up:boot 2024-04-23T18:23:45.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:45 smithi019 ceph-mon[29215]: fsmap 10 up:standby 2024-04-23T18:23:45.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:23:45.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:45.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:45 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:45.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:45 smithi019 ceph-mon[29215]: Deploying daemon mds.a on smithi019 2024-04-23T18:23:45.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:45 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2193025353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:45.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:45.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:45.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:45 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.19:6838/716138605,v1:172.21.15.19:6839/716138605] up:boot 2024-04-23T18:23:45.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:45 smithi044 ceph-mon[32458]: fsmap 10 up:standby 2024-04-23T18:23:45.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:23:45.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:45.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:45 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:45.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:45 smithi044 ceph-mon[32458]: Deploying daemon mds.a on smithi019 2024-04-23T18:23:45.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:45 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2193025353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:45.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:45 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.19:6838/716138605,v1:172.21.15.19:6839/716138605] up:boot 2024-04-23T18:23:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:45 smithi155 ceph-mon[33468]: fsmap 10 up:standby 2024-04-23T18:23:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:23:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:45 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:45 smithi155 ceph-mon[33468]: Deploying daemon mds.a on smithi019 2024-04-23T18:23:45.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:45 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2193025353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:46.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:46 smithi019 ceph-mon[29215]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:46.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:46 smithi044 ceph-mon[32458]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:46.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:46 smithi155 ceph-mon[33468]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:46.895 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:46.920 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:23:46 smithi019 systemd[1]: Starting Ceph mds.a for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.009+0000 7fa3ba879640 1 -- 172.21.15.19:0/745850455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4101fe0 msgr2=0x7fa3b4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.009+0000 7fa3ba879640 1 --2- 172.21.15.19:0/745850455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4101fe0 0x7fa3b4102460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009900 tx=0x7fa3a802a410 comp rx=0 tx=0).stop 2024-04-23T18:23:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 -- 172.21.15.19:0/745850455 shutdown_connections 2024-04-23T18:23:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 --2- 172.21.15.19:0/745850455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3b41029a0 0x7fa3b41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 --2- 172.21.15.19:0/745850455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4101fe0 0x7fa3b4102460 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 --2- 172.21.15.19:0/745850455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3b4100e90 0x7fa3b41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 -- 172.21.15.19:0/745850455 >> 172.21.15.19:0/745850455 conn(0x7fa3b40fc400 msgr2=0x7fa3b40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 -- 172.21.15.19:0/745850455 shutdown_connections 2024-04-23T18:23:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 -- 172.21.15.19:0/745850455 wait complete. 2024-04-23T18:23:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 Processor -- start 2024-04-23T18:23:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 -- start start 2024-04-23T18:23:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3b4100e90 0x7fa3b4197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3b4101fe0 0x7fa3b41976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b41029a0 0x7fa3b419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b410ca60 con 0x7fa3b41029a0 2024-04-23T18:23:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b410c8e0 con 0x7fa3b4100e90 2024-04-23T18:23:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.010+0000 7fa3ba879640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b410cbe0 con 0x7fa3b4101fe0 2024-04-23T18:23:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b8e16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b41029a0 0x7fa3b419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:47.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b8e16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b41029a0 0x7fa3b419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59508/0 (socket says 172.21.15.19:59508) 2024-04-23T18:23:47.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b8e16640 1 -- 172.21.15.19:0/1297041072 learned_addr learned my addr 172.21.15.19:0/1297041072 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:47.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b37fe640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3b4101fe0 0x7fa3b41976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:47.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b3fff640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3b4100e90 0x7fa3b4197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:47.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b8e16640 1 -- 172.21.15.19:0/1297041072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3b4101fe0 msgr2=0x7fa3b41976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:47.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b8e16640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3b4101fe0 0x7fa3b41976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:47.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b8e16640 1 -- 172.21.15.19:0/1297041072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3b4100e90 msgr2=0x7fa3b4197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:47.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b8e16640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3b4100e90 0x7fa3b4197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:47.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b8e16640 1 -- 172.21.15.19:0/1297041072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b40691e0 con 0x7fa3b41029a0 2024-04-23T18:23:47.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b8e16640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b41029a0 0x7fa3b419e710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00b970 tx=0x7fa3ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:47.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b17fa640 1 -- 172.21.15.19:0/1297041072 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac019070 con 0x7fa3b41029a0 2024-04-23T18:23:47.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b17fa640 1 -- 172.21.15.19:0/1297041072 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3ac014440 con 0x7fa3b41029a0 2024-04-23T18:23:47.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3ba879640 1 -- 172.21.15.19:0/1297041072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b40694d0 con 0x7fa3b41029a0 2024-04-23T18:23:47.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3b17fa640 1 -- 172.21.15.19:0/1297041072 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0075d0 con 0x7fa3b41029a0 2024-04-23T18:23:47.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.011+0000 7fa3ba879640 1 -- 172.21.15.19:0/1297041072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b4069a20 con 0x7fa3b41029a0 2024-04-23T18:23:47.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.012+0000 7fa3ba879640 1 -- 172.21.15.19:0/1297041072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b4108210 con 0x7fa3b41029a0 2024-04-23T18:23:47.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.014+0000 7fa3b17fa640 1 -- 172.21.15.19:0/1297041072 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa3ac01c070 con 0x7fa3b41029a0 2024-04-23T18:23:47.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.014+0000 7fa3b17fa640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa39c0a3bb0 0x7fa39c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:47.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.014+0000 7fa3b17fa640 1 -- 172.21.15.19:0/1297041072 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa3ac0cfac0 con 0x7fa3b41029a0 2024-04-23T18:23:47.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.014+0000 7fa3b3fff640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa39c0a3bb0 0x7fa39c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:47.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.015+0000 7fa3b3fff640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa39c0a3bb0 0x7fa39c0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4005fd0 tx=0x7fa3a4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:47.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.018+0000 7fa3b17fa640 1 -- 172.21.15.19:0/1297041072 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3ac09c250 con 0x7fa3b41029a0 2024-04-23T18:23:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.247+0000 7fa3ba879640 1 -- 172.21.15.19:0/1297041072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3b4069ce0 con 0x7fa3b41029a0 2024-04-23T18:23:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.248+0000 7fa3b17fa640 1 -- 172.21.15.19:0/1297041072 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8815 (secure 0 0 0) 0x7fa3ac0a0550 con 0x7fa3b41029a0 2024-04-23T18:23:47.241 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:47.242 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T18:23:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-04-23T18:23:47.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 -- 172.21.15.19:0/1297041072 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa39c0a3bb0 msgr2=0x7fa39c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa39c0a3bb0 0x7fa39c0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4005fd0 tx=0x7fa3a4005850 comp rx=0 tx=0).stop 2024-04-23T18:23:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 -- 172.21.15.19:0/1297041072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b41029a0 msgr2=0x7fa3b419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b41029a0 0x7fa3b419e710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00b970 tx=0x7fa3ac00be30 comp rx=0 tx=0).stop 2024-04-23T18:23:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 -- 172.21.15.19:0/1297041072 shutdown_connections 2024-04-23T18:23:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa39c0a3bb0 0x7fa39c0a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b41029a0 0x7fa3b419e710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3b4101fe0 0x7fa3b41976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 --2- 172.21.15.19:0/1297041072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3b4100e90 0x7fa3b4197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 -- 172.21.15.19:0/1297041072 >> 172.21.15.19:0/1297041072 conn(0x7fa3b40fc400 msgr2=0x7fa3b41053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:47.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 -- 172.21.15.19:0/1297041072 shutdown_connections 2024-04-23T18:23:47.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:47.250+0000 7fa3ba879640 1 -- 172.21.15.19:0/1297041072 wait complete. 2024-04-23T18:23:47.253 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:23:46 smithi019 podman[70704]: 2024-04-23 18:23:46.873282376 +0000 UTC m=+0.018445070 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:47.253 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:23:47 smithi019 podman[70704]: 2024-04-23 18:23:47.00329995 +0000 UTC m=+0.148462634 container create 14eec03debed59c5f93cb6719a35caba19ca68d11a0090e5bc4664d02d0857a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True) 2024-04-23T18:23:47.506 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:23:47 smithi019 podman[70704]: 2024-04-23 18:23:47.26438973 +0000 UTC m=+0.409552415 container init 14eec03debed59c5f93cb6719a35caba19ca68d11a0090e5bc4664d02d0857a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-23T18:23:47.506 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:23:47 smithi019 podman[70704]: 2024-04-23 18:23:47.267699992 +0000 UTC m=+0.412862684 container start 14eec03debed59c5f93cb6719a35caba19ca68d11a0090e5bc4664d02d0857a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:23:47.506 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:23:47 smithi019 ceph-mds[70766]: -- 172.21.15.19:0/4151756391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56147f37da40 con 0x56147f350800 2024-04-23T18:23:47.506 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:23:47 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a[70760]: starting mds.a at 2024-04-23T18:23:47.506 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:23:47 smithi019 bash[70704]: 14eec03debed59c5f93cb6719a35caba19ca68d11a0090e5bc4664d02d0857a1 2024-04-23T18:23:47.506 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:23:47 smithi019 systemd[1]: Started Ceph mds.a for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:47.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:47 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1297041072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:47.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:47.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:47.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:47.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:47.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:47.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:47 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:47.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:47 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1297041072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:47.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:47.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:47.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:47.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:47.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:47.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:47 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:47.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:47 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1297041072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:47.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:47.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:47.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:47.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:47.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:47.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:47 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:48.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:48 smithi019 ceph-mon[29215]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:48.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:48 smithi019 ceph-mon[29215]: Deploying daemon mds.b on smithi044 2024-04-23T18:23:48.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:48 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.19:6840/2593314342,v1:172.21.15.19:6841/2593314342] up:boot 2024-04-23T18:23:48.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:48 smithi019 ceph-mon[29215]: fsmap 11 up:standby 2024-04-23T18:23:48.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:48 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:23:48.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:48 smithi044 ceph-mon[32458]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:48.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:48 smithi044 ceph-mon[32458]: Deploying daemon mds.b on smithi044 2024-04-23T18:23:48.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:48 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.19:6840/2593314342,v1:172.21.15.19:6841/2593314342] up:boot 2024-04-23T18:23:48.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:48 smithi044 ceph-mon[32458]: fsmap 11 up:standby 2024-04-23T18:23:48.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:48 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:23:48.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:48 smithi155 ceph-mon[33468]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:48.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:48 smithi155 ceph-mon[33468]: Deploying daemon mds.b on smithi044 2024-04-23T18:23:48.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:48 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.19:6840/2593314342,v1:172.21.15.19:6841/2593314342] up:boot 2024-04-23T18:23:48.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:48 smithi155 ceph-mon[33468]: fsmap 11 up:standby 2024-04-23T18:23:48.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:48 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:23:49.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.379+0000 7f273f76f640 1 -- 172.21.15.19:0/241746406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2738101fc0 msgr2=0x7f27381023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.379+0000 7f273f76f640 1 --2- 172.21.15.19:0/241746406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2738101fc0 0x7f27381023e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2734009900 tx=0x7f273402a410 comp rx=0 tx=0).stop 2024-04-23T18:23:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.379+0000 7f273f76f640 1 -- 172.21.15.19:0/241746406 shutdown_connections 2024-04-23T18:23:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.379+0000 7f273f76f640 1 --2- 172.21.15.19:0/241746406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27381031b0 0x7f2738103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.379+0000 7f273f76f640 1 --2- 172.21.15.19:0/241746406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2738103b70 0x7f273810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.379+0000 7f273f76f640 1 --2- 172.21.15.19:0/241746406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2738101fc0 0x7f27381023e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.379+0000 7f273f76f640 1 -- 172.21.15.19:0/241746406 >> 172.21.15.19:0/241746406 conn(0x7f27380fd550 msgr2=0x7f27380ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.379+0000 7f273f76f640 1 -- 172.21.15.19:0/241746406 shutdown_connections 2024-04-23T18:23:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.379+0000 7f273f76f640 1 -- 172.21.15.19:0/241746406 wait complete. 2024-04-23T18:23:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273f76f640 1 Processor -- start 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273f76f640 1 -- start start 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273f76f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2738101fc0 0x7f27381981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273f76f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27381031b0 0x7f2738198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273f76f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2738103b70 0x7f273819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273f76f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f273810db00 con 0x7f2738103b70 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273f76f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f273810d980 con 0x7f27381031b0 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273f76f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f273810dc80 con 0x7f2738101fc0 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273dd0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2738103b70 0x7f273819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273dd0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2738103b70 0x7f273819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59542/0 (socket says 172.21.15.19:59542) 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273dd0c640 1 -- 172.21.15.19:0/2517392280 learned_addr learned my addr 172.21.15.19:0/2517392280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273d50b640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2738101fc0 0x7f27381981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.380+0000 7f273cd0a640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27381031b0 0x7f2738198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f273dd0c640 1 -- 172.21.15.19:0/2517392280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2738101fc0 msgr2=0x7f27381981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f273dd0c640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2738101fc0 0x7f27381981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f273dd0c640 1 -- 172.21.15.19:0/2517392280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27381031b0 msgr2=0x7f2738198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f273dd0c640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27381031b0 0x7f2738198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f273dd0c640 1 -- 172.21.15.19:0/2517392280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273819fe00 con 0x7f2738103b70 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f273cd0a640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27381031b0 0x7f2738198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f273dd0c640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2738103b70 0x7f273819f790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f273000eaa0 tx=0x7f273000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f272e7fc640 1 -- 172.21.15.19:0/2517392280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273000cdb0 con 0x7f2738103b70 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f272e7fc640 1 -- 172.21.15.19:0/2517392280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2730005020 con 0x7f2738103b70 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f273f76f640 1 -- 172.21.15.19:0/2517392280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27381a00f0 con 0x7f2738103b70 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f272e7fc640 1 -- 172.21.15.19:0/2517392280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27300116d0 con 0x7f2738103b70 2024-04-23T18:23:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.381+0000 7f273f76f640 1 -- 172.21.15.19:0/2517392280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27381a0640 con 0x7f2738103b70 2024-04-23T18:23:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.382+0000 7f273f76f640 1 -- 172.21.15.19:0/2517392280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27381093b0 con 0x7f2738103b70 2024-04-23T18:23:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.383+0000 7f272e7fc640 1 -- 172.21.15.19:0/2517392280 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f27300051d0 con 0x7f2738103b70 2024-04-23T18:23:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.384+0000 7f272e7fc640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27240a3bb0 0x7f27240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.384+0000 7f273d50b640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27240a3bb0 0x7f27240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.384+0000 7f272e7fc640 1 -- 172.21.15.19:0/2517392280 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f27300d04e0 con 0x7f2738103b70 2024-04-23T18:23:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.384+0000 7f273d50b640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27240a3bb0 0x7f27240a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2734009230 tx=0x7f27340314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.388+0000 7f272e7fc640 1 -- 172.21.15.19:0/2517392280 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2730016030 con 0x7f2738103b70 2024-04-23T18:23:49.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.616+0000 7f273f76f640 1 -- 172.21.15.19:0/2517392280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f273802d5c0 con 0x7f2738103b70 2024-04-23T18:23:49.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.617+0000 7f272e7fc640 1 -- 172.21.15.19:0/2517392280 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9641 (secure 0 0 0) 0x7f27300bbe20 con 0x7f2738103b70 2024-04-23T18:23:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:49.610 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T18:23:49.610 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 12 2024-04-23T18:23:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 -- 172.21.15.19:0/2517392280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27240a3bb0 msgr2=0x7f27240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27240a3bb0 0x7f27240a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2734009230 tx=0x7f27340314a0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 -- 172.21.15.19:0/2517392280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2738103b70 msgr2=0x7f273819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2738103b70 0x7f273819f790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f273000eaa0 tx=0x7f273000ef60 comp rx=0 tx=0).stop 2024-04-23T18:23:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 -- 172.21.15.19:0/2517392280 shutdown_connections 2024-04-23T18:23:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27240a3bb0 0x7f27240a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27381031b0 0x7f2738198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2738103b70 0x7f273819f790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 --2- 172.21.15.19:0/2517392280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2738101fc0 0x7f27381981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 -- 172.21.15.19:0/2517392280 >> 172.21.15.19:0/2517392280 conn(0x7f27380fd550 msgr2=0x7f2738106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:49.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 -- 172.21.15.19:0/2517392280 shutdown_connections 2024-04-23T18:23:49.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:49.619+0000 7f273f76f640 1 -- 172.21.15.19:0/2517392280 wait complete. 2024-04-23T18:23:50.014 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:23:49 smithi044 systemd[1]: Starting Ceph mds.b for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:50.270 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:23:50 smithi044 podman[62792]: 2024-04-23 18:23:49.92485174 +0000 UTC m=+0.019362806 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:50.271 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:23:50 smithi044 podman[62792]: 2024-04-23 18:23:50.032430021 +0000 UTC m=+0.126941082 container create bb88eed0142d8cb8372119ae9a0398184fa00587bb2eee45ccda7e5b94856bac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-23T18:23:50.271 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:23:50 smithi044 podman[62792]: 2024-04-23 18:23:50.25155307 +0000 UTC m=+0.346064127 container init bb88eed0142d8cb8372119ae9a0398184fa00587bb2eee45ccda7e5b94856bac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T18:23:50.271 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:23:50 smithi044 podman[62792]: 2024-04-23 18:23:50.254141089 +0000 UTC m=+0.348652154 container start bb88eed0142d8cb8372119ae9a0398184fa00587bb2eee45ccda7e5b94856bac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-23T18:23:50.562 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:23:50 smithi044 ceph-mds[62811]: -- 172.21.15.44:0/3775883093 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560398827a40 con 0x5603987fac00 2024-04-23T18:23:50.562 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:23:50 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b[62807]: starting mds.b at 2024-04-23T18:23:50.562 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:23:50 smithi044 bash[62792]: bb88eed0142d8cb8372119ae9a0398184fa00587bb2eee45ccda7e5b94856bac 2024-04-23T18:23:50.563 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:23:50 smithi044 systemd[1]: Started Ceph mds.b for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:50 smithi044 ceph-mon[32458]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:50 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2517392280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:50.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:50 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:50.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:50 smithi155 ceph-mon[33468]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:50 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2517392280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:50.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:50 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:51.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:50 smithi019 ceph-mon[29215]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:51.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:50 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2517392280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:51.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:51.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:51.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:51.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:51.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:50 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:51.623 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:51.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.738+0000 7f6ab50bb640 1 -- 172.21.15.19:0/3716472147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab0107c80 msgr2=0x7f6ab010a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:51.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.738+0000 7f6ab50bb640 1 --2- 172.21.15.19:0/3716472147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab0107c80 0x7f6ab010a0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009ca0 tx=0x7f6aa802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.738+0000 7f6ab50bb640 1 -- 172.21.15.19:0/3716472147 shutdown_connections 2024-04-23T18:23:51.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.738+0000 7f6ab50bb640 1 --2- 172.21.15.19:0/3716472147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ab0105350 0x7f6ab0107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.738+0000 7f6ab50bb640 1 --2- 172.21.15.19:0/3716472147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab0107c80 0x7f6ab010a0d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.738+0000 7f6ab50bb640 1 --2- 172.21.15.19:0/3716472147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ab0102a20 0x7f6ab0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.738+0000 7f6ab50bb640 1 -- 172.21.15.19:0/3716472147 >> 172.21.15.19:0/3716472147 conn(0x7f6ab00fc430 msgr2=0x7f6ab00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:51.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.738+0000 7f6ab50bb640 1 -- 172.21.15.19:0/3716472147 shutdown_connections 2024-04-23T18:23:51.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.738+0000 7f6ab50bb640 1 -- 172.21.15.19:0/3716472147 wait complete. 2024-04-23T18:23:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6ab50bb640 1 Processor -- start 2024-04-23T18:23:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6ab50bb640 1 -- start start 2024-04-23T18:23:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6ab50bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ab0102a20 0x7f6ab0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6ab50bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab0105350 0x7f6ab0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6ab50bb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ab0107c80 0x7f6ab019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6ab50bb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab010dae0 con 0x7f6ab0105350 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6ab50bb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab010d960 con 0x7f6ab0102a20 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6ab50bb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab010dc60 con 0x7f6ab0107c80 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab0105350 0x7f6ab0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab0105350 0x7f6ab0198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53272/0 (socket says 172.21.15.19:53272) 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6aae59c640 1 -- 172.21.15.19:0/86716684 learned_addr learned my addr 172.21.15.19:0/86716684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6aaed9d640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ab0102a20 0x7f6ab0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6aaf59e640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ab0107c80 0x7f6ab019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6aae59c640 1 -- 172.21.15.19:0/86716684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ab0107c80 msgr2=0x7f6ab019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6aae59c640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ab0107c80 0x7f6ab019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6aae59c640 1 -- 172.21.15.19:0/86716684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ab0102a20 msgr2=0x7f6ab0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.739+0000 7f6aae59c640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ab0102a20 0x7f6ab0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.740+0000 7f6aae59c640 1 -- 172.21.15.19:0/86716684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab019fd20 con 0x7f6ab0105350 2024-04-23T18:23:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.740+0000 7f6aae59c640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab0105350 0x7f6ab0198640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400d770 tx=0x7f6aa400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.740+0000 7f6a97fff640 1 -- 172.21.15.19:0/86716684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4017070 con 0x7f6ab0105350 2024-04-23T18:23:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.740+0000 7f6a97fff640 1 -- 172.21.15.19:0/86716684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa4010460 con 0x7f6ab0105350 2024-04-23T18:23:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.740+0000 7f6ab50bb640 1 -- 172.21.15.19:0/86716684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab019ffb0 con 0x7f6ab0105350 2024-04-23T18:23:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.740+0000 7f6a97fff640 1 -- 172.21.15.19:0/86716684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa400f620 con 0x7f6ab0105350 2024-04-23T18:23:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.740+0000 7f6ab50bb640 1 -- 172.21.15.19:0/86716684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab01a0520 con 0x7f6ab0105350 2024-04-23T18:23:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.741+0000 7f6ab50bb640 1 -- 172.21.15.19:0/86716684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab00fdf20 con 0x7f6ab0105350 2024-04-23T18:23:51.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.742+0000 7f6a97fff640 1 -- 172.21.15.19:0/86716684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6aa401a070 con 0x7f6ab0105350 2024-04-23T18:23:51.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.742+0000 7f6a97fff640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6a980a3c30 0x7f6a980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:51.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.743+0000 7f6aaed9d640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6a980a3c30 0x7f6a980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:51.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.743+0000 7f6a97fff640 1 -- 172.21.15.19:0/86716684 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6aa40cf960 con 0x7f6ab0105350 2024-04-23T18:23:51.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.743+0000 7f6aaed9d640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6a980a3c30 0x7f6a980a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000abe0 tx=0x7f6aa000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.747+0000 7f6a97fff640 1 -- 172.21.15.19:0/86716684 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6aa409c740 con 0x7f6ab0105350 2024-04-23T18:23:51.744 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:51 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] up:boot 2024-04-23T18:23:51.744 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:51 smithi155 ceph-mon[33468]: fsmap 12 up:standby 2024-04-23T18:23:51.744 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:51 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:23:51.744 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:51 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:51.787 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:51 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] up:boot 2024-04-23T18:23:51.787 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:51 smithi044 ceph-mon[32458]: fsmap 12 up:standby 2024-04-23T18:23:51.787 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:51 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:23:51.787 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:51 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:51.852 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:51 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] up:boot 2024-04-23T18:23:51.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:51 smithi019 ceph-mon[29215]: fsmap 12 up:standby 2024-04-23T18:23:51.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:51 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:23:51.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:51 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.975+0000 7f6ab50bb640 1 -- 172.21.15.19:0/86716684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ab0062bf0 con 0x7f6ab0105350 2024-04-23T18:23:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.976+0000 7f6a97fff640 1 -- 172.21.15.19:0/86716684 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10467 (secure 0 0 0) 0x7f6aa409c040 con 0x7f6ab0105350 2024-04-23T18:23:51.969 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:51.969 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-23T18:23:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 13 2024-04-23T18:23:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 -- 172.21.15.19:0/86716684 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6a980a3c30 msgr2=0x7f6a980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6a980a3c30 0x7f6a980a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000abe0 tx=0x7f6aa000a2d0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 -- 172.21.15.19:0/86716684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab0105350 msgr2=0x7f6ab0198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab0105350 0x7f6ab0198640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400d770 tx=0x7f6aa400db70 comp rx=0 tx=0).stop 2024-04-23T18:23:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 -- 172.21.15.19:0/86716684 shutdown_connections 2024-04-23T18:23:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6a980a3c30 0x7f6a980a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ab0107c80 0x7f6ab019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab0105350 0x7f6ab0198640 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 --2- 172.21.15.19:0/86716684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ab0102a20 0x7f6ab0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 -- 172.21.15.19:0/86716684 >> 172.21.15.19:0/86716684 conn(0x7f6ab00fc430 msgr2=0x7f6ab0103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 -- 172.21.15.19:0/86716684 shutdown_connections 2024-04-23T18:23:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:51.978+0000 7f6ab50bb640 1 -- 172.21.15.19:0/86716684 wait complete. 2024-04-23T18:23:51.982 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:23:52.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.093+0000 7f40aaf15640 1 -- 172.21.15.19:0/2271699407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40a4101fe0 msgr2=0x7f40a4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.093+0000 7f40aaf15640 1 --2- 172.21.15.19:0/2271699407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40a4101fe0 0x7f40a4102460 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4094009ca0 tx=0x7f409402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.093+0000 7f40aaf15640 1 -- 172.21.15.19:0/2271699407 shutdown_connections 2024-04-23T18:23:52.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.093+0000 7f40aaf15640 1 --2- 172.21.15.19:0/2271699407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40a41029a0 0x7f40a41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.093+0000 7f40aaf15640 1 --2- 172.21.15.19:0/2271699407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40a4101fe0 0x7f40a4102460 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.093+0000 7f40aaf15640 1 --2- 172.21.15.19:0/2271699407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40a4100e90 0x7f40a41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.093+0000 7f40aaf15640 1 -- 172.21.15.19:0/2271699407 >> 172.21.15.19:0/2271699407 conn(0x7f40a40fc400 msgr2=0x7f40a40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.093+0000 7f40aaf15640 1 -- 172.21.15.19:0/2271699407 shutdown_connections 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.093+0000 7f40aaf15640 1 -- 172.21.15.19:0/2271699407 wait complete. 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40aaf15640 1 Processor -- start 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40aaf15640 1 -- start start 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40aaf15640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40a4100e90 0x7f40a4069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40aaf15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40a4101fe0 0x7f40a419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40aaf15640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40a41029a0 0x7f40a419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40aaf15640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40a410ca20 con 0x7f40a4101fe0 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40aaf15640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40a410c8a0 con 0x7f40a41029a0 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40aaf15640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f40a410cba0 con 0x7f40a4100e90 2024-04-23T18:23:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40a4101fe0 0x7f40a419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40a4101fe0 0x7f40a419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53294/0 (socket says 172.21.15.19:53294) 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a3fff640 1 -- 172.21.15.19:0/144447626 learned_addr learned my addr 172.21.15.19:0/144447626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a8cb1640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40a4100e90 0x7f40a4069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a94b2640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40a41029a0 0x7f40a419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a3fff640 1 -- 172.21.15.19:0/144447626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40a4100e90 msgr2=0x7f40a4069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a3fff640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40a4100e90 0x7f40a4069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a3fff640 1 -- 172.21.15.19:0/144447626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40a41029a0 msgr2=0x7f40a419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a3fff640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40a41029a0 0x7f40a419f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a3fff640 1 -- 172.21.15.19:0/144447626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a419fde0 con 0x7f40a4101fe0 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a3fff640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40a4101fe0 0x7f40a419ceb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4094005e30 tx=0x7f40940317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.094+0000 7f40a1ffb640 1 -- 172.21.15.19:0/144447626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094003bf0 con 0x7f40a4101fe0 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.095+0000 7f40aaf15640 1 -- 172.21.15.19:0/144447626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a41a0070 con 0x7f40a4101fe0 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.095+0000 7f40a1ffb640 1 -- 172.21.15.19:0/144447626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4094003d90 con 0x7f40a4101fe0 2024-04-23T18:23:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.095+0000 7f40a1ffb640 1 -- 172.21.15.19:0/144447626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409402fc50 con 0x7f40a4101fe0 2024-04-23T18:23:52.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.095+0000 7f40aaf15640 1 -- 172.21.15.19:0/144447626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a41a0610 con 0x7f40a4101fe0 2024-04-23T18:23:52.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.096+0000 7f40aaf15640 1 -- 172.21.15.19:0/144447626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a41081a0 con 0x7f40a4101fe0 2024-04-23T18:23:52.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.097+0000 7f40a1ffb640 1 -- 172.21.15.19:0/144447626 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4094004040 con 0x7f40a4101fe0 2024-04-23T18:23:52.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.097+0000 7f40a1ffb640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40900a3bb0 0x7f40900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.098+0000 7f40a8cb1640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40900a3bb0 0x7f40900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.098+0000 7f40a1ffb640 1 -- 172.21.15.19:0/144447626 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f40940f1310 con 0x7f40a4101fe0 2024-04-23T18:23:52.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.098+0000 7f40a8cb1640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40900a3bb0 0x7f40900a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f408c009bd0 tx=0x7f408c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:52.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.102+0000 7f40a1ffb640 1 -- 172.21.15.19:0/144447626 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40940be220 con 0x7f40a4101fe0 2024-04-23T18:23:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.296+0000 7f40aaf15640 1 -- 172.21.15.19:0/144447626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40a402d770 con 0x7f40a4101fe0 2024-04-23T18:23:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.297+0000 7f40a1ffb640 1 -- 172.21.15.19:0/144447626 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18446 (secure 0 0 0) 0x7f40940bdb20 con 0x7f40a4101fe0 2024-04-23T18:23:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:52.291 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:23:03.173026+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.299+0000 7f40aaf15640 1 -- 172.21.15.19:0/144447626 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40900a3bb0 msgr2=0x7f40900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.299+0000 7f40aaf15640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40900a3bb0 0x7f40900a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f408c009bd0 tx=0x7f408c005f90 comp rx=0 tx=0).stop 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.299+0000 7f40aaf15640 1 -- 172.21.15.19:0/144447626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40a4101fe0 msgr2=0x7f40a419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.299+0000 7f40aaf15640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40a4101fe0 0x7f40a419ceb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4094005e30 tx=0x7f40940317e0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.299+0000 7f40aaf15640 1 -- 172.21.15.19:0/144447626 shutdown_connections 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.299+0000 7f40aaf15640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40900a3bb0 0x7f40900a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.299+0000 7f40aaf15640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40a41029a0 0x7f40a419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.299+0000 7f40aaf15640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40a4101fe0 0x7f40a419ceb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.299+0000 7f40aaf15640 1 --2- 172.21.15.19:0/144447626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40a4100e90 0x7f40a4069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.299+0000 7f40aaf15640 1 -- 172.21.15.19:0/144447626 >> 172.21.15.19:0/144447626 conn(0x7f40a40fc400 msgr2=0x7f40a4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:52.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.300+0000 7f40aaf15640 1 -- 172.21.15.19:0/144447626 shutdown_connections 2024-04-23T18:23:52.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.300+0000 7f40aaf15640 1 -- 172.21.15.19:0/144447626 wait complete. 2024-04-23T18:23:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:23:52.306 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:23:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.437+0000 7f810fd4f640 1 -- 172.21.15.19:0/109497640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8108101fe0 msgr2=0x7f8108102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.437+0000 7f810fd4f640 1 --2- 172.21.15.19:0/109497640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8108101fe0 0x7f8108102460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009860 tx=0x7f80f802a340 comp rx=0 tx=0).stop 2024-04-23T18:23:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.438+0000 7f810fd4f640 1 -- 172.21.15.19:0/109497640 shutdown_connections 2024-04-23T18:23:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.438+0000 7f810fd4f640 1 --2- 172.21.15.19:0/109497640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81081029a0 0x7f81081092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.438+0000 7f810fd4f640 1 --2- 172.21.15.19:0/109497640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8108101fe0 0x7f8108102460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.438+0000 7f810fd4f640 1 --2- 172.21.15.19:0/109497640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108100e90 0x7f81081012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.438+0000 7f810fd4f640 1 -- 172.21.15.19:0/109497640 >> 172.21.15.19:0/109497640 conn(0x7f81080fc400 msgr2=0x7f81080fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.438+0000 7f810fd4f640 1 -- 172.21.15.19:0/109497640 shutdown_connections 2024-04-23T18:23:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.438+0000 7f810fd4f640 1 -- 172.21.15.19:0/109497640 wait complete. 2024-04-23T18:23:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.438+0000 7f810fd4f640 1 Processor -- start 2024-04-23T18:23:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.438+0000 7f810fd4f640 1 -- start start 2024-04-23T18:23:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810fd4f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108100e90 0x7f81081981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810fd4f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8108101fe0 0x7f8108198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810fd4f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81081029a0 0x7f810819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810fd4f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f810810c9e0 con 0x7f8108100e90 2024-04-23T18:23:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810fd4f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f810810c860 con 0x7f81081029a0 2024-04-23T18:23:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810fd4f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f810810cb60 con 0x7f8108101fe0 2024-04-23T18:23:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810daeb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108100e90 0x7f81081981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810daeb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108100e90 0x7f81081981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53316/0 (socket says 172.21.15.19:53316) 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810daeb640 1 -- 172.21.15.19:0/3507238090 learned_addr learned my addr 172.21.15.19:0/3507238090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810d2ea640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8108101fe0 0x7f8108198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810e2ec640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81081029a0 0x7f810819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810daeb640 1 -- 172.21.15.19:0/3507238090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8108101fe0 msgr2=0x7f8108198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810daeb640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8108101fe0 0x7f8108198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810daeb640 1 -- 172.21.15.19:0/3507238090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81081029a0 msgr2=0x7f810819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810daeb640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81081029a0 0x7f810819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810daeb640 1 -- 172.21.15.19:0/3507238090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810819fe00 con 0x7f8108100e90 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810e2ec640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81081029a0 0x7f810819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810daeb640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108100e90 0x7f81081981e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f810400e9c0 tx=0x7f810400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f80feffd640 1 -- 172.21.15.19:0/3507238090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810400ccc0 con 0x7f8108100e90 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.439+0000 7f810fd4f640 1 -- 172.21.15.19:0/3507238090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81081a00f0 con 0x7f8108100e90 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.440+0000 7f80feffd640 1 -- 172.21.15.19:0/3507238090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81040054d0 con 0x7f8108100e90 2024-04-23T18:23:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.440+0000 7f80feffd640 1 -- 172.21.15.19:0/3507238090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104010430 con 0x7f8108100e90 2024-04-23T18:23:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.440+0000 7f810fd4f640 1 -- 172.21.15.19:0/3507238090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81081a0660 con 0x7f8108100e90 2024-04-23T18:23:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.441+0000 7f810fd4f640 1 -- 172.21.15.19:0/3507238090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81081a0920 con 0x7f8108100e90 2024-04-23T18:23:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.442+0000 7f80feffd640 1 -- 172.21.15.19:0/3507238090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8104004f80 con 0x7f8108100e90 2024-04-23T18:23:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.442+0000 7f80feffd640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80f40a3bb0 0x7f80f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.443+0000 7f810d2ea640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80f40a3bb0 0x7f80f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.443+0000 7f80feffd640 1 -- 172.21.15.19:0/3507238090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f81040d0760 con 0x7f8108100e90 2024-04-23T18:23:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.443+0000 7f810d2ea640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80f40a3bb0 0x7f80f40a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f80f802a930 tx=0x7f80f8030470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.446+0000 7f80feffd640 1 -- 172.21.15.19:0/3507238090 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f810409d5e0 con 0x7f8108100e90 2024-04-23T18:23:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.653+0000 7f810fd4f640 1 -- 172.21.15.19:0/3507238090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81081a0af0 con 0x7f8108100e90 2024-04-23T18:23:52.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.654+0000 7f80feffd640 1 -- 172.21.15.19:0/3507238090 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f810409cee0 con 0x7f8108100e90 2024-04-23T18:23:52.647 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:23:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 -- 172.21.15.19:0/3507238090 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80f40a3bb0 msgr2=0x7f80f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80f40a3bb0 0x7f80f40a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f80f802a930 tx=0x7f80f8030470 comp rx=0 tx=0).stop 2024-04-23T18:23:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 -- 172.21.15.19:0/3507238090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108100e90 msgr2=0x7f81081981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108100e90 0x7f81081981e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f810400e9c0 tx=0x7f810400ee80 comp rx=0 tx=0).stop 2024-04-23T18:23:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 -- 172.21.15.19:0/3507238090 shutdown_connections 2024-04-23T18:23:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80f40a3bb0 0x7f80f40a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81081029a0 0x7f810819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8108101fe0 0x7f8108198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 --2- 172.21.15.19:0/3507238090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108100e90 0x7f81081981e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 -- 172.21.15.19:0/3507238090 >> 172.21.15.19:0/3507238090 conn(0x7f81080fc400 msgr2=0x7f81080690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 -- 172.21.15.19:0/3507238090 shutdown_connections 2024-04-23T18:23:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.656+0000 7f810fd4f640 1 -- 172.21.15.19:0/3507238090 wait complete. 2024-04-23T18:23:52.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:52 smithi019 ceph-mon[29215]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:52.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:52 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/86716684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:52.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:52 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/144447626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:52.660 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:52.660 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:52 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:52.661 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.772+0000 7f1fd0efd640 1 -- 172.21.15.19:0/3601927489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fcc1029a0 msgr2=0x7f1fcc1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.772+0000 7f1fd0efd640 1 --2- 172.21.15.19:0/3601927489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fcc1029a0 0x7f1fcc1092c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009ca0 tx=0x7f1fc402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.773+0000 7f1fd0efd640 1 -- 172.21.15.19:0/3601927489 shutdown_connections 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.773+0000 7f1fd0efd640 1 --2- 172.21.15.19:0/3601927489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fcc101fe0 0x7f1fcc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.773+0000 7f1fd0efd640 1 --2- 172.21.15.19:0/3601927489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fcc1029a0 0x7f1fcc1092c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.773+0000 7f1fd0efd640 1 --2- 172.21.15.19:0/3601927489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc100e90 0x7f1fcc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.773+0000 7f1fd0efd640 1 -- 172.21.15.19:0/3601927489 >> 172.21.15.19:0/3601927489 conn(0x7f1fcc0fc400 msgr2=0x7f1fcc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.773+0000 7f1fd0efd640 1 -- 172.21.15.19:0/3601927489 shutdown_connections 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.773+0000 7f1fd0efd640 1 -- 172.21.15.19:0/3601927489 wait complete. 2024-04-23T18:23:52.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.773+0000 7f1fd0efd640 1 Processor -- start 2024-04-23T18:23:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.773+0000 7f1fd0efd640 1 -- start start 2024-04-23T18:23:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.773+0000 7f1fd0efd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc100e90 0x7f1fcc197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fd0efd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fcc101fe0 0x7f1fcc1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fd0efd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fcc1029a0 0x7f1fcc19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fd0efd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc10c9f0 con 0x7f1fcc1029a0 2024-04-23T18:23:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fd0efd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc10c870 con 0x7f1fcc101fe0 2024-04-23T18:23:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fd0efd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc10cb70 con 0x7f1fcc100e90 2024-04-23T18:23:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fcad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fcc1029a0 0x7f1fcc19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fc9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fcc101fe0 0x7f1fcc1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fc9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fcc101fe0 0x7f1fcc1975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:50836/0 (socket says 172.21.15.19:50836) 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fcad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fcc1029a0 0x7f1fcc19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53342/0 (socket says 172.21.15.19:53342) 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fc9d9b640 1 -- 172.21.15.19:0/1678748386 learned_addr learned my addr 172.21.15.19:0/1678748386 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fca59c640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc100e90 0x7f1fcc197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fc9d9b640 1 -- 172.21.15.19:0/1678748386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc100e90 msgr2=0x7f1fcc197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fc9d9b640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc100e90 0x7f1fcc197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fc9d9b640 1 -- 172.21.15.19:0/1678748386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fcc1029a0 msgr2=0x7f1fcc19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fc9d9b640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fcc1029a0 0x7f1fcc19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fc9d9b640 1 -- 172.21.15.19:0/1678748386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc19ecb0 con 0x7f1fcc101fe0 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fca59c640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc100e90 0x7f1fcc197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fc9d9b640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fcc101fe0 0x7f1fcc1975d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000e9c0 tx=0x7f1fc000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fbb7fe640 1 -- 172.21.15.19:0/1678748386 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc000cde0 con 0x7f1fcc101fe0 2024-04-23T18:23:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fd0efd640 1 -- 172.21.15.19:0/1678748386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc19efa0 con 0x7f1fcc101fe0 2024-04-23T18:23:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fbb7fe640 1 -- 172.21.15.19:0/1678748386 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fc00108e0 con 0x7f1fcc101fe0 2024-04-23T18:23:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fbb7fe640 1 -- 172.21.15.19:0/1678748386 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00188d0 con 0x7f1fcc101fe0 2024-04-23T18:23:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.774+0000 7f1fd0efd640 1 -- 172.21.15.19:0/1678748386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc19f4b0 con 0x7f1fcc101fe0 2024-04-23T18:23:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.775+0000 7f1fd0efd640 1 -- 172.21.15.19:0/1678748386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc1081a0 con 0x7f1fcc101fe0 2024-04-23T18:23:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.777+0000 7f1fbb7fe640 1 -- 172.21.15.19:0/1678748386 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1fc001f020 con 0x7f1fcc101fe0 2024-04-23T18:23:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.777+0000 7f1fbb7fe640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1fb40a3bb0 0x7f1fb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.777+0000 7f1fca59c640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1fb40a3bb0 0x7f1fb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.777+0000 7f1fbb7fe640 1 -- 172.21.15.19:0/1678748386 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1fc00d03f0 con 0x7f1fcc101fe0 2024-04-23T18:23:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.778+0000 7f1fca59c640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1fb40a3bb0 0x7f1fb40a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009b50 tx=0x7f1fbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.781+0000 7f1fbb7fe640 1 -- 172.21.15.19:0/1678748386 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fc009d250 con 0x7f1fcc101fe0 2024-04-23T18:23:52.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:52 smithi044 ceph-mon[32458]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:52.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:52 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/86716684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:52.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:52 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/144447626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:52.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:52.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:52 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:52.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:52 smithi155 ceph-mon[33468]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:52.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:52 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/86716684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:52.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:52 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/144447626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:52.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:52.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:52 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.970+0000 7f1fd0efd640 1 -- 172.21.15.19:0/1678748386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1fcc02d740 con 0x7f1fcc101fe0 2024-04-23T18:23:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.970+0000 7f1fbb7fe640 1 -- 172.21.15.19:0/1678748386 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f1fc009cb50 con 0x7f1fcc101fe0 2024-04-23T18:23:52.964 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:52.964 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-23T18:23:52.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:23:52.964 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-23T18:23:52.964 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-23T18:23:52.964 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:23:52.964 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.973+0000 7f1fd0efd640 1 -- 172.21.15.19:0/1678748386 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1fb40a3bb0 msgr2=0x7f1fb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.973+0000 7f1fd0efd640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1fb40a3bb0 0x7f1fb40a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009b50 tx=0x7f1fbc005f90 comp rx=0 tx=0).stop 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.973+0000 7f1fd0efd640 1 -- 172.21.15.19:0/1678748386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fcc101fe0 msgr2=0x7f1fcc1975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.973+0000 7f1fd0efd640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fcc101fe0 0x7f1fcc1975d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000e9c0 tx=0x7f1fc000ee80 comp rx=0 tx=0).stop 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.973+0000 7f1fd0efd640 1 -- 172.21.15.19:0/1678748386 shutdown_connections 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.973+0000 7f1fd0efd640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1fb40a3bb0 0x7f1fb40a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.973+0000 7f1fd0efd640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fcc101fe0 0x7f1fcc1975d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.973+0000 7f1fd0efd640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fcc1029a0 0x7f1fcc19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.973+0000 7f1fd0efd640 1 --2- 172.21.15.19:0/1678748386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc100e90 0x7f1fcc197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.973+0000 7f1fd0efd640 1 -- 172.21.15.19:0/1678748386 >> 172.21.15.19:0/1678748386 conn(0x7f1fcc0fc400 msgr2=0x7f1fcc105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.974+0000 7f1fd0efd640 1 -- 172.21.15.19:0/1678748386 shutdown_connections 2024-04-23T18:23:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:52.974+0000 7f1fd0efd640 1 -- 172.21.15.19:0/1678748386 wait complete. 2024-04-23T18:23:52.978 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T18:23:52.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T18:23:53.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.092+0000 7f5dd89e8640 1 -- 172.21.15.19:0/169093044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4101fe0 msgr2=0x7f5dd4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:53.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.092+0000 7f5dd89e8640 1 --2- 172.21.15.19:0/169093044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4101fe0 0x7f5dd4102460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009900 tx=0x7f5dc802a410 comp rx=0 tx=0).stop 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.093+0000 7f5dd89e8640 1 -- 172.21.15.19:0/169093044 shutdown_connections 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.093+0000 7f5dd89e8640 1 --2- 172.21.15.19:0/169093044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dd41029a0 0x7f5dd41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.093+0000 7f5dd89e8640 1 --2- 172.21.15.19:0/169093044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4101fe0 0x7f5dd4102460 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.093+0000 7f5dd89e8640 1 --2- 172.21.15.19:0/169093044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd4100e90 0x7f5dd41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.093+0000 7f5dd89e8640 1 -- 172.21.15.19:0/169093044 >> 172.21.15.19:0/169093044 conn(0x7f5dd40fc400 msgr2=0x7f5dd40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.093+0000 7f5dd89e8640 1 -- 172.21.15.19:0/169093044 shutdown_connections 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.093+0000 7f5dd89e8640 1 -- 172.21.15.19:0/169093044 wait complete. 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.093+0000 7f5dd89e8640 1 Processor -- start 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.093+0000 7f5dd89e8640 1 -- start start 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd89e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4100e90 0x7f5dd4069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd89e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dd4101fe0 0x7f5dd419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd89e8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd41029a0 0x7f5dd419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd89e8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd410c9f0 con 0x7f5dd4100e90 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd89e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd410c870 con 0x7f5dd4101fe0 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd89e8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd410cb70 con 0x7f5dd41029a0 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4100e90 0x7f5dd4069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4100e90 0x7f5dd4069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53372/0 (socket says 172.21.15.19:53372) 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd259c640 1 -- 172.21.15.19:0/2410742041 learned_addr learned my addr 172.21.15.19:0/2410742041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd1d9b640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dd4101fe0 0x7f5dd419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd2d9d640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd41029a0 0x7f5dd419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd259c640 1 -- 172.21.15.19:0/2410742041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd41029a0 msgr2=0x7f5dd419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd259c640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd41029a0 0x7f5dd419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd259c640 1 -- 172.21.15.19:0/2410742041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dd4101fe0 msgr2=0x7f5dd419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd259c640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dd4101fe0 0x7f5dd419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:53.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd259c640 1 -- 172.21.15.19:0/2410742041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd419fed0 con 0x7f5dd4100e90 2024-04-23T18:23:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd2d9d640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd41029a0 0x7f5dd419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.094+0000 7f5dd259c640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4100e90 0x7f5dd4069c50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00cce0 tx=0x7f5dcc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.095+0000 7f5dc37fe640 1 -- 172.21.15.19:0/2410742041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc004b20 con 0x7f5dd4100e90 2024-04-23T18:23:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.095+0000 7f5dd89e8640 1 -- 172.21.15.19:0/2410742041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd41a0160 con 0x7f5dd4100e90 2024-04-23T18:23:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.095+0000 7f5dc37fe640 1 -- 172.21.15.19:0/2410742041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dcc004cc0 con 0x7f5dd4100e90 2024-04-23T18:23:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.095+0000 7f5dc37fe640 1 -- 172.21.15.19:0/2410742041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00f7c0 con 0x7f5dd4100e90 2024-04-23T18:23:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.095+0000 7f5dd89e8640 1 -- 172.21.15.19:0/2410742041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd41a0630 con 0x7f5dd4100e90 2024-04-23T18:23:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.096+0000 7f5dd89e8640 1 -- 172.21.15.19:0/2410742041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd4108310 con 0x7f5dd4100e90 2024-04-23T18:23:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.097+0000 7f5dc37fe640 1 -- 172.21.15.19:0/2410742041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5dcc005560 con 0x7f5dd4100e90 2024-04-23T18:23:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.097+0000 7f5dc37fe640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5dbc0a3bb0 0x7f5dbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.098+0000 7f5dd1d9b640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5dbc0a3bb0 0x7f5dbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.098+0000 7f5dc37fe640 1 -- 172.21.15.19:0/2410742041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5dcc0d0400 con 0x7f5dd4100e90 2024-04-23T18:23:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.098+0000 7f5dd1d9b640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5dbc0a3bb0 0x7f5dbc0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009230 tx=0x7f5dc80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:53.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.101+0000 7f5dc37fe640 1 -- 172.21.15.19:0/2410742041 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dcc014030 con 0x7f5dd4100e90 2024-04-23T18:23:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:53.292+0000 7f5dd89e8640 1 -- 172.21.15.19:0/2410742041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5dd402d5c0 con 0x7f5dd4100e90 2024-04-23T18:23:54.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3507238090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:54.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1678748386' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:23:54.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:54.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:54.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2410742041' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:23:54.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:53 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:54.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3507238090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:54.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1678748386' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:23:54.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:54.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:54.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2410742041' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:23:54.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:53 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:54.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3507238090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:54.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1678748386' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:23:54.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:54.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:54.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:54.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2410742041' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:23:54.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:53 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.241+0000 7f5dc37fe640 1 -- 172.21.15.19:0/2410742041 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f5dcc09d280 con 0x7f5dd4100e90 2024-04-23T18:23:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-23T18:23:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.244+0000 7f5dd89e8640 1 -- 172.21.15.19:0/2410742041 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5dbc0a3bb0 msgr2=0x7f5dbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.244+0000 7f5dd89e8640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5dbc0a3bb0 0x7f5dbc0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009230 tx=0x7f5dc80314a0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.244+0000 7f5dd89e8640 1 -- 172.21.15.19:0/2410742041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4100e90 msgr2=0x7f5dd4069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.244+0000 7f5dd89e8640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4100e90 0x7f5dd4069c50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00cce0 tx=0x7f5dcc00a620 comp rx=0 tx=0).stop 2024-04-23T18:23:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.245+0000 7f5dd89e8640 1 -- 172.21.15.19:0/2410742041 shutdown_connections 2024-04-23T18:23:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.245+0000 7f5dd89e8640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5dbc0a3bb0 0x7f5dbc0a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.245+0000 7f5dd89e8640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dd4101fe0 0x7f5dd419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.245+0000 7f5dd89e8640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd41029a0 0x7f5dd419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.245+0000 7f5dd89e8640 1 --2- 172.21.15.19:0/2410742041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4100e90 0x7f5dd4069c50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.245+0000 7f5dd89e8640 1 -- 172.21.15.19:0/2410742041 >> 172.21.15.19:0/2410742041 conn(0x7f5dd40fc400 msgr2=0x7f5dd41054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.245+0000 7f5dd89e8640 1 -- 172.21.15.19:0/2410742041 shutdown_connections 2024-04-23T18:23:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.245+0000 7f5dd89e8640 1 -- 172.21.15.19:0/2410742041 wait complete. 2024-04-23T18:23:54.252 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T18:23:54.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.372+0000 7f944c405640 1 -- 172.21.15.19:0/3934693927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444103190 msgr2=0x7f9444103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:54.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.372+0000 7f944c405640 1 --2- 172.21.15.19:0/3934693927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444103190 0x7f9444103610 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9438009900 tx=0x7f943802a410 comp rx=0 tx=0).stop 2024-04-23T18:23:54.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.372+0000 7f944c405640 1 -- 172.21.15.19:0/3934693927 shutdown_connections 2024-04-23T18:23:54.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.372+0000 7f944c405640 1 --2- 172.21.15.19:0/3934693927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9444103b50 0x7f944410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.372+0000 7f944c405640 1 --2- 172.21.15.19:0/3934693927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444103190 0x7f9444103610 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.372+0000 7f944c405640 1 --2- 172.21.15.19:0/3934693927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9444101fa0 0x7f94441023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.372+0000 7f944c405640 1 -- 172.21.15.19:0/3934693927 >> 172.21.15.19:0/3934693927 conn(0x7f94440fd530 msgr2=0x7f94440ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.372+0000 7f944c405640 1 -- 172.21.15.19:0/3934693927 shutdown_connections 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.372+0000 7f944c405640 1 -- 172.21.15.19:0/3934693927 wait complete. 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.372+0000 7f944c405640 1 Processor -- start 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944c405640 1 -- start start 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944c405640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9444101fa0 0x7f94441981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944c405640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9444103190 0x7f9444198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944c405640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444103b50 0x7f944419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944c405640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f944410dae0 con 0x7f9444103b50 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944c405640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f944410d960 con 0x7f9444103190 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944c405640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f944410dc60 con 0x7f9444101fa0 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944a9a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444103b50 0x7f944419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944a9a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444103b50 0x7f944419f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53400/0 (socket says 172.21.15.19:53400) 2024-04-23T18:23:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944a9a2640 1 -- 172.21.15.19:0/3016855847 learned_addr learned my addr 172.21.15.19:0/3016855847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944a1a1640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9444101fa0 0x7f94441981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f94499a0640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9444103190 0x7f9444198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944a9a2640 1 -- 172.21.15.19:0/3016855847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9444101fa0 msgr2=0x7f94441981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944a9a2640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9444101fa0 0x7f94441981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944a9a2640 1 -- 172.21.15.19:0/3016855847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9444103190 msgr2=0x7f9444198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944a9a2640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9444103190 0x7f9444198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944a9a2640 1 -- 172.21.15.19:0/3016855847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944419fe10 con 0x7f9444103b50 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.373+0000 7f944a9a2640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444103b50 0x7f944419f7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f943400eaa0 tx=0x7f943400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.374+0000 7f94437fe640 1 -- 172.21.15.19:0/3016855847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943400cdb0 con 0x7f9444103b50 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.374+0000 7f944c405640 1 -- 172.21.15.19:0/3016855847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94441a0100 con 0x7f9444103b50 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.374+0000 7f94437fe640 1 -- 172.21.15.19:0/3016855847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9434005020 con 0x7f9444103b50 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.374+0000 7f94437fe640 1 -- 172.21.15.19:0/3016855847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94340116d0 con 0x7f9444103b50 2024-04-23T18:23:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.374+0000 7f944c405640 1 -- 172.21.15.19:0/3016855847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94441a0620 con 0x7f9444103b50 2024-04-23T18:23:54.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.375+0000 7f944c405640 1 -- 172.21.15.19:0/3016855847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94441093e0 con 0x7f9444103b50 2024-04-23T18:23:54.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.376+0000 7f94437fe640 1 -- 172.21.15.19:0/3016855847 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f94340051d0 con 0x7f9444103b50 2024-04-23T18:23:54.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.376+0000 7f94437fe640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f942c0a3bb0 0x7f942c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:54.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.377+0000 7f94437fe640 1 -- 172.21.15.19:0/3016855847 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f94340d0610 con 0x7f9444103b50 2024-04-23T18:23:54.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.377+0000 7f944a1a1640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f942c0a3bb0 0x7f942c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:54.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.377+0000 7f944a1a1640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f942c0a3bb0 0x7f942c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f943c0058c0 tx=0x7f943c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:54.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.381+0000 7f94437fe640 1 -- 172.21.15.19:0/3016855847 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9434016030 con 0x7f9444103b50 2024-04-23T18:23:54.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:54.569+0000 7f944c405640 1 -- 172.21.15.19:0/3016855847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f944402d5c0 con 0x7f9444103b50 2024-04-23T18:23:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:54 smithi019 ceph-mon[29215]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:54 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2410742041' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:23:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:54 smithi019 ceph-mon[29215]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:23:54.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:54 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3016855847' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:23:54.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:54 smithi044 ceph-mon[32458]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:54.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:54 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2410742041' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:23:54.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:54 smithi044 ceph-mon[32458]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:23:54.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:54 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3016855847' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:23:54.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:54 smithi155 ceph-mon[33468]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:54 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2410742041' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:23:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:54 smithi155 ceph-mon[33468]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:23:54.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:54 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3016855847' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:23:55.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.243+0000 7f94437fe640 1 -- 172.21.15.19:0/3016855847 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f943409ccd0 con 0x7f9444103b50 2024-04-23T18:23:55.236 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-23T18:23:55.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.246+0000 7f944c405640 1 -- 172.21.15.19:0/3016855847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f942c0a3bb0 msgr2=0x7f942c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:55.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.246+0000 7f944c405640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f942c0a3bb0 0x7f942c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f943c0058c0 tx=0x7f943c00a2d0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.246+0000 7f944c405640 1 -- 172.21.15.19:0/3016855847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444103b50 msgr2=0x7f944419f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:55.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.246+0000 7f944c405640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444103b50 0x7f944419f7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f943400eaa0 tx=0x7f943400ef60 comp rx=0 tx=0).stop 2024-04-23T18:23:55.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.246+0000 7f944c405640 1 -- 172.21.15.19:0/3016855847 shutdown_connections 2024-04-23T18:23:55.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.247+0000 7f944c405640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f942c0a3bb0 0x7f942c0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.247+0000 7f944c405640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9444103190 0x7f9444198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.247+0000 7f944c405640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444103b50 0x7f944419f7a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.247+0000 7f944c405640 1 --2- 172.21.15.19:0/3016855847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9444101fa0 0x7f94441981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.247+0000 7f944c405640 1 -- 172.21.15.19:0/3016855847 >> 172.21.15.19:0/3016855847 conn(0x7f94440fd530 msgr2=0x7f94441065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:55.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.248+0000 7f944c405640 1 -- 172.21.15.19:0/3016855847 shutdown_connections 2024-04-23T18:23:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.248+0000 7f944c405640 1 -- 172.21.15.19:0/3016855847 wait complete. 2024-04-23T18:23:55.262 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T18:23:55.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.375+0000 7f44d8864640 1 -- 172.21.15.19:0/630008651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44d41011e0 msgr2=0x7f44d4101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:55.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.375+0000 7f44d8864640 1 --2- 172.21.15.19:0/630008651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44d41011e0 0x7f44d4101640 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009900 tx=0x7f44c802a410 comp rx=0 tx=0).stop 2024-04-23T18:23:55.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.375+0000 7f44d8864640 1 -- 172.21.15.19:0/630008651 shutdown_connections 2024-04-23T18:23:55.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.375+0000 7f44d8864640 1 --2- 172.21.15.19:0/630008651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d4101b80 0x7f44d4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.375+0000 7f44d8864640 1 --2- 172.21.15.19:0/630008651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44d41011e0 0x7f44d4101640 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.375+0000 7f44d8864640 1 --2- 172.21.15.19:0/630008651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d4100880 0x7f44d4100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.375+0000 7f44d8864640 1 -- 172.21.15.19:0/630008651 >> 172.21.15.19:0/630008651 conn(0x7f44d40fc440 msgr2=0x7f44d40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:55.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.375+0000 7f44d8864640 1 -- 172.21.15.19:0/630008651 shutdown_connections 2024-04-23T18:23:55.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.375+0000 7f44d8864640 1 -- 172.21.15.19:0/630008651 wait complete. 2024-04-23T18:23:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d8864640 1 Processor -- start 2024-04-23T18:23:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d8864640 1 -- start start 2024-04-23T18:23:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d8864640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44d4100880 0x7f44d419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d8864640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d41011e0 0x7f44d419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d8864640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d4101b80 0x7f44d41a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d8864640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410fd50 con 0x7f44d4100880 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d8864640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410fbd0 con 0x7f44d41011e0 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d8864640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410fed0 con 0x7f44d4101b80 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44d4100880 0x7f44d419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44d4100880 0x7f44d419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53416/0 (socket says 172.21.15.19:53416) 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d259c640 1 -- 172.21.15.19:0/4131898875 learned_addr learned my addr 172.21.15.19:0/4131898875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d1d9b640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d41011e0 0x7f44d419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d2d9d640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d4101b80 0x7f44d41a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d259c640 1 -- 172.21.15.19:0/4131898875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d4101b80 msgr2=0x7f44d41a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d259c640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d4101b80 0x7f44d41a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d259c640 1 -- 172.21.15.19:0/4131898875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d41011e0 msgr2=0x7f44d419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d259c640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d41011e0 0x7f44d419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.376+0000 7f44d259c640 1 -- 172.21.15.19:0/4131898875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d41a2060 con 0x7f44d4100880 2024-04-23T18:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.377+0000 7f44d259c640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44d4100880 0x7f44d419a440 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f44c400d770 tx=0x7f44c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:55.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.377+0000 7f44c37fe640 1 -- 172.21.15.19:0/4131898875 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4017070 con 0x7f44d4100880 2024-04-23T18:23:55.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.377+0000 7f44c37fe640 1 -- 172.21.15.19:0/4131898875 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44c40054d0 con 0x7f44d4100880 2024-04-23T18:23:55.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.377+0000 7f44d8864640 1 -- 172.21.15.19:0/4131898875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d41a2350 con 0x7f44d4100880 2024-04-23T18:23:55.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.377+0000 7f44c37fe640 1 -- 172.21.15.19:0/4131898875 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4004c30 con 0x7f44d4100880 2024-04-23T18:23:55.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.377+0000 7f44d8864640 1 -- 172.21.15.19:0/4131898875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d41a2870 con 0x7f44d4100880 2024-04-23T18:23:55.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.378+0000 7f44d8864640 1 -- 172.21.15.19:0/4131898875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d410b5d0 con 0x7f44d4100880 2024-04-23T18:23:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.379+0000 7f44c37fe640 1 -- 172.21.15.19:0/4131898875 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f44c4004dd0 con 0x7f44d4100880 2024-04-23T18:23:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.380+0000 7f44c37fe640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44bc0a3c30 0x7f44bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.380+0000 7f44d1d9b640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44bc0a3c30 0x7f44bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.380+0000 7f44c37fe640 1 -- 172.21.15.19:0/4131898875 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f44c40d0160 con 0x7f44d4100880 2024-04-23T18:23:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.380+0000 7f44d1d9b640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44bc0a3c30 0x7f44bc0a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f44c8031500 tx=0x7f44c8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.384+0000 7f44c37fe640 1 -- 172.21.15.19:0/4131898875 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44c409ccd0 con 0x7f44d4100880 2024-04-23T18:23:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:55.610+0000 7f44d8864640 1 -- 172.21.15.19:0/4131898875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f44d402d5c0 con 0x7f44d4100880 2024-04-23T18:23:56.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:56.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:56.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3016855847' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:23:56.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:23:56.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4131898875' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:23:56.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:56 smithi155 ceph-mon[33468]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:23:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.346+0000 7f44c37fe640 1 -- 172.21.15.19:0/4131898875 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f44c409c5d0 con 0x7f44d4100880 2024-04-23T18:23:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-04-23T18:23:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 -- 172.21.15.19:0/4131898875 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44bc0a3c30 msgr2=0x7f44bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44bc0a3c30 0x7f44bc0a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f44c8031500 tx=0x7f44c8005a20 comp rx=0 tx=0).stop 2024-04-23T18:23:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 -- 172.21.15.19:0/4131898875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44d4100880 msgr2=0x7f44d419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:56.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44d4100880 0x7f44d419a440 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f44c400d770 tx=0x7f44c400dc30 comp rx=0 tx=0).stop 2024-04-23T18:23:56.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 -- 172.21.15.19:0/4131898875 shutdown_connections 2024-04-23T18:23:56.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44bc0a3c30 0x7f44bc0a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d41011e0 0x7f44d419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d4101b80 0x7f44d41a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 --2- 172.21.15.19:0/4131898875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44d4100880 0x7f44d419a440 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 -- 172.21.15.19:0/4131898875 >> 172.21.15.19:0/4131898875 conn(0x7f44d40fc440 msgr2=0x7f44d40ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:56.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 -- 172.21.15.19:0/4131898875 shutdown_connections 2024-04-23T18:23:56.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.348+0000 7f44d8864640 1 -- 172.21.15.19:0/4131898875 wait complete. 2024-04-23T18:23:56.355 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.355 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.355 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3016855847' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4131898875' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:23:56.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:56 smithi019 ceph-mon[29215]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:23:56.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:23:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.474+0000 7fa63ea8a640 1 -- 172.21.15.19:0/801203338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381029a0 msgr2=0x7fa6381092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.474+0000 7fa63ea8a640 1 --2- 172.21.15.19:0/801203338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381029a0 0x7fa6381092c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa630009ca0 tx=0x7fa63002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.474+0000 7fa63ea8a640 1 -- 172.21.15.19:0/801203338 shutdown_connections 2024-04-23T18:23:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.474+0000 7fa63ea8a640 1 --2- 172.21.15.19:0/801203338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381029a0 0x7fa6381092c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.474+0000 7fa63ea8a640 1 --2- 172.21.15.19:0/801203338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa638101fe0 0x7fa638102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.474+0000 7fa63ea8a640 1 --2- 172.21.15.19:0/801203338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa638100e90 0x7fa6381012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.474+0000 7fa63ea8a640 1 -- 172.21.15.19:0/801203338 >> 172.21.15.19:0/801203338 conn(0x7fa6380fc400 msgr2=0x7fa6380fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.474+0000 7fa63ea8a640 1 -- 172.21.15.19:0/801203338 shutdown_connections 2024-04-23T18:23:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.474+0000 7fa63ea8a640 1 -- 172.21.15.19:0/801203338 wait complete. 2024-04-23T18:23:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63ea8a640 1 Processor -- start 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63ea8a640 1 -- start start 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63ea8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa638100e90 0x7fa638197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63ea8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa638101fe0 0x7fa638197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63ea8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6381029a0 0x7fa63819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63ea8a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa63810ca20 con 0x7fa638101fe0 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63ea8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa63810c8a0 con 0x7fa6381029a0 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63ea8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa63810cba0 con 0x7fa638100e90 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa637fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa638101fe0 0x7fa638197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa637fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa638101fe0 0x7fa638197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53434/0 (socket says 172.21.15.19:53434) 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa637fff640 1 -- 172.21.15.19:0/3334565822 learned_addr learned my addr 172.21.15.19:0/3334565822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63c826640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa638100e90 0x7fa638197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa637fff640 1 -- 172.21.15.19:0/3334565822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa638100e90 msgr2=0x7fa638197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63d027640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6381029a0 0x7fa63819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa637fff640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa638100e90 0x7fa638197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa637fff640 1 -- 172.21.15.19:0/3334565822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6381029a0 msgr2=0x7fa63819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa637fff640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6381029a0 0x7fa63819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa637fff640 1 -- 172.21.15.19:0/3334565822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63819ec60 con 0x7fa638101fe0 2024-04-23T18:23:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa63c826640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa638100e90 0x7fa638197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.475+0000 7fa637fff640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa638101fe0 0x7fa638197580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa62400d770 tx=0x7fa62400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.476+0000 7fa635ffb640 1 -- 172.21.15.19:0/3334565822 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624017070 con 0x7fa638101fe0 2024-04-23T18:23:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.476+0000 7fa635ffb640 1 -- 172.21.15.19:0/3334565822 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa624009c30 con 0x7fa638101fe0 2024-04-23T18:23:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.476+0000 7fa63ea8a640 1 -- 172.21.15.19:0/3334565822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63819ef50 con 0x7fa638101fe0 2024-04-23T18:23:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.476+0000 7fa635ffb640 1 -- 172.21.15.19:0/3334565822 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624003cc0 con 0x7fa638101fe0 2024-04-23T18:23:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.476+0000 7fa63ea8a640 1 -- 172.21.15.19:0/3334565822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63819f410 con 0x7fa638101fe0 2024-04-23T18:23:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.477+0000 7fa63ea8a640 1 -- 172.21.15.19:0/3334565822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6381081a0 con 0x7fa638101fe0 2024-04-23T18:23:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.478+0000 7fa635ffb640 1 -- 172.21.15.19:0/3334565822 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa624003e60 con 0x7fa638101fe0 2024-04-23T18:23:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.479+0000 7fa635ffb640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa6280a3bb0 0x7fa6280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.479+0000 7fa63c826640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa6280a3bb0 0x7fa6280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.479+0000 7fa635ffb640 1 -- 172.21.15.19:0/3334565822 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa6240cfd00 con 0x7fa638101fe0 2024-04-23T18:23:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.479+0000 7fa63c826640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa6280a3bb0 0x7fa6280a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009b40 tx=0x7fa62c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:56.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.484+0000 7fa635ffb640 1 -- 172.21.15.19:0/3334565822 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa62409c830 con 0x7fa638101fe0 2024-04-23T18:23:56.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3016855847' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:23:56.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4131898875' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:23:56.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:56 smithi044 ceph-mon[32458]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:23:56.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.673+0000 7fa63ea8a640 1 -- 172.21.15.19:0/3334565822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa63802d850 con 0x7fa638101fe0 2024-04-23T18:23:56.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.676+0000 7fa635ffb640 1 -- 172.21.15.19:0/3334565822 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22222 (secure 0 0 0) 0x7fa624009de0 con 0x7fa638101fe0 2024-04-23T18:23:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:56.669 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:23:55.611512+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:23:56.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 -- 172.21.15.19:0/3334565822 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa6280a3bb0 msgr2=0x7fa6280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:56.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa6280a3bb0 0x7fa6280a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009b40 tx=0x7fa62c005e50 comp rx=0 tx=0).stop 2024-04-23T18:23:56.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 -- 172.21.15.19:0/3334565822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa638101fe0 msgr2=0x7fa638197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:56.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa638101fe0 0x7fa638197580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa62400d770 tx=0x7fa62400dc30 comp rx=0 tx=0).stop 2024-04-23T18:23:56.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 -- 172.21.15.19:0/3334565822 shutdown_connections 2024-04-23T18:23:56.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa6280a3bb0 0x7fa6280a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6381029a0 0x7fa63819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa638101fe0 0x7fa638197580 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 --2- 172.21.15.19:0/3334565822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa638100e90 0x7fa638197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 -- 172.21.15.19:0/3334565822 >> 172.21.15.19:0/3334565822 conn(0x7fa6380fc400 msgr2=0x7fa638105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:56.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 -- 172.21.15.19:0/3334565822 shutdown_connections 2024-04-23T18:23:56.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.678+0000 7fa63ea8a640 1 -- 172.21.15.19:0/3334565822 wait complete. 2024-04-23T18:23:56.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:23:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.793+0000 7f0830a3a640 1 -- 172.21.15.19:0/1286967501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c101fe0 msgr2=0x7f082c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.793+0000 7f0830a3a640 1 --2- 172.21.15.19:0/1286967501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c101fe0 0x7f082c102460 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f081c009900 tx=0x7f081c02a410 comp rx=0 tx=0).stop 2024-04-23T18:23:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.793+0000 7f0830a3a640 1 -- 172.21.15.19:0/1286967501 shutdown_connections 2024-04-23T18:23:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.793+0000 7f0830a3a640 1 --2- 172.21.15.19:0/1286967501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082c1029a0 0x7f082c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.793+0000 7f0830a3a640 1 --2- 172.21.15.19:0/1286967501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c101fe0 0x7f082c102460 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.793+0000 7f0830a3a640 1 --2- 172.21.15.19:0/1286967501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f082c100e90 0x7f082c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.793+0000 7f0830a3a640 1 -- 172.21.15.19:0/1286967501 >> 172.21.15.19:0/1286967501 conn(0x7f082c0fc400 msgr2=0x7f082c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:56.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.793+0000 7f0830a3a640 1 -- 172.21.15.19:0/1286967501 shutdown_connections 2024-04-23T18:23:56.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.793+0000 7f0830a3a640 1 -- 172.21.15.19:0/1286967501 wait complete. 2024-04-23T18:23:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0830a3a640 1 Processor -- start 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0830a3a640 1 -- start start 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0830a3a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f082c100e90 0x7f082c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0830a3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c101fe0 0x7f082c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0830a3a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082c1029a0 0x7f082c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0830a3a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f082c10c9d0 con 0x7f082c101fe0 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0830a3a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f082c10c850 con 0x7f082c100e90 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0830a3a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f082c10cb50 con 0x7f082c1029a0 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0829d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c101fe0 0x7f082c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0829d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c101fe0 0x7f082c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53448/0 (socket says 172.21.15.19:53448) 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0829d9b640 1 -- 172.21.15.19:0/1056640178 learned_addr learned my addr 172.21.15.19:0/1056640178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0829d9b640 1 -- 172.21.15.19:0/1056640178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082c1029a0 msgr2=0x7f082c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0829d9b640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082c1029a0 0x7f082c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f082a59c640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f082c100e90 0x7f082c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0829d9b640 1 -- 172.21.15.19:0/1056640178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f082c100e90 msgr2=0x7f082c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0829d9b640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f082c100e90 0x7f082c198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0829d9b640 1 -- 172.21.15.19:0/1056640178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c19fd20 con 0x7f082c101fe0 2024-04-23T18:23:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.794+0000 7f0829d9b640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c101fe0 0x7f082c198640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f081c006010 tx=0x7f081c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.795+0000 7f081b7fe640 1 -- 172.21.15.19:0/1056640178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c03f030 con 0x7f082c101fe0 2024-04-23T18:23:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.795+0000 7f0830a3a640 1 -- 172.21.15.19:0/1056640178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082c19ffb0 con 0x7f082c101fe0 2024-04-23T18:23:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.795+0000 7f081b7fe640 1 -- 172.21.15.19:0/1056640178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f081c003cb0 con 0x7f082c101fe0 2024-04-23T18:23:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.795+0000 7f081b7fe640 1 -- 172.21.15.19:0/1056640178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c035070 con 0x7f082c101fe0 2024-04-23T18:23:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.795+0000 7f0830a3a640 1 -- 172.21.15.19:0/1056640178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082c1a03b0 con 0x7f082c101fe0 2024-04-23T18:23:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.796+0000 7f0830a3a640 1 -- 172.21.15.19:0/1056640178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082c1a0610 con 0x7f082c101fe0 2024-04-23T18:23:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.797+0000 7f081b7fe640 1 -- 172.21.15.19:0/1056640178 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f081c042030 con 0x7f082c101fe0 2024-04-23T18:23:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.798+0000 7f081b7fe640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f08140a3bb0 0x7f08140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.798+0000 7f082a59c640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f08140a3bb0 0x7f08140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.798+0000 7f081b7fe640 1 -- 172.21.15.19:0/1056640178 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f081c0f0a50 con 0x7f082c101fe0 2024-04-23T18:23:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.798+0000 7f082a59c640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f08140a3bb0 0x7f08140a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f082000ba40 tx=0x7f0820005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:56.803+0000 7f081b7fe640 1 -- 172.21.15.19:0/1056640178 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f081c003e60 con 0x7f082c101fe0 2024-04-23T18:23:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.026+0000 7f0830a3a640 1 -- 172.21.15.19:0/1056640178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f082c1a0a00 con 0x7f082c101fe0 2024-04-23T18:23:57.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.028+0000 7f081b7fe640 1 -- 172.21.15.19:0/1056640178 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22222 (secure 0 0 0) 0x7f081c0bd580 con 0x7f082c101fe0 2024-04-23T18:23:57.021 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:23:57.022 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:23:55.611512+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.031+0000 7f0830a3a640 1 -- 172.21.15.19:0/1056640178 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f08140a3bb0 msgr2=0x7f08140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.031+0000 7f0830a3a640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f08140a3bb0 0x7f08140a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f082000ba40 tx=0x7f0820005cc0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.031+0000 7f0830a3a640 1 -- 172.21.15.19:0/1056640178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c101fe0 msgr2=0x7f082c198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.031+0000 7f0830a3a640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c101fe0 0x7f082c198640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f081c006010 tx=0x7f081c004430 comp rx=0 tx=0).stop 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.031+0000 7f0830a3a640 1 -- 172.21.15.19:0/1056640178 shutdown_connections 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.031+0000 7f0830a3a640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f08140a3bb0 0x7f08140a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.031+0000 7f0830a3a640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082c1029a0 0x7f082c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.032+0000 7f0830a3a640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c101fe0 0x7f082c198640 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.032+0000 7f0830a3a640 1 --2- 172.21.15.19:0/1056640178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f082c100e90 0x7f082c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.032+0000 7f0830a3a640 1 -- 172.21.15.19:0/1056640178 >> 172.21.15.19:0/1056640178 conn(0x7f082c0fc400 msgr2=0x7f082c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.032+0000 7f0830a3a640 1 -- 172.21.15.19:0/1056640178 shutdown_connections 2024-04-23T18:23:57.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.032+0000 7f0830a3a640 1 -- 172.21.15.19:0/1056640178 wait complete. 2024-04-23T18:23:57.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T18:23:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.153+0000 7f1706322640 1 -- 172.21.15.19:0/1593903267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1700107cd0 msgr2=0x7f170010a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.153+0000 7f1706322640 1 --2- 172.21.15.19:0/1593903267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1700107cd0 0x7f170010a0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009ca0 tx=0x7f16f802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.153+0000 7f1706322640 1 -- 172.21.15.19:0/1593903267 shutdown_connections 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.153+0000 7f1706322640 1 --2- 172.21.15.19:0/1593903267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17001053a0 0x7f1700107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.153+0000 7f1706322640 1 --2- 172.21.15.19:0/1593903267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1700107cd0 0x7f170010a0c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.153+0000 7f1706322640 1 --2- 172.21.15.19:0/1593903267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1700102a70 0x7f1700104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.153+0000 7f1706322640 1 -- 172.21.15.19:0/1593903267 >> 172.21.15.19:0/1593903267 conn(0x7f17000fc400 msgr2=0x7f17000fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.153+0000 7f1706322640 1 -- 172.21.15.19:0/1593903267 shutdown_connections 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.153+0000 7f1706322640 1 -- 172.21.15.19:0/1593903267 wait complete. 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f1706322640 1 Processor -- start 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f1706322640 1 -- start start 2024-04-23T18:23:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f1706322640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1700102a70 0x7f17001023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f1706322640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17001053a0 0x7f1700100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f1706322640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1700107cd0 0x7f1700100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f1706322640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f170010dae0 con 0x7f1700107cd0 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f1706322640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f170010d960 con 0x7f1700102a70 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f1706322640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f170010dc60 con 0x7f17001053a0 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f17048bf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1700107cd0 0x7f1700100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f17048bf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1700107cd0 0x7f1700100f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53472/0 (socket says 172.21.15.19:53472) 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.154+0000 7f17048bf640 1 -- 172.21.15.19:0/1688742240 learned_addr learned my addr 172.21.15.19:0/1688742240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f16fffff640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1700102a70 0x7f17001023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f16ff7fe640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17001053a0 0x7f1700100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f17048bf640 1 -- 172.21.15.19:0/1688742240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17001053a0 msgr2=0x7f1700100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f17048bf640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17001053a0 0x7f1700100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f17048bf640 1 -- 172.21.15.19:0/1688742240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1700102a70 msgr2=0x7f17001023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f17048bf640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1700102a70 0x7f17001023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f17048bf640 1 -- 172.21.15.19:0/1688742240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17001017f0 con 0x7f1700107cd0 2024-04-23T18:23:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f16fffff640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1700102a70 0x7f17001023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f17048bf640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1700107cd0 0x7f1700100f60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f16f8031a20 tx=0x7f16f8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f16fd7fa640 1 -- 172.21.15.19:0/1688742240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f8004240 con 0x7f1700107cd0 2024-04-23T18:23:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f1706322640 1 -- 172.21.15.19:0/1688742240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17001a22b0 con 0x7f1700107cd0 2024-04-23T18:23:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f16fd7fa640 1 -- 172.21.15.19:0/1688742240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f8003e10 con 0x7f1700107cd0 2024-04-23T18:23:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f16fd7fa640 1 -- 172.21.15.19:0/1688742240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f80305c0 con 0x7f1700107cd0 2024-04-23T18:23:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.155+0000 7f1706322640 1 -- 172.21.15.19:0/1688742240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17001a2800 con 0x7f1700107cd0 2024-04-23T18:23:57.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.156+0000 7f1706322640 1 -- 172.21.15.19:0/1688742240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17000fdef0 con 0x7f1700107cd0 2024-04-23T18:23:57.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.157+0000 7f16fd7fa640 1 -- 172.21.15.19:0/1688742240 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f16f802f660 con 0x7f1700107cd0 2024-04-23T18:23:57.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.158+0000 7f16fd7fa640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f16e80a3bb0 0x7f16e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:57.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.158+0000 7f16fffff640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f16e80a3bb0 0x7f16e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:57.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.158+0000 7f16fd7fa640 1 -- 172.21.15.19:0/1688742240 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f16f80f1c40 con 0x7f1700107cd0 2024-04-23T18:23:57.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.158+0000 7f16fffff640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f16e80a3bb0 0x7f16e80a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005fd0 tx=0x7f16f0005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:57.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.162+0000 7f16fd7fa640 1 -- 172.21.15.19:0/1688742240 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16f80be770 con 0x7f1700107cd0 2024-04-23T18:23:57.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:57 smithi155 ceph-mon[33468]: pgmap v294: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:57.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:57 smithi155 ceph-mon[33468]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:23:57.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:57 smithi155 ceph-mon[33468]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:23:57.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:57 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4131898875' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:23:57.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:57 smithi155 ceph-mon[33468]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:23:57.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:57 smithi155 ceph-mon[33468]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:23:57.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:57 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3334565822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:57.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:57 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1056640178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:57.391+0000 7f1706322640 1 -- 172.21.15.19:0/1688742240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f170002d770 con 0x7f1700107cd0 2024-04-23T18:23:57.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:57 smithi019 ceph-mon[29215]: pgmap v294: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:57.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:57 smithi019 ceph-mon[29215]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:23:57.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:57 smithi019 ceph-mon[29215]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:23:57.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:57 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4131898875' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:23:57.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:57 smithi019 ceph-mon[29215]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:23:57.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:57 smithi019 ceph-mon[29215]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:23:57.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:57 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3334565822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:57.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:57 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1056640178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:57.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:57 smithi044 ceph-mon[32458]: pgmap v294: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:57.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:57 smithi044 ceph-mon[32458]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:23:57.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:57 smithi044 ceph-mon[32458]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:23:57.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:57 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4131898875' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:23:57.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:57 smithi044 ceph-mon[32458]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:23:57.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:57 smithi044 ceph-mon[32458]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:23:57.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:57 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3334565822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:57.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:57 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1056640178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:58.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:58 smithi155 ceph-mon[33468]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:23:58.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:58 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] up:active 2024-04-23T18:23:58.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:58 smithi155 ceph-mon[33468]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:23:58.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:58 smithi155 ceph-mon[33468]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:23:58.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:58 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1688742240' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:23:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.354+0000 7f16fd7fa640 1 -- 172.21.15.19:0/1688742240 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f16f80be070 con 0x7f1700107cd0 2024-04-23T18:23:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 -- 172.21.15.19:0/1688742240 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f16e80a3bb0 msgr2=0x7f16e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f16e80a3bb0 0x7f16e80a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005fd0 tx=0x7f16f0005ea0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 -- 172.21.15.19:0/1688742240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1700107cd0 msgr2=0x7f1700100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1700107cd0 0x7f1700100f60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f16f8031a20 tx=0x7f16f8003c20 comp rx=0 tx=0).stop 2024-04-23T18:23:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 -- 172.21.15.19:0/1688742240 shutdown_connections 2024-04-23T18:23:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f16e80a3bb0 0x7f16e80a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1700107cd0 0x7f1700100f60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17001053a0 0x7f1700100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 --2- 172.21.15.19:0/1688742240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1700102a70 0x7f17001023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 -- 172.21.15.19:0/1688742240 >> 172.21.15.19:0/1688742240 conn(0x7f17000fc400 msgr2=0x7f17001033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 -- 172.21.15.19:0/1688742240 shutdown_connections 2024-04-23T18:23:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.357+0000 7f1706322640 1 -- 172.21.15.19:0/1688742240 wait complete. 2024-04-23T18:23:58.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:58 smithi019 ceph-mon[29215]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:23:58.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:58 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] up:active 2024-04-23T18:23:58.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:58 smithi019 ceph-mon[29215]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:23:58.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:58 smithi019 ceph-mon[29215]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:23:58.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:58 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1688742240' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:23:58.362 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-23T18:23:58.363 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-23T18:23:58.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.478+0000 7fc353143640 1 -- 172.21.15.19:0/2431598728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc34c102a00 msgr2=0x7fc34c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:58.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.478+0000 7fc353143640 1 --2- 172.21.15.19:0/2431598728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc34c102a00 0x7fc34c109320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc344009ca0 tx=0x7fc34402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 -- 172.21.15.19:0/2431598728 shutdown_connections 2024-04-23T18:23:58.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 --2- 172.21.15.19:0/2431598728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc34c102a00 0x7fc34c109320 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 --2- 172.21.15.19:0/2431598728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102040 0x7fc34c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 --2- 172.21.15.19:0/2431598728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc34c100e60 0x7fc34c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 -- 172.21.15.19:0/2431598728 >> 172.21.15.19:0/2431598728 conn(0x7fc34c0fc440 msgr2=0x7fc34c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 -- 172.21.15.19:0/2431598728 shutdown_connections 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 -- 172.21.15.19:0/2431598728 wait complete. 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 Processor -- start 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 -- start start 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc34c100e60 0x7fc34c06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc34c102040 0x7fc34c19f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102a00 0x7fc34c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c10ca60 con 0x7fc34c100e60 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c10c8e0 con 0x7fc34c102040 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc353143640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c10cbe0 con 0x7fc34c102a00 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc350edf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc34c100e60 0x7fc34c06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc350edf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc34c100e60 0x7fc34c06b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53496/0 (socket says 172.21.15.19:53496) 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.479+0000 7fc350edf640 1 -- 172.21.15.19:0/4263599283 learned_addr learned my addr 172.21.15.19:0/4263599283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc34bfff640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc34c102040 0x7fc34c19f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc3516e0640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102a00 0x7fc34c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc350edf640 1 -- 172.21.15.19:0/4263599283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102a00 msgr2=0x7fc34c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc350edf640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102a00 0x7fc34c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc350edf640 1 -- 172.21.15.19:0/4263599283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc34c102040 msgr2=0x7fc34c19f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc350edf640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc34c102040 0x7fc34c19f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc350edf640 1 -- 172.21.15.19:0/4263599283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c1a2090 con 0x7fc34c100e60 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc3516e0640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102a00 0x7fc34c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc34bfff640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc34c102040 0x7fc34c19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc350edf640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc34c100e60 0x7fc34c06b0c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc34000d770 tx=0x7fc34000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc349ffb640 1 -- 172.21.15.19:0/4263599283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340017070 con 0x7fc34c100e60 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc349ffb640 1 -- 172.21.15.19:0/4263599283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc340009c30 con 0x7fc34c100e60 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc353143640 1 -- 172.21.15.19:0/4263599283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34c1a2320 con 0x7fc34c100e60 2024-04-23T18:23:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc349ffb640 1 -- 172.21.15.19:0/4263599283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340003cc0 con 0x7fc34c100e60 2024-04-23T18:23:58.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.480+0000 7fc353143640 1 -- 172.21.15.19:0/4263599283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34c1a2940 con 0x7fc34c100e60 2024-04-23T18:23:58.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.481+0000 7fc353143640 1 -- 172.21.15.19:0/4263599283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34c108200 con 0x7fc34c100e60 2024-04-23T18:23:58.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.482+0000 7fc349ffb640 1 -- 172.21.15.19:0/4263599283 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc340003e60 con 0x7fc34c100e60 2024-04-23T18:23:58.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.483+0000 7fc349ffb640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc33c0a3c60 0x7fc33c0a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:58.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.483+0000 7fc34bfff640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc33c0a3c60 0x7fc33c0a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:58.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.483+0000 7fc349ffb640 1 -- 172.21.15.19:0/4263599283 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc3400cfcf0 con 0x7fc34c100e60 2024-04-23T18:23:58.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.483+0000 7fc34bfff640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc33c0a3c60 0x7fc33c0a6120 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc338006fd0 tx=0x7fc338008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:58.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.488+0000 7fc349ffb640 1 -- 172.21.15.19:0/4263599283 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc34009c780 con 0x7fc34c100e60 2024-04-23T18:23:58.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:58 smithi044 ceph-mon[32458]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:23:58.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:58 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] up:active 2024-04-23T18:23:58.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:58 smithi044 ceph-mon[32458]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:23:58.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:58 smithi044 ceph-mon[32458]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:23:58.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:58 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1688742240' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:23:58.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:58.713+0000 7fc353143640 1 -- 172.21.15.19:0/4263599283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fc34c062b80 con 0x7fc34c100e60 2024-04-23T18:23:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.355+0000 7fc349ffb640 1 -- 172.21.15.19:0/4263599283 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fc34009c080 con 0x7fc34c100e60 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.359+0000 7fc353143640 1 -- 172.21.15.19:0/4263599283 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc33c0a3c60 msgr2=0x7fc33c0a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.359+0000 7fc353143640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc33c0a3c60 0x7fc33c0a6120 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc338006fd0 tx=0x7fc338008040 comp rx=0 tx=0).stop 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.359+0000 7fc353143640 1 -- 172.21.15.19:0/4263599283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc34c100e60 msgr2=0x7fc34c06b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.359+0000 7fc353143640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc34c100e60 0x7fc34c06b0c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc34000d770 tx=0x7fc34000dc30 comp rx=0 tx=0).stop 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.359+0000 7fc353143640 1 -- 172.21.15.19:0/4263599283 shutdown_connections 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.359+0000 7fc353143640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc33c0a3c60 0x7fc33c0a6120 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.359+0000 7fc353143640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc34c102040 0x7fc34c19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.359+0000 7fc353143640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102a00 0x7fc34c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.359+0000 7fc353143640 1 --2- 172.21.15.19:0/4263599283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc34c100e60 0x7fc34c06b0c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.359+0000 7fc353143640 1 -- 172.21.15.19:0/4263599283 >> 172.21.15.19:0/4263599283 conn(0x7fc34c0fc440 msgr2=0x7fc34c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.360+0000 7fc353143640 1 -- 172.21.15.19:0/4263599283 shutdown_connections 2024-04-23T18:23:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.360+0000 7fc353143640 1 -- 172.21.15.19:0/4263599283 wait complete. 2024-04-23T18:23:59.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:59 smithi019 ceph-mon[29215]: pgmap v297: 97 pgs: 9 creating+peering, 39 active+clean, 49 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-04-23T18:23:59.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:59 smithi019 ceph-mon[29215]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-23T18:23:59.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:59 smithi019 ceph-mon[29215]: Cluster is now healthy 2024-04-23T18:23:59.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:59 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1688742240' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:23:59.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:59 smithi019 ceph-mon[29215]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:23:59.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:23:59 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4263599283' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T18:23:59.368 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-23T18:23:59.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.485+0000 7fe2beee1640 1 -- 172.21.15.19:0/3068288409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b8102af0 msgr2=0x7fe2b8104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.485+0000 7fe2beee1640 1 --2- 172.21.15.19:0/3068288409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b8102af0 0x7fe2b8104ee0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009900 tx=0x7fe2ac02a410 comp rx=0 tx=0).stop 2024-04-23T18:23:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 -- 172.21.15.19:0/3068288409 shutdown_connections 2024-04-23T18:23:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 --2- 172.21.15.19:0/3068288409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2b8105420 0x7fe2b8107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 --2- 172.21.15.19:0/3068288409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8107d50 0x7fe2b810a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 --2- 172.21.15.19:0/3068288409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b8102af0 0x7fe2b8104ee0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 -- 172.21.15.19:0/3068288409 >> 172.21.15.19:0/3068288409 conn(0x7fe2b80fc440 msgr2=0x7fe2b80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 -- 172.21.15.19:0/3068288409 shutdown_connections 2024-04-23T18:23:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 -- 172.21.15.19:0/3068288409 wait complete. 2024-04-23T18:23:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 Processor -- start 2024-04-23T18:23:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 -- start start 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8102af0 0x7fe2b8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2b8105420 0x7fe2b81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b8107d50 0x7fe2b819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810db00 con 0x7fe2b8107d50 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810d980 con 0x7fe2b8105420 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2beee1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810dc80 con 0x7fe2b8102af0 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.486+0000 7fe2bd47e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b8107d50 0x7fe2b819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2bd47e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b8107d50 0x7fe2b819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53512/0 (socket says 172.21.15.19:53512) 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2bd47e640 1 -- 172.21.15.19:0/2869250671 learned_addr learned my addr 172.21.15.19:0/2869250671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2b7fff640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2b8105420 0x7fe2b81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2bcc7d640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8102af0 0x7fe2b8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2bd47e640 1 -- 172.21.15.19:0/2869250671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8102af0 msgr2=0x7fe2b8198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2bd47e640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8102af0 0x7fe2b8198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2bd47e640 1 -- 172.21.15.19:0/2869250671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2b8105420 msgr2=0x7fe2b81987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2bd47e640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2b8105420 0x7fe2b81987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2bd47e640 1 -- 172.21.15.19:0/2869250671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b819fe90 con 0x7fe2b8107d50 2024-04-23T18:23:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2bd47e640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b8107d50 0x7fe2b819f820 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009fd0 tx=0x7fe2b000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2b5ffb640 1 -- 172.21.15.19:0/2869250671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0017070 con 0x7fe2b8107d50 2024-04-23T18:23:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2b5ffb640 1 -- 172.21.15.19:0/2869250671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b0003e20 con 0x7fe2b8107d50 2024-04-23T18:23:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2beee1640 1 -- 172.21.15.19:0/2869250671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b81a0180 con 0x7fe2b8107d50 2024-04-23T18:23:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2b5ffb640 1 -- 172.21.15.19:0/2869250671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0010460 con 0x7fe2b8107d50 2024-04-23T18:23:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.487+0000 7fe2beee1640 1 -- 172.21.15.19:0/2869250671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b81a06d0 con 0x7fe2b8107d50 2024-04-23T18:23:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.488+0000 7fe2beee1640 1 -- 172.21.15.19:0/2869250671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b80fdf30 con 0x7fe2b8107d50 2024-04-23T18:23:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.489+0000 7fe2b5ffb640 1 -- 172.21.15.19:0/2869250671 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe2b0003970 con 0x7fe2b8107d50 2024-04-23T18:23:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.490+0000 7fe2b5ffb640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe2a80a3c30 0x7fe2a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.490+0000 7fe2b5ffb640 1 -- 172.21.15.19:0/2869250671 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe2b00cffe0 con 0x7fe2b8107d50 2024-04-23T18:23:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.490+0000 7fe2bcc7d640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe2a80a3c30 0x7fe2a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.490+0000 7fe2bcc7d640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe2a80a3c30 0x7fe2a80a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac000c00 tx=0x7fe2ac0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.494+0000 7fe2b5ffb640 1 -- 172.21.15.19:0/2869250671 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe2b009c420 con 0x7fe2b8107d50 2024-04-23T18:23:59.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:59 smithi044 ceph-mon[32458]: pgmap v297: 97 pgs: 9 creating+peering, 39 active+clean, 49 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-04-23T18:23:59.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:59 smithi044 ceph-mon[32458]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-23T18:23:59.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:59 smithi044 ceph-mon[32458]: Cluster is now healthy 2024-04-23T18:23:59.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:59 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1688742240' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:23:59.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:59 smithi044 ceph-mon[32458]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:23:59.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:23:59 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4263599283' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T18:23:59.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:59 smithi155 ceph-mon[33468]: pgmap v297: 97 pgs: 9 creating+peering, 39 active+clean, 49 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-04-23T18:23:59.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:59 smithi155 ceph-mon[33468]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-23T18:23:59.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:59 smithi155 ceph-mon[33468]: Cluster is now healthy 2024-04-23T18:23:59.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:59 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1688742240' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:23:59.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:59 smithi155 ceph-mon[33468]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:23:59.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:23:59 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4263599283' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T18:23:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:23:59.722+0000 7fe2beee1640 1 -- 172.21.15.19:0/2869250671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe2b8062b50 con 0x7fe2b8107d50 2024-04-23T18:24:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.365+0000 7fe2b5ffb640 1 -- 172.21.15.19:0/2869250671 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7fe2b00a0720 con 0x7fe2b8107d50 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.368+0000 7fe2beee1640 1 -- 172.21.15.19:0/2869250671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe2a80a3c30 msgr2=0x7fe2a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.368+0000 7fe2beee1640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe2a80a3c30 0x7fe2a80a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac000c00 tx=0x7fe2ac0314a0 comp rx=0 tx=0).stop 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.368+0000 7fe2beee1640 1 -- 172.21.15.19:0/2869250671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b8107d50 msgr2=0x7fe2b819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.368+0000 7fe2beee1640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b8107d50 0x7fe2b819f820 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009fd0 tx=0x7fe2b000b260 comp rx=0 tx=0).stop 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.369+0000 7fe2beee1640 1 -- 172.21.15.19:0/2869250671 shutdown_connections 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.369+0000 7fe2beee1640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe2a80a3c30 0x7fe2a80a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.369+0000 7fe2beee1640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2b8105420 0x7fe2b81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.369+0000 7fe2beee1640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b8107d50 0x7fe2b819f820 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.369+0000 7fe2beee1640 1 --2- 172.21.15.19:0/2869250671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8102af0 0x7fe2b8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.369+0000 7fe2beee1640 1 -- 172.21.15.19:0/2869250671 >> 172.21.15.19:0/2869250671 conn(0x7fe2b80fc440 msgr2=0x7fe2b81035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.369+0000 7fe2beee1640 1 -- 172.21.15.19:0/2869250671 shutdown_connections 2024-04-23T18:24:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.369+0000 7fe2beee1640 1 -- 172.21.15.19:0/2869250671 wait complete. 2024-04-23T18:24:00.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:00 smithi019 ceph-mon[29215]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:24:00.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:00 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4263599283' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T18:24:00.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:00 smithi019 ceph-mon[29215]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:24:00.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:00 smithi019 ceph-mon[29215]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:00.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:00 smithi019 ceph-mon[29215]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-23T18:24:00.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:00 smithi019 ceph-mon[29215]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-23T18:24:00.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:00 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2869250671' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:24:00.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-23T18:24:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.496+0000 7fae35195640 1 -- 172.21.15.19:0/258113891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae30101fb0 msgr2=0x7fae301023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.496+0000 7fae35195640 1 --2- 172.21.15.19:0/258113891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae30101fb0 0x7fae301023d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fae20009900 tx=0x7fae2002a410 comp rx=0 tx=0).stop 2024-04-23T18:24:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.496+0000 7fae35195640 1 -- 172.21.15.19:0/258113891 shutdown_connections 2024-04-23T18:24:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.496+0000 7fae35195640 1 --2- 172.21.15.19:0/258113891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae301031a0 0x7fae30103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.496+0000 7fae35195640 1 --2- 172.21.15.19:0/258113891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30103b60 0x7fae3010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.496+0000 7fae35195640 1 --2- 172.21.15.19:0/258113891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae30101fb0 0x7fae301023d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.496+0000 7fae35195640 1 -- 172.21.15.19:0/258113891 >> 172.21.15.19:0/258113891 conn(0x7fae300fd520 msgr2=0x7fae300ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.496+0000 7fae35195640 1 -- 172.21.15.19:0/258113891 shutdown_connections 2024-04-23T18:24:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.496+0000 7fae35195640 1 -- 172.21.15.19:0/258113891 wait complete. 2024-04-23T18:24:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae35195640 1 Processor -- start 2024-04-23T18:24:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae35195640 1 -- start start 2024-04-23T18:24:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae35195640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30101fb0 0x7fae30075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae35195640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae301031a0 0x7fae30074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae35195640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae30103b60 0x7fae30074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae35195640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae3010daf0 con 0x7fae301031a0 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae35195640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae3010d970 con 0x7fae30103b60 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae35195640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fae3010dc70 con 0x7fae30101fb0 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae301031a0 0x7fae30074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae301031a0 0x7fae30074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60456/0 (socket says 172.21.15.19:60456) 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae27fff640 1 -- 172.21.15.19:0/2856459697 learned_addr learned my addr 172.21.15.19:0/2856459697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.497+0000 7fae2f59e640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae30103b60 0x7fae30074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae2ed9d640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30101fb0 0x7fae30075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae27fff640 1 -- 172.21.15.19:0/2856459697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30101fb0 msgr2=0x7fae30075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae27fff640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30101fb0 0x7fae30075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae27fff640 1 -- 172.21.15.19:0/2856459697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae30103b60 msgr2=0x7fae30074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae27fff640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae30103b60 0x7fae30074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae27fff640 1 -- 172.21.15.19:0/2856459697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae30075110 con 0x7fae301031a0 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae27fff640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae301031a0 0x7fae30074340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fae1800ea60 tx=0x7fae1800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae2cd99640 1 -- 172.21.15.19:0/2856459697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1800cde0 con 0x7fae301031a0 2024-04-23T18:24:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae2cd99640 1 -- 172.21.15.19:0/2856459697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae180035f0 con 0x7fae301031a0 2024-04-23T18:24:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae35195640 1 -- 172.21.15.19:0/2856459697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae301a22f0 con 0x7fae301031a0 2024-04-23T18:24:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae2cd99640 1 -- 172.21.15.19:0/2856459697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18010690 con 0x7fae301031a0 2024-04-23T18:24:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.498+0000 7fae35195640 1 -- 172.21.15.19:0/2856459697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae301a2840 con 0x7fae301031a0 2024-04-23T18:24:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.499+0000 7fae35195640 1 -- 172.21.15.19:0/2856459697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae30109350 con 0x7fae301031a0 2024-04-23T18:24:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.500+0000 7fae2cd99640 1 -- 172.21.15.19:0/2856459697 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fae180037a0 con 0x7fae301031a0 2024-04-23T18:24:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.501+0000 7fae2cd99640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fae1c0a3bb0 0x7fae1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.501+0000 7fae2ed9d640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fae1c0a3bb0 0x7fae1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.501+0000 7fae2cd99640 1 -- 172.21.15.19:0/2856459697 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fae180d08a0 con 0x7fae301031a0 2024-04-23T18:24:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.502+0000 7fae2ed9d640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fae1c0a3bb0 0x7fae1c0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fae20000c00 tx=0x7fae200314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:00.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.505+0000 7fae2cd99640 1 -- 172.21.15.19:0/2856459697 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae18053ff0 con 0x7fae301031a0 2024-04-23T18:24:00.522 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:00 smithi044 ceph-mon[32458]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:24:00.522 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:00 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4263599283' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T18:24:00.523 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:00 smithi044 ceph-mon[32458]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:24:00.523 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:00 smithi044 ceph-mon[32458]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:00.523 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:00 smithi044 ceph-mon[32458]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-23T18:24:00.523 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:00 smithi044 ceph-mon[32458]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-23T18:24:00.523 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:00 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2869250671' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:24:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:00 smithi155 ceph-mon[33468]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:24:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:00 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4263599283' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T18:24:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:00 smithi155 ceph-mon[33468]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:24:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:00 smithi155 ceph-mon[33468]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:00 smithi155 ceph-mon[33468]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-23T18:24:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:00 smithi155 ceph-mon[33468]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-23T18:24:00.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:00 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2869250671' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:24:00.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:00.730+0000 7fae35195640 1 -- 172.21.15.19:0/2856459697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fae3002d810 con 0x7fae301031a0 2024-04-23T18:24:00.812 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:24:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b[62807]: 2024-04-23T18:24:00.527+0000 7fb570850700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T18:24:00.812 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:24:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b[62807]: 2024-04-23T18:24:00.527+0000 7fb570850700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-23T18:24:01.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.374+0000 7fae2cd99640 1 -- 172.21.15.19:0/2856459697 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7fae1809d3d0 con 0x7fae301031a0 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.376+0000 7fae35195640 1 -- 172.21.15.19:0/2856459697 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fae1c0a3bb0 msgr2=0x7fae1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.376+0000 7fae35195640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fae1c0a3bb0 0x7fae1c0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fae20000c00 tx=0x7fae200314a0 comp rx=0 tx=0).stop 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.376+0000 7fae35195640 1 -- 172.21.15.19:0/2856459697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae301031a0 msgr2=0x7fae30074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.376+0000 7fae35195640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae301031a0 0x7fae30074340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fae1800ea60 tx=0x7fae1800ef20 comp rx=0 tx=0).stop 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.377+0000 7fae35195640 1 -- 172.21.15.19:0/2856459697 shutdown_connections 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.377+0000 7fae35195640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fae1c0a3bb0 0x7fae1c0a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.377+0000 7fae35195640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae30103b60 0x7fae30074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.377+0000 7fae35195640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae301031a0 0x7fae30074340 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.377+0000 7fae35195640 1 --2- 172.21.15.19:0/2856459697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30101fb0 0x7fae30075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.377+0000 7fae35195640 1 -- 172.21.15.19:0/2856459697 >> 172.21.15.19:0/2856459697 conn(0x7fae300fd520 msgr2=0x7fae30106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:01.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.377+0000 7fae35195640 1 -- 172.21.15.19:0/2856459697 shutdown_connections 2024-04-23T18:24:01.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.377+0000 7fae35195640 1 -- 172.21.15.19:0/2856459697 wait complete. 2024-04-23T18:24:01.385 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-mon[29215]: pgmap v298: 97 pgs: 9 creating+peering, 84 active+clean, 4 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-23T18:24:01.385 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2869250671' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:24:01.385 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] up:active 2024-04-23T18:24:01.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-mon[29215]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:24:01.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-mon[29215]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-23T18:24:01.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-mon[29215]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-23T18:24:01.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-mon[29215]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-23T18:24:01.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2856459697' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:24:01.387 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-23T18:24:01.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-23T18:24:01.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.504+0000 7fc83a831640 1 -- 172.21.15.19:0/926424211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8341031a0 msgr2=0x7fc834103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:01.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.504+0000 7fc83a831640 1 --2- 172.21.15.19:0/926424211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8341031a0 0x7fc834103620 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc828009900 tx=0x7fc82802a410 comp rx=0 tx=0).stop 2024-04-23T18:24:01.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.504+0000 7fc83a831640 1 -- 172.21.15.19:0/926424211 shutdown_connections 2024-04-23T18:24:01.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.504+0000 7fc83a831640 1 --2- 172.21.15.19:0/926424211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc834103b60 0x7fc83410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:01.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.504+0000 7fc83a831640 1 --2- 172.21.15.19:0/926424211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8341031a0 0x7fc834103620 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:01.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.504+0000 7fc83a831640 1 --2- 172.21.15.19:0/926424211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc834101fb0 0x7fc8341023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:01.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.504+0000 7fc83a831640 1 -- 172.21.15.19:0/926424211 >> 172.21.15.19:0/926424211 conn(0x7fc8340fd520 msgr2=0x7fc8340ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:01.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.504+0000 7fc83a831640 1 -- 172.21.15.19:0/926424211 shutdown_connections 2024-04-23T18:24:01.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.504+0000 7fc83a831640 1 -- 172.21.15.19:0/926424211 wait complete. 2024-04-23T18:24:01.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc83a831640 1 Processor -- start 2024-04-23T18:24:01.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc83a831640 1 -- start start 2024-04-23T18:24:01.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc83a831640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc834101fb0 0x7fc8341981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:01.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc83a831640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8341031a0 0x7fc8341986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:01.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc83a831640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc834103b60 0x7fc83419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:01.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc83a831640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc83410daf0 con 0x7fc834103b60 2024-04-23T18:24:01.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc83a831640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc83410d970 con 0x7fc8341031a0 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc83a831640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc83410dc70 con 0x7fc834101fb0 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc838dce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc834103b60 0x7fc83419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc838dce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc834103b60 0x7fc83419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60486/0 (socket says 172.21.15.19:60486) 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc838dce640 1 -- 172.21.15.19:0/2427249030 learned_addr learned my addr 172.21.15.19:0/2427249030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc8337fe640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8341031a0 0x7fc8341986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.505+0000 7fc833fff640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc834101fb0 0x7fc8341981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc838dce640 1 -- 172.21.15.19:0/2427249030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc834101fb0 msgr2=0x7fc8341981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc838dce640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc834101fb0 0x7fc8341981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc838dce640 1 -- 172.21.15.19:0/2427249030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8341031a0 msgr2=0x7fc8341986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc838dce640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8341031a0 0x7fc8341986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:01.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc838dce640 1 -- 172.21.15.19:0/2427249030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83419fdc0 con 0x7fc834103b60 2024-04-23T18:24:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc8337fe640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8341031a0 0x7fc8341986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc838dce640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc834103b60 0x7fc83419f750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00eaa0 tx=0x7fc82c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc8317fa640 1 -- 172.21.15.19:0/2427249030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c00cdb0 con 0x7fc834103b60 2024-04-23T18:24:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc8317fa640 1 -- 172.21.15.19:0/2427249030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82c005020 con 0x7fc834103b60 2024-04-23T18:24:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc83a831640 1 -- 172.21.15.19:0/2427249030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8341a00b0 con 0x7fc834103b60 2024-04-23T18:24:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc8317fa640 1 -- 172.21.15.19:0/2427249030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0116d0 con 0x7fc834103b60 2024-04-23T18:24:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.506+0000 7fc83a831640 1 -- 172.21.15.19:0/2427249030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8341a0620 con 0x7fc834103b60 2024-04-23T18:24:01.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.507+0000 7fc83a831640 1 -- 172.21.15.19:0/2427249030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc834109350 con 0x7fc834103b60 2024-04-23T18:24:01.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.508+0000 7fc8317fa640 1 -- 172.21.15.19:0/2427249030 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc82c0051d0 con 0x7fc834103b60 2024-04-23T18:24:01.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.509+0000 7fc8317fa640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc81c0a3c30 0x7fc81c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:01.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.509+0000 7fc833fff640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc81c0a3c30 0x7fc81c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:01.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.509+0000 7fc8317fa640 1 -- 172.21.15.19:0/2427249030 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc82c0d03b0 con 0x7fc834103b60 2024-04-23T18:24:01.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.510+0000 7fc833fff640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc81c0a3c30 0x7fc81c0a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc834103000 tx=0x7fc824005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.514+0000 7fc8317fa640 1 -- 172.21.15.19:0/2427249030 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc82c016030 con 0x7fc834103b60 2024-04-23T18:24:01.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:01 smithi044 ceph-mon[32458]: pgmap v298: 97 pgs: 9 creating+peering, 84 active+clean, 4 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-23T18:24:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:01 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2869250671' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:24:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:01 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] up:active 2024-04-23T18:24:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:01 smithi044 ceph-mon[32458]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:24:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:01 smithi044 ceph-mon[32458]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-23T18:24:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:01 smithi044 ceph-mon[32458]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-23T18:24:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:01 smithi044 ceph-mon[32458]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-23T18:24:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:01 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2856459697' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:24:01.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:01 smithi155 ceph-mon[33468]: pgmap v298: 97 pgs: 9 creating+peering, 84 active+clean, 4 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-23T18:24:01.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:01 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2869250671' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:24:01.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:01 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] up:active 2024-04-23T18:24:01.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:01 smithi155 ceph-mon[33468]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:24:01.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:01 smithi155 ceph-mon[33468]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-23T18:24:01.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:01 smithi155 ceph-mon[33468]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-23T18:24:01.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:01 smithi155 ceph-mon[33468]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-23T18:24:01.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:01 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2856459697' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:24:01.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:01.727+0000 7fc83a831640 1 -- 172.21.15.19:0/2427249030 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc834062390 con 0x7fc81c0a3c30 2024-04-23T18:24:02.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:24:01.732+0000 7f881fd21700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:24:02.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:24:01.732+0000 7f881fd21700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:24:02.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:24:01.732+0000 7f881fd21700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:24:02.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:24:01.732+0000 7f881fd21700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:24:02.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:24:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:24:01.732+0000 7f881fd21700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:24:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.400+0000 7fc8317fa640 1 -- 172.21.15.19:0/2427249030 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc834062390 con 0x7fc81c0a3c30 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 -- 172.21.15.19:0/2427249030 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc81c0a3c30 msgr2=0x7fc81c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc81c0a3c30 0x7fc81c0a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc834103000 tx=0x7fc824005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 -- 172.21.15.19:0/2427249030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc834103b60 msgr2=0x7fc83419f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc834103b60 0x7fc83419f750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00eaa0 tx=0x7fc82c00ef60 comp rx=0 tx=0).stop 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 -- 172.21.15.19:0/2427249030 shutdown_connections 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc81c0a3c30 0x7fc81c0a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8341031a0 0x7fc8341986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc834103b60 0x7fc83419f750 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 --2- 172.21.15.19:0/2427249030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc834101fb0 0x7fc8341981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 -- 172.21.15.19:0/2427249030 >> 172.21.15.19:0/2427249030 conn(0x7fc8340fd520 msgr2=0x7fc834106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 -- 172.21.15.19:0/2427249030 shutdown_connections 2024-04-23T18:24:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.403+0000 7fc83a831640 1 -- 172.21.15.19:0/2427249030 wait complete. 2024-04-23T18:24:02.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-23T18:24:02.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.529+0000 7f9d36e13640 1 -- 172.21.15.19:0/2096077720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d30103b60 msgr2=0x7f9d3010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.529+0000 7f9d36e13640 1 --2- 172.21.15.19:0/2096077720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d30103b60 0x7f9d3010a3f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009900 tx=0x7f9d2002a410 comp rx=0 tx=0).stop 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.529+0000 7f9d36e13640 1 -- 172.21.15.19:0/2096077720 shutdown_connections 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.529+0000 7f9d36e13640 1 --2- 172.21.15.19:0/2096077720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d30103b60 0x7f9d3010a3f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.529+0000 7f9d36e13640 1 --2- 172.21.15.19:0/2096077720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d301031a0 0x7f9d30103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.529+0000 7f9d36e13640 1 --2- 172.21.15.19:0/2096077720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d30101fb0 0x7f9d301023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.529+0000 7f9d36e13640 1 -- 172.21.15.19:0/2096077720 >> 172.21.15.19:0/2096077720 conn(0x7f9d300fd520 msgr2=0x7f9d300ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.529+0000 7f9d36e13640 1 -- 172.21.15.19:0/2096077720 shutdown_connections 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.529+0000 7f9d36e13640 1 -- 172.21.15.19:0/2096077720 wait complete. 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d36e13640 1 Processor -- start 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d36e13640 1 -- start start 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d36e13640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d30101fb0 0x7f9d30198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d36e13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d301031a0 0x7f9d30198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d36e13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d30103b60 0x7f9d3019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d36e13640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3010db80 con 0x7f9d301031a0 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d36e13640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3010da00 con 0x7f9d30103b60 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d36e13640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3010dd00 con 0x7f9d30101fb0 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d2ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d301031a0 0x7f9d30198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d2ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d301031a0 0x7f9d30198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60528/0 (socket says 172.21.15.19:60528) 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d2ffff640 1 -- 172.21.15.19:0/881278812 learned_addr learned my addr 172.21.15.19:0/881278812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d353b0640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d30103b60 0x7f9d3019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d34baf640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d30101fb0 0x7f9d30198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d2ffff640 1 -- 172.21.15.19:0/881278812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d30101fb0 msgr2=0x7f9d30198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d2ffff640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d30101fb0 0x7f9d30198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.530+0000 7f9d2ffff640 1 -- 172.21.15.19:0/881278812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d30103b60 msgr2=0x7f9d3019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.531+0000 7f9d2ffff640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d30103b60 0x7f9d3019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.531+0000 7f9d2ffff640 1 -- 172.21.15.19:0/881278812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3019fe20 con 0x7f9d301031a0 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.531+0000 7f9d34baf640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d30101fb0 0x7f9d30198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.531+0000 7f9d2ffff640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d301031a0 0x7f9d30198740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400b4a0 tx=0x7f9d2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:02.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.531+0000 7f9d2dffb640 1 -- 172.21.15.19:0/881278812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24014070 con 0x7f9d301031a0 2024-04-23T18:24:02.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.531+0000 7f9d2dffb640 1 -- 172.21.15.19:0/881278812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d24010ac0 con 0x7f9d301031a0 2024-04-23T18:24:02.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.531+0000 7f9d36e13640 1 -- 172.21.15.19:0/881278812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d301a0110 con 0x7f9d301031a0 2024-04-23T18:24:02.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.531+0000 7f9d2dffb640 1 -- 172.21.15.19:0/881278812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24017c30 con 0x7f9d301031a0 2024-04-23T18:24:02.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.531+0000 7f9d36e13640 1 -- 172.21.15.19:0/881278812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d301a0730 con 0x7f9d301031a0 2024-04-23T18:24:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.532+0000 7f9d36e13640 1 -- 172.21.15.19:0/881278812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d301092f0 con 0x7f9d301031a0 2024-04-23T18:24:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.533+0000 7f9d2dffb640 1 -- 172.21.15.19:0/881278812 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d24017070 con 0x7f9d301031a0 2024-04-23T18:24:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.534+0000 7f9d2dffb640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9d180a3c30 0x7f9d180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.534+0000 7f9d34baf640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9d180a3c30 0x7f9d180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.534+0000 7f9d2dffb640 1 -- 172.21.15.19:0/881278812 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9d240cf920 con 0x7f9d301031a0 2024-04-23T18:24:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.534+0000 7f9d34baf640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9d180a3c30 0x7f9d180a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d30103000 tx=0x7f9d28009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:02.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.538+0000 7f9d2dffb640 1 -- 172.21.15.19:0/881278812 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d2409c450 con 0x7f9d301031a0 2024-04-23T18:24:02.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.754+0000 7f9d36e13640 1 -- 172.21.15.19:0/881278812 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9d30062390 con 0x7f9d180a3c30 2024-04-23T18:24:02.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:02 smithi019 ceph-mon[29215]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 9 op/s 2024-04-23T18:24:02.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:02 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2856459697' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:24:02.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:02 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] up:active 2024-04-23T18:24:02.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:02 smithi019 ceph-mon[29215]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T18:24:02.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:02 smithi019 ceph-mon[29215]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-23T18:24:02.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:02 smithi019 ceph-mon[29215]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-23T18:24:02.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:02 smithi019 ceph-mon[29215]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-23T18:24:02.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:02 smithi019 ceph-mon[29215]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.779+0000 7f9d2dffb640 1 -- 172.21.15.19:0/881278812 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d30062390 con 0x7f9d180a3c30 2024-04-23T18:24:02.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 -- 172.21.15.19:0/881278812 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9d180a3c30 msgr2=0x7f9d180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9d180a3c30 0x7f9d180a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d30103000 tx=0x7f9d28009500 comp rx=0 tx=0).stop 2024-04-23T18:24:02.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 -- 172.21.15.19:0/881278812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d301031a0 msgr2=0x7f9d30198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d301031a0 0x7f9d30198740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400b4a0 tx=0x7f9d2400b960 comp rx=0 tx=0).stop 2024-04-23T18:24:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 -- 172.21.15.19:0/881278812 shutdown_connections 2024-04-23T18:24:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9d180a3c30 0x7f9d180a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d30103b60 0x7f9d3019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d301031a0 0x7f9d30198740 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 --2- 172.21.15.19:0/881278812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d30101fb0 0x7f9d30198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 -- 172.21.15.19:0/881278812 >> 172.21.15.19:0/881278812 conn(0x7f9d300fd520 msgr2=0x7f9d301064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 -- 172.21.15.19:0/881278812 shutdown_connections 2024-04-23T18:24:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.781+0000 7f9d36e13640 1 -- 172.21.15.19:0/881278812 wait complete. 2024-04-23T18:24:02.785 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-23T18:24:02.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:02 smithi044 ceph-mon[32458]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 9 op/s 2024-04-23T18:24:02.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:02 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2856459697' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:24:02.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:02 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] up:active 2024-04-23T18:24:02.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:02 smithi044 ceph-mon[32458]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T18:24:02.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:02 smithi044 ceph-mon[32458]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-23T18:24:02.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:02 smithi044 ceph-mon[32458]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-23T18:24:02.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:02 smithi044 ceph-mon[32458]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-23T18:24:02.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:02 smithi044 ceph-mon[32458]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:02.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:02 smithi155 ceph-mon[33468]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 9 op/s 2024-04-23T18:24:02.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:02 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2856459697' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:24:02.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:02 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] up:active 2024-04-23T18:24:02.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:02 smithi155 ceph-mon[33468]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T18:24:02.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:02 smithi155 ceph-mon[33468]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-23T18:24:02.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:02 smithi155 ceph-mon[33468]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-23T18:24:02.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:02 smithi155 ceph-mon[33468]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-23T18:24:02.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:02 smithi155 ceph-mon[33468]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.898+0000 7f60e308a640 1 -- 172.21.15.19:0/862655926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc102a90 msgr2=0x7f60dc104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.898+0000 7f60e308a640 1 --2- 172.21.15.19:0/862655926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc102a90 0x7f60dc104e80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009ca0 tx=0x7f60d402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.898+0000 7f60e308a640 1 -- 172.21.15.19:0/862655926 shutdown_connections 2024-04-23T18:24:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.898+0000 7f60e308a640 1 --2- 172.21.15.19:0/862655926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60dc1053c0 0x7f60dc1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.898+0000 7f60e308a640 1 --2- 172.21.15.19:0/862655926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60dc107cf0 0x7f60dc10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.898+0000 7f60e308a640 1 --2- 172.21.15.19:0/862655926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc102a90 0x7f60dc104e80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.898+0000 7f60e308a640 1 -- 172.21.15.19:0/862655926 >> 172.21.15.19:0/862655926 conn(0x7f60dc0fc440 msgr2=0x7f60dc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.898+0000 7f60e308a640 1 -- 172.21.15.19:0/862655926 shutdown_connections 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.898+0000 7f60e308a640 1 -- 172.21.15.19:0/862655926 wait complete. 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e308a640 1 Processor -- start 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e308a640 1 -- start start 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e308a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60dc102a90 0x7f60dc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e308a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc1053c0 0x7f60dc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e308a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60dc107cf0 0x7f60dc19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e308a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc10daf0 con 0x7f60dc1053c0 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e308a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc10d970 con 0x7f60dc102a90 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e308a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc10dc70 con 0x7f60dc107cf0 2024-04-23T18:24:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc1053c0 0x7f60dc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc1053c0 0x7f60dc198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60538/0 (socket says 172.21.15.19:60538) 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60dbfff640 1 -- 172.21.15.19:0/1724184611 learned_addr learned my addr 172.21.15.19:0/1724184611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e0e26640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60dc102a90 0x7f60dc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e1627640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60dc107cf0 0x7f60dc19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60dbfff640 1 -- 172.21.15.19:0/1724184611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60dc107cf0 msgr2=0x7f60dc19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60dbfff640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60dc107cf0 0x7f60dc19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60dbfff640 1 -- 172.21.15.19:0/1724184611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60dc102a90 msgr2=0x7f60dc198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60dbfff640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60dc102a90 0x7f60dc198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60dbfff640 1 -- 172.21.15.19:0/1724184611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc19fe50 con 0x7f60dc1053c0 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.899+0000 7f60e0e26640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60dc102a90 0x7f60dc198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.900+0000 7f60dbfff640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc1053c0 0x7f60dc198770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f60d000e9c0 tx=0x7f60d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.900+0000 7f60d9ffb640 1 -- 172.21.15.19:0/1724184611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d000cd80 con 0x7f60dc1053c0 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.900+0000 7f60d9ffb640 1 -- 172.21.15.19:0/1724184611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d00054d0 con 0x7f60dc1053c0 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.900+0000 7f60e308a640 1 -- 172.21.15.19:0/1724184611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc1a0140 con 0x7f60dc1053c0 2024-04-23T18:24:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.900+0000 7f60d9ffb640 1 -- 172.21.15.19:0/1724184611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0010430 con 0x7f60dc1053c0 2024-04-23T18:24:02.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.900+0000 7f60e308a640 1 -- 172.21.15.19:0/1724184611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc1a0690 con 0x7f60dc1053c0 2024-04-23T18:24:02.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.901+0000 7f60e308a640 1 -- 172.21.15.19:0/1724184611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60dc0fdf30 con 0x7f60dc1053c0 2024-04-23T18:24:02.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.902+0000 7f60d9ffb640 1 -- 172.21.15.19:0/1724184611 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f60d0004fd0 con 0x7f60dc1053c0 2024-04-23T18:24:02.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.903+0000 7f60d9ffb640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60c40a3bb0 0x7f60c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.903+0000 7f60d9ffb640 1 -- 172.21.15.19:0/1724184611 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f60d00d06a0 con 0x7f60dc1053c0 2024-04-23T18:24:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.903+0000 7f60e0e26640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60c40a3bb0 0x7f60c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.903+0000 7f60e0e26640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60c40a3bb0 0x7f60c40a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f60d4033040 tx=0x7f60d4032fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:02.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:02.908+0000 7f60d9ffb640 1 -- 172.21.15.19:0/1724184611 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60d009d1d0 con 0x7f60dc1053c0 2024-04-23T18:24:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.126+0000 7f60e308a640 1 -- 172.21.15.19:0/1724184611 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f60dc062390 con 0x7f60c40a3bb0 2024-04-23T18:24:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.131+0000 7f60d9ffb640 1 -- 172.21.15.19:0/1724184611 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f60dc062390 con 0x7f60c40a3bb0 2024-04-23T18:24:03.124 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_0/e556f177-d6b5-4d3d-97a2-0a204ae03e74 2024-04-23T18:24:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 -- 172.21.15.19:0/1724184611 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60c40a3bb0 msgr2=0x7f60c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60c40a3bb0 0x7f60c40a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f60d4033040 tx=0x7f60d4032fb0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 -- 172.21.15.19:0/1724184611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc1053c0 msgr2=0x7f60dc198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc1053c0 0x7f60dc198770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f60d000e9c0 tx=0x7f60d000ee80 comp rx=0 tx=0).stop 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 -- 172.21.15.19:0/1724184611 shutdown_connections 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f60c40a3bb0 0x7f60c40a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60dc107cf0 0x7f60dc19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc1053c0 0x7f60dc198770 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 --2- 172.21.15.19:0/1724184611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60dc102a90 0x7f60dc198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 -- 172.21.15.19:0/1724184611 >> 172.21.15.19:0/1724184611 conn(0x7f60dc0fc440 msgr2=0x7f60dc103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 -- 172.21.15.19:0/1724184611 shutdown_connections 2024-04-23T18:24:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.133+0000 7f60e308a640 1 -- 172.21.15.19:0/1724184611 wait complete. 2024-04-23T18:24:03.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2469666553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc107c70 msgr2=0x7f59dc10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 --2- 172.21.15.19:0/2469666553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc107c70 0x7f59dc10a0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009ca0 tx=0x7f59cc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2469666553 shutdown_connections 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 --2- 172.21.15.19:0/2469666553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc107c70 0x7f59dc10a0c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 --2- 172.21.15.19:0/2469666553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc105340 0x7f59dc107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 --2- 172.21.15.19:0/2469666553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc102a10 0x7f59dc104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2469666553 >> 172.21.15.19:0/2469666553 conn(0x7f59dc0fc400 msgr2=0x7f59dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2469666553 shutdown_connections 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2469666553 wait complete. 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 Processor -- start 2024-04-23T18:24:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.252+0000 7f59e0ce8640 1 -- start start 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59e0ce8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc102a10 0x7f59dc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59e0ce8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc105340 0x7f59dc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59e0ce8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc107c70 0x7f59dc19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59e0ce8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc10db20 con 0x7f59dc102a10 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59e0ce8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc10d9a0 con 0x7f59dc105340 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59e0ce8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc10dca0 con 0x7f59dc107c70 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59da59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc102a10 0x7f59dc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59da59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc102a10 0x7f59dc1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60554/0 (socket says 172.21.15.19:60554) 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59da59c640 1 -- 172.21.15.19:0/2110227951 learned_addr learned my addr 172.21.15.19:0/2110227951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59d9d9b640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc105340 0x7f59dc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59da59c640 1 -- 172.21.15.19:0/2110227951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc107c70 msgr2=0x7f59dc19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59da59c640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc107c70 0x7f59dc19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59da59c640 1 -- 172.21.15.19:0/2110227951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc105340 msgr2=0x7f59dc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59da59c640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc105340 0x7f59dc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59da59c640 1 -- 172.21.15.19:0/2110227951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc19fdc0 con 0x7f59dc102a10 2024-04-23T18:24:03.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59da59c640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc102a10 0x7f59dc1981a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f59d400e9f0 tx=0x7f59d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59d37fe640 1 -- 172.21.15.19:0/2110227951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d400cd80 con 0x7f59dc102a10 2024-04-23T18:24:03.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.253+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2110227951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc1a00b0 con 0x7f59dc102a10 2024-04-23T18:24:03.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.254+0000 7f59d37fe640 1 -- 172.21.15.19:0/2110227951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d40108e0 con 0x7f59dc102a10 2024-04-23T18:24:03.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.254+0000 7f59d37fe640 1 -- 172.21.15.19:0/2110227951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4018980 con 0x7f59dc102a10 2024-04-23T18:24:03.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.254+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2110227951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc1a0650 con 0x7f59dc102a10 2024-04-23T18:24:03.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.255+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2110227951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59dc0fdef0 con 0x7f59dc102a10 2024-04-23T18:24:03.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.256+0000 7f59d37fe640 1 -- 172.21.15.19:0/2110227951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f59d4010430 con 0x7f59dc102a10 2024-04-23T18:24:03.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.257+0000 7f59d37fe640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3bb0 0x7f59c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.257+0000 7f59d9d9b640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3bb0 0x7f59c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.257+0000 7f59d37fe640 1 -- 172.21.15.19:0/2110227951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f59d40d0b30 con 0x7f59dc102a10 2024-04-23T18:24:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.258+0000 7f59d9d9b640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3bb0 0x7f59c80a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59dc1996b0 tx=0x7f59c400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.261+0000 7f59d37fe640 1 -- 172.21.15.19:0/2110227951 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59d409d660 con 0x7f59dc102a10 2024-04-23T18:24:03.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.478+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2110227951 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f59dc062420 con 0x7f59c80a3bb0 2024-04-23T18:24:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.507+0000 7f59d37fe640 1 -- 172.21.15.19:0/2110227951 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59dc062420 con 0x7f59c80a3bb0 2024-04-23T18:24:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2110227951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3bb0 msgr2=0x7f59c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3bb0 0x7f59c80a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59dc1996b0 tx=0x7f59c400a250 comp rx=0 tx=0).stop 2024-04-23T18:24:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2110227951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc102a10 msgr2=0x7f59dc1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc102a10 0x7f59dc1981a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f59d400e9f0 tx=0x7f59d400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2110227951 shutdown_connections 2024-04-23T18:24:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f59c80a3bb0 0x7f59c80a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59dc105340 0x7f59dc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59dc107c70 0x7f59dc19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 --2- 172.21.15.19:0/2110227951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59dc102a10 0x7f59dc1981a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2110227951 >> 172.21.15.19:0/2110227951 conn(0x7f59dc0fc400 msgr2=0x7f59dc103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2110227951 shutdown_connections 2024-04-23T18:24:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.509+0000 7f59e0ce8640 1 -- 172.21.15.19:0/2110227951 wait complete. 2024-04-23T18:24:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] up:active 2024-04-23T18:24:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T18:24:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:24:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: Cluster is now healthy 2024-04-23T18:24:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-04-23T18:24:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-23T18:24:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-23T18:24:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] up:active 2024-04-23T18:24:03.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:24:03.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:03.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:03 smithi019 ceph-mon[29215]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:03.513 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-23T18:24:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.627+0000 7f7118d8a640 1 -- 172.21.15.19:0/3415158894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114105360 msgr2=0x7f7114107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.627+0000 7f7118d8a640 1 --2- 172.21.15.19:0/3415158894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114105360 0x7f7114107750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7108009900 tx=0x7f710802a410 comp rx=0 tx=0).stop 2024-04-23T18:24:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.627+0000 7f7118d8a640 1 -- 172.21.15.19:0/3415158894 shutdown_connections 2024-04-23T18:24:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.627+0000 7f7118d8a640 1 --2- 172.21.15.19:0/3415158894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7114107c90 0x7f711410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.627+0000 7f7118d8a640 1 --2- 172.21.15.19:0/3415158894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114105360 0x7f7114107750 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.627+0000 7f7118d8a640 1 --2- 172.21.15.19:0/3415158894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114102a30 0x7f7114104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.627+0000 7f7118d8a640 1 -- 172.21.15.19:0/3415158894 >> 172.21.15.19:0/3415158894 conn(0x7f71140fc420 msgr2=0x7f71140fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.627+0000 7f7118d8a640 1 -- 172.21.15.19:0/3415158894 shutdown_connections 2024-04-23T18:24:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.628+0000 7f7118d8a640 1 -- 172.21.15.19:0/3415158894 wait complete. 2024-04-23T18:24:03.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.628+0000 7f7118d8a640 1 Processor -- start 2024-04-23T18:24:03.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.628+0000 7f7118d8a640 1 -- start start 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.628+0000 7f7118d8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114102a30 0x7f711419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.628+0000 7f7118d8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114105360 0x7f711419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.628+0000 7f7118d8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7114107c90 0x7f71141a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.628+0000 7f7118d8a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f711410dab0 con 0x7f7114105360 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.628+0000 7f7118d8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f711410d930 con 0x7f7114102a30 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.628+0000 7f7118d8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f711410dc30 con 0x7f7114107c90 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f7111d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114105360 0x7f711419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f711259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114102a30 0x7f711419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f711259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114102a30 0x7f711419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:57130/0 (socket says 172.21.15.19:57130) 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f711259c640 1 -- 172.21.15.19:0/2440496604 learned_addr learned my addr 172.21.15.19:0/2440496604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f7111d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114105360 0x7f711419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60572/0 (socket says 172.21.15.19:60572) 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f7112d9d640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7114107c90 0x7f71141a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f711259c640 1 -- 172.21.15.19:0/2440496604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7114107c90 msgr2=0x7f71141a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f711259c640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7114107c90 0x7f71141a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f711259c640 1 -- 172.21.15.19:0/2440496604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114105360 msgr2=0x7f711419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f711259c640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114105360 0x7f711419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f711259c640 1 -- 172.21.15.19:0/2440496604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71141a2070 con 0x7f7114102a30 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f7112d9d640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7114107c90 0x7f71141a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f7111d9b640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114105360 0x7f711419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.629+0000 7f711259c640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114102a30 0x7f711419a450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f710400e9f0 tx=0x7f710400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.631+0000 7f71037fe640 1 -- 172.21.15.19:0/2440496604 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710400cd80 con 0x7f7114102a30 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.631+0000 7f7118d8a640 1 -- 172.21.15.19:0/2440496604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71141a2360 con 0x7f7114102a30 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.631+0000 7f71037fe640 1 -- 172.21.15.19:0/2440496604 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71040108e0 con 0x7f7114102a30 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.631+0000 7f71037fe640 1 -- 172.21.15.19:0/2440496604 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104018980 con 0x7f7114102a30 2024-04-23T18:24:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.631+0000 7f7118d8a640 1 -- 172.21.15.19:0/2440496604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71141a2880 con 0x7f7114102a30 2024-04-23T18:24:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.632+0000 7f7118d8a640 1 -- 172.21.15.19:0/2440496604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71141097e0 con 0x7f7114102a30 2024-04-23T18:24:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.633+0000 7f71037fe640 1 -- 172.21.15.19:0/2440496604 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7104010430 con 0x7f7114102a30 2024-04-23T18:24:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.633+0000 7f71037fe640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f70fc0a3bb0 0x7f70fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.633+0000 7f7111d9b640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f70fc0a3bb0 0x7f70fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.633+0000 7f71037fe640 1 -- 172.21.15.19:0/2440496604 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f71040d0650 con 0x7f7114102a30 2024-04-23T18:24:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.634+0000 7f7111d9b640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f70fc0a3bb0 0x7f70fc0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7108009230 tx=0x7f71080314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.639+0000 7f71037fe640 1 -- 172.21.15.19:0/2440496604 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f710409d160 con 0x7f7114102a30 2024-04-23T18:24:03.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] up:active 2024-04-23T18:24:03.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T18:24:03.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:24:03.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: Cluster is now healthy 2024-04-23T18:24:03.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-04-23T18:24:03.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-23T18:24:03.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-23T18:24:03.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] up:active 2024-04-23T18:24:03.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:24:03.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:03.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:03 smithi044 ceph-mon[32458]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:03.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] up:active 2024-04-23T18:24:03.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T18:24:03.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:24:03.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: Cluster is now healthy 2024-04-23T18:24:03.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-04-23T18:24:03.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-23T18:24:03.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-23T18:24:03.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] up:active 2024-04-23T18:24:03.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:24:03.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:03.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:03 smithi155 ceph-mon[33468]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.854+0000 7f7118d8a640 1 -- 172.21.15.19:0/2440496604 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f7114062420 con 0x7f70fc0a3bb0 2024-04-23T18:24:03.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.860+0000 7f71037fe640 1 -- 172.21.15.19:0/2440496604 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7114062420 con 0x7f70fc0a3bb0 2024-04-23T18:24:03.853 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_1/897ae47f-ad36-4531-8493-4f897c3ecfab 2024-04-23T18:24:03.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 -- 172.21.15.19:0/2440496604 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f70fc0a3bb0 msgr2=0x7f70fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f70fc0a3bb0 0x7f70fc0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7108009230 tx=0x7f71080314a0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 -- 172.21.15.19:0/2440496604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114102a30 msgr2=0x7f711419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114102a30 0x7f711419a450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f710400e9f0 tx=0x7f710400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 -- 172.21.15.19:0/2440496604 shutdown_connections 2024-04-23T18:24:03.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f70fc0a3bb0 0x7f70fc0a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7114107c90 0x7f71141a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114105360 0x7f711419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 --2- 172.21.15.19:0/2440496604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114102a30 0x7f711419a450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 -- 172.21.15.19:0/2440496604 >> 172.21.15.19:0/2440496604 conn(0x7f71140fc420 msgr2=0x7f7114102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 -- 172.21.15.19:0/2440496604 shutdown_connections 2024-04-23T18:24:03.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.862+0000 7f7118d8a640 1 -- 172.21.15.19:0/2440496604 wait complete. 2024-04-23T18:24:03.866 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.974+0000 7fd86cdd8640 1 -- 172.21.15.19:0/3273427819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868107c80 msgr2=0x7fd86810a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.974+0000 7fd86cdd8640 1 --2- 172.21.15.19:0/3273427819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868107c80 0x7fd86810a0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd860009ca0 tx=0x7fd86002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.974+0000 7fd86cdd8640 1 -- 172.21.15.19:0/3273427819 shutdown_connections 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.974+0000 7fd86cdd8640 1 --2- 172.21.15.19:0/3273427819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868107c80 0x7fd86810a0d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.974+0000 7fd86cdd8640 1 --2- 172.21.15.19:0/3273427819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd868105350 0x7fd868107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.974+0000 7fd86cdd8640 1 --2- 172.21.15.19:0/3273427819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd868102a20 0x7fd868104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.974+0000 7fd86cdd8640 1 -- 172.21.15.19:0/3273427819 >> 172.21.15.19:0/3273427819 conn(0x7fd8680fc430 msgr2=0x7fd8680fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.974+0000 7fd86cdd8640 1 -- 172.21.15.19:0/3273427819 shutdown_connections 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.974+0000 7fd86cdd8640 1 -- 172.21.15.19:0/3273427819 wait complete. 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86cdd8640 1 Processor -- start 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86cdd8640 1 -- start start 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86cdd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868102a20 0x7fd8681981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86cdd8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd868105350 0x7fd868198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86cdd8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd868107c80 0x7fd86819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86cdd8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd86810dae0 con 0x7fd868102a20 2024-04-23T18:24:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86cdd8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd86810d960 con 0x7fd868107c80 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86cdd8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd86810dc60 con 0x7fd868105350 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868102a20 0x7fd8681981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868102a20 0x7fd8681981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60584/0 (socket says 172.21.15.19:60584) 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86659c640 1 -- 172.21.15.19:0/1268870657 learned_addr learned my addr 172.21.15.19:0/1268870657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd866d9d640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd868107c80 0x7fd86819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd865d9b640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd868105350 0x7fd868198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86659c640 1 -- 172.21.15.19:0/1268870657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd868105350 msgr2=0x7fd868198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86659c640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd868105350 0x7fd868198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86659c640 1 -- 172.21.15.19:0/1268870657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd868107c80 msgr2=0x7fd86819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86659c640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd868107c80 0x7fd86819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.975+0000 7fd86659c640 1 -- 172.21.15.19:0/1268870657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86819fe00 con 0x7fd868102a20 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.976+0000 7fd86659c640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868102a20 0x7fd8681981e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd85800d950 tx=0x7fd85800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.976+0000 7fd8577fe640 1 -- 172.21.15.19:0/1268870657 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858017030 con 0x7fd868102a20 2024-04-23T18:24:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.976+0000 7fd86cdd8640 1 -- 172.21.15.19:0/1268870657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8681a00f0 con 0x7fd868102a20 2024-04-23T18:24:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.976+0000 7fd8577fe640 1 -- 172.21.15.19:0/1268870657 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd858004020 con 0x7fd868102a20 2024-04-23T18:24:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.976+0000 7fd8577fe640 1 -- 172.21.15.19:0/1268870657 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8580047a0 con 0x7fd868102a20 2024-04-23T18:24:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.976+0000 7fd86cdd8640 1 -- 172.21.15.19:0/1268870657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8681a0690 con 0x7fd868102a20 2024-04-23T18:24:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.977+0000 7fd86cdd8640 1 -- 172.21.15.19:0/1268870657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8680fdf20 con 0x7fd868102a20 2024-04-23T18:24:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.978+0000 7fd8577fe640 1 -- 172.21.15.19:0/1268870657 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd858009a30 con 0x7fd868102a20 2024-04-23T18:24:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.979+0000 7fd8577fe640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd8500a3bb0 0x7fd8500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.979+0000 7fd865d9b640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd8500a3bb0 0x7fd8500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.979+0000 7fd8577fe640 1 -- 172.21.15.19:0/1268870657 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd8580cfe10 con 0x7fd868102a20 2024-04-23T18:24:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.979+0000 7fd865d9b640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd8500a3bb0 0x7fd8500a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd85c005d70 tx=0x7fd85c005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:03.983+0000 7fd8577fe640 1 -- 172.21.15.19:0/1268870657 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8580d3030 con 0x7fd868102a20 2024-04-23T18:24:04.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.213+0000 7fd86cdd8640 1 -- 172.21.15.19:0/1268870657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd86802d5c0 con 0x7fd868102a20 2024-04-23T18:24:04.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.214+0000 7fd8577fe640 1 -- 172.21.15.19:0/1268870657 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7fd85809c8a0 con 0x7fd868102a20 2024-04-23T18:24:04.207 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:24:04.208 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:24:02.624724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24508,24469,24521,24491]},"id":1}]} 2024-04-23T18:24:04.208 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-23T18:24:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 -- 172.21.15.19:0/1268870657 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd8500a3bb0 msgr2=0x7fd8500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd8500a3bb0 0x7fd8500a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd85c005d70 tx=0x7fd85c005d00 comp rx=0 tx=0).stop 2024-04-23T18:24:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 -- 172.21.15.19:0/1268870657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868102a20 msgr2=0x7fd8681981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868102a20 0x7fd8681981e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd85800d950 tx=0x7fd85800de10 comp rx=0 tx=0).stop 2024-04-23T18:24:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 -- 172.21.15.19:0/1268870657 shutdown_connections 2024-04-23T18:24:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd8500a3bb0 0x7fd8500a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd868107c80 0x7fd86819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd868105350 0x7fd868198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 --2- 172.21.15.19:0/1268870657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868102a20 0x7fd8681981e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 -- 172.21.15.19:0/1268870657 >> 172.21.15.19:0/1268870657 conn(0x7fd8680fc430 msgr2=0x7fd868103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 -- 172.21.15.19:0/1268870657 shutdown_connections 2024-04-23T18:24:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.216+0000 7fd86cdd8640 1 -- 172.21.15.19:0/1268870657 wait complete. 2024-04-23T18:24:04.220 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.328+0000 7f4455c2a640 1 -- 172.21.15.19:0/3229909370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4450107c90 msgr2=0x7f445010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.328+0000 7f4455c2a640 1 --2- 172.21.15.19:0/3229909370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4450107c90 0x7f445010a0e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4448009ca0 tx=0x7f444802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.328+0000 7f4455c2a640 1 -- 172.21.15.19:0/3229909370 shutdown_connections 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.328+0000 7f4455c2a640 1 --2- 172.21.15.19:0/3229909370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4450105360 0x7f4450107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.328+0000 7f4455c2a640 1 --2- 172.21.15.19:0/3229909370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4450107c90 0x7f445010a0e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.328+0000 7f4455c2a640 1 --2- 172.21.15.19:0/3229909370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450102a30 0x7f4450104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.328+0000 7f4455c2a640 1 -- 172.21.15.19:0/3229909370 >> 172.21.15.19:0/3229909370 conn(0x7f44500fc440 msgr2=0x7f44500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.328+0000 7f4455c2a640 1 -- 172.21.15.19:0/3229909370 shutdown_connections 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.328+0000 7f4455c2a640 1 -- 172.21.15.19:0/3229909370 wait complete. 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.328+0000 7f4455c2a640 1 Processor -- start 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f4455c2a640 1 -- start start 2024-04-23T18:24:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f4455c2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4450102a30 0x7f44501981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f4455c2a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4450105360 0x7f4450198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f4455c2a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450107c90 0x7f445019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f4455c2a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f445010daf0 con 0x7f4450102a30 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f4455c2a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f445010d970 con 0x7f4450105360 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f4455c2a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f445010dc70 con 0x7f4450107c90 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4450102a30 0x7f44501981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4450102a30 0x7f44501981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60600/0 (socket says 172.21.15.19:60600) 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444f7fe640 1 -- 172.21.15.19:0/3963238972 learned_addr learned my addr 172.21.15.19:0/3963238972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444ffff640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450107c90 0x7f445019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444f7fe640 1 -- 172.21.15.19:0/3963238972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450107c90 msgr2=0x7f445019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444f7fe640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450107c90 0x7f445019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444effd640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4450105360 0x7f4450198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444f7fe640 1 -- 172.21.15.19:0/3963238972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4450105360 msgr2=0x7f4450198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444f7fe640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4450105360 0x7f4450198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444f7fe640 1 -- 172.21.15.19:0/3963238972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445019fde0 con 0x7f4450102a30 2024-04-23T18:24:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.329+0000 7f444effd640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4450105360 0x7f4450198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.330+0000 7f444f7fe640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4450102a30 0x7f44501981c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f444000d770 tx=0x7f444000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.330+0000 7f444cff9640 1 -- 172.21.15.19:0/3963238972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440017070 con 0x7f4450102a30 2024-04-23T18:24:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.330+0000 7f444cff9640 1 -- 172.21.15.19:0/3963238972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44400054d0 con 0x7f4450102a30 2024-04-23T18:24:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.330+0000 7f4455c2a640 1 -- 172.21.15.19:0/3963238972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44501a00d0 con 0x7f4450102a30 2024-04-23T18:24:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.330+0000 7f444cff9640 1 -- 172.21.15.19:0/3963238972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440004c30 con 0x7f4450102a30 2024-04-23T18:24:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.330+0000 7f4455c2a640 1 -- 172.21.15.19:0/3963238972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44501a05d0 con 0x7f4450102a30 2024-04-23T18:24:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.331+0000 7f4455c2a640 1 -- 172.21.15.19:0/3963238972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44500fdf30 con 0x7f4450102a30 2024-04-23T18:24:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.332+0000 7f444cff9640 1 -- 172.21.15.19:0/3963238972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4440004dd0 con 0x7f4450102a30 2024-04-23T18:24:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.332+0000 7f444cff9640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44380a3bb0 0x7f44380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.332+0000 7f444effd640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44380a3bb0 0x7f44380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.333+0000 7f444cff9640 1 -- 172.21.15.19:0/3963238972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f44400d0250 con 0x7f4450102a30 2024-04-23T18:24:04.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.333+0000 7f444effd640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44380a3bb0 0x7f44380a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f44501996d0 tx=0x7f4444005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:04.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.336+0000 7f444cff9640 1 -- 172.21.15.19:0/3963238972 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f444009cce0 con 0x7f4450102a30 2024-04-23T18:24:04.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.394+0000 7f444cff9640 1 -- 172.21.15.19:0/3963238972 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4440098c50 con 0x7f4450102a30 2024-04-23T18:24:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.529+0000 7f4455c2a640 1 -- 172.21.15.19:0/3963238972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4450062b50 con 0x7f4450102a30 2024-04-23T18:24:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.531+0000 7f444cff9640 1 -- 172.21.15.19:0/3963238972 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22209 (secure 0 0 0) 0x7f44400a54a0 con 0x7f4450102a30 2024-04-23T18:24:04.524 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:24:04.524 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":86,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:23:57.076186+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 -- 172.21.15.19:0/3963238972 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44380a3bb0 msgr2=0x7f44380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44380a3bb0 0x7f44380a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f44501996d0 tx=0x7f4444005d30 comp rx=0 tx=0).stop 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 -- 172.21.15.19:0/3963238972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4450102a30 msgr2=0x7f44501981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4450102a30 0x7f44501981c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f444000d770 tx=0x7f444000dc30 comp rx=0 tx=0).stop 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 -- 172.21.15.19:0/3963238972 shutdown_connections 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f44380a3bb0 0x7f44380a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4450105360 0x7f4450198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450107c90 0x7f445019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 --2- 172.21.15.19:0/3963238972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4450102a30 0x7f44501981c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 -- 172.21.15.19:0/3963238972 >> 172.21.15.19:0/3963238972 conn(0x7f44500fc440 msgr2=0x7f4450103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 -- 172.21.15.19:0/3963238972 shutdown_connections 2024-04-23T18:24:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.533+0000 7f4455c2a640 1 -- 172.21.15.19:0/3963238972 wait complete. 2024-04-23T18:24:04.538 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T18:24:04.538 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:24:04.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.647+0000 7f2ccd707640 1 -- 172.21.15.19:0/1719263557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8102a30 msgr2=0x7f2cc8104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.647+0000 7f2ccd707640 1 --2- 172.21.15.19:0/1719263557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8102a30 0x7f2cc8104e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009900 tx=0x7f2cb802a410 comp rx=0 tx=0).stop 2024-04-23T18:24:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.647+0000 7f2ccd707640 1 -- 172.21.15.19:0/1719263557 shutdown_connections 2024-04-23T18:24:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.647+0000 7f2ccd707640 1 --2- 172.21.15.19:0/1719263557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc8107c90 0x7f2cc810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.647+0000 7f2ccd707640 1 --2- 172.21.15.19:0/1719263557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cc8105360 0x7f2cc8107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.647+0000 7f2ccd707640 1 --2- 172.21.15.19:0/1719263557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8102a30 0x7f2cc8104e20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.647+0000 7f2ccd707640 1 -- 172.21.15.19:0/1719263557 >> 172.21.15.19:0/1719263557 conn(0x7f2cc80fc440 msgr2=0x7f2cc80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.647+0000 7f2ccd707640 1 -- 172.21.15.19:0/1719263557 shutdown_connections 2024-04-23T18:24:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.647+0000 7f2ccd707640 1 -- 172.21.15.19:0/1719263557 wait complete. 2024-04-23T18:24:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2ccd707640 1 Processor -- start 2024-04-23T18:24:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2ccd707640 1 -- start start 2024-04-23T18:24:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2ccd707640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cc8102a30 0x7f2cc8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2ccd707640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc8105360 0x7f2cc81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2ccd707640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8107c90 0x7f2cc819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2ccd707640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc810daf0 con 0x7f2cc8107c90 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2ccd707640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc810d970 con 0x7f2cc8105360 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2ccd707640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc810dc70 con 0x7f2cc8102a30 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2cc77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8107c90 0x7f2cc819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2cc77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8107c90 0x7f2cc819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60610/0 (socket says 172.21.15.19:60610) 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2cc6ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cc8102a30 0x7f2cc8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2cc77fe640 1 -- 172.21.15.19:0/840808450 learned_addr learned my addr 172.21.15.19:0/840808450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.648+0000 7f2cc67fc640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc8105360 0x7f2cc81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2cc77fe640 1 -- 172.21.15.19:0/840808450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cc8102a30 msgr2=0x7f2cc8198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2cc77fe640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cc8102a30 0x7f2cc8198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2cc77fe640 1 -- 172.21.15.19:0/840808450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc8105360 msgr2=0x7f2cc81987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2cc77fe640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc8105360 0x7f2cc81987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2cc77fe640 1 -- 172.21.15.19:0/840808450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc819fe40 con 0x7f2cc8107c90 2024-04-23T18:24:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2cc67fc640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc8105360 0x7f2cc81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2cc77fe640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8107c90 0x7f2cc819f7d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000eb10 tx=0x7f2cc000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2caffff640 1 -- 172.21.15.19:0/840808450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0004b20 con 0x7f2cc8107c90 2024-04-23T18:24:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2ccd707640 1 -- 172.21.15.19:0/840808450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc81a0130 con 0x7f2cc8107c90 2024-04-23T18:24:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2caffff640 1 -- 172.21.15.19:0/840808450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc0004cc0 con 0x7f2cc8107c90 2024-04-23T18:24:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2caffff640 1 -- 172.21.15.19:0/840808450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0010670 con 0x7f2cc8107c90 2024-04-23T18:24:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.649+0000 7f2ccd707640 1 -- 172.21.15.19:0/840808450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc81a0680 con 0x7f2cc8107c90 2024-04-23T18:24:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.650+0000 7f2ccd707640 1 -- 172.21.15.19:0/840808450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc80fdf30 con 0x7f2cc8107c90 2024-04-23T18:24:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.651+0000 7f2caffff640 1 -- 172.21.15.19:0/840808450 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2cc0005020 con 0x7f2cc8107c90 2024-04-23T18:24:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.652+0000 7f2caffff640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2cb00a3c40 0x7f2cb00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.652+0000 7f2cc6ffd640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2cb00a3c40 0x7f2cb00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.652+0000 7f2caffff640 1 -- 172.21.15.19:0/840808450 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2cc00d0980 con 0x7f2cc8107c90 2024-04-23T18:24:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.652+0000 7f2cc6ffd640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2cb00a3c40 0x7f2cb00a6100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8000c00 tx=0x7f2cb80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.656+0000 7f2caffff640 1 -- 172.21.15.19:0/840808450 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cc009d3b0 con 0x7f2cc8107c90 2024-04-23T18:24:04.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.839+0000 7f2ccd707640 1 -- 172.21.15.19:0/840808450 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cc8062390 con 0x7f2cb00a3c40 2024-04-23T18:24:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.847+0000 7f2caffff640 1 -- 172.21.15.19:0/840808450 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258889 (secure 0 0 0) 0x7f2cc8062390 con 0x7f2cb00a3c40 2024-04-23T18:24:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:24:04.845 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-23T18:24:04.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 -- 172.21.15.19:0/840808450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2cb00a3c40 msgr2=0x7f2cb00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2cb00a3c40 0x7f2cb00a6100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8000c00 tx=0x7f2cb80314a0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 -- 172.21.15.19:0/840808450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8107c90 msgr2=0x7f2cc819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8107c90 0x7f2cc819f7d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000eb10 tx=0x7f2cc000c2d0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 -- 172.21.15.19:0/840808450 shutdown_connections 2024-04-23T18:24:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2cb00a3c40 0x7f2cb00a6100 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cc8105360 0x7f2cc81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc8107c90 0x7f2cc819f7d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 --2- 172.21.15.19:0/840808450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cc8102a30 0x7f2cc8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 -- 172.21.15.19:0/840808450 >> 172.21.15.19:0/840808450 conn(0x7f2cc80fc440 msgr2=0x7f2cc81034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 -- 172.21.15.19:0/840808450 shutdown_connections 2024-04-23T18:24:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:04.850+0000 7f2ccd707640 1 -- 172.21.15.19:0/840808450 wait complete. 2024-04-23T18:24:04.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:04 smithi019 ceph-mon[29215]: pgmap v300: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 13 op/s 2024-04-23T18:24:04.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:04 smithi019 ceph-mon[29215]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:04.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:04 smithi019 ceph-mon[29215]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:04.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:04 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1268870657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:04.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:04 smithi019 ceph-mon[29215]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:24:04.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:04 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3963238972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:04.854 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-04-23T18:24:03.353944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594386,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":291,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":267,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381164,"kb_used_data":11180,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560660,"statfs":{"total":1151940427776,"available":1151550115840,"internally_reserved":0,"allocated":11448320,"data_stored":7101719,"data_compressed":106755,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4018,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002289"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368375+0000","last_change":"2024-04-23T18:23:56.098292+0000","last_active":"2024-04-23T18:23:57.368375+0000","last_peered":"2024-04-23T18:23:57.368375+0000","last_clean":"2024-04-23T18:23:57.368375+0000","last_became_active":"2024-04-23T18:23:56.096623+0000","last_became_peered":"2024-04-23T18:23:56.096623+0000","last_unstale":"2024-04-23T18:23:57.368375+0000","last_undegraded":"2024-04-23T18:23:57.368375+0000","last_fullsized":"2024-04-23T18:23:57.368375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:06.186581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367418+0000","last_change":"2024-04-23T18:23:56.096807+0000","last_active":"2024-04-23T18:23:57.367418+0000","last_peered":"2024-04-23T18:23:57.367418+0000","last_clean":"2024-04-23T18:23:57.367418+0000","last_became_active":"2024-04-23T18:23:56.096327+0000","last_became_peered":"2024-04-23T18:23:56.096327+0000","last_unstale":"2024-04-23T18:23:57.367418+0000","last_undegraded":"2024-04-23T18:23:57.367418+0000","last_fullsized":"2024-04-23T18:23:57.367418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:56:16.391821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367300+0000","last_change":"2024-04-23T18:23:56.101615+0000","last_active":"2024-04-23T18:23:57.367300+0000","last_peered":"2024-04-23T18:23:57.367300+0000","last_clean":"2024-04-23T18:23:57.367300+0000","last_became_active":"2024-04-23T18:23:56.100929+0000","last_became_peered":"2024-04-23T18:23:56.100929+0000","last_unstale":"2024-04-23T18:23:57.367300+0000","last_undegraded":"2024-04-23T18:23:57.367300+0000","last_fullsized":"2024-04-23T18:23:57.367300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:25.547029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086535+0000","last_change":"2024-04-23T18:23:56.103627+0000","last_active":"2024-04-23T18:23:57.086535+0000","last_peered":"2024-04-23T18:23:57.086535+0000","last_clean":"2024-04-23T18:23:57.086535+0000","last_became_active":"2024-04-23T18:23:56.102752+0000","last_became_peered":"2024-04-23T18:23:56.102752+0000","last_unstale":"2024-04-23T18:23:57.086535+0000","last_undegraded":"2024-04-23T18:23:57.086535+0000","last_fullsized":"2024-04-23T18:23:57.086535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:28.460342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355590+0000","last_change":"2024-04-23T18:23:56.094889+0000","last_active":"2024-04-23T18:23:57.355590+0000","last_peered":"2024-04-23T18:23:57.355590+0000","last_clean":"2024-04-23T18:23:57.355590+0000","last_became_active":"2024-04-23T18:23:56.094262+0000","last_became_peered":"2024-04-23T18:23:56.094262+0000","last_unstale":"2024-04-23T18:23:57.355590+0000","last_undegraded":"2024-04-23T18:23:57.355590+0000","last_fullsized":"2024-04-23T18:23:57.355590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:22.332114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.118532+0000","last_change":"2024-04-23T18:23:56.108893+0000","last_active":"2024-04-23T18:23:57.118532+0000","last_peered":"2024-04-23T18:23:57.118532+0000","last_clean":"2024-04-23T18:23:57.118532+0000","last_became_active":"2024-04-23T18:23:56.108229+0000","last_became_peered":"2024-04-23T18:23:56.108229+0000","last_unstale":"2024-04-23T18:23:57.118532+0000","last_undegraded":"2024-04-23T18:23:57.118532+0000","last_fullsized":"2024-04-23T18:23:57.118532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:50.930321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368511+0000","last_change":"2024-04-23T18:23:56.098674+0000","last_active":"2024-04-23T18:23:57.368511+0000","last_peered":"2024-04-23T18:23:57.368511+0000","last_clean":"2024-04-23T18:23:57.368511+0000","last_became_active":"2024-04-23T18:23:56.098299+0000","last_became_peered":"2024-04-23T18:23:56.098299+0000","last_unstale":"2024-04-23T18:23:57.368511+0000","last_undegraded":"2024-04-23T18:23:57.368511+0000","last_fullsized":"2024-04-23T18:23:57.368511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:05.711410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085754+0000","last_change":"2024-04-23T18:23:56.097437+0000","last_active":"2024-04-23T18:23:57.085754+0000","last_peered":"2024-04-23T18:23:57.085754+0000","last_clean":"2024-04-23T18:23:57.085754+0000","last_became_active":"2024-04-23T18:23:56.095702+0000","last_became_peered":"2024-04-23T18:23:56.095702+0000","last_unstale":"2024-04-23T18:23:57.085754+0000","last_undegraded":"2024-04-23T18:23:57.085754+0000","last_fullsized":"2024-04-23T18:23:57.085754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:35.754832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.369096+0000","last_change":"2024-04-23T18:23:56.096060+0000","last_active":"2024-04-23T18:23:57.369096+0000","last_peered":"2024-04-23T18:23:57.369096+0000","last_clean":"2024-04-23T18:23:57.369096+0000","last_became_active":"2024-04-23T18:23:56.095597+0000","last_became_peered":"2024-04-23T18:23:56.095597+0000","last_unstale":"2024-04-23T18:23:57.369096+0000","last_undegraded":"2024-04-23T18:23:57.369096+0000","last_fullsized":"2024-04-23T18:23:57.369096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:59.735347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.118863+0000","last_change":"2024-04-23T18:23:56.109033+0000","last_active":"2024-04-23T18:23:57.118863+0000","last_peered":"2024-04-23T18:23:57.118863+0000","last_clean":"2024-04-23T18:23:57.118863+0000","last_became_active":"2024-04-23T18:23:56.108472+0000","last_became_peered":"2024-04-23T18:23:56.108472+0000","last_unstale":"2024-04-23T18:23:57.118863+0000","last_undegraded":"2024-04-23T18:23:57.118863+0000","last_fullsized":"2024-04-23T18:23:57.118863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:32.491181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368729+0000","last_change":"2024-04-23T18:23:56.097771+0000","last_active":"2024-04-23T18:23:57.368729+0000","last_peered":"2024-04-23T18:23:57.368729+0000","last_clean":"2024-04-23T18:23:57.368729+0000","last_became_active":"2024-04-23T18:23:56.097403+0000","last_became_peered":"2024-04-23T18:23:56.097403+0000","last_unstale":"2024-04-23T18:23:57.368729+0000","last_undegraded":"2024-04-23T18:23:57.368729+0000","last_fullsized":"2024-04-23T18:23:57.368729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:48:36.261543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086554+0000","last_change":"2024-04-23T18:23:56.108550+0000","last_active":"2024-04-23T18:23:57.086554+0000","last_peered":"2024-04-23T18:23:57.086554+0000","last_clean":"2024-04-23T18:23:57.086554+0000","last_became_active":"2024-04-23T18:23:56.108191+0000","last_became_peered":"2024-04-23T18:23:56.108191+0000","last_unstale":"2024-04-23T18:23:57.086554+0000","last_undegraded":"2024-04-23T18:23:57.086554+0000","last_fullsized":"2024-04-23T18:23:57.086554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:25.142466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368262+0000","last_change":"2024-04-23T18:23:56.092573+0000","last_active":"2024-04-23T18:23:57.368262+0000","last_peered":"2024-04-23T18:23:57.368262+0000","last_clean":"2024-04-23T18:23:57.368262+0000","last_became_active":"2024-04-23T18:23:56.092095+0000","last_became_peered":"2024-04-23T18:23:56.092095+0000","last_unstale":"2024-04-23T18:23:57.368262+0000","last_undegraded":"2024-04-23T18:23:57.368262+0000","last_fullsized":"2024-04-23T18:23:57.368262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:01.601206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367207+0000","last_change":"2024-04-23T18:23:56.102073+0000","last_active":"2024-04-23T18:23:57.367207+0000","last_peered":"2024-04-23T18:23:57.367207+0000","last_clean":"2024-04-23T18:23:57.367207+0000","last_became_active":"2024-04-23T18:23:56.101059+0000","last_became_peered":"2024-04-23T18:23:56.101059+0000","last_unstale":"2024-04-23T18:23:57.367207+0000","last_undegraded":"2024-04-23T18:23:57.367207+0000","last_fullsized":"2024-04-23T18:23:57.367207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:37.904099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367122+0000","last_change":"2024-04-23T18:23:56.108482+0000","last_active":"2024-04-23T18:23:57.367122+0000","last_peered":"2024-04-23T18:23:57.367122+0000","last_clean":"2024-04-23T18:23:57.367122+0000","last_became_active":"2024-04-23T18:23:56.108128+0000","last_became_peered":"2024-04-23T18:23:56.108128+0000","last_unstale":"2024-04-23T18:23:57.367122+0000","last_undegraded":"2024-04-23T18:23:57.367122+0000","last_fullsized":"2024-04-23T18:23:57.367122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:51.925881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367953+0000","last_change":"2024-04-23T18:23:56.099691+0000","last_active":"2024-04-23T18:23:57.367953+0000","last_peered":"2024-04-23T18:23:57.367953+0000","last_clean":"2024-04-23T18:23:57.367953+0000","last_became_active":"2024-04-23T18:23:56.099272+0000","last_became_peered":"2024-04-23T18:23:56.099272+0000","last_unstale":"2024-04-23T18:23:57.367953+0000","last_undegraded":"2024-04-23T18:23:57.367953+0000","last_fullsized":"2024-04-23T18:23:57.367953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:41.834104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368476+0000","last_change":"2024-04-23T18:23:56.097965+0000","last_active":"2024-04-23T18:23:57.368476+0000","last_peered":"2024-04-23T18:23:57.368476+0000","last_clean":"2024-04-23T18:23:57.368476+0000","last_became_active":"2024-04-23T18:23:56.096766+0000","last_became_peered":"2024-04-23T18:23:56.096766+0000","last_unstale":"2024-04-23T18:23:57.368476+0000","last_undegraded":"2024-04-23T18:23:57.368476+0000","last_fullsized":"2024-04-23T18:23:57.368476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:40:00.076939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.353400+0000","last_change":"2024-04-23T18:23:56.095158+0000","last_active":"2024-04-23T18:23:57.353400+0000","last_peered":"2024-04-23T18:23:57.353400+0000","last_clean":"2024-04-23T18:23:57.353400+0000","last_became_active":"2024-04-23T18:23:56.094812+0000","last_became_peered":"2024-04-23T18:23:56.094812+0000","last_unstale":"2024-04-23T18:23:57.353400+0000","last_undegraded":"2024-04-23T18:23:57.353400+0000","last_fullsized":"2024-04-23T18:23:57.353400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:40.874698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367851+0000","last_change":"2024-04-23T18:23:56.099358+0000","last_active":"2024-04-23T18:23:57.367851+0000","last_peered":"2024-04-23T18:23:57.367851+0000","last_clean":"2024-04-23T18:23:57.367851+0000","last_became_active":"2024-04-23T18:23:56.099031+0000","last_became_peered":"2024-04-23T18:23:56.099031+0000","last_unstale":"2024-04-23T18:23:57.367851+0000","last_undegraded":"2024-04-23T18:23:57.367851+0000","last_fullsized":"2024-04-23T18:23:57.367851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:29.634932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.353239+0000","last_change":"2024-04-23T18:23:55.262939+0000","last_active":"2024-04-23T18:23:57.353239+0000","last_peered":"2024-04-23T18:23:57.353239+0000","last_clean":"2024-04-23T18:23:57.353239+0000","last_became_active":"2024-04-23T18:23:55.262609+0000","last_became_peered":"2024-04-23T18:23:55.262609+0000","last_unstale":"2024-04-23T18:23:57.353239+0000","last_undegraded":"2024-04-23T18:23:57.353239+0000","last_fullsized":"2024-04-23T18:23:57.353239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:12.418349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085869+0000","last_change":"2024-04-23T18:23:56.092182+0000","last_active":"2024-04-23T18:23:57.085869+0000","last_peered":"2024-04-23T18:23:57.085869+0000","last_clean":"2024-04-23T18:23:57.085869+0000","last_became_active":"2024-04-23T18:23:56.091444+0000","last_became_peered":"2024-04-23T18:23:56.091444+0000","last_unstale":"2024-04-23T18:23:57.085869+0000","last_undegraded":"2024-04-23T18:23:57.085869+0000","last_fullsized":"2024-04-23T18:23:57.085869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:00.775498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.392295+0000","last_change":"2024-04-23T18:23:55.259880+0000","last_active":"2024-04-23T18:23:57.392295+0000","last_peered":"2024-04-23T18:23:57.392295+0000","last_clean":"2024-04-23T18:23:57.392295+0000","last_became_active":"2024-04-23T18:23:55.259576+0000","last_became_peered":"2024-04-23T18:23:55.259576+0000","last_unstale":"2024-04-23T18:23:57.392295+0000","last_undegraded":"2024-04-23T18:23:57.392295+0000","last_fullsized":"2024-04-23T18:23:57.392295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:13.783776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367518+0000","last_change":"2024-04-23T18:23:56.094307+0000","last_active":"2024-04-23T18:23:57.367518+0000","last_peered":"2024-04-23T18:23:57.367518+0000","last_clean":"2024-04-23T18:23:57.367518+0000","last_became_active":"2024-04-23T18:23:56.093893+0000","last_became_peered":"2024-04-23T18:23:56.093893+0000","last_unstale":"2024-04-23T18:23:57.367518+0000","last_undegraded":"2024-04-23T18:23:57.367518+0000","last_fullsized":"2024-04-23T18:23:57.367518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:12:35.385497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:01.394701+0000","last_change":"2024-04-23T18:23:55.263137+0000","last_active":"2024-04-23T18:24:01.394701+0000","last_peered":"2024-04-23T18:24:01.394701+0000","last_clean":"2024-04-23T18:24:01.394701+0000","last_became_active":"2024-04-23T18:23:55.262553+0000","last_became_peered":"2024-04-23T18:23:55.262553+0000","last_unstale":"2024-04-23T18:24:01.394701+0000","last_undegraded":"2024-04-23T18:24:01.394701+0000","last_fullsized":"2024-04-23T18:24:01.394701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:33:42.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368304+0000","last_change":"2024-04-23T18:23:56.091098+0000","last_active":"2024-04-23T18:23:57.368304+0000","last_peered":"2024-04-23T18:23:57.368304+0000","last_clean":"2024-04-23T18:23:57.368304+0000","last_became_active":"2024-04-23T18:23:56.090635+0000","last_became_peered":"2024-04-23T18:23:56.090635+0000","last_unstale":"2024-04-23T18:23:57.368304+0000","last_undegraded":"2024-04-23T18:23:57.368304+0000","last_fullsized":"2024-04-23T18:23:57.368304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:17.787937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355858+0000","last_change":"2024-04-23T18:23:55.262146+0000","last_active":"2024-04-23T18:23:57.355858+0000","last_peered":"2024-04-23T18:23:57.355858+0000","last_clean":"2024-04-23T18:23:57.355858+0000","last_became_active":"2024-04-23T18:23:55.261838+0000","last_became_peered":"2024-04-23T18:23:55.261838+0000","last_unstale":"2024-04-23T18:23:57.355858+0000","last_undegraded":"2024-04-23T18:23:57.355858+0000","last_fullsized":"2024-04-23T18:23:57.355858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:42:39.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086635+0000","last_change":"2024-04-23T18:23:56.100550+0000","last_active":"2024-04-23T18:23:57.086635+0000","last_peered":"2024-04-23T18:23:57.086635+0000","last_clean":"2024-04-23T18:23:57.086635+0000","last_became_active":"2024-04-23T18:23:56.099709+0000","last_became_peered":"2024-04-23T18:23:56.099709+0000","last_unstale":"2024-04-23T18:23:57.086635+0000","last_undegraded":"2024-04-23T18:23:57.086635+0000","last_fullsized":"2024-04-23T18:23:57.086635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:23.797920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086301+0000","last_change":"2024-04-23T18:23:55.261399+0000","last_active":"2024-04-23T18:23:57.086301+0000","last_peered":"2024-04-23T18:23:57.086301+0000","last_clean":"2024-04-23T18:23:57.086301+0000","last_became_active":"2024-04-23T18:23:55.261012+0000","last_became_peered":"2024-04-23T18:23:55.261012+0000","last_unstale":"2024-04-23T18:23:57.086301+0000","last_undegraded":"2024-04-23T18:23:57.086301+0000","last_fullsized":"2024-04-23T18:23:57.086301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:31.377776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355568+0000","last_change":"2024-04-23T18:23:56.096785+0000","last_active":"2024-04-23T18:23:57.355568+0000","last_peered":"2024-04-23T18:23:57.355568+0000","last_clean":"2024-04-23T18:23:57.355568+0000","last_became_active":"2024-04-23T18:23:56.096420+0000","last_became_peered":"2024-04-23T18:23:56.096420+0000","last_unstale":"2024-04-23T18:23:57.355568+0000","last_undegraded":"2024-04-23T18:23:57.355568+0000","last_fullsized":"2024-04-23T18:23:57.355568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:20:49.215071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.118210+0000","last_change":"2024-04-23T18:23:55.260645+0000","last_active":"2024-04-23T18:23:57.118210+0000","last_peered":"2024-04-23T18:23:57.118210+0000","last_clean":"2024-04-23T18:23:57.118210+0000","last_became_active":"2024-04-23T18:23:55.260118+0000","last_became_peered":"2024-04-23T18:23:55.260118+0000","last_unstale":"2024-04-23T18:23:57.118210+0000","last_undegraded":"2024-04-23T18:23:57.118210+0000","last_fullsized":"2024-04-23T18:23:57.118210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:14:12.066297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086647+0000","last_change":"2024-04-23T18:23:56.100618+0000","last_active":"2024-04-23T18:23:57.086647+0000","last_peered":"2024-04-23T18:23:57.086647+0000","last_clean":"2024-04-23T18:23:57.086647+0000","last_became_active":"2024-04-23T18:23:56.100126+0000","last_became_peered":"2024-04-23T18:23:56.100126+0000","last_unstale":"2024-04-23T18:23:57.086647+0000","last_undegraded":"2024-04-23T18:23:57.086647+0000","last_fullsized":"2024-04-23T18:23:57.086647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:41:11.481893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.384298+0000","last_change":"2024-04-23T18:23:55.258636+0000","last_active":"2024-04-23T18:24:00.384298+0000","last_peered":"2024-04-23T18:24:00.384298+0000","last_clean":"2024-04-23T18:24:00.384298+0000","last_became_active":"2024-04-23T18:23:55.257744+0000","last_became_peered":"2024-04-23T18:23:55.257744+0000","last_unstale":"2024-04-23T18:24:00.384298+0000","last_undegraded":"2024-04-23T18:24:00.384298+0000","last_fullsized":"2024-04-23T18:24:00.384298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:46:07.454725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085920+0000","last_change":"2024-04-23T18:23:56.094822+0000","last_active":"2024-04-23T18:23:57.085920+0000","last_peered":"2024-04-23T18:23:57.085920+0000","last_clean":"2024-04-23T18:23:57.085920+0000","last_became_active":"2024-04-23T18:23:56.094489+0000","last_became_peered":"2024-04-23T18:23:56.094489+0000","last_unstale":"2024-04-23T18:23:57.085920+0000","last_undegraded":"2024-04-23T18:23:57.085920+0000","last_fullsized":"2024-04-23T18:23:57.085920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:21.234363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085978+0000","last_change":"2024-04-23T18:23:55.261322+0000","last_active":"2024-04-23T18:23:57.085978+0000","last_peered":"2024-04-23T18:23:57.085978+0000","last_clean":"2024-04-23T18:23:57.085978+0000","last_became_active":"2024-04-23T18:23:55.260990+0000","last_became_peered":"2024-04-23T18:23:55.260990+0000","last_unstale":"2024-04-23T18:23:57.085978+0000","last_undegraded":"2024-04-23T18:23:57.085978+0000","last_fullsized":"2024-04-23T18:23:57.085978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:31.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.353308+0000","last_change":"2024-04-23T18:23:56.093105+0000","last_active":"2024-04-23T18:23:57.353308+0000","last_peered":"2024-04-23T18:23:57.353308+0000","last_clean":"2024-04-23T18:23:57.353308+0000","last_became_active":"2024-04-23T18:23:56.092717+0000","last_became_peered":"2024-04-23T18:23:56.092717+0000","last_unstale":"2024-04-23T18:23:57.353308+0000","last_undegraded":"2024-04-23T18:23:57.353308+0000","last_fullsized":"2024-04-23T18:23:57.353308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:13:59.574057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:59.377727+0000","last_change":"2024-04-23T18:23:55.256307+0000","last_active":"2024-04-23T18:23:59.377727+0000","last_peered":"2024-04-23T18:23:59.377727+0000","last_clean":"2024-04-23T18:23:59.377727+0000","last_became_active":"2024-04-23T18:23:55.255959+0000","last_became_peered":"2024-04-23T18:23:55.255959+0000","last_unstale":"2024-04-23T18:23:59.377727+0000","last_undegraded":"2024-04-23T18:23:59.377727+0000","last_fullsized":"2024-04-23T18:23:59.377727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:04.945919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.369003+0000","last_change":"2024-04-23T18:23:56.095177+0000","last_active":"2024-04-23T18:23:57.369003+0000","last_peered":"2024-04-23T18:23:57.369003+0000","last_clean":"2024-04-23T18:23:57.369003+0000","last_became_active":"2024-04-23T18:23:56.094825+0000","last_became_peered":"2024-04-23T18:23:56.094825+0000","last_unstale":"2024-04-23T18:23:57.369003+0000","last_undegraded":"2024-04-23T18:23:57.369003+0000","last_fullsized":"2024-04-23T18:23:57.369003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:07:33.499817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:59.374520+0000","last_change":"2024-04-23T18:23:55.264258+0000","last_active":"2024-04-23T18:23:59.374520+0000","last_peered":"2024-04-23T18:23:59.374520+0000","last_clean":"2024-04-23T18:23:59.374520+0000","last_became_active":"2024-04-23T18:23:55.263698+0000","last_became_peered":"2024-04-23T18:23:55.263698+0000","last_unstale":"2024-04-23T18:23:59.374520+0000","last_undegraded":"2024-04-23T18:23:59.374520+0000","last_fullsized":"2024-04-23T18:23:59.374520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:00.566948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367840+0000","last_change":"2024-04-23T18:23:56.109085+0000","last_active":"2024-04-23T18:23:57.367840+0000","last_peered":"2024-04-23T18:23:57.367840+0000","last_clean":"2024-04-23T18:23:57.367840+0000","last_became_active":"2024-04-23T18:23:56.108033+0000","last_became_peered":"2024-04-23T18:23:56.108033+0000","last_unstale":"2024-04-23T18:23:57.367840+0000","last_undegraded":"2024-04-23T18:23:57.367840+0000","last_fullsized":"2024-04-23T18:23:57.367840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:59.980512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086679+0000","last_change":"2024-04-23T18:23:55.256728+0000","last_active":"2024-04-23T18:23:57.086679+0000","last_peered":"2024-04-23T18:23:57.086679+0000","last_clean":"2024-04-23T18:23:57.086679+0000","last_became_active":"2024-04-23T18:23:55.256367+0000","last_became_peered":"2024-04-23T18:23:55.256367+0000","last_unstale":"2024-04-23T18:23:57.086679+0000","last_undegraded":"2024-04-23T18:23:57.086679+0000","last_fullsized":"2024-04-23T18:23:57.086679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:08.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368605+0000","last_change":"2024-04-23T18:23:56.102898+0000","last_active":"2024-04-23T18:23:57.368605+0000","last_peered":"2024-04-23T18:23:57.368605+0000","last_clean":"2024-04-23T18:23:57.368605+0000","last_became_active":"2024-04-23T18:23:56.102373+0000","last_became_peered":"2024-04-23T18:23:56.102373+0000","last_unstale":"2024-04-23T18:23:57.368605+0000","last_undegraded":"2024-04-23T18:23:57.368605+0000","last_fullsized":"2024-04-23T18:23:57.368605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:49.316219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:59.376964+0000","last_change":"2024-04-23T18:23:55.255094+0000","last_active":"2024-04-23T18:23:59.376964+0000","last_peered":"2024-04-23T18:23:59.376964+0000","last_clean":"2024-04-23T18:23:59.376964+0000","last_became_active":"2024-04-23T18:23:55.254726+0000","last_became_peered":"2024-04-23T18:23:55.254726+0000","last_unstale":"2024-04-23T18:23:59.376964+0000","last_undegraded":"2024-04-23T18:23:59.376964+0000","last_fullsized":"2024-04-23T18:23:59.376964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:26.988184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355944+0000","last_change":"2024-04-23T18:23:56.099866+0000","last_active":"2024-04-23T18:23:57.355944+0000","last_peered":"2024-04-23T18:23:57.355944+0000","last_clean":"2024-04-23T18:23:57.355944+0000","last_became_active":"2024-04-23T18:23:56.098606+0000","last_became_peered":"2024-04-23T18:23:56.098606+0000","last_unstale":"2024-04-23T18:23:57.355944+0000","last_undegraded":"2024-04-23T18:23:57.355944+0000","last_fullsized":"2024-04-23T18:23:57.355944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:26.661857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.386218+0000","last_change":"2024-04-23T18:23:55.256602+0000","last_active":"2024-04-23T18:24:00.386218+0000","last_peered":"2024-04-23T18:24:00.386218+0000","last_clean":"2024-04-23T18:24:00.386218+0000","last_became_active":"2024-04-23T18:23:55.256073+0000","last_became_peered":"2024-04-23T18:23:55.256073+0000","last_unstale":"2024-04-23T18:24:00.386218+0000","last_undegraded":"2024-04-23T18:24:00.386218+0000","last_fullsized":"2024-04-23T18:24:00.386218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:47.454305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368095+0000","last_change":"2024-04-23T18:23:56.099109+0000","last_active":"2024-04-23T18:23:57.368095+0000","last_peered":"2024-04-23T18:23:57.368095+0000","last_clean":"2024-04-23T18:23:57.368095+0000","last_became_active":"2024-04-23T18:23:56.098608+0000","last_became_peered":"2024-04-23T18:23:56.098608+0000","last_unstale":"2024-04-23T18:23:57.368095+0000","last_undegraded":"2024-04-23T18:23:57.368095+0000","last_fullsized":"2024-04-23T18:23:57.368095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:43.720160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086446+0000","last_change":"2024-04-23T18:23:55.262523+0000","last_active":"2024-04-23T18:23:57.086446+0000","last_peered":"2024-04-23T18:23:57.086446+0000","last_clean":"2024-04-23T18:23:57.086446+0000","last_became_active":"2024-04-23T18:23:55.262216+0000","last_became_peered":"2024-04-23T18:23:55.262216+0000","last_unstale":"2024-04-23T18:23:57.086446+0000","last_undegraded":"2024-04-23T18:23:57.086446+0000","last_fullsized":"2024-04-23T18:23:57.086446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:35.268435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368363+0000","last_change":"2024-04-23T18:23:56.106573+0000","last_active":"2024-04-23T18:23:57.368363+0000","last_peered":"2024-04-23T18:23:57.368363+0000","last_clean":"2024-04-23T18:23:57.368363+0000","last_became_active":"2024-04-23T18:23:56.106181+0000","last_became_peered":"2024-04-23T18:23:56.106181+0000","last_unstale":"2024-04-23T18:23:57.368363+0000","last_undegraded":"2024-04-23T18:23:57.368363+0000","last_fullsized":"2024-04-23T18:23:57.368363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:15.422647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086171+0000","last_change":"2024-04-23T18:23:55.262813+0000","last_active":"2024-04-23T18:23:57.086171+0000","last_peered":"2024-04-23T18:23:57.086171+0000","last_clean":"2024-04-23T18:23:57.086171+0000","last_became_active":"2024-04-23T18:23:55.262365+0000","last_became_peered":"2024-04-23T18:23:55.262365+0000","last_unstale":"2024-04-23T18:23:57.086171+0000","last_undegraded":"2024-04-23T18:23:57.086171+0000","last_fullsized":"2024-04-23T18:23:57.086171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:15.234355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367791+0000","last_change":"2024-04-23T18:23:56.101491+0000","last_active":"2024-04-23T18:23:57.367791+0000","last_peered":"2024-04-23T18:23:57.367791+0000","last_clean":"2024-04-23T18:23:57.367791+0000","last_became_active":"2024-04-23T18:23:56.101069+0000","last_became_peered":"2024-04-23T18:23:56.101069+0000","last_unstale":"2024-04-23T18:23:57.367791+0000","last_undegraded":"2024-04-23T18:23:57.367791+0000","last_fullsized":"2024-04-23T18:23:57.367791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:42.484653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'16","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.385356+0000","last_change":"2024-04-23T18:23:55.265947+0000","last_active":"2024-04-23T18:24:00.385356+0000","last_peered":"2024-04-23T18:24:00.385356+0000","last_clean":"2024-04-23T18:24:00.385356+0000","last_became_active":"2024-04-23T18:23:55.265174+0000","last_became_peered":"2024-04-23T18:23:55.265174+0000","last_unstale":"2024-04-23T18:24:00.385356+0000","last_undegraded":"2024-04-23T18:24:00.385356+0000","last_fullsized":"2024-04-23T18:24:00.385356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:41.996928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086155+0000","last_change":"2024-04-23T18:23:56.105691+0000","last_active":"2024-04-23T18:23:57.086155+0000","last_peered":"2024-04-23T18:23:57.086155+0000","last_clean":"2024-04-23T18:23:57.086155+0000","last_became_active":"2024-04-23T18:23:56.105205+0000","last_became_peered":"2024-04-23T18:23:56.105205+0000","last_unstale":"2024-04-23T18:23:57.086155+0000","last_undegraded":"2024-04-23T18:23:57.086155+0000","last_fullsized":"2024-04-23T18:23:57.086155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:30.779821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.356097+0000","last_change":"2024-04-23T18:23:55.261198+0000","last_active":"2024-04-23T18:23:57.356097+0000","last_peered":"2024-04-23T18:23:57.356097+0000","last_clean":"2024-04-23T18:23:57.356097+0000","last_became_active":"2024-04-23T18:23:55.260884+0000","last_became_peered":"2024-04-23T18:23:55.260884+0000","last_unstale":"2024-04-23T18:23:57.356097+0000","last_undegraded":"2024-04-23T18:23:57.356097+0000","last_fullsized":"2024-04-23T18:23:57.356097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:00.841671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368002+0000","last_change":"2024-04-23T18:23:56.099450+0000","last_active":"2024-04-23T18:23:57.368002+0000","last_peered":"2024-04-23T18:23:57.368002+0000","last_clean":"2024-04-23T18:23:57.368002+0000","last_became_active":"2024-04-23T18:23:56.098723+0000","last_became_peered":"2024-04-23T18:23:56.098723+0000","last_unstale":"2024-04-23T18:23:57.368002+0000","last_undegraded":"2024-04-23T18:23:57.368002+0000","last_fullsized":"2024-04-23T18:23:57.368002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:53.027495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.353256+0000","last_change":"2024-04-23T18:23:55.262030+0000","last_active":"2024-04-23T18:23:57.353256+0000","last_peered":"2024-04-23T18:23:57.353256+0000","last_clean":"2024-04-23T18:23:57.353256+0000","last_became_active":"2024-04-23T18:23:55.261659+0000","last_became_peered":"2024-04-23T18:23:55.261659+0000","last_unstale":"2024-04-23T18:23:57.353256+0000","last_undegraded":"2024-04-23T18:23:57.353256+0000","last_fullsized":"2024-04-23T18:23:57.353256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:17.036709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.118783+0000","last_change":"2024-04-23T18:23:56.096511+0000","last_active":"2024-04-23T18:23:57.118783+0000","last_peered":"2024-04-23T18:23:57.118783+0000","last_clean":"2024-04-23T18:23:57.118783+0000","last_became_active":"2024-04-23T18:23:56.096154+0000","last_became_peered":"2024-04-23T18:23:56.096154+0000","last_unstale":"2024-04-23T18:23:57.118783+0000","last_undegraded":"2024-04-23T18:23:57.118783+0000","last_fullsized":"2024-04-23T18:23:57.118783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:03.926178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368166+0000","last_change":"2024-04-23T18:23:55.263618+0000","last_active":"2024-04-23T18:23:57.368166+0000","last_peered":"2024-04-23T18:23:57.368166+0000","last_clean":"2024-04-23T18:23:57.368166+0000","last_became_active":"2024-04-23T18:23:55.261758+0000","last_became_peered":"2024-04-23T18:23:55.261758+0000","last_unstale":"2024-04-23T18:23:57.368166+0000","last_undegraded":"2024-04-23T18:23:57.368166+0000","last_fullsized":"2024-04-23T18:23:57.368166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:23:27.745668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.118709+0000","last_change":"2024-04-23T18:23:56.109292+0000","last_active":"2024-04-23T18:23:57.118709+0000","last_peered":"2024-04-23T18:23:57.118709+0000","last_clean":"2024-04-23T18:23:57.118709+0000","last_became_active":"2024-04-23T18:23:56.108414+0000","last_became_peered":"2024-04-23T18:23:56.108414+0000","last_unstale":"2024-04-23T18:23:57.118709+0000","last_undegraded":"2024-04-23T18:23:57.118709+0000","last_fullsized":"2024-04-23T18:23:57.118709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:13.428782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086381+0000","last_change":"2024-04-23T18:23:56.097122+0000","last_active":"2024-04-23T18:23:57.086381+0000","last_peered":"2024-04-23T18:23:57.086381+0000","last_clean":"2024-04-23T18:23:57.086381+0000","last_became_active":"2024-04-23T18:23:56.096751+0000","last_became_peered":"2024-04-23T18:23:56.096751+0000","last_unstale":"2024-04-23T18:23:57.086381+0000","last_undegraded":"2024-04-23T18:23:57.086381+0000","last_fullsized":"2024-04-23T18:23:57.086381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:00:55.090370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.385003+0000","last_change":"2024-04-23T18:23:55.256458+0000","last_active":"2024-04-23T18:24:00.385003+0000","last_peered":"2024-04-23T18:24:00.385003+0000","last_clean":"2024-04-23T18:24:00.385003+0000","last_became_active":"2024-04-23T18:23:55.256127+0000","last_became_peered":"2024-04-23T18:23:55.256127+0000","last_unstale":"2024-04-23T18:24:00.385003+0000","last_undegraded":"2024-04-23T18:24:00.385003+0000","last_fullsized":"2024-04-23T18:24:00.385003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:03.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085840+0000","last_change":"2024-04-23T18:23:56.103743+0000","last_active":"2024-04-23T18:23:57.085840+0000","last_peered":"2024-04-23T18:23:57.085840+0000","last_clean":"2024-04-23T18:23:57.085840+0000","last_became_active":"2024-04-23T18:23:56.103041+0000","last_became_peered":"2024-04-23T18:23:56.103041+0000","last_unstale":"2024-04-23T18:23:57.085840+0000","last_undegraded":"2024-04-23T18:23:57.085840+0000","last_fullsized":"2024-04-23T18:23:57.085840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:22.419555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086678+0000","last_change":"2024-04-23T18:23:55.257008+0000","last_active":"2024-04-23T18:23:57.086678+0000","last_peered":"2024-04-23T18:23:57.086678+0000","last_clean":"2024-04-23T18:23:57.086678+0000","last_became_active":"2024-04-23T18:23:55.256648+0000","last_became_peered":"2024-04-23T18:23:55.256648+0000","last_unstale":"2024-04-23T18:23:57.086678+0000","last_undegraded":"2024-04-23T18:23:57.086678+0000","last_fullsized":"2024-04-23T18:23:57.086678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:00.867083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367814+0000","last_change":"2024-04-23T18:22:38.610948+0000","last_active":"2024-04-23T18:23:57.367814+0000","last_peered":"2024-04-23T18:23:57.367814+0000","last_clean":"2024-04-23T18:23:57.367814+0000","last_became_active":"2024-04-23T18:22:38.610475+0000","last_became_peered":"2024-04-23T18:22:38.610475+0000","last_unstale":"2024-04-23T18:23:57.367814+0000","last_undegraded":"2024-04-23T18:23:57.367814+0000","last_fullsized":"2024-04-23T18:23:57.367814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_clean_scrub_stamp":"2024-04-23T18:19:07.980519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:08.209087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367829+0000","last_change":"2024-04-23T18:23:56.108706+0000","last_active":"2024-04-23T18:23:57.367829+0000","last_peered":"2024-04-23T18:23:57.367829+0000","last_clean":"2024-04-23T18:23:57.367829+0000","last_became_active":"2024-04-23T18:23:56.108336+0000","last_became_peered":"2024-04-23T18:23:56.108336+0000","last_unstale":"2024-04-23T18:23:57.367829+0000","last_undegraded":"2024-04-23T18:23:57.367829+0000","last_fullsized":"2024-04-23T18:23:57.367829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:41.641623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.382342+0000","last_change":"2024-04-23T18:23:55.263713+0000","last_active":"2024-04-23T18:24:00.382342+0000","last_peered":"2024-04-23T18:24:00.382342+0000","last_clean":"2024-04-23T18:24:00.382342+0000","last_became_active":"2024-04-23T18:23:55.261703+0000","last_became_peered":"2024-04-23T18:23:55.261703+0000","last_unstale":"2024-04-23T18:24:00.382342+0000","last_undegraded":"2024-04-23T18:24:00.382342+0000","last_fullsized":"2024-04-23T18:24:00.382342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:31.676269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:01.393519+0000","last_change":"2024-04-23T18:23:55.265853+0000","last_active":"2024-04-23T18:24:01.393519+0000","last_peered":"2024-04-23T18:24:01.393519+0000","last_clean":"2024-04-23T18:24:01.393519+0000","last_became_active":"2024-04-23T18:23:55.265435+0000","last_became_peered":"2024-04-23T18:23:55.265435+0000","last_unstale":"2024-04-23T18:24:01.393519+0000","last_undegraded":"2024-04-23T18:24:01.393519+0000","last_fullsized":"2024-04-23T18:24:01.393519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.969720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368295+0000","last_change":"2024-04-23T18:23:56.085609+0000","last_active":"2024-04-23T18:23:57.368295+0000","last_peered":"2024-04-23T18:23:57.368295+0000","last_clean":"2024-04-23T18:23:57.368295+0000","last_became_active":"2024-04-23T18:23:56.085244+0000","last_became_peered":"2024-04-23T18:23:56.085244+0000","last_unstale":"2024-04-23T18:23:57.368295+0000","last_undegraded":"2024-04-23T18:23:57.368295+0000","last_fullsized":"2024-04-23T18:23:57.368295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.211717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355120+0000","last_change":"2024-04-23T18:23:55.260260+0000","last_active":"2024-04-23T18:23:57.355120+0000","last_peered":"2024-04-23T18:23:57.355120+0000","last_clean":"2024-04-23T18:23:57.355120+0000","last_became_active":"2024-04-23T18:23:55.259862+0000","last_became_peered":"2024-04-23T18:23:55.259862+0000","last_unstale":"2024-04-23T18:23:57.355120+0000","last_undegraded":"2024-04-23T18:23:57.355120+0000","last_fullsized":"2024-04-23T18:23:57.355120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:57.833293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085956+0000","last_change":"2024-04-23T18:23:56.098826+0000","last_active":"2024-04-23T18:23:57.085956+0000","last_peered":"2024-04-23T18:23:57.085956+0000","last_clean":"2024-04-23T18:23:57.085956+0000","last_became_active":"2024-04-23T18:23:56.098494+0000","last_became_peered":"2024-04-23T18:23:56.098494+0000","last_unstale":"2024-04-23T18:23:57.085956+0000","last_undegraded":"2024-04-23T18:23:57.085956+0000","last_fullsized":"2024-04-23T18:23:57.085956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:12:24.953330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086592+0000","last_change":"2024-04-23T18:23:55.258174+0000","last_active":"2024-04-23T18:23:57.086592+0000","last_peered":"2024-04-23T18:23:57.086592+0000","last_clean":"2024-04-23T18:23:57.086592+0000","last_became_active":"2024-04-23T18:23:55.257846+0000","last_became_peered":"2024-04-23T18:23:55.257846+0000","last_unstale":"2024-04-23T18:23:57.086592+0000","last_undegraded":"2024-04-23T18:23:57.086592+0000","last_fullsized":"2024-04-23T18:23:57.086592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:35.131566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367450+0000","last_change":"2024-04-23T18:23:56.110758+0000","last_active":"2024-04-23T18:23:57.367450+0000","last_peered":"2024-04-23T18:23:57.367450+0000","last_clean":"2024-04-23T18:23:57.367450+0000","last_became_active":"2024-04-23T18:23:56.110372+0000","last_became_peered":"2024-04-23T18:23:56.110372+0000","last_unstale":"2024-04-23T18:23:57.367450+0000","last_undegraded":"2024-04-23T18:23:57.367450+0000","last_fullsized":"2024-04-23T18:23:57.367450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:40.193088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:59.375841+0000","last_change":"2024-04-23T18:23:55.265714+0000","last_active":"2024-04-23T18:23:59.375841+0000","last_peered":"2024-04-23T18:23:59.375841+0000","last_clean":"2024-04-23T18:23:59.375841+0000","last_became_active":"2024-04-23T18:23:55.265198+0000","last_became_peered":"2024-04-23T18:23:55.265198+0000","last_unstale":"2024-04-23T18:23:59.375841+0000","last_undegraded":"2024-04-23T18:23:59.375841+0000","last_fullsized":"2024-04-23T18:23:59.375841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:10.770748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355537+0000","last_change":"2024-04-23T18:23:56.102347+0000","last_active":"2024-04-23T18:23:57.355537+0000","last_peered":"2024-04-23T18:23:57.355537+0000","last_clean":"2024-04-23T18:23:57.355537+0000","last_became_active":"2024-04-23T18:23:56.100687+0000","last_became_peered":"2024-04-23T18:23:56.100687+0000","last_unstale":"2024-04-23T18:23:57.355537+0000","last_undegraded":"2024-04-23T18:23:57.355537+0000","last_fullsized":"2024-04-23T18:23:57.355537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:20.378730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:01.394611+0000","last_change":"2024-04-23T18:23:55.255943+0000","last_active":"2024-04-23T18:24:01.394611+0000","last_peered":"2024-04-23T18:24:01.394611+0000","last_clean":"2024-04-23T18:24:01.394611+0000","last_became_active":"2024-04-23T18:23:55.255618+0000","last_became_peered":"2024-04-23T18:23:55.255618+0000","last_unstale":"2024-04-23T18:24:01.394611+0000","last_undegraded":"2024-04-23T18:24:01.394611+0000","last_fullsized":"2024-04-23T18:24:01.394611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:01.408533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367714+0000","last_change":"2024-04-23T18:23:56.091664+0000","last_active":"2024-04-23T18:23:57.367714+0000","last_peered":"2024-04-23T18:23:57.367714+0000","last_clean":"2024-04-23T18:23:57.367714+0000","last_became_active":"2024-04-23T18:23:56.091308+0000","last_became_peered":"2024-04-23T18:23:56.091308+0000","last_unstale":"2024-04-23T18:23:57.367714+0000","last_undegraded":"2024-04-23T18:23:57.367714+0000","last_fullsized":"2024-04-23T18:23:57.367714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:31.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.371497+0000","last_change":"2024-04-23T18:23:55.260992+0000","last_active":"2024-04-23T18:24:00.371497+0000","last_peered":"2024-04-23T18:24:00.371497+0000","last_clean":"2024-04-23T18:24:00.371497+0000","last_became_active":"2024-04-23T18:23:55.260679+0000","last_became_peered":"2024-04-23T18:23:55.260679+0000","last_unstale":"2024-04-23T18:24:00.371497+0000","last_undegraded":"2024-04-23T18:24:00.371497+0000","last_fullsized":"2024-04-23T18:24:00.371497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:02:50.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085855+0000","last_change":"2024-04-23T18:23:56.097956+0000","last_active":"2024-04-23T18:23:57.085855+0000","last_peered":"2024-04-23T18:23:57.085855+0000","last_clean":"2024-04-23T18:23:57.085855+0000","last_became_active":"2024-04-23T18:23:56.097231+0000","last_became_peered":"2024-04-23T18:23:56.097231+0000","last_unstale":"2024-04-23T18:23:57.085855+0000","last_undegraded":"2024-04-23T18:23:57.085855+0000","last_fullsized":"2024-04-23T18:23:57.085855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:10.945857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:59.375748+0000","last_change":"2024-04-23T18:23:55.262860+0000","last_active":"2024-04-23T18:23:59.375748+0000","last_peered":"2024-04-23T18:23:59.375748+0000","last_clean":"2024-04-23T18:23:59.375748+0000","last_became_active":"2024-04-23T18:23:55.262492+0000","last_became_peered":"2024-04-23T18:23:55.262492+0000","last_unstale":"2024-04-23T18:23:59.375748+0000","last_undegraded":"2024-04-23T18:23:59.375748+0000","last_fullsized":"2024-04-23T18:23:59.375748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:43.836017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368298+0000","last_change":"2024-04-23T18:23:56.096889+0000","last_active":"2024-04-23T18:23:57.368298+0000","last_peered":"2024-04-23T18:23:57.368298+0000","last_clean":"2024-04-23T18:23:57.368298+0000","last_became_active":"2024-04-23T18:23:56.096520+0000","last_became_peered":"2024-04-23T18:23:56.096520+0000","last_unstale":"2024-04-23T18:23:57.368298+0000","last_undegraded":"2024-04-23T18:23:57.368298+0000","last_fullsized":"2024-04-23T18:23:57.368298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:34:53.785481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:01.394602+0000","last_change":"2024-04-23T18:23:55.258710+0000","last_active":"2024-04-23T18:24:01.394602+0000","last_peered":"2024-04-23T18:24:01.394602+0000","last_clean":"2024-04-23T18:24:01.394602+0000","last_became_active":"2024-04-23T18:23:55.256247+0000","last_became_peered":"2024-04-23T18:23:55.256247+0000","last_unstale":"2024-04-23T18:24:01.394602+0000","last_undegraded":"2024-04-23T18:24:01.394602+0000","last_fullsized":"2024-04-23T18:24:01.394602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:20.551614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368796+0000","last_change":"2024-04-23T18:23:56.101862+0000","last_active":"2024-04-23T18:23:57.368796+0000","last_peered":"2024-04-23T18:23:57.368796+0000","last_clean":"2024-04-23T18:23:57.368796+0000","last_became_active":"2024-04-23T18:23:56.101458+0000","last_became_peered":"2024-04-23T18:23:56.101458+0000","last_unstale":"2024-04-23T18:23:57.368796+0000","last_undegraded":"2024-04-23T18:23:57.368796+0000","last_fullsized":"2024-04-23T18:23:57.368796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:59.292077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086547+0000","last_change":"2024-04-23T18:23:55.258292+0000","last_active":"2024-04-23T18:23:57.086547+0000","last_peered":"2024-04-23T18:23:57.086547+0000","last_clean":"2024-04-23T18:23:57.086547+0000","last_became_active":"2024-04-23T18:23:55.257954+0000","last_became_peered":"2024-04-23T18:23:55.257954+0000","last_unstale":"2024-04-23T18:23:57.086547+0000","last_undegraded":"2024-04-23T18:23:57.086547+0000","last_fullsized":"2024-04-23T18:23:57.086547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:39.105967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368267+0000","last_change":"2024-04-23T18:23:56.105977+0000","last_active":"2024-04-23T18:23:57.368267+0000","last_peered":"2024-04-23T18:23:57.368267+0000","last_clean":"2024-04-23T18:23:57.368267+0000","last_became_active":"2024-04-23T18:23:56.105582+0000","last_became_peered":"2024-04-23T18:23:56.105582+0000","last_unstale":"2024-04-23T18:23:57.368267+0000","last_undegraded":"2024-04-23T18:23:57.368267+0000","last_fullsized":"2024-04-23T18:23:57.368267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:28:10.700684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368121+0000","last_change":"2024-04-23T18:23:55.252095+0000","last_active":"2024-04-23T18:23:57.368121+0000","last_peered":"2024-04-23T18:23:57.368121+0000","last_clean":"2024-04-23T18:23:57.368121+0000","last_became_active":"2024-04-23T18:23:55.251702+0000","last_became_peered":"2024-04-23T18:23:55.251702+0000","last_unstale":"2024-04-23T18:23:57.368121+0000","last_undegraded":"2024-04-23T18:23:57.368121+0000","last_fullsized":"2024-04-23T18:23:57.368121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:21.625905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368390+0000","last_change":"2024-04-23T18:23:56.101997+0000","last_active":"2024-04-23T18:23:57.368390+0000","last_peered":"2024-04-23T18:23:57.368390+0000","last_clean":"2024-04-23T18:23:57.368390+0000","last_became_active":"2024-04-23T18:23:56.101629+0000","last_became_peered":"2024-04-23T18:23:56.101629+0000","last_unstale":"2024-04-23T18:23:57.368390+0000","last_undegraded":"2024-04-23T18:23:57.368390+0000","last_fullsized":"2024-04-23T18:23:57.368390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:42.484413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367778+0000","last_change":"2024-04-23T18:23:56.090543+0000","last_active":"2024-04-23T18:23:57.367778+0000","last_peered":"2024-04-23T18:23:57.367778+0000","last_clean":"2024-04-23T18:23:57.367778+0000","last_became_active":"2024-04-23T18:23:56.090191+0000","last_became_peered":"2024-04-23T18:23:56.090191+0000","last_unstale":"2024-04-23T18:23:57.367778+0000","last_undegraded":"2024-04-23T18:23:57.367778+0000","last_fullsized":"2024-04-23T18:23:57.367778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:01.401822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367986+0000","last_change":"2024-04-23T18:23:56.105537+0000","last_active":"2024-04-23T18:23:57.367986+0000","last_peered":"2024-04-23T18:23:57.367986+0000","last_clean":"2024-04-23T18:23:57.367986+0000","last_became_active":"2024-04-23T18:23:56.105130+0000","last_became_peered":"2024-04-23T18:23:56.105130+0000","last_unstale":"2024-04-23T18:23:57.367986+0000","last_undegraded":"2024-04-23T18:23:57.367986+0000","last_fullsized":"2024-04-23T18:23:57.367986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:59.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367800+0000","last_change":"2024-04-23T18:23:56.097765+0000","last_active":"2024-04-23T18:23:57.367800+0000","last_peered":"2024-04-23T18:23:57.367800+0000","last_clean":"2024-04-23T18:23:57.367800+0000","last_became_active":"2024-04-23T18:23:56.097439+0000","last_became_peered":"2024-04-23T18:23:56.097439+0000","last_unstale":"2024-04-23T18:23:57.367800+0000","last_undegraded":"2024-04-23T18:23:57.367800+0000","last_fullsized":"2024-04-23T18:23:57.367800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:31.529751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367919+0000","last_change":"2024-04-23T18:23:56.108548+0000","last_active":"2024-04-23T18:23:57.367919+0000","last_peered":"2024-04-23T18:23:57.367919+0000","last_clean":"2024-04-23T18:23:57.367919+0000","last_became_active":"2024-04-23T18:23:56.108214+0000","last_became_peered":"2024-04-23T18:23:56.108214+0000","last_unstale":"2024-04-23T18:23:57.367919+0000","last_undegraded":"2024-04-23T18:23:57.367919+0000","last_fullsized":"2024-04-23T18:23:57.367919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:41.738754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368206+0000","last_change":"2024-04-23T18:23:56.108551+0000","last_active":"2024-04-23T18:23:57.368206+0000","last_peered":"2024-04-23T18:23:57.368206+0000","last_clean":"2024-04-23T18:23:57.368206+0000","last_became_active":"2024-04-23T18:23:56.108227+0000","last_became_peered":"2024-04-23T18:23:56.108227+0000","last_unstale":"2024-04-23T18:23:57.368206+0000","last_undegraded":"2024-04-23T18:23:57.368206+0000","last_fullsized":"2024-04-23T18:23:57.368206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:19.226718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368213+0000","last_change":"2024-04-23T18:23:56.093969+0000","last_active":"2024-04-23T18:23:57.368213+0000","last_peered":"2024-04-23T18:23:57.368213+0000","last_clean":"2024-04-23T18:23:57.368213+0000","last_became_active":"2024-04-23T18:23:56.093568+0000","last_became_peered":"2024-04-23T18:23:56.093568+0000","last_unstale":"2024-04-23T18:23:57.368213+0000","last_undegraded":"2024-04-23T18:23:57.368213+0000","last_fullsized":"2024-04-23T18:23:57.368213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:08.361305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368654+0000","last_change":"2024-04-23T18:23:56.096106+0000","last_active":"2024-04-23T18:23:57.368654+0000","last_peered":"2024-04-23T18:23:57.368654+0000","last_clean":"2024-04-23T18:23:57.368654+0000","last_became_active":"2024-04-23T18:23:56.095745+0000","last_became_peered":"2024-04-23T18:23:56.095745+0000","last_unstale":"2024-04-23T18:23:57.368654+0000","last_undegraded":"2024-04-23T18:23:57.368654+0000","last_fullsized":"2024-04-23T18:23:57.368654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:01.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085867+0000","last_change":"2024-04-23T18:23:56.095474+0000","last_active":"2024-04-23T18:23:57.085867+0000","last_peered":"2024-04-23T18:23:57.085867+0000","last_clean":"2024-04-23T18:23:57.085867+0000","last_became_active":"2024-04-23T18:23:56.095116+0000","last_became_peered":"2024-04-23T18:23:56.095116+0000","last_unstale":"2024-04-23T18:23:57.085867+0000","last_undegraded":"2024-04-23T18:23:57.085867+0000","last_fullsized":"2024-04-23T18:23:57.085867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:50:00.339558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086661+0000","last_change":"2024-04-23T18:23:56.096022+0000","last_active":"2024-04-23T18:23:57.086661+0000","last_peered":"2024-04-23T18:23:57.086661+0000","last_clean":"2024-04-23T18:23:57.086661+0000","last_became_active":"2024-04-23T18:23:56.095493+0000","last_became_peered":"2024-04-23T18:23:56.095493+0000","last_unstale":"2024-04-23T18:23:57.086661+0000","last_undegraded":"2024-04-23T18:23:57.086661+0000","last_fullsized":"2024-04-23T18:23:57.086661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:42.209072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085942+0000","last_change":"2024-04-23T18:23:56.101697+0000","last_active":"2024-04-23T18:23:57.085942+0000","last_peered":"2024-04-23T18:23:57.085942+0000","last_clean":"2024-04-23T18:23:57.085942+0000","last_became_active":"2024-04-23T18:23:56.098708+0000","last_became_peered":"2024-04-23T18:23:56.098708+0000","last_unstale":"2024-04-23T18:23:57.085942+0000","last_undegraded":"2024-04-23T18:23:57.085942+0000","last_fullsized":"2024-04-23T18:23:57.085942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:14.170700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368786+0000","last_change":"2024-04-23T18:23:56.096982+0000","last_active":"2024-04-23T18:23:57.368786+0000","last_peered":"2024-04-23T18:23:57.368786+0000","last_clean":"2024-04-23T18:23:57.368786+0000","last_became_active":"2024-04-23T18:23:56.096608+0000","last_became_peered":"2024-04-23T18:23:56.096608+0000","last_unstale":"2024-04-23T18:23:57.368786+0000","last_undegraded":"2024-04-23T18:23:57.368786+0000","last_fullsized":"2024-04-23T18:23:57.368786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:50.329337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368268+0000","last_change":"2024-04-23T18:23:56.100345+0000","last_active":"2024-04-23T18:23:57.368268+0000","last_peered":"2024-04-23T18:23:57.368268+0000","last_clean":"2024-04-23T18:23:57.368268+0000","last_became_active":"2024-04-23T18:23:56.100043+0000","last_became_peered":"2024-04-23T18:23:56.100043+0000","last_unstale":"2024-04-23T18:23:57.368268+0000","last_undegraded":"2024-04-23T18:23:57.368268+0000","last_fullsized":"2024-04-23T18:23:57.368268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:36.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367936+0000","last_change":"2024-04-23T18:23:56.101947+0000","last_active":"2024-04-23T18:23:57.367936+0000","last_peered":"2024-04-23T18:23:57.367936+0000","last_clean":"2024-04-23T18:23:57.367936+0000","last_became_active":"2024-04-23T18:23:56.101627+0000","last_became_peered":"2024-04-23T18:23:56.101627+0000","last_unstale":"2024-04-23T18:23:57.367936+0000","last_undegraded":"2024-04-23T18:23:57.367936+0000","last_fullsized":"2024-04-23T18:23:57.367936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:21:43.703653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279760,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4018,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":23448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":444207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":999424,"data_stored":1035189,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":929792,"data_stored":443597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":443753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1036629,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":443015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":445600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":929792,"data_stored":443105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":446599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":443423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":443037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":1033565,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":9964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:24:04.860 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:24:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.003+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2069017239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd684102a10 msgr2=0x7fd684104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.003+0000 7fd68ae6b640 1 --2- 172.21.15.19:0/2069017239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd684102a10 0x7fd684104e00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009ca0 tx=0x7fd67c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.003+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2069017239 shutdown_connections 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.003+0000 7fd68ae6b640 1 --2- 172.21.15.19:0/2069017239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd684107c70 0x7fd68410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.003+0000 7fd68ae6b640 1 --2- 172.21.15.19:0/2069017239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd684105340 0x7fd684107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.003+0000 7fd68ae6b640 1 --2- 172.21.15.19:0/2069017239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd684102a10 0x7fd684104e00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.003+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2069017239 >> 172.21.15.19:0/2069017239 conn(0x7fd6840fc400 msgr2=0x7fd6840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.003+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2069017239 shutdown_connections 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.003+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2069017239 wait complete. 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd68ae6b640 1 Processor -- start 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd68ae6b640 1 -- start start 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd68ae6b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd684102a10 0x7fd684102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd68ae6b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd684105340 0x7fd6841009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd68ae6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd684107c70 0x7fd684100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd68ae6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410dac0 con 0x7fd684107c70 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd68ae6b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410d940 con 0x7fd684105340 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd68ae6b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410dc40 con 0x7fd684102a10 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd689408640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd684107c70 0x7fd684100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd689408640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd684107c70 0x7fd684100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60642/0 (socket says 172.21.15.19:60642) 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd689408640 1 -- 172.21.15.19:0/2030703161 learned_addr learned my addr 172.21.15.19:0/2030703161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd688c07640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd684102a10 0x7fd684102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.004+0000 7fd683fff640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd684105340 0x7fd6841009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd689408640 1 -- 172.21.15.19:0/2030703161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd684102a10 msgr2=0x7fd684102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd689408640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd684102a10 0x7fd684102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd689408640 1 -- 172.21.15.19:0/2030703161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd684105340 msgr2=0x7fd6841009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd689408640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd684105340 0x7fd6841009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd689408640 1 -- 172.21.15.19:0/2030703161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd684101790 con 0x7fd684107c70 2024-04-23T18:24:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd688c07640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd684102a10 0x7fd684102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd683fff640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd684105340 0x7fd6841009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd689408640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd684107c70 0x7fd684100f00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd67400d950 tx=0x7fd67400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd681ffb640 1 -- 172.21.15.19:0/2030703161 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674017030 con 0x7fd684107c70 2024-04-23T18:24:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd681ffb640 1 -- 172.21.15.19:0/2030703161 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd674004020 con 0x7fd684107c70 2024-04-23T18:24:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2030703161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6841a22e0 con 0x7fd684107c70 2024-04-23T18:24:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd681ffb640 1 -- 172.21.15.19:0/2030703161 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674003b50 con 0x7fd684107c70 2024-04-23T18:24:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.005+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2030703161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6841a2830 con 0x7fd684107c70 2024-04-23T18:24:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.006+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2030703161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6840fdef0 con 0x7fd684107c70 2024-04-23T18:24:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.007+0000 7fd681ffb640 1 -- 172.21.15.19:0/2030703161 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd67401a070 con 0x7fd684107c70 2024-04-23T18:24:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.008+0000 7fd681ffb640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd6700a3cc0 0x7fd6700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.008+0000 7fd688c07640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd6700a3cc0 0x7fd6700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.008+0000 7fd681ffb640 1 -- 172.21.15.19:0/2030703161 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd6740cfc10 con 0x7fd684107c70 2024-04-23T18:24:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.008+0000 7fd688c07640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd6700a3cc0 0x7fd6700a6180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd67c031b50 tx=0x7fd67c005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:05.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.012+0000 7fd681ffb640 1 -- 172.21.15.19:0/2030703161 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd674021070 con 0x7fd684107c70 2024-04-23T18:24:05.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:04 smithi044 ceph-mon[32458]: pgmap v300: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 13 op/s 2024-04-23T18:24:05.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:04 smithi044 ceph-mon[32458]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:05.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:04 smithi044 ceph-mon[32458]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:05.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:04 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1268870657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:05.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:04 smithi044 ceph-mon[32458]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:24:05.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:04 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3963238972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:05.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:04 smithi155 ceph-mon[33468]: pgmap v300: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 13 op/s 2024-04-23T18:24:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:04 smithi155 ceph-mon[33468]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:04 smithi155 ceph-mon[33468]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:04 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1268870657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:04 smithi155 ceph-mon[33468]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:24:05.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:04 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3963238972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:05.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.197+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2030703161 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd684062390 con 0x7fd6700a3cc0 2024-04-23T18:24:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.204+0000 7fd681ffb640 1 -- 172.21.15.19:0/2030703161 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258889 (secure 0 0 0) 0x7fd684062390 con 0x7fd6700a3cc0 2024-04-23T18:24:05.200 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:24:05.201 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-23T18:24:05.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.207+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2030703161 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd6700a3cc0 msgr2=0x7fd6700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:05.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.207+0000 7fd68ae6b640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd6700a3cc0 0x7fd6700a6180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd67c031b50 tx=0x7fd67c005ef0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.207+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2030703161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd684107c70 msgr2=0x7fd684100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:05.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.207+0000 7fd68ae6b640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd684107c70 0x7fd684100f00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd67400d950 tx=0x7fd67400de10 comp rx=0 tx=0).stop 2024-04-23T18:24:05.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.207+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2030703161 shutdown_connections 2024-04-23T18:24:05.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.207+0000 7fd68ae6b640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd6700a3cc0 0x7fd6700a6180 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.207+0000 7fd68ae6b640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd684105340 0x7fd6841009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.207+0000 7fd68ae6b640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd684107c70 0x7fd684100f00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.207+0000 7fd68ae6b640 1 --2- 172.21.15.19:0/2030703161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd684102a10 0x7fd684102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.207+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2030703161 >> 172.21.15.19:0/2030703161 conn(0x7fd6840fc400 msgr2=0x7fd684103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:05.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.208+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2030703161 shutdown_connections 2024-04-23T18:24:05.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:05.208+0000 7fd68ae6b640 1 -- 172.21.15.19:0/2030703161 wait complete. 2024-04-23T18:24:05.211 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-04-23T18:24:03.353944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594386,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":291,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":267,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381164,"kb_used_data":11180,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560660,"statfs":{"total":1151940427776,"available":1151550115840,"internally_reserved":0,"allocated":11448320,"data_stored":7101719,"data_compressed":106755,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4018,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002289"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368375+0000","last_change":"2024-04-23T18:23:56.098292+0000","last_active":"2024-04-23T18:23:57.368375+0000","last_peered":"2024-04-23T18:23:57.368375+0000","last_clean":"2024-04-23T18:23:57.368375+0000","last_became_active":"2024-04-23T18:23:56.096623+0000","last_became_peered":"2024-04-23T18:23:56.096623+0000","last_unstale":"2024-04-23T18:23:57.368375+0000","last_undegraded":"2024-04-23T18:23:57.368375+0000","last_fullsized":"2024-04-23T18:23:57.368375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:06.186581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367418+0000","last_change":"2024-04-23T18:23:56.096807+0000","last_active":"2024-04-23T18:23:57.367418+0000","last_peered":"2024-04-23T18:23:57.367418+0000","last_clean":"2024-04-23T18:23:57.367418+0000","last_became_active":"2024-04-23T18:23:56.096327+0000","last_became_peered":"2024-04-23T18:23:56.096327+0000","last_unstale":"2024-04-23T18:23:57.367418+0000","last_undegraded":"2024-04-23T18:23:57.367418+0000","last_fullsized":"2024-04-23T18:23:57.367418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:56:16.391821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367300+0000","last_change":"2024-04-23T18:23:56.101615+0000","last_active":"2024-04-23T18:23:57.367300+0000","last_peered":"2024-04-23T18:23:57.367300+0000","last_clean":"2024-04-23T18:23:57.367300+0000","last_became_active":"2024-04-23T18:23:56.100929+0000","last_became_peered":"2024-04-23T18:23:56.100929+0000","last_unstale":"2024-04-23T18:23:57.367300+0000","last_undegraded":"2024-04-23T18:23:57.367300+0000","last_fullsized":"2024-04-23T18:23:57.367300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:25.547029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086535+0000","last_change":"2024-04-23T18:23:56.103627+0000","last_active":"2024-04-23T18:23:57.086535+0000","last_peered":"2024-04-23T18:23:57.086535+0000","last_clean":"2024-04-23T18:23:57.086535+0000","last_became_active":"2024-04-23T18:23:56.102752+0000","last_became_peered":"2024-04-23T18:23:56.102752+0000","last_unstale":"2024-04-23T18:23:57.086535+0000","last_undegraded":"2024-04-23T18:23:57.086535+0000","last_fullsized":"2024-04-23T18:23:57.086535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:28.460342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355590+0000","last_change":"2024-04-23T18:23:56.094889+0000","last_active":"2024-04-23T18:23:57.355590+0000","last_peered":"2024-04-23T18:23:57.355590+0000","last_clean":"2024-04-23T18:23:57.355590+0000","last_became_active":"2024-04-23T18:23:56.094262+0000","last_became_peered":"2024-04-23T18:23:56.094262+0000","last_unstale":"2024-04-23T18:23:57.355590+0000","last_undegraded":"2024-04-23T18:23:57.355590+0000","last_fullsized":"2024-04-23T18:23:57.355590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:22.332114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.118532+0000","last_change":"2024-04-23T18:23:56.108893+0000","last_active":"2024-04-23T18:23:57.118532+0000","last_peered":"2024-04-23T18:23:57.118532+0000","last_clean":"2024-04-23T18:23:57.118532+0000","last_became_active":"2024-04-23T18:23:56.108229+0000","last_became_peered":"2024-04-23T18:23:56.108229+0000","last_unstale":"2024-04-23T18:23:57.118532+0000","last_undegraded":"2024-04-23T18:23:57.118532+0000","last_fullsized":"2024-04-23T18:23:57.118532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:50.930321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368511+0000","last_change":"2024-04-23T18:23:56.098674+0000","last_active":"2024-04-23T18:23:57.368511+0000","last_peered":"2024-04-23T18:23:57.368511+0000","last_clean":"2024-04-23T18:23:57.368511+0000","last_became_active":"2024-04-23T18:23:56.098299+0000","last_became_peered":"2024-04-23T18:23:56.098299+0000","last_unstale":"2024-04-23T18:23:57.368511+0000","last_undegraded":"2024-04-23T18:23:57.368511+0000","last_fullsized":"2024-04-23T18:23:57.368511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:05.711410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085754+0000","last_change":"2024-04-23T18:23:56.097437+0000","last_active":"2024-04-23T18:23:57.085754+0000","last_peered":"2024-04-23T18:23:57.085754+0000","last_clean":"2024-04-23T18:23:57.085754+0000","last_became_active":"2024-04-23T18:23:56.095702+0000","last_became_peered":"2024-04-23T18:23:56.095702+0000","last_unstale":"2024-04-23T18:23:57.085754+0000","last_undegraded":"2024-04-23T18:23:57.085754+0000","last_fullsized":"2024-04-23T18:23:57.085754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:35.754832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.369096+0000","last_change":"2024-04-23T18:23:56.096060+0000","last_active":"2024-04-23T18:23:57.369096+0000","last_peered":"2024-04-23T18:23:57.369096+0000","last_clean":"2024-04-23T18:23:57.369096+0000","last_became_active":"2024-04-23T18:23:56.095597+0000","last_became_peered":"2024-04-23T18:23:56.095597+0000","last_unstale":"2024-04-23T18:23:57.369096+0000","last_undegraded":"2024-04-23T18:23:57.369096+0000","last_fullsized":"2024-04-23T18:23:57.369096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:59.735347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.118863+0000","last_change":"2024-04-23T18:23:56.109033+0000","last_active":"2024-04-23T18:23:57.118863+0000","last_peered":"2024-04-23T18:23:57.118863+0000","last_clean":"2024-04-23T18:23:57.118863+0000","last_became_active":"2024-04-23T18:23:56.108472+0000","last_became_peered":"2024-04-23T18:23:56.108472+0000","last_unstale":"2024-04-23T18:23:57.118863+0000","last_undegraded":"2024-04-23T18:23:57.118863+0000","last_fullsized":"2024-04-23T18:23:57.118863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:32.491181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368729+0000","last_change":"2024-04-23T18:23:56.097771+0000","last_active":"2024-04-23T18:23:57.368729+0000","last_peered":"2024-04-23T18:23:57.368729+0000","last_clean":"2024-04-23T18:23:57.368729+0000","last_became_active":"2024-04-23T18:23:56.097403+0000","last_became_peered":"2024-04-23T18:23:56.097403+0000","last_unstale":"2024-04-23T18:23:57.368729+0000","last_undegraded":"2024-04-23T18:23:57.368729+0000","last_fullsized":"2024-04-23T18:23:57.368729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:48:36.261543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086554+0000","last_change":"2024-04-23T18:23:56.108550+0000","last_active":"2024-04-23T18:23:57.086554+0000","last_peered":"2024-04-23T18:23:57.086554+0000","last_clean":"2024-04-23T18:23:57.086554+0000","last_became_active":"2024-04-23T18:23:56.108191+0000","last_became_peered":"2024-04-23T18:23:56.108191+0000","last_unstale":"2024-04-23T18:23:57.086554+0000","last_undegraded":"2024-04-23T18:23:57.086554+0000","last_fullsized":"2024-04-23T18:23:57.086554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:25.142466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368262+0000","last_change":"2024-04-23T18:23:56.092573+0000","last_active":"2024-04-23T18:23:57.368262+0000","last_peered":"2024-04-23T18:23:57.368262+0000","last_clean":"2024-04-23T18:23:57.368262+0000","last_became_active":"2024-04-23T18:23:56.092095+0000","last_became_peered":"2024-04-23T18:23:56.092095+0000","last_unstale":"2024-04-23T18:23:57.368262+0000","last_undegraded":"2024-04-23T18:23:57.368262+0000","last_fullsized":"2024-04-23T18:23:57.368262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:01.601206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367207+0000","last_change":"2024-04-23T18:23:56.102073+0000","last_active":"2024-04-23T18:23:57.367207+0000","last_peered":"2024-04-23T18:23:57.367207+0000","last_clean":"2024-04-23T18:23:57.367207+0000","last_became_active":"2024-04-23T18:23:56.101059+0000","last_became_peered":"2024-04-23T18:23:56.101059+0000","last_unstale":"2024-04-23T18:23:57.367207+0000","last_undegraded":"2024-04-23T18:23:57.367207+0000","last_fullsized":"2024-04-23T18:23:57.367207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:37.904099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367122+0000","last_change":"2024-04-23T18:23:56.108482+0000","last_active":"2024-04-23T18:23:57.367122+0000","last_peered":"2024-04-23T18:23:57.367122+0000","last_clean":"2024-04-23T18:23:57.367122+0000","last_became_active":"2024-04-23T18:23:56.108128+0000","last_became_peered":"2024-04-23T18:23:56.108128+0000","last_unstale":"2024-04-23T18:23:57.367122+0000","last_undegraded":"2024-04-23T18:23:57.367122+0000","last_fullsized":"2024-04-23T18:23:57.367122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:51.925881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367953+0000","last_change":"2024-04-23T18:23:56.099691+0000","last_active":"2024-04-23T18:23:57.367953+0000","last_peered":"2024-04-23T18:23:57.367953+0000","last_clean":"2024-04-23T18:23:57.367953+0000","last_became_active":"2024-04-23T18:23:56.099272+0000","last_became_peered":"2024-04-23T18:23:56.099272+0000","last_unstale":"2024-04-23T18:23:57.367953+0000","last_undegraded":"2024-04-23T18:23:57.367953+0000","last_fullsized":"2024-04-23T18:23:57.367953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:41.834104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368476+0000","last_change":"2024-04-23T18:23:56.097965+0000","last_active":"2024-04-23T18:23:57.368476+0000","last_peered":"2024-04-23T18:23:57.368476+0000","last_clean":"2024-04-23T18:23:57.368476+0000","last_became_active":"2024-04-23T18:23:56.096766+0000","last_became_peered":"2024-04-23T18:23:56.096766+0000","last_unstale":"2024-04-23T18:23:57.368476+0000","last_undegraded":"2024-04-23T18:23:57.368476+0000","last_fullsized":"2024-04-23T18:23:57.368476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:40:00.076939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.353400+0000","last_change":"2024-04-23T18:23:56.095158+0000","last_active":"2024-04-23T18:23:57.353400+0000","last_peered":"2024-04-23T18:23:57.353400+0000","last_clean":"2024-04-23T18:23:57.353400+0000","last_became_active":"2024-04-23T18:23:56.094812+0000","last_became_peered":"2024-04-23T18:23:56.094812+0000","last_unstale":"2024-04-23T18:23:57.353400+0000","last_undegraded":"2024-04-23T18:23:57.353400+0000","last_fullsized":"2024-04-23T18:23:57.353400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:40.874698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367851+0000","last_change":"2024-04-23T18:23:56.099358+0000","last_active":"2024-04-23T18:23:57.367851+0000","last_peered":"2024-04-23T18:23:57.367851+0000","last_clean":"2024-04-23T18:23:57.367851+0000","last_became_active":"2024-04-23T18:23:56.099031+0000","last_became_peered":"2024-04-23T18:23:56.099031+0000","last_unstale":"2024-04-23T18:23:57.367851+0000","last_undegraded":"2024-04-23T18:23:57.367851+0000","last_fullsized":"2024-04-23T18:23:57.367851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:29.634932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.353239+0000","last_change":"2024-04-23T18:23:55.262939+0000","last_active":"2024-04-23T18:23:57.353239+0000","last_peered":"2024-04-23T18:23:57.353239+0000","last_clean":"2024-04-23T18:23:57.353239+0000","last_became_active":"2024-04-23T18:23:55.262609+0000","last_became_peered":"2024-04-23T18:23:55.262609+0000","last_unstale":"2024-04-23T18:23:57.353239+0000","last_undegraded":"2024-04-23T18:23:57.353239+0000","last_fullsized":"2024-04-23T18:23:57.353239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:12.418349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085869+0000","last_change":"2024-04-23T18:23:56.092182+0000","last_active":"2024-04-23T18:23:57.085869+0000","last_peered":"2024-04-23T18:23:57.085869+0000","last_clean":"2024-04-23T18:23:57.085869+0000","last_became_active":"2024-04-23T18:23:56.091444+0000","last_became_peered":"2024-04-23T18:23:56.091444+0000","last_unstale":"2024-04-23T18:23:57.085869+0000","last_undegraded":"2024-04-23T18:23:57.085869+0000","last_fullsized":"2024-04-23T18:23:57.085869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:00.775498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.392295+0000","last_change":"2024-04-23T18:23:55.259880+0000","last_active":"2024-04-23T18:23:57.392295+0000","last_peered":"2024-04-23T18:23:57.392295+0000","last_clean":"2024-04-23T18:23:57.392295+0000","last_became_active":"2024-04-23T18:23:55.259576+0000","last_became_peered":"2024-04-23T18:23:55.259576+0000","last_unstale":"2024-04-23T18:23:57.392295+0000","last_undegraded":"2024-04-23T18:23:57.392295+0000","last_fullsized":"2024-04-23T18:23:57.392295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:13.783776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367518+0000","last_change":"2024-04-23T18:23:56.094307+0000","last_active":"2024-04-23T18:23:57.367518+0000","last_peered":"2024-04-23T18:23:57.367518+0000","last_clean":"2024-04-23T18:23:57.367518+0000","last_became_active":"2024-04-23T18:23:56.093893+0000","last_became_peered":"2024-04-23T18:23:56.093893+0000","last_unstale":"2024-04-23T18:23:57.367518+0000","last_undegraded":"2024-04-23T18:23:57.367518+0000","last_fullsized":"2024-04-23T18:23:57.367518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:12:35.385497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:01.394701+0000","last_change":"2024-04-23T18:23:55.263137+0000","last_active":"2024-04-23T18:24:01.394701+0000","last_peered":"2024-04-23T18:24:01.394701+0000","last_clean":"2024-04-23T18:24:01.394701+0000","last_became_active":"2024-04-23T18:23:55.262553+0000","last_became_peered":"2024-04-23T18:23:55.262553+0000","last_unstale":"2024-04-23T18:24:01.394701+0000","last_undegraded":"2024-04-23T18:24:01.394701+0000","last_fullsized":"2024-04-23T18:24:01.394701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:33:42.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368304+0000","last_change":"2024-04-23T18:23:56.091098+0000","last_active":"2024-04-23T18:23:57.368304+0000","last_peered":"2024-04-23T18:23:57.368304+0000","last_clean":"2024-04-23T18:23:57.368304+0000","last_became_active":"2024-04-23T18:23:56.090635+0000","last_became_peered":"2024-04-23T18:23:56.090635+0000","last_unstale":"2024-04-23T18:23:57.368304+0000","last_undegraded":"2024-04-23T18:23:57.368304+0000","last_fullsized":"2024-04-23T18:23:57.368304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:17.787937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355858+0000","last_change":"2024-04-23T18:23:55.262146+0000","last_active":"2024-04-23T18:23:57.355858+0000","last_peered":"2024-04-23T18:23:57.355858+0000","last_clean":"2024-04-23T18:23:57.355858+0000","last_became_active":"2024-04-23T18:23:55.261838+0000","last_became_peered":"2024-04-23T18:23:55.261838+0000","last_unstale":"2024-04-23T18:23:57.355858+0000","last_undegraded":"2024-04-23T18:23:57.355858+0000","last_fullsized":"2024-04-23T18:23:57.355858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:42:39.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086635+0000","last_change":"2024-04-23T18:23:56.100550+0000","last_active":"2024-04-23T18:23:57.086635+0000","last_peered":"2024-04-23T18:23:57.086635+0000","last_clean":"2024-04-23T18:23:57.086635+0000","last_became_active":"2024-04-23T18:23:56.099709+0000","last_became_peered":"2024-04-23T18:23:56.099709+0000","last_unstale":"2024-04-23T18:23:57.086635+0000","last_undegraded":"2024-04-23T18:23:57.086635+0000","last_fullsized":"2024-04-23T18:23:57.086635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:23.797920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086301+0000","last_change":"2024-04-23T18:23:55.261399+0000","last_active":"2024-04-23T18:23:57.086301+0000","last_peered":"2024-04-23T18:23:57.086301+0000","last_clean":"2024-04-23T18:23:57.086301+0000","last_became_active":"2024-04-23T18:23:55.261012+0000","last_became_peered":"2024-04-23T18:23:55.261012+0000","last_unstale":"2024-04-23T18:23:57.086301+0000","last_undegraded":"2024-04-23T18:23:57.086301+0000","last_fullsized":"2024-04-23T18:23:57.086301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:31.377776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355568+0000","last_change":"2024-04-23T18:23:56.096785+0000","last_active":"2024-04-23T18:23:57.355568+0000","last_peered":"2024-04-23T18:23:57.355568+0000","last_clean":"2024-04-23T18:23:57.355568+0000","last_became_active":"2024-04-23T18:23:56.096420+0000","last_became_peered":"2024-04-23T18:23:56.096420+0000","last_unstale":"2024-04-23T18:23:57.355568+0000","last_undegraded":"2024-04-23T18:23:57.355568+0000","last_fullsized":"2024-04-23T18:23:57.355568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:20:49.215071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.118210+0000","last_change":"2024-04-23T18:23:55.260645+0000","last_active":"2024-04-23T18:23:57.118210+0000","last_peered":"2024-04-23T18:23:57.118210+0000","last_clean":"2024-04-23T18:23:57.118210+0000","last_became_active":"2024-04-23T18:23:55.260118+0000","last_became_peered":"2024-04-23T18:23:55.260118+0000","last_unstale":"2024-04-23T18:23:57.118210+0000","last_undegraded":"2024-04-23T18:23:57.118210+0000","last_fullsized":"2024-04-23T18:23:57.118210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:14:12.066297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086647+0000","last_change":"2024-04-23T18:23:56.100618+0000","last_active":"2024-04-23T18:23:57.086647+0000","last_peered":"2024-04-23T18:23:57.086647+0000","last_clean":"2024-04-23T18:23:57.086647+0000","last_became_active":"2024-04-23T18:23:56.100126+0000","last_became_peered":"2024-04-23T18:23:56.100126+0000","last_unstale":"2024-04-23T18:23:57.086647+0000","last_undegraded":"2024-04-23T18:23:57.086647+0000","last_fullsized":"2024-04-23T18:23:57.086647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:41:11.481893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.384298+0000","last_change":"2024-04-23T18:23:55.258636+0000","last_active":"2024-04-23T18:24:00.384298+0000","last_peered":"2024-04-23T18:24:00.384298+0000","last_clean":"2024-04-23T18:24:00.384298+0000","last_became_active":"2024-04-23T18:23:55.257744+0000","last_became_peered":"2024-04-23T18:23:55.257744+0000","last_unstale":"2024-04-23T18:24:00.384298+0000","last_undegraded":"2024-04-23T18:24:00.384298+0000","last_fullsized":"2024-04-23T18:24:00.384298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:46:07.454725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085920+0000","last_change":"2024-04-23T18:23:56.094822+0000","last_active":"2024-04-23T18:23:57.085920+0000","last_peered":"2024-04-23T18:23:57.085920+0000","last_clean":"2024-04-23T18:23:57.085920+0000","last_became_active":"2024-04-23T18:23:56.094489+0000","last_became_peered":"2024-04-23T18:23:56.094489+0000","last_unstale":"2024-04-23T18:23:57.085920+0000","last_undegraded":"2024-04-23T18:23:57.085920+0000","last_fullsized":"2024-04-23T18:23:57.085920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:21.234363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085978+0000","last_change":"2024-04-23T18:23:55.261322+0000","last_active":"2024-04-23T18:23:57.085978+0000","last_peered":"2024-04-23T18:23:57.085978+0000","last_clean":"2024-04-23T18:23:57.085978+0000","last_became_active":"2024-04-23T18:23:55.260990+0000","last_became_peered":"2024-04-23T18:23:55.260990+0000","last_unstale":"2024-04-23T18:23:57.085978+0000","last_undegraded":"2024-04-23T18:23:57.085978+0000","last_fullsized":"2024-04-23T18:23:57.085978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:31.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.353308+0000","last_change":"2024-04-23T18:23:56.093105+0000","last_active":"2024-04-23T18:23:57.353308+0000","last_peered":"2024-04-23T18:23:57.353308+0000","last_clean":"2024-04-23T18:23:57.353308+0000","last_became_active":"2024-04-23T18:23:56.092717+0000","last_became_peered":"2024-04-23T18:23:56.092717+0000","last_unstale":"2024-04-23T18:23:57.353308+0000","last_undegraded":"2024-04-23T18:23:57.353308+0000","last_fullsized":"2024-04-23T18:23:57.353308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:13:59.574057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:59.377727+0000","last_change":"2024-04-23T18:23:55.256307+0000","last_active":"2024-04-23T18:23:59.377727+0000","last_peered":"2024-04-23T18:23:59.377727+0000","last_clean":"2024-04-23T18:23:59.377727+0000","last_became_active":"2024-04-23T18:23:55.255959+0000","last_became_peered":"2024-04-23T18:23:55.255959+0000","last_unstale":"2024-04-23T18:23:59.377727+0000","last_undegraded":"2024-04-23T18:23:59.377727+0000","last_fullsized":"2024-04-23T18:23:59.377727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:04.945919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.369003+0000","last_change":"2024-04-23T18:23:56.095177+0000","last_active":"2024-04-23T18:23:57.369003+0000","last_peered":"2024-04-23T18:23:57.369003+0000","last_clean":"2024-04-23T18:23:57.369003+0000","last_became_active":"2024-04-23T18:23:56.094825+0000","last_became_peered":"2024-04-23T18:23:56.094825+0000","last_unstale":"2024-04-23T18:23:57.369003+0000","last_undegraded":"2024-04-23T18:23:57.369003+0000","last_fullsized":"2024-04-23T18:23:57.369003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:07:33.499817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:59.374520+0000","last_change":"2024-04-23T18:23:55.264258+0000","last_active":"2024-04-23T18:23:59.374520+0000","last_peered":"2024-04-23T18:23:59.374520+0000","last_clean":"2024-04-23T18:23:59.374520+0000","last_became_active":"2024-04-23T18:23:55.263698+0000","last_became_peered":"2024-04-23T18:23:55.263698+0000","last_unstale":"2024-04-23T18:23:59.374520+0000","last_undegraded":"2024-04-23T18:23:59.374520+0000","last_fullsized":"2024-04-23T18:23:59.374520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:00.566948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367840+0000","last_change":"2024-04-23T18:23:56.109085+0000","last_active":"2024-04-23T18:23:57.367840+0000","last_peered":"2024-04-23T18:23:57.367840+0000","last_clean":"2024-04-23T18:23:57.367840+0000","last_became_active":"2024-04-23T18:23:56.108033+0000","last_became_peered":"2024-04-23T18:23:56.108033+0000","last_unstale":"2024-04-23T18:23:57.367840+0000","last_undegraded":"2024-04-23T18:23:57.367840+0000","last_fullsized":"2024-04-23T18:23:57.367840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:59.980512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086679+0000","last_change":"2024-04-23T18:23:55.256728+0000","last_active":"2024-04-23T18:23:57.086679+0000","last_peered":"2024-04-23T18:23:57.086679+0000","last_clean":"2024-04-23T18:23:57.086679+0000","last_became_active":"2024-04-23T18:23:55.256367+0000","last_became_peered":"2024-04-23T18:23:55.256367+0000","last_unstale":"2024-04-23T18:23:57.086679+0000","last_undegraded":"2024-04-23T18:23:57.086679+0000","last_fullsized":"2024-04-23T18:23:57.086679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:08.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368605+0000","last_change":"2024-04-23T18:23:56.102898+0000","last_active":"2024-04-23T18:23:57.368605+0000","last_peered":"2024-04-23T18:23:57.368605+0000","last_clean":"2024-04-23T18:23:57.368605+0000","last_became_active":"2024-04-23T18:23:56.102373+0000","last_became_peered":"2024-04-23T18:23:56.102373+0000","last_unstale":"2024-04-23T18:23:57.368605+0000","last_undegraded":"2024-04-23T18:23:57.368605+0000","last_fullsized":"2024-04-23T18:23:57.368605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:49.316219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:59.376964+0000","last_change":"2024-04-23T18:23:55.255094+0000","last_active":"2024-04-23T18:23:59.376964+0000","last_peered":"2024-04-23T18:23:59.376964+0000","last_clean":"2024-04-23T18:23:59.376964+0000","last_became_active":"2024-04-23T18:23:55.254726+0000","last_became_peered":"2024-04-23T18:23:55.254726+0000","last_unstale":"2024-04-23T18:23:59.376964+0000","last_undegraded":"2024-04-23T18:23:59.376964+0000","last_fullsized":"2024-04-23T18:23:59.376964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:26.988184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355944+0000","last_change":"2024-04-23T18:23:56.099866+0000","last_active":"2024-04-23T18:23:57.355944+0000","last_peered":"2024-04-23T18:23:57.355944+0000","last_clean":"2024-04-23T18:23:57.355944+0000","last_became_active":"2024-04-23T18:23:56.098606+0000","last_became_peered":"2024-04-23T18:23:56.098606+0000","last_unstale":"2024-04-23T18:23:57.355944+0000","last_undegraded":"2024-04-23T18:23:57.355944+0000","last_fullsized":"2024-04-23T18:23:57.355944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:26.661857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.386218+0000","last_change":"2024-04-23T18:23:55.256602+0000","last_active":"2024-04-23T18:24:00.386218+0000","last_peered":"2024-04-23T18:24:00.386218+0000","last_clean":"2024-04-23T18:24:00.386218+0000","last_became_active":"2024-04-23T18:23:55.256073+0000","last_became_peered":"2024-04-23T18:23:55.256073+0000","last_unstale":"2024-04-23T18:24:00.386218+0000","last_undegraded":"2024-04-23T18:24:00.386218+0000","last_fullsized":"2024-04-23T18:24:00.386218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:47.454305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368095+0000","last_change":"2024-04-23T18:23:56.099109+0000","last_active":"2024-04-23T18:23:57.368095+0000","last_peered":"2024-04-23T18:23:57.368095+0000","last_clean":"2024-04-23T18:23:57.368095+0000","last_became_active":"2024-04-23T18:23:56.098608+0000","last_became_peered":"2024-04-23T18:23:56.098608+0000","last_unstale":"2024-04-23T18:23:57.368095+0000","last_undegraded":"2024-04-23T18:23:57.368095+0000","last_fullsized":"2024-04-23T18:23:57.368095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:43.720160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086446+0000","last_change":"2024-04-23T18:23:55.262523+0000","last_active":"2024-04-23T18:23:57.086446+0000","last_peered":"2024-04-23T18:23:57.086446+0000","last_clean":"2024-04-23T18:23:57.086446+0000","last_became_active":"2024-04-23T18:23:55.262216+0000","last_became_peered":"2024-04-23T18:23:55.262216+0000","last_unstale":"2024-04-23T18:23:57.086446+0000","last_undegraded":"2024-04-23T18:23:57.086446+0000","last_fullsized":"2024-04-23T18:23:57.086446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:35.268435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368363+0000","last_change":"2024-04-23T18:23:56.106573+0000","last_active":"2024-04-23T18:23:57.368363+0000","last_peered":"2024-04-23T18:23:57.368363+0000","last_clean":"2024-04-23T18:23:57.368363+0000","last_became_active":"2024-04-23T18:23:56.106181+0000","last_became_peered":"2024-04-23T18:23:56.106181+0000","last_unstale":"2024-04-23T18:23:57.368363+0000","last_undegraded":"2024-04-23T18:23:57.368363+0000","last_fullsized":"2024-04-23T18:23:57.368363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:15.422647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086171+0000","last_change":"2024-04-23T18:23:55.262813+0000","last_active":"2024-04-23T18:23:57.086171+0000","last_peered":"2024-04-23T18:23:57.086171+0000","last_clean":"2024-04-23T18:23:57.086171+0000","last_became_active":"2024-04-23T18:23:55.262365+0000","last_became_peered":"2024-04-23T18:23:55.262365+0000","last_unstale":"2024-04-23T18:23:57.086171+0000","last_undegraded":"2024-04-23T18:23:57.086171+0000","last_fullsized":"2024-04-23T18:23:57.086171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:15.234355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367791+0000","last_change":"2024-04-23T18:23:56.101491+0000","last_active":"2024-04-23T18:23:57.367791+0000","last_peered":"2024-04-23T18:23:57.367791+0000","last_clean":"2024-04-23T18:23:57.367791+0000","last_became_active":"2024-04-23T18:23:56.101069+0000","last_became_peered":"2024-04-23T18:23:56.101069+0000","last_unstale":"2024-04-23T18:23:57.367791+0000","last_undegraded":"2024-04-23T18:23:57.367791+0000","last_fullsized":"2024-04-23T18:23:57.367791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:42.484653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'16","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.385356+0000","last_change":"2024-04-23T18:23:55.265947+0000","last_active":"2024-04-23T18:24:00.385356+0000","last_peered":"2024-04-23T18:24:00.385356+0000","last_clean":"2024-04-23T18:24:00.385356+0000","last_became_active":"2024-04-23T18:23:55.265174+0000","last_became_peered":"2024-04-23T18:23:55.265174+0000","last_unstale":"2024-04-23T18:24:00.385356+0000","last_undegraded":"2024-04-23T18:24:00.385356+0000","last_fullsized":"2024-04-23T18:24:00.385356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:41.996928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086155+0000","last_change":"2024-04-23T18:23:56.105691+0000","last_active":"2024-04-23T18:23:57.086155+0000","last_peered":"2024-04-23T18:23:57.086155+0000","last_clean":"2024-04-23T18:23:57.086155+0000","last_became_active":"2024-04-23T18:23:56.105205+0000","last_became_peered":"2024-04-23T18:23:56.105205+0000","last_unstale":"2024-04-23T18:23:57.086155+0000","last_undegraded":"2024-04-23T18:23:57.086155+0000","last_fullsized":"2024-04-23T18:23:57.086155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:30.779821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.356097+0000","last_change":"2024-04-23T18:23:55.261198+0000","last_active":"2024-04-23T18:23:57.356097+0000","last_peered":"2024-04-23T18:23:57.356097+0000","last_clean":"2024-04-23T18:23:57.356097+0000","last_became_active":"2024-04-23T18:23:55.260884+0000","last_became_peered":"2024-04-23T18:23:55.260884+0000","last_unstale":"2024-04-23T18:23:57.356097+0000","last_undegraded":"2024-04-23T18:23:57.356097+0000","last_fullsized":"2024-04-23T18:23:57.356097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:00.841671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368002+0000","last_change":"2024-04-23T18:23:56.099450+0000","last_active":"2024-04-23T18:23:57.368002+0000","last_peered":"2024-04-23T18:23:57.368002+0000","last_clean":"2024-04-23T18:23:57.368002+0000","last_became_active":"2024-04-23T18:23:56.098723+0000","last_became_peered":"2024-04-23T18:23:56.098723+0000","last_unstale":"2024-04-23T18:23:57.368002+0000","last_undegraded":"2024-04-23T18:23:57.368002+0000","last_fullsized":"2024-04-23T18:23:57.368002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:53.027495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.353256+0000","last_change":"2024-04-23T18:23:55.262030+0000","last_active":"2024-04-23T18:23:57.353256+0000","last_peered":"2024-04-23T18:23:57.353256+0000","last_clean":"2024-04-23T18:23:57.353256+0000","last_became_active":"2024-04-23T18:23:55.261659+0000","last_became_peered":"2024-04-23T18:23:55.261659+0000","last_unstale":"2024-04-23T18:23:57.353256+0000","last_undegraded":"2024-04-23T18:23:57.353256+0000","last_fullsized":"2024-04-23T18:23:57.353256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:17.036709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.118783+0000","last_change":"2024-04-23T18:23:56.096511+0000","last_active":"2024-04-23T18:23:57.118783+0000","last_peered":"2024-04-23T18:23:57.118783+0000","last_clean":"2024-04-23T18:23:57.118783+0000","last_became_active":"2024-04-23T18:23:56.096154+0000","last_became_peered":"2024-04-23T18:23:56.096154+0000","last_unstale":"2024-04-23T18:23:57.118783+0000","last_undegraded":"2024-04-23T18:23:57.118783+0000","last_fullsized":"2024-04-23T18:23:57.118783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:03.926178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368166+0000","last_change":"2024-04-23T18:23:55.263618+0000","last_active":"2024-04-23T18:23:57.368166+0000","last_peered":"2024-04-23T18:23:57.368166+0000","last_clean":"2024-04-23T18:23:57.368166+0000","last_became_active":"2024-04-23T18:23:55.261758+0000","last_became_peered":"2024-04-23T18:23:55.261758+0000","last_unstale":"2024-04-23T18:23:57.368166+0000","last_undegraded":"2024-04-23T18:23:57.368166+0000","last_fullsized":"2024-04-23T18:23:57.368166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:23:27.745668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.118709+0000","last_change":"2024-04-23T18:23:56.109292+0000","last_active":"2024-04-23T18:23:57.118709+0000","last_peered":"2024-04-23T18:23:57.118709+0000","last_clean":"2024-04-23T18:23:57.118709+0000","last_became_active":"2024-04-23T18:23:56.108414+0000","last_became_peered":"2024-04-23T18:23:56.108414+0000","last_unstale":"2024-04-23T18:23:57.118709+0000","last_undegraded":"2024-04-23T18:23:57.118709+0000","last_fullsized":"2024-04-23T18:23:57.118709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:13.428782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086381+0000","last_change":"2024-04-23T18:23:56.097122+0000","last_active":"2024-04-23T18:23:57.086381+0000","last_peered":"2024-04-23T18:23:57.086381+0000","last_clean":"2024-04-23T18:23:57.086381+0000","last_became_active":"2024-04-23T18:23:56.096751+0000","last_became_peered":"2024-04-23T18:23:56.096751+0000","last_unstale":"2024-04-23T18:23:57.086381+0000","last_undegraded":"2024-04-23T18:23:57.086381+0000","last_fullsized":"2024-04-23T18:23:57.086381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:00:55.090370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.385003+0000","last_change":"2024-04-23T18:23:55.256458+0000","last_active":"2024-04-23T18:24:00.385003+0000","last_peered":"2024-04-23T18:24:00.385003+0000","last_clean":"2024-04-23T18:24:00.385003+0000","last_became_active":"2024-04-23T18:23:55.256127+0000","last_became_peered":"2024-04-23T18:23:55.256127+0000","last_unstale":"2024-04-23T18:24:00.385003+0000","last_undegraded":"2024-04-23T18:24:00.385003+0000","last_fullsized":"2024-04-23T18:24:00.385003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:03.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085840+0000","last_change":"2024-04-23T18:23:56.103743+0000","last_active":"2024-04-23T18:23:57.085840+0000","last_peered":"2024-04-23T18:23:57.085840+0000","last_clean":"2024-04-23T18:23:57.085840+0000","last_became_active":"2024-04-23T18:23:56.103041+0000","last_became_peered":"2024-04-23T18:23:56.103041+0000","last_unstale":"2024-04-23T18:23:57.085840+0000","last_undegraded":"2024-04-23T18:23:57.085840+0000","last_fullsized":"2024-04-23T18:23:57.085840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:22.419555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086678+0000","last_change":"2024-04-23T18:23:55.257008+0000","last_active":"2024-04-23T18:23:57.086678+0000","last_peered":"2024-04-23T18:23:57.086678+0000","last_clean":"2024-04-23T18:23:57.086678+0000","last_became_active":"2024-04-23T18:23:55.256648+0000","last_became_peered":"2024-04-23T18:23:55.256648+0000","last_unstale":"2024-04-23T18:23:57.086678+0000","last_undegraded":"2024-04-23T18:23:57.086678+0000","last_fullsized":"2024-04-23T18:23:57.086678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:00.867083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367814+0000","last_change":"2024-04-23T18:22:38.610948+0000","last_active":"2024-04-23T18:23:57.367814+0000","last_peered":"2024-04-23T18:23:57.367814+0000","last_clean":"2024-04-23T18:23:57.367814+0000","last_became_active":"2024-04-23T18:22:38.610475+0000","last_became_peered":"2024-04-23T18:22:38.610475+0000","last_unstale":"2024-04-23T18:23:57.367814+0000","last_undegraded":"2024-04-23T18:23:57.367814+0000","last_fullsized":"2024-04-23T18:23:57.367814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_clean_scrub_stamp":"2024-04-23T18:19:07.980519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:08.209087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367829+0000","last_change":"2024-04-23T18:23:56.108706+0000","last_active":"2024-04-23T18:23:57.367829+0000","last_peered":"2024-04-23T18:23:57.367829+0000","last_clean":"2024-04-23T18:23:57.367829+0000","last_became_active":"2024-04-23T18:23:56.108336+0000","last_became_peered":"2024-04-23T18:23:56.108336+0000","last_unstale":"2024-04-23T18:23:57.367829+0000","last_undegraded":"2024-04-23T18:23:57.367829+0000","last_fullsized":"2024-04-23T18:23:57.367829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:41.641623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.382342+0000","last_change":"2024-04-23T18:23:55.263713+0000","last_active":"2024-04-23T18:24:00.382342+0000","last_peered":"2024-04-23T18:24:00.382342+0000","last_clean":"2024-04-23T18:24:00.382342+0000","last_became_active":"2024-04-23T18:23:55.261703+0000","last_became_peered":"2024-04-23T18:23:55.261703+0000","last_unstale":"2024-04-23T18:24:00.382342+0000","last_undegraded":"2024-04-23T18:24:00.382342+0000","last_fullsized":"2024-04-23T18:24:00.382342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:31.676269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:01.393519+0000","last_change":"2024-04-23T18:23:55.265853+0000","last_active":"2024-04-23T18:24:01.393519+0000","last_peered":"2024-04-23T18:24:01.393519+0000","last_clean":"2024-04-23T18:24:01.393519+0000","last_became_active":"2024-04-23T18:23:55.265435+0000","last_became_peered":"2024-04-23T18:23:55.265435+0000","last_unstale":"2024-04-23T18:24:01.393519+0000","last_undegraded":"2024-04-23T18:24:01.393519+0000","last_fullsized":"2024-04-23T18:24:01.393519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.969720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368295+0000","last_change":"2024-04-23T18:23:56.085609+0000","last_active":"2024-04-23T18:23:57.368295+0000","last_peered":"2024-04-23T18:23:57.368295+0000","last_clean":"2024-04-23T18:23:57.368295+0000","last_became_active":"2024-04-23T18:23:56.085244+0000","last_became_peered":"2024-04-23T18:23:56.085244+0000","last_unstale":"2024-04-23T18:23:57.368295+0000","last_undegraded":"2024-04-23T18:23:57.368295+0000","last_fullsized":"2024-04-23T18:23:57.368295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.211717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355120+0000","last_change":"2024-04-23T18:23:55.260260+0000","last_active":"2024-04-23T18:23:57.355120+0000","last_peered":"2024-04-23T18:23:57.355120+0000","last_clean":"2024-04-23T18:23:57.355120+0000","last_became_active":"2024-04-23T18:23:55.259862+0000","last_became_peered":"2024-04-23T18:23:55.259862+0000","last_unstale":"2024-04-23T18:23:57.355120+0000","last_undegraded":"2024-04-23T18:23:57.355120+0000","last_fullsized":"2024-04-23T18:23:57.355120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:57.833293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085956+0000","last_change":"2024-04-23T18:23:56.098826+0000","last_active":"2024-04-23T18:23:57.085956+0000","last_peered":"2024-04-23T18:23:57.085956+0000","last_clean":"2024-04-23T18:23:57.085956+0000","last_became_active":"2024-04-23T18:23:56.098494+0000","last_became_peered":"2024-04-23T18:23:56.098494+0000","last_unstale":"2024-04-23T18:23:57.085956+0000","last_undegraded":"2024-04-23T18:23:57.085956+0000","last_fullsized":"2024-04-23T18:23:57.085956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:12:24.953330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086592+0000","last_change":"2024-04-23T18:23:55.258174+0000","last_active":"2024-04-23T18:23:57.086592+0000","last_peered":"2024-04-23T18:23:57.086592+0000","last_clean":"2024-04-23T18:23:57.086592+0000","last_became_active":"2024-04-23T18:23:55.257846+0000","last_became_peered":"2024-04-23T18:23:55.257846+0000","last_unstale":"2024-04-23T18:23:57.086592+0000","last_undegraded":"2024-04-23T18:23:57.086592+0000","last_fullsized":"2024-04-23T18:23:57.086592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:35.131566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367450+0000","last_change":"2024-04-23T18:23:56.110758+0000","last_active":"2024-04-23T18:23:57.367450+0000","last_peered":"2024-04-23T18:23:57.367450+0000","last_clean":"2024-04-23T18:23:57.367450+0000","last_became_active":"2024-04-23T18:23:56.110372+0000","last_became_peered":"2024-04-23T18:23:56.110372+0000","last_unstale":"2024-04-23T18:23:57.367450+0000","last_undegraded":"2024-04-23T18:23:57.367450+0000","last_fullsized":"2024-04-23T18:23:57.367450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:40.193088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:59.375841+0000","last_change":"2024-04-23T18:23:55.265714+0000","last_active":"2024-04-23T18:23:59.375841+0000","last_peered":"2024-04-23T18:23:59.375841+0000","last_clean":"2024-04-23T18:23:59.375841+0000","last_became_active":"2024-04-23T18:23:55.265198+0000","last_became_peered":"2024-04-23T18:23:55.265198+0000","last_unstale":"2024-04-23T18:23:59.375841+0000","last_undegraded":"2024-04-23T18:23:59.375841+0000","last_fullsized":"2024-04-23T18:23:59.375841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:10.770748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.355537+0000","last_change":"2024-04-23T18:23:56.102347+0000","last_active":"2024-04-23T18:23:57.355537+0000","last_peered":"2024-04-23T18:23:57.355537+0000","last_clean":"2024-04-23T18:23:57.355537+0000","last_became_active":"2024-04-23T18:23:56.100687+0000","last_became_peered":"2024-04-23T18:23:56.100687+0000","last_unstale":"2024-04-23T18:23:57.355537+0000","last_undegraded":"2024-04-23T18:23:57.355537+0000","last_fullsized":"2024-04-23T18:23:57.355537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:20.378730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:01.394611+0000","last_change":"2024-04-23T18:23:55.255943+0000","last_active":"2024-04-23T18:24:01.394611+0000","last_peered":"2024-04-23T18:24:01.394611+0000","last_clean":"2024-04-23T18:24:01.394611+0000","last_became_active":"2024-04-23T18:23:55.255618+0000","last_became_peered":"2024-04-23T18:23:55.255618+0000","last_unstale":"2024-04-23T18:24:01.394611+0000","last_undegraded":"2024-04-23T18:24:01.394611+0000","last_fullsized":"2024-04-23T18:24:01.394611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:01.408533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367714+0000","last_change":"2024-04-23T18:23:56.091664+0000","last_active":"2024-04-23T18:23:57.367714+0000","last_peered":"2024-04-23T18:23:57.367714+0000","last_clean":"2024-04-23T18:23:57.367714+0000","last_became_active":"2024-04-23T18:23:56.091308+0000","last_became_peered":"2024-04-23T18:23:56.091308+0000","last_unstale":"2024-04-23T18:23:57.367714+0000","last_undegraded":"2024-04-23T18:23:57.367714+0000","last_fullsized":"2024-04-23T18:23:57.367714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:31.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:00.371497+0000","last_change":"2024-04-23T18:23:55.260992+0000","last_active":"2024-04-23T18:24:00.371497+0000","last_peered":"2024-04-23T18:24:00.371497+0000","last_clean":"2024-04-23T18:24:00.371497+0000","last_became_active":"2024-04-23T18:23:55.260679+0000","last_became_peered":"2024-04-23T18:23:55.260679+0000","last_unstale":"2024-04-23T18:24:00.371497+0000","last_undegraded":"2024-04-23T18:24:00.371497+0000","last_fullsized":"2024-04-23T18:24:00.371497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:02:50.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085855+0000","last_change":"2024-04-23T18:23:56.097956+0000","last_active":"2024-04-23T18:23:57.085855+0000","last_peered":"2024-04-23T18:23:57.085855+0000","last_clean":"2024-04-23T18:23:57.085855+0000","last_became_active":"2024-04-23T18:23:56.097231+0000","last_became_peered":"2024-04-23T18:23:56.097231+0000","last_unstale":"2024-04-23T18:23:57.085855+0000","last_undegraded":"2024-04-23T18:23:57.085855+0000","last_fullsized":"2024-04-23T18:23:57.085855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:10.945857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:59.375748+0000","last_change":"2024-04-23T18:23:55.262860+0000","last_active":"2024-04-23T18:23:59.375748+0000","last_peered":"2024-04-23T18:23:59.375748+0000","last_clean":"2024-04-23T18:23:59.375748+0000","last_became_active":"2024-04-23T18:23:55.262492+0000","last_became_peered":"2024-04-23T18:23:55.262492+0000","last_unstale":"2024-04-23T18:23:59.375748+0000","last_undegraded":"2024-04-23T18:23:59.375748+0000","last_fullsized":"2024-04-23T18:23:59.375748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:43.836017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368298+0000","last_change":"2024-04-23T18:23:56.096889+0000","last_active":"2024-04-23T18:23:57.368298+0000","last_peered":"2024-04-23T18:23:57.368298+0000","last_clean":"2024-04-23T18:23:57.368298+0000","last_became_active":"2024-04-23T18:23:56.096520+0000","last_became_peered":"2024-04-23T18:23:56.096520+0000","last_unstale":"2024-04-23T18:23:57.368298+0000","last_undegraded":"2024-04-23T18:23:57.368298+0000","last_fullsized":"2024-04-23T18:23:57.368298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:34:53.785481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:24:01.394602+0000","last_change":"2024-04-23T18:23:55.258710+0000","last_active":"2024-04-23T18:24:01.394602+0000","last_peered":"2024-04-23T18:24:01.394602+0000","last_clean":"2024-04-23T18:24:01.394602+0000","last_became_active":"2024-04-23T18:23:55.256247+0000","last_became_peered":"2024-04-23T18:23:55.256247+0000","last_unstale":"2024-04-23T18:24:01.394602+0000","last_undegraded":"2024-04-23T18:24:01.394602+0000","last_fullsized":"2024-04-23T18:24:01.394602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:20.551614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368796+0000","last_change":"2024-04-23T18:23:56.101862+0000","last_active":"2024-04-23T18:23:57.368796+0000","last_peered":"2024-04-23T18:23:57.368796+0000","last_clean":"2024-04-23T18:23:57.368796+0000","last_became_active":"2024-04-23T18:23:56.101458+0000","last_became_peered":"2024-04-23T18:23:56.101458+0000","last_unstale":"2024-04-23T18:23:57.368796+0000","last_undegraded":"2024-04-23T18:23:57.368796+0000","last_fullsized":"2024-04-23T18:23:57.368796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:59.292077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086547+0000","last_change":"2024-04-23T18:23:55.258292+0000","last_active":"2024-04-23T18:23:57.086547+0000","last_peered":"2024-04-23T18:23:57.086547+0000","last_clean":"2024-04-23T18:23:57.086547+0000","last_became_active":"2024-04-23T18:23:55.257954+0000","last_became_peered":"2024-04-23T18:23:55.257954+0000","last_unstale":"2024-04-23T18:23:57.086547+0000","last_undegraded":"2024-04-23T18:23:57.086547+0000","last_fullsized":"2024-04-23T18:23:57.086547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:39.105967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368267+0000","last_change":"2024-04-23T18:23:56.105977+0000","last_active":"2024-04-23T18:23:57.368267+0000","last_peered":"2024-04-23T18:23:57.368267+0000","last_clean":"2024-04-23T18:23:57.368267+0000","last_became_active":"2024-04-23T18:23:56.105582+0000","last_became_peered":"2024-04-23T18:23:56.105582+0000","last_unstale":"2024-04-23T18:23:57.368267+0000","last_undegraded":"2024-04-23T18:23:57.368267+0000","last_fullsized":"2024-04-23T18:23:57.368267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:28:10.700684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368121+0000","last_change":"2024-04-23T18:23:55.252095+0000","last_active":"2024-04-23T18:23:57.368121+0000","last_peered":"2024-04-23T18:23:57.368121+0000","last_clean":"2024-04-23T18:23:57.368121+0000","last_became_active":"2024-04-23T18:23:55.251702+0000","last_became_peered":"2024-04-23T18:23:55.251702+0000","last_unstale":"2024-04-23T18:23:57.368121+0000","last_undegraded":"2024-04-23T18:23:57.368121+0000","last_fullsized":"2024-04-23T18:23:57.368121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:21.625905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368390+0000","last_change":"2024-04-23T18:23:56.101997+0000","last_active":"2024-04-23T18:23:57.368390+0000","last_peered":"2024-04-23T18:23:57.368390+0000","last_clean":"2024-04-23T18:23:57.368390+0000","last_became_active":"2024-04-23T18:23:56.101629+0000","last_became_peered":"2024-04-23T18:23:56.101629+0000","last_unstale":"2024-04-23T18:23:57.368390+0000","last_undegraded":"2024-04-23T18:23:57.368390+0000","last_fullsized":"2024-04-23T18:23:57.368390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:42.484413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367778+0000","last_change":"2024-04-23T18:23:56.090543+0000","last_active":"2024-04-23T18:23:57.367778+0000","last_peered":"2024-04-23T18:23:57.367778+0000","last_clean":"2024-04-23T18:23:57.367778+0000","last_became_active":"2024-04-23T18:23:56.090191+0000","last_became_peered":"2024-04-23T18:23:56.090191+0000","last_unstale":"2024-04-23T18:23:57.367778+0000","last_undegraded":"2024-04-23T18:23:57.367778+0000","last_fullsized":"2024-04-23T18:23:57.367778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:01.401822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367986+0000","last_change":"2024-04-23T18:23:56.105537+0000","last_active":"2024-04-23T18:23:57.367986+0000","last_peered":"2024-04-23T18:23:57.367986+0000","last_clean":"2024-04-23T18:23:57.367986+0000","last_became_active":"2024-04-23T18:23:56.105130+0000","last_became_peered":"2024-04-23T18:23:56.105130+0000","last_unstale":"2024-04-23T18:23:57.367986+0000","last_undegraded":"2024-04-23T18:23:57.367986+0000","last_fullsized":"2024-04-23T18:23:57.367986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:59.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367800+0000","last_change":"2024-04-23T18:23:56.097765+0000","last_active":"2024-04-23T18:23:57.367800+0000","last_peered":"2024-04-23T18:23:57.367800+0000","last_clean":"2024-04-23T18:23:57.367800+0000","last_became_active":"2024-04-23T18:23:56.097439+0000","last_became_peered":"2024-04-23T18:23:56.097439+0000","last_unstale":"2024-04-23T18:23:57.367800+0000","last_undegraded":"2024-04-23T18:23:57.367800+0000","last_fullsized":"2024-04-23T18:23:57.367800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:31.529751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367919+0000","last_change":"2024-04-23T18:23:56.108548+0000","last_active":"2024-04-23T18:23:57.367919+0000","last_peered":"2024-04-23T18:23:57.367919+0000","last_clean":"2024-04-23T18:23:57.367919+0000","last_became_active":"2024-04-23T18:23:56.108214+0000","last_became_peered":"2024-04-23T18:23:56.108214+0000","last_unstale":"2024-04-23T18:23:57.367919+0000","last_undegraded":"2024-04-23T18:23:57.367919+0000","last_fullsized":"2024-04-23T18:23:57.367919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:41.738754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368206+0000","last_change":"2024-04-23T18:23:56.108551+0000","last_active":"2024-04-23T18:23:57.368206+0000","last_peered":"2024-04-23T18:23:57.368206+0000","last_clean":"2024-04-23T18:23:57.368206+0000","last_became_active":"2024-04-23T18:23:56.108227+0000","last_became_peered":"2024-04-23T18:23:56.108227+0000","last_unstale":"2024-04-23T18:23:57.368206+0000","last_undegraded":"2024-04-23T18:23:57.368206+0000","last_fullsized":"2024-04-23T18:23:57.368206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:19.226718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368213+0000","last_change":"2024-04-23T18:23:56.093969+0000","last_active":"2024-04-23T18:23:57.368213+0000","last_peered":"2024-04-23T18:23:57.368213+0000","last_clean":"2024-04-23T18:23:57.368213+0000","last_became_active":"2024-04-23T18:23:56.093568+0000","last_became_peered":"2024-04-23T18:23:56.093568+0000","last_unstale":"2024-04-23T18:23:57.368213+0000","last_undegraded":"2024-04-23T18:23:57.368213+0000","last_fullsized":"2024-04-23T18:23:57.368213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:08.361305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368654+0000","last_change":"2024-04-23T18:23:56.096106+0000","last_active":"2024-04-23T18:23:57.368654+0000","last_peered":"2024-04-23T18:23:57.368654+0000","last_clean":"2024-04-23T18:23:57.368654+0000","last_became_active":"2024-04-23T18:23:56.095745+0000","last_became_peered":"2024-04-23T18:23:56.095745+0000","last_unstale":"2024-04-23T18:23:57.368654+0000","last_undegraded":"2024-04-23T18:23:57.368654+0000","last_fullsized":"2024-04-23T18:23:57.368654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:01.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085867+0000","last_change":"2024-04-23T18:23:56.095474+0000","last_active":"2024-04-23T18:23:57.085867+0000","last_peered":"2024-04-23T18:23:57.085867+0000","last_clean":"2024-04-23T18:23:57.085867+0000","last_became_active":"2024-04-23T18:23:56.095116+0000","last_became_peered":"2024-04-23T18:23:56.095116+0000","last_unstale":"2024-04-23T18:23:57.085867+0000","last_undegraded":"2024-04-23T18:23:57.085867+0000","last_fullsized":"2024-04-23T18:23:57.085867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:50:00.339558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.086661+0000","last_change":"2024-04-23T18:23:56.096022+0000","last_active":"2024-04-23T18:23:57.086661+0000","last_peered":"2024-04-23T18:23:57.086661+0000","last_clean":"2024-04-23T18:23:57.086661+0000","last_became_active":"2024-04-23T18:23:56.095493+0000","last_became_peered":"2024-04-23T18:23:56.095493+0000","last_unstale":"2024-04-23T18:23:57.086661+0000","last_undegraded":"2024-04-23T18:23:57.086661+0000","last_fullsized":"2024-04-23T18:23:57.086661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:42.209072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.085942+0000","last_change":"2024-04-23T18:23:56.101697+0000","last_active":"2024-04-23T18:23:57.085942+0000","last_peered":"2024-04-23T18:23:57.085942+0000","last_clean":"2024-04-23T18:23:57.085942+0000","last_became_active":"2024-04-23T18:23:56.098708+0000","last_became_peered":"2024-04-23T18:23:56.098708+0000","last_unstale":"2024-04-23T18:23:57.085942+0000","last_undegraded":"2024-04-23T18:23:57.085942+0000","last_fullsized":"2024-04-23T18:23:57.085942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:14.170700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368786+0000","last_change":"2024-04-23T18:23:56.096982+0000","last_active":"2024-04-23T18:23:57.368786+0000","last_peered":"2024-04-23T18:23:57.368786+0000","last_clean":"2024-04-23T18:23:57.368786+0000","last_became_active":"2024-04-23T18:23:56.096608+0000","last_became_peered":"2024-04-23T18:23:56.096608+0000","last_unstale":"2024-04-23T18:23:57.368786+0000","last_undegraded":"2024-04-23T18:23:57.368786+0000","last_fullsized":"2024-04-23T18:23:57.368786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:50.329337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.368268+0000","last_change":"2024-04-23T18:23:56.100345+0000","last_active":"2024-04-23T18:23:57.368268+0000","last_peered":"2024-04-23T18:23:57.368268+0000","last_clean":"2024-04-23T18:23:57.368268+0000","last_became_active":"2024-04-23T18:23:56.100043+0000","last_became_peered":"2024-04-23T18:23:56.100043+0000","last_unstale":"2024-04-23T18:23:57.368268+0000","last_undegraded":"2024-04-23T18:23:57.368268+0000","last_fullsized":"2024-04-23T18:23:57.368268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:36.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:23:57.367936+0000","last_change":"2024-04-23T18:23:56.101947+0000","last_active":"2024-04-23T18:23:57.367936+0000","last_peered":"2024-04-23T18:23:57.367936+0000","last_clean":"2024-04-23T18:23:57.367936+0000","last_became_active":"2024-04-23T18:23:56.101627+0000","last_became_peered":"2024-04-23T18:23:56.101627+0000","last_unstale":"2024-04-23T18:23:57.367936+0000","last_undegraded":"2024-04-23T18:23:57.367936+0000","last_fullsized":"2024-04-23T18:23:57.367936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:21:43.703653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279760,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4018,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":23448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":444207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":999424,"data_stored":1035189,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":929792,"data_stored":443597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":443753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1036629,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":443015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":445600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":929792,"data_stored":443105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":446599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":443423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":443037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":1033565,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":9964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:24:05.215 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T18:24:05.215 INFO:tasks.cephadm:Setting up client nodes... 2024-04-23T18:24:05.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-23T18:24:05.499 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:05.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:05 smithi044 ceph-mon[32458]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:05.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:05 smithi044 ceph-mon[32458]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:05.813 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:24:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b[62807]: 2024-04-23T18:24:05.528+0000 7fb570850700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-23T18:24:05.813 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:24:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b[62807]: 2024-04-23T18:24:05.528+0000 7fb570850700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-23T18:24:05.913 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:05 smithi019 ceph-mon[29215]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:05.913 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:05 smithi019 ceph-mon[29215]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:06.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:05 smithi155 ceph-mon[33468]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:06.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:05 smithi155 ceph-mon[33468]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:07.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:06 smithi019 ceph-mon[29215]: pgmap v301: 97 pgs: 97 active+clean; 620 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-04-23T18:24:07.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:06 smithi044 ceph-mon[32458]: pgmap v301: 97 pgs: 97 active+clean; 620 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-04-23T18:24:07.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:06 smithi155 ceph-mon[33468]: pgmap v301: 97 pgs: 97 active+clean; 620 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-04-23T18:24:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.564+0000 7f976c573700 1 -- 172.21.15.19:0/3033917434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9764103570 msgr2=0x7f9764105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.564+0000 7f976c573700 1 --2- 172.21.15.19:0/3033917434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9764103570 0x7f9764105960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9760009990 tx=0x7f9760009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.564+0000 7f976c573700 1 -- 172.21.15.19:0/3033917434 shutdown_connections 2024-04-23T18:24:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.564+0000 7f976c573700 1 --2- 172.21.15.19:0/3033917434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97641087d0 0x7f976410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.564+0000 7f976c573700 1 --2- 172.21.15.19:0/3033917434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9764103570 0x7f9764105960 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.564+0000 7f976c573700 1 --2- 172.21.15.19:0/3033917434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9764105ea0 0x7f9764108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.564+0000 7f976c573700 1 -- 172.21.15.19:0/3033917434 >> 172.21.15.19:0/3033917434 conn(0x7f97640fcfa0 msgr2=0x7f97640ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.564+0000 7f976c573700 1 -- 172.21.15.19:0/3033917434 shutdown_connections 2024-04-23T18:24:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.564+0000 7f976c573700 1 -- 172.21.15.19:0/3033917434 wait complete. 2024-04-23T18:24:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.564+0000 7f976c573700 1 Processor -- start 2024-04-23T18:24:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976c573700 1 -- start start 2024-04-23T18:24:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976c573700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9764103570 0x7f976419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:07.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976c573700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9764105ea0 0x7f976419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:07.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976c573700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97641087d0 0x7f97641a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:07.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976c573700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f976410e580 con 0x7f97641087d0 2024-04-23T18:24:07.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976c573700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f976410e400 con 0x7f9764103570 2024-04-23T18:24:07.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976c573700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f976410e700 con 0x7f9764105ea0 2024-04-23T18:24:07.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976ab10700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97641087d0 0x7f97641a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976ab10700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97641087d0 0x7f97641a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60658/0 (socket says 172.21.15.19:60658) 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976ab10700 1 -- 172.21.15.19:0/696904502 learned_addr learned my addr 172.21.15.19:0/696904502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976a30f700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9764103570 0x7f976419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f9769b0e700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9764105ea0 0x7f976419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976ab10700 1 -- 172.21.15.19:0/696904502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9764105ea0 msgr2=0x7f976419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976ab10700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9764105ea0 0x7f976419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976ab10700 1 -- 172.21.15.19:0/696904502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9764103570 msgr2=0x7f976419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976ab10700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9764103570 0x7f976419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976ab10700 1 -- 172.21.15.19:0/696904502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97641a2990 con 0x7f97641087d0 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976a30f700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9764103570 0x7f976419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f9769b0e700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9764105ea0 0x7f976419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.565+0000 7f976ab10700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97641087d0 0x7f97641a22c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f975c0094f0 tx=0x7f975c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.566+0000 7f975b7fe700 1 -- 172.21.15.19:0/696904502 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c004db0 con 0x7f97641087d0 2024-04-23T18:24:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.566+0000 7f976c573700 1 -- 172.21.15.19:0/696904502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97641a2c80 con 0x7f97641087d0 2024-04-23T18:24:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.566+0000 7f975b7fe700 1 -- 172.21.15.19:0/696904502 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c00db60 con 0x7f97641087d0 2024-04-23T18:24:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.566+0000 7f975b7fe700 1 -- 172.21.15.19:0/696904502 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c015400 con 0x7f97641087d0 2024-04-23T18:24:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.566+0000 7f976c573700 1 -- 172.21.15.19:0/696904502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97641a3170 con 0x7f97641087d0 2024-04-23T18:24:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.567+0000 7f976c573700 1 -- 172.21.15.19:0/696904502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9764050f80 con 0x7f97641087d0 2024-04-23T18:24:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.568+0000 7f975b7fe700 1 -- 172.21.15.19:0/696904502 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f975c00dd10 con 0x7f97641087d0 2024-04-23T18:24:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.568+0000 7f975b7fe700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f97500a3d80 0x7f97500a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.569+0000 7f976a30f700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f97500a3d80 0x7f97500a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.569+0000 7f975b7fe700 1 -- 172.21.15.19:0/696904502 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f975c0cffe0 con 0x7f97641087d0 2024-04-23T18:24:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.569+0000 7f976a30f700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f97500a3d80 0x7f97500a6240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f97600038a0 tx=0x7f9760003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.573+0000 7f975b7fe700 1 -- 172.21.15.19:0/696904502 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f975c010070 con 0x7f97641087d0 2024-04-23T18:24:07.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.867+0000 7f976c573700 1 -- 172.21.15.19:0/696904502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f976402e0a0 con 0x7f97641087d0 2024-04-23T18:24:07.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.869+0000 7f975b7fe700 1 -- 172.21.15.19:0/696904502 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f976402e0a0 con 0x7f97641087d0 2024-04-23T18:24:07.862 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-23T18:24:07.863 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDH/Cdm66XKMxAAmUlBpLz+/GsXcy5uNe8few== 2024-04-23T18:24:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 -- 172.21.15.19:0/696904502 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f97500a3d80 msgr2=0x7f97500a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f97500a3d80 0x7f97500a6240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f97600038a0 tx=0x7f9760003680 comp rx=0 tx=0).stop 2024-04-23T18:24:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 -- 172.21.15.19:0/696904502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97641087d0 msgr2=0x7f97641a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97641087d0 0x7f97641a22c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f975c0094f0 tx=0x7f975c007f90 comp rx=0 tx=0).stop 2024-04-23T18:24:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 -- 172.21.15.19:0/696904502 shutdown_connections 2024-04-23T18:24:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f97500a3d80 0x7f97500a6240 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97641087d0 0x7f97641a22c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9764103570 0x7f976419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 --2- 172.21.15.19:0/696904502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9764105ea0 0x7f976419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 -- 172.21.15.19:0/696904502 >> 172.21.15.19:0/696904502 conn(0x7f97640fcfa0 msgr2=0x7f9764102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 -- 172.21.15.19:0/696904502 shutdown_connections 2024-04-23T18:24:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:07.872+0000 7f976c573700 1 -- 172.21.15.19:0/696904502 wait complete. 2024-04-23T18:24:08.696 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:24:08.696 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-23T18:24:08.696 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T18:24:08.737 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T18:24:08.738 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-23T18:24:08.738 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-23T18:24:09.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:08 smithi019 ceph-mon[29215]: pgmap v302: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.4 KiB/s wr, 25 op/s 2024-04-23T18:24:09.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:08 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/696904502' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:24:09.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:08 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/696904502' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:24:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:09.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:08 smithi044 ceph-mon[32458]: pgmap v302: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.4 KiB/s wr, 25 op/s 2024-04-23T18:24:09.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:08 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/696904502' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:24:09.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:08 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/696904502' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:24:09.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:08 smithi155 ceph-mon[33468]: pgmap v302: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.4 KiB/s wr, 25 op/s 2024-04-23T18:24:09.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:08 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/696904502' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:24:09.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:08 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/696904502' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:24:10.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:10 smithi019 ceph-mon[29215]: pgmap v303: 97 pgs: 97 active+clean; 621 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-23T18:24:10.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.978+0000 7ff2afa66700 1 -- 172.21.15.19:0/2143015011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a8102010 msgr2=0x7ff2a8105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:10.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.978+0000 7ff2afa66700 1 --2- 172.21.15.19:0/2143015011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a8102010 0x7ff2a8105dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff298009990 tx=0x7ff298009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:10.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.978+0000 7ff2afa66700 1 -- 172.21.15.19:0/2143015011 shutdown_connections 2024-04-23T18:24:10.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.978+0000 7ff2afa66700 1 --2- 172.21.15.19:0/2143015011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a8106500 0x7ff2a810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:10.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.978+0000 7ff2afa66700 1 --2- 172.21.15.19:0/2143015011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2a8101660 0x7ff2a8101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:10.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.978+0000 7ff2afa66700 1 --2- 172.21.15.19:0/2143015011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a8102010 0x7ff2a8105dd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:10.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.978+0000 7ff2afa66700 1 -- 172.21.15.19:0/2143015011 >> 172.21.15.19:0/2143015011 conn(0x7ff2a8076c60 msgr2=0x7ff2a8077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:10.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.978+0000 7ff2afa66700 1 -- 172.21.15.19:0/2143015011 shutdown_connections 2024-04-23T18:24:10.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.978+0000 7ff2afa66700 1 -- 172.21.15.19:0/2143015011 wait complete. 2024-04-23T18:24:10.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.979+0000 7ff2afa66700 1 Processor -- start 2024-04-23T18:24:10.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.979+0000 7ff2afa66700 1 -- start start 2024-04-23T18:24:10.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.979+0000 7ff2afa66700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a8101660 0x7ff2a8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:10.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.979+0000 7ff2afa66700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a8102010 0x7ff2a8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:10.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.979+0000 7ff2afa66700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2a8106500 0x7ff2a819d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:10.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.979+0000 7ff2afa66700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a8110ac0 con 0x7ff2a8102010 2024-04-23T18:24:10.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.979+0000 7ff2afa66700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a8110940 con 0x7ff2a8106500 2024-04-23T18:24:10.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.979+0000 7ff2afa66700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a8110c40 con 0x7ff2a8101660 2024-04-23T18:24:10.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ad001700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a8102010 0x7ff2a8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:10.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ad001700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a8102010 0x7ff2a8199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52678/0 (socket says 172.21.15.19:52678) 2024-04-23T18:24:10.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ad001700 1 -- 172.21.15.19:0/1470034404 learned_addr learned my addr 172.21.15.19:0/1470034404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:10.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ae003700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2a8106500 0x7ff2a819d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:10.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ad802700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a8101660 0x7ff2a8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:10.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ad001700 1 -- 172.21.15.19:0/1470034404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a8101660 msgr2=0x7ff2a8199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:10.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ad001700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a8101660 0x7ff2a8199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:10.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ad001700 1 -- 172.21.15.19:0/1470034404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2a8106500 msgr2=0x7ff2a819d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:10.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ad001700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2a8106500 0x7ff2a819d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:10.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ad001700 1 -- 172.21.15.19:0/1470034404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a819e110 con 0x7ff2a8102010 2024-04-23T18:24:10.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2ad001700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a8102010 0x7ff2a8199640 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff298006010 tx=0x7ff2980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:10.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff29effd700 1 -- 172.21.15.19:0/1470034404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298027070 con 0x7ff2a8102010 2024-04-23T18:24:10.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2afa66700 1 -- 172.21.15.19:0/1470034404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a819e3a0 con 0x7ff2a8102010 2024-04-23T18:24:10.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff29effd700 1 -- 172.21.15.19:0/1470034404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff298022470 con 0x7ff2a8102010 2024-04-23T18:24:10.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff29effd700 1 -- 172.21.15.19:0/1470034404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298017650 con 0x7ff2a8102010 2024-04-23T18:24:10.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.980+0000 7ff2afa66700 1 -- 172.21.15.19:0/1470034404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a81a5c80 con 0x7ff2a8102010 2024-04-23T18:24:10.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.981+0000 7ff2afa66700 1 -- 172.21.15.19:0/1470034404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a8069730 con 0x7ff2a8102010 2024-04-23T18:24:10.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.982+0000 7ff29effd700 1 -- 172.21.15.19:0/1470034404 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff29802a070 con 0x7ff2a8102010 2024-04-23T18:24:10.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.983+0000 7ff29effd700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ff2940a3cd0 0x7ff2940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:10.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.983+0000 7ff2ad802700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ff2940a3cd0 0x7ff2940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:10.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.983+0000 7ff29effd700 1 -- 172.21.15.19:0/1470034404 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff2980d1060 con 0x7ff2a8102010 2024-04-23T18:24:10.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.983+0000 7ff2ad802700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ff2940a3cd0 0x7ff2940a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400ac60 tx=0x7ff2a400a370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:10.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:10.987+0000 7ff29effd700 1 -- 172.21.15.19:0/1470034404 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff298013070 con 0x7ff2a8102010 2024-04-23T18:24:11.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:10 smithi044 ceph-mon[32458]: pgmap v303: 97 pgs: 97 active+clean; 621 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-23T18:24:11.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:10 smithi155 ceph-mon[33468]: pgmap v303: 97 pgs: 97 active+clean; 621 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-23T18:24:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.257+0000 7ff2afa66700 1 -- 172.21.15.19:0/1470034404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff2a802df80 con 0x7ff2a8102010 2024-04-23T18:24:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.262+0000 7ff29effd700 1 -- 172.21.15.19:0/1470034404 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264996 (secure 0 0 0) 0x7ff29809db20 con 0x7ff2a8102010 2024-04-23T18:24:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.266+0000 7ff2afa66700 1 -- 172.21.15.19:0/1470034404 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ff2940a3cd0 msgr2=0x7ff2940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.266+0000 7ff2afa66700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ff2940a3cd0 0x7ff2940a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400ac60 tx=0x7ff2a400a370 comp rx=0 tx=0).stop 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.266+0000 7ff2afa66700 1 -- 172.21.15.19:0/1470034404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a8102010 msgr2=0x7ff2a8199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.266+0000 7ff2afa66700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a8102010 0x7ff2a8199640 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff298006010 tx=0x7ff2980047b0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.266+0000 7ff2afa66700 1 -- 172.21.15.19:0/1470034404 shutdown_connections 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.266+0000 7ff2afa66700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ff2940a3cd0 0x7ff2940a6190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.266+0000 7ff2afa66700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2a8106500 0x7ff2a819d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.266+0000 7ff2afa66700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a8101660 0x7ff2a8199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.266+0000 7ff2afa66700 1 --2- 172.21.15.19:0/1470034404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a8102010 0x7ff2a8199640 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.266+0000 7ff2afa66700 1 -- 172.21.15.19:0/1470034404 >> 172.21.15.19:0/1470034404 conn(0x7ff2a8076c60 msgr2=0x7ff2a8104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.267+0000 7ff2afa66700 1 -- 172.21.15.19:0/1470034404 shutdown_connections 2024-04-23T18:24:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:11.267+0000 7ff2afa66700 1 -- 172.21.15.19:0/1470034404 wait complete. 2024-04-23T18:24:11.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:11 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1470034404' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:24:11.993 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":1993809272},{"type":"v1","addr":"172.21.15.19:6801","nonce":1993809272}]},"active_addr":"172.21.15.19:6801/1993809272","active_change":"2024-04-23T18:16:35.264303+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24118,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1108384152}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":983971095}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":304874449}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2556512605}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1619481720}]}]} 2024-04-23T18:24:12.000 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-23T18:24:12.000 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-23T18:24:12.000 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:24:12.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:11 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1470034404' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:24:12.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:11 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1470034404' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:24:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:13.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:12 smithi019 ceph-mon[29215]: pgmap v304: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.5 KiB/s wr, 24 op/s 2024-04-23T18:24:13.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:12 smithi044 ceph-mon[32458]: pgmap v304: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.5 KiB/s wr, 24 op/s 2024-04-23T18:24:13.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:12 smithi155 ceph-mon[33468]: pgmap v304: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.5 KiB/s wr, 24 op/s 2024-04-23T18:24:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.014+0000 7f23d007f700 1 -- 172.21.15.19:0/1754022486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23c8102a90 msgr2=0x7f23c8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.014+0000 7f23d007f700 1 --2- 172.21.15.19:0/1754022486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23c8102a90 0x7f23c8102f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009990 tx=0x7f23bc009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.014+0000 7f23d007f700 1 -- 172.21.15.19:0/1754022486 shutdown_connections 2024-04-23T18:24:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.014+0000 7f23d007f700 1 --2- 172.21.15.19:0/1754022486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c8103450 0x7f23c810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.014+0000 7f23d007f700 1 --2- 172.21.15.19:0/1754022486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8108a50 0x7f23c8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.014+0000 7f23d007f700 1 --2- 172.21.15.19:0/1754022486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23c8102a90 0x7f23c8102f10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.014+0000 7f23d007f700 1 -- 172.21.15.19:0/1754022486 >> 172.21.15.19:0/1754022486 conn(0x7f23c8079ba0 msgr2=0x7f23c8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.015+0000 7f23d007f700 1 -- 172.21.15.19:0/1754022486 shutdown_connections 2024-04-23T18:24:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.015+0000 7f23d007f700 1 -- 172.21.15.19:0/1754022486 wait complete. 2024-04-23T18:24:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.015+0000 7f23d007f700 1 Processor -- start 2024-04-23T18:24:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.015+0000 7f23d007f700 1 -- start start 2024-04-23T18:24:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23d007f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c8102a90 0x7f23c8199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23d007f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8103450 0x7f23c8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23d007f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23c8108a50 0x7f23c819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23d007f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8112be0 con 0x7f23c8108a50 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23d007f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8112a60 con 0x7f23c8102a90 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23d007f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8112d60 con 0x7f23c8103450 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23ce61c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23c8108a50 0x7f23c819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23ce61c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23c8108a50 0x7f23c819da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52694/0 (socket says 172.21.15.19:52694) 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23ce61c700 1 -- 172.21.15.19:0/1927093606 learned_addr learned my addr 172.21.15.19:0/1927093606 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23cd61a700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8103450 0x7f23c8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23cde1b700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c8102a90 0x7f23c8199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23ce61c700 1 -- 172.21.15.19:0/1927093606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8103450 msgr2=0x7f23c8199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23ce61c700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8103450 0x7f23c8199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23ce61c700 1 -- 172.21.15.19:0/1927093606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c8102a90 msgr2=0x7f23c8199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23ce61c700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c8102a90 0x7f23c8199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23ce61c700 1 -- 172.21.15.19:0/1927093606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c819e160 con 0x7f23c8108a50 2024-04-23T18:24:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23cde1b700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c8102a90 0x7f23c8199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.016+0000 7f23ce61c700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23c8108a50 0x7f23c819da20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f23b800d980 tx=0x7f23b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.017+0000 7f23c6ffd700 1 -- 172.21.15.19:0/1927093606 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8016070 con 0x7f23c8108a50 2024-04-23T18:24:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.017+0000 7f23c6ffd700 1 -- 172.21.15.19:0/1927093606 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23b8004830 con 0x7f23c8108a50 2024-04-23T18:24:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.017+0000 7f23d007f700 1 -- 172.21.15.19:0/1927093606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c819e450 con 0x7f23c8108a50 2024-04-23T18:24:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.017+0000 7f23c6ffd700 1 -- 172.21.15.19:0/1927093606 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80054b0 con 0x7f23c8108a50 2024-04-23T18:24:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.017+0000 7f23d007f700 1 -- 172.21.15.19:0/1927093606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c81a5cd0 con 0x7f23c8108a50 2024-04-23T18:24:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.018+0000 7f23d007f700 1 -- 172.21.15.19:0/1927093606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c8104180 con 0x7f23c8108a50 2024-04-23T18:24:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.020+0000 7f23c6ffd700 1 -- 172.21.15.19:0/1927093606 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f23b80049e0 con 0x7f23c8108a50 2024-04-23T18:24:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.020+0000 7f23c6ffd700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23b00a3cd0 0x7f23b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.020+0000 7f23cde1b700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23b00a3cd0 0x7f23b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.020+0000 7f23c6ffd700 1 -- 172.21.15.19:0/1927093606 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f23b80d0250 con 0x7f23c8108a50 2024-04-23T18:24:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.021+0000 7f23cde1b700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23b00a3cd0 0x7f23b00a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009d50 tx=0x7f23c0009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.025+0000 7f23c6ffd700 1 -- 172.21.15.19:0/1927093606 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23b801f020 con 0x7f23c8108a50 2024-04-23T18:24:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.234+0000 7f23d007f700 1 -- 172.21.15.19:0/1927093606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23c819a4b0 con 0x7f23c8108a50 2024-04-23T18:24:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.236+0000 7f23c6ffd700 1 -- 172.21.15.19:0/1927093606 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22209 (secure 0 0 0) 0x7f23b800f3c0 con 0x7f23c8108a50 2024-04-23T18:24:14.229 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:24:14.229 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":86,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:23:57.076186+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:24:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 -- 172.21.15.19:0/1927093606 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23b00a3cd0 msgr2=0x7f23b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23b00a3cd0 0x7f23b00a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009d50 tx=0x7f23c0009040 comp rx=0 tx=0).stop 2024-04-23T18:24:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 -- 172.21.15.19:0/1927093606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23c8108a50 msgr2=0x7f23c819da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23c8108a50 0x7f23c819da20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f23b800d980 tx=0x7f23b800dc90 comp rx=0 tx=0).stop 2024-04-23T18:24:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 -- 172.21.15.19:0/1927093606 shutdown_connections 2024-04-23T18:24:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23b00a3cd0 0x7f23b00a6190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23c8108a50 0x7f23c819da20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c8102a90 0x7f23c8199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 --2- 172.21.15.19:0/1927093606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8103450 0x7f23c8199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 -- 172.21.15.19:0/1927093606 >> 172.21.15.19:0/1927093606 conn(0x7f23c8079ba0 msgr2=0x7f23c81017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:14.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 -- 172.21.15.19:0/1927093606 shutdown_connections 2024-04-23T18:24:14.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:14.238+0000 7f23d007f700 1 -- 172.21.15.19:0/1927093606 wait complete. 2024-04-23T18:24:14.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:14 smithi019 ceph-mon[29215]: pgmap v305: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.1 KiB/s wr, 23 op/s 2024-04-23T18:24:14.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:14 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1927093606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:14.993 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-23T18:24:14.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:24:15.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:14 smithi044 ceph-mon[32458]: pgmap v305: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.1 KiB/s wr, 23 op/s 2024-04-23T18:24:15.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:14 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1927093606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:15.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:14 smithi155 ceph-mon[33468]: pgmap v305: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.1 KiB/s wr, 23 op/s 2024-04-23T18:24:15.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:14 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1927093606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:15.249 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:16.902 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:16 smithi019 ceph-mon[29215]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-23T18:24:17.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:16 smithi044 ceph-mon[32458]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-23T18:24:17.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:16 smithi155 ceph-mon[33468]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-23T18:24:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.176+0000 7f215f897700 1 -- 172.21.15.19:0/1873964625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2158107b00 msgr2=0x7f2158107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.176+0000 7f215f897700 1 --2- 172.21.15.19:0/1873964625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2158107b00 0x7f2158107ee0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2154009990 tx=0x7f2154009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.176+0000 7f215f897700 1 -- 172.21.15.19:0/1873964625 shutdown_connections 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.176+0000 7f215f897700 1 --2- 172.21.15.19:0/1873964625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2158101da0 0x7f215810f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.176+0000 7f215f897700 1 --2- 172.21.15.19:0/1873964625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2158107b00 0x7f2158107ee0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.176+0000 7f215f897700 1 --2- 172.21.15.19:0/1873964625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21581013e0 0x7f2158101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.176+0000 7f215f897700 1 -- 172.21.15.19:0/1873964625 >> 172.21.15.19:0/1873964625 conn(0x7f2158076c60 msgr2=0x7f2158077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.176+0000 7f215f897700 1 -- 172.21.15.19:0/1873964625 shutdown_connections 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.176+0000 7f215f897700 1 -- 172.21.15.19:0/1873964625 wait complete. 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.177+0000 7f215f897700 1 Processor -- start 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.177+0000 7f215f897700 1 -- start start 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215f897700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21581013e0 0x7f21581025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215f897700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2158101da0 0x7f2158102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215f897700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2158107b00 0x7f2158103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215f897700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2158112c60 con 0x7f2158107b00 2024-04-23T18:24:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215f897700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2158112ae0 con 0x7f21581013e0 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215f897700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2158112de0 con 0x7f2158101da0 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215ce32700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2158101da0 0x7f2158102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215ce32700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2158101da0 0x7f2158102b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:45748/0 (socket says 172.21.15.19:45748) 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215ce32700 1 -- 172.21.15.19:0/2074431329 learned_addr learned my addr 172.21.15.19:0/2074431329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215d633700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21581013e0 0x7f21581025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215de34700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2158107b00 0x7f2158103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215ce32700 1 -- 172.21.15.19:0/2074431329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21581013e0 msgr2=0x7f21581025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215ce32700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21581013e0 0x7f21581025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215ce32700 1 -- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2158107b00 msgr2=0x7f2158103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215ce32700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2158107b00 0x7f2158103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215ce32700 1 -- 172.21.15.19:0/2074431329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21581abb40 con 0x7f2158101da0 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215de34700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2158107b00 0x7f2158103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.178+0000 7f215ce32700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2158101da0 0x7f2158102b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f214800d950 tx=0x7f214800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:17.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.179+0000 7f214e7fc700 1 -- 172.21.15.19:0/2074431329 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148016070 con 0x7f2158101da0 2024-04-23T18:24:17.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.179+0000 7f215f897700 1 -- 172.21.15.19:0/2074431329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21581abdd0 con 0x7f2158101da0 2024-04-23T18:24:17.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.179+0000 7f214e7fc700 1 -- 172.21.15.19:0/2074431329 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2148009df0 con 0x7f2158101da0 2024-04-23T18:24:17.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.179+0000 7f214e7fc700 1 -- 172.21.15.19:0/2074431329 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214800f3c0 con 0x7f2158101da0 2024-04-23T18:24:17.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.179+0000 7f215f897700 1 -- 172.21.15.19:0/2074431329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21581ac230 con 0x7f2158101da0 2024-04-23T18:24:17.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.180+0000 7f215f897700 1 -- 172.21.15.19:0/2074431329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215806ac90 con 0x7f2158101da0 2024-04-23T18:24:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.181+0000 7f214e7fc700 1 -- 172.21.15.19:0/2074431329 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2148019070 con 0x7f2158101da0 2024-04-23T18:24:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.182+0000 7f214e7fc700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f21440a3cd0 0x7f21440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.182+0000 7f215d633700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f21440a3cd0 0x7f21440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.182+0000 7f214e7fc700 1 -- 172.21.15.19:0/2074431329 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f21480cfff0 con 0x7f2158101da0 2024-04-23T18:24:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.182+0000 7f215d633700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f21440a3cd0 0x7f21440a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2154006010 tx=0x7f2154005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.186+0000 7f214e7fc700 1 -- 172.21.15.19:0/2074431329 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f214809c920 con 0x7f2158101da0 2024-04-23T18:24:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.394+0000 7f215f897700 1 -- 172.21.15.19:0/2074431329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f215802df80 con 0x7f2158101da0 2024-04-23T18:24:17.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.396+0000 7f214e7fc700 1 -- 172.21.15.19:0/2074431329 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22209 (secure 0 0 0) 0x7f214809c220 con 0x7f2158101da0 2024-04-23T18:24:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:24:17.390 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":86,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:23:57.076186+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:24:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 -- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f21440a3cd0 msgr2=0x7f21440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f21440a3cd0 0x7f21440a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2154006010 tx=0x7f2154005960 comp rx=0 tx=0).stop 2024-04-23T18:24:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 -- 172.21.15.19:0/2074431329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2158101da0 msgr2=0x7f2158102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2158101da0 0x7f2158102b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f214800d950 tx=0x7f214800dc60 comp rx=0 tx=0).stop 2024-04-23T18:24:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 -- 172.21.15.19:0/2074431329 shutdown_connections 2024-04-23T18:24:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f21440a3cd0 0x7f21440a6190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2158107b00 0x7f2158103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21581013e0 0x7f21581025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 --2- 172.21.15.19:0/2074431329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2158101da0 0x7f2158102b00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 -- 172.21.15.19:0/2074431329 >> 172.21.15.19:0/2074431329 conn(0x7f2158076c60 msgr2=0x7f2158100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 -- 172.21.15.19:0/2074431329 shutdown_connections 2024-04-23T18:24:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:17.398+0000 7f215f897700 1 -- 172.21.15.19:0/2074431329 wait complete. 2024-04-23T18:24:18.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:17 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2074431329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:18.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:17 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2074431329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:18.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:17 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2074431329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:18.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-23T18:24:18.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-23T18:24:18.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-23T18:24:18.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-23T18:24:18.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-23T18:24:18.257 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-23T18:24:18.257 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-23T18:24:18.258 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-23T18:24:18.258 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-23T18:24:18.258 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-23T18:24:18.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-23T18:24:18.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-23T18:24:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:18.900 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:18.953 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:19.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:18 smithi019 ceph-mon[29215]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 14 op/s 2024-04-23T18:24:19.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:18 smithi044 ceph-mon[32458]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 14 op/s 2024-04-23T18:24:19.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:18 smithi155 ceph-mon[33468]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 14 op/s 2024-04-23T18:24:21.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:20 smithi019 ceph-mon[29215]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-04-23T18:24:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:21.048 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:21.048 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:21.051 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:21.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:20 smithi044 ceph-mon[32458]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-04-23T18:24:21.063 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:21.070 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:21.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:20 smithi155 ceph-mon[33468]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-04-23T18:24:22.922 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:22 smithi019 ceph-mon[29215]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-23T18:24:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.039+0000 7f27ce7c0700 1 -- 172.21.15.19:0/1678197488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c8108a50 msgr2=0x7f27c8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.039+0000 7f27ce7c0700 1 --2- 172.21.15.19:0/1678197488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c8108a50 0x7f27c8108e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009990 tx=0x7f27b8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.039+0000 7f27ce7c0700 1 -- 172.21.15.19:0/1678197488 shutdown_connections 2024-04-23T18:24:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.039+0000 7f27ce7c0700 1 --2- 172.21.15.19:0/1678197488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c8103450 0x7f27c810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.039+0000 7f27ce7c0700 1 --2- 172.21.15.19:0/1678197488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c8108a50 0x7f27c8108e30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.039+0000 7f27ce7c0700 1 --2- 172.21.15.19:0/1678197488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c8102a90 0x7f27c8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.040+0000 7f27ce7c0700 1 -- 172.21.15.19:0/1678197488 >> 172.21.15.19:0/1678197488 conn(0x7f27c8079ba0 msgr2=0x7f27c8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.040+0000 7f27ce7c0700 1 -- 172.21.15.19:0/1678197488 shutdown_connections 2024-04-23T18:24:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.040+0000 7f27ce7c0700 1 -- 172.21.15.19:0/1678197488 wait complete. 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.040+0000 7f27ce7c0700 1 Processor -- start 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.040+0000 7f27ce7c0700 1 -- start start 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27ce7c0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c8102a90 0x7f27c8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27ce7c0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c8103450 0x7f27c8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27ce7c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c8108a50 0x7f27c819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27ce7c0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8112be0 con 0x7f27c8102a90 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27ce7c0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8112a60 con 0x7f27c8103450 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27ce7c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8112d60 con 0x7f27c8108a50 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c8102a90 0x7f27c8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c8102a90 0x7f27c8199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60340/0 (socket says 172.21.15.19:60340) 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c7fff700 1 -- 172.21.15.19:0/725630770 learned_addr learned my addr 172.21.15.19:0/725630770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c77fe700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c8103450 0x7f27c8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27ccd5d700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c8108a50 0x7f27c819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c7fff700 1 -- 172.21.15.19:0/725630770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c8108a50 msgr2=0x7f27c819d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c7fff700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c8108a50 0x7f27c819d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c7fff700 1 -- 172.21.15.19:0/725630770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c8103450 msgr2=0x7f27c8199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c7fff700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c8103450 0x7f27c8199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c7fff700 1 -- 172.21.15.19:0/725630770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c819e120 con 0x7f27c8102a90 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c77fe700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c8103450 0x7f27c8199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27ccd5d700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c8108a50 0x7f27c819d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c7fff700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c8102a90 0x7f27c8199110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f27b80057f0 tx=0x7f27b8003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.041+0000 7f27c57fa700 1 -- 172.21.15.19:0/725630770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8027070 con 0x7f27c8102a90 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.042+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c819e3b0 con 0x7f27c8102a90 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.042+0000 7f27c57fa700 1 -- 172.21.15.19:0/725630770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b80041c0 con 0x7f27c8102a90 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.042+0000 7f27c57fa700 1 -- 172.21.15.19:0/725630770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b800f460 con 0x7f27c8102a90 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.042+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c81a5c90 con 0x7f27c8102a90 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.042+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27c8063150 con 0x7f27c8102a90 2024-04-23T18:24:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.044+0000 7f27c57fa700 1 -- 172.21.15.19:0/725630770 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f27b802a070 con 0x7f27c8102a90 2024-04-23T18:24:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.044+0000 7f27c57fa700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27b00a3cd0 0x7f27b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.044+0000 7f27c77fe700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27b00a3cd0 0x7f27b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.044+0000 7f27c57fa700 1 -- 172.21.15.19:0/725630770 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f27b80d1e60 con 0x7f27c8102a90 2024-04-23T18:24:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.045+0000 7f27c57fa700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] conn(0x7f27b00ad4b0 0x7f27b00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.045+0000 7f27c57fa700 1 -- 172.21.15.19:0/725630770 --> [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27b800b0b0 con 0x7f27b00ad4b0 2024-04-23T18:24:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.045+0000 7f27c57fa700 1 -- 172.21.15.19:0/725630770 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f27b80d2340 con 0x7f27c8102a90 2024-04-23T18:24:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.045+0000 7f27c77fe700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27b00a3cd0 0x7f27b00a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f27bc005fd0 tx=0x7f27bc005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.045+0000 7f27ccd5d700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] conn(0x7f27b00ad4b0 0x7f27b00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.045+0000 7f27ccd5d700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] conn(0x7f27b00ad4b0 0x7f27b00af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:23.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.046+0000 7f27c57fa700 1 -- 172.21.15.19:0/725630770 <== osd.10 v2:172.21.15.44:6824/2033313090 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27b800b0b0 con 0x7f27b00ad4b0 2024-04-23T18:24:23.048 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.065+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 --> [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f27c80635d0 con 0x7f27b00ad4b0 2024-04-23T18:24:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.065+0000 7f27c57fa700 1 -- 172.21.15.19:0/725630770 <== osd.10 v2:172.21.15.44:6824/2033313090 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f27c80635d0 con 0x7f27b00ad4b0 2024-04-23T18:24:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 >> [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] conn(0x7f27b00ad4b0 msgr2=0x7f27b00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ce7c0700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] conn(0x7f27b00ad4b0 0x7f27b00af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27b00a3cd0 msgr2=0x7f27b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ce7c0700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f27b00a3cd0 0x7f27b00a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f27bc005fd0 tx=0x7f27bc005e20 comp rx=0 tx=0).stop 2024-04-23T18:24:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c8102a90 msgr2=0x7f27c8199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ce7c0700 1 --2- 172.21.15.19:0/725630770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c8102a90 0x7f27c8199110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f27b80057f0 tx=0x7f27b8003730 comp rx=0 tx=0).stop 2024-04-23T18:24:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ccd5d700 1 -- 172.21.15.19:0/725630770 reap_dead start 2024-04-23T18:24:23.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 shutdown_connections 2024-04-23T18:24:23.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 >> 172.21.15.19:0/725630770 conn(0x7f27c8079ba0 msgr2=0x7f27c8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:23.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 shutdown_connections 2024-04-23T18:24:23.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:23.066+0000 7f27ce7c0700 1 -- 172.21.15.19:0/725630770 wait complete. 2024-04-23T18:24:23.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:22 smithi044 ceph-mon[32458]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-23T18:24:23.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:22 smithi155 ceph-mon[33468]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-23T18:24:24.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:24 smithi019 ceph-mon[29215]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-23T18:24:25.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:24 smithi044 ceph-mon[32458]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-23T18:24:25.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:24 smithi155 ceph-mon[33468]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-23T18:24:25.102 INFO:teuthology.orchestra.run.smithi019.stdout:313532612632 2024-04-23T18:24:25.103 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-23T18:24:26.933 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:26 smithi019 ceph-mon[29215]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-23T18:24:27.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:26 smithi044 ceph-mon[32458]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-23T18:24:27.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:26 smithi155 ceph-mon[33468]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-23T18:24:28.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:28 smithi019 ceph-mon[29215]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T18:24:29.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:28 smithi044 ceph-mon[32458]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T18:24:29.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:28 smithi155 ceph-mon[33468]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T18:24:30.947 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:30 smithi019 ceph-mon[29215]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T18:24:31.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:30 smithi044 ceph-mon[32458]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T18:24:31.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:30 smithi155 ceph-mon[33468]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T18:24:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.616+0000 7f99523df700 1 -- 172.21.15.19:0/433714781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994c06a750 msgr2=0x7f994c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.616+0000 7f99523df700 1 --2- 172.21.15.19:0/433714781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994c06a750 0x7f994c06abb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9934009990 tx=0x7f9934009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.616+0000 7f99523df700 1 -- 172.21.15.19:0/433714781 shutdown_connections 2024-04-23T18:24:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.616+0000 7f99523df700 1 --2- 172.21.15.19:0/433714781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f994c06b0f0 0x7f994c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.616+0000 7f99523df700 1 --2- 172.21.15.19:0/433714781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c108a30 0x7f994c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.616+0000 7f99523df700 1 --2- 172.21.15.19:0/433714781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994c06a750 0x7f994c06abb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.616+0000 7f99523df700 1 -- 172.21.15.19:0/433714781 >> 172.21.15.19:0/433714781 conn(0x7f994c0788f0 msgr2=0x7f994c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.616+0000 7f99523df700 1 -- 172.21.15.19:0/433714781 shutdown_connections 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.616+0000 7f99523df700 1 -- 172.21.15.19:0/433714781 wait complete. 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f99523df700 1 Processor -- start 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f99523df700 1 -- start start 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f99523df700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c06a750 0x7f994c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f99523df700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994c06b0f0 0x7f994c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f99523df700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f994c108a30 0x7f994c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f99523df700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f994c112cc0 con 0x7f994c06b0f0 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f99523df700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f994c112b40 con 0x7f994c06a750 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f99523df700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f994c112e40 con 0x7f994c108a30 2024-04-23T18:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f994b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994c06b0f0 0x7f994c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f994b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994c06b0f0 0x7f994c199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53960/0 (socket says 172.21.15.19:53960) 2024-04-23T18:24:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f994b7fe700 1 -- 172.21.15.19:0/3224641170 learned_addr learned my addr 172.21.15.19:0/3224641170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f994bfff700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c06a750 0x7f994c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.617+0000 7f995097c700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f994c108a30 0x7f994c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f994b7fe700 1 -- 172.21.15.19:0/3224641170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f994c108a30 msgr2=0x7f994c19db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f994b7fe700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f994c108a30 0x7f994c19db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f994b7fe700 1 -- 172.21.15.19:0/3224641170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c06a750 msgr2=0x7f994c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f994b7fe700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c06a750 0x7f994c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f994b7fe700 1 -- 172.21.15.19:0/3224641170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c19e240 con 0x7f994c06b0f0 2024-04-23T18:24:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f994b7fe700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994c06b0f0 0x7f994c199770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9934005910 tx=0x7f99340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f99497fa700 1 -- 172.21.15.19:0/3224641170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934027070 con 0x7f994c06b0f0 2024-04-23T18:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f99497fa700 1 -- 172.21.15.19:0/3224641170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9934003f20 con 0x7f994c06b0f0 2024-04-23T18:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c19e470 con 0x7f994c06b0f0 2024-04-23T18:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f99497fa700 1 -- 172.21.15.19:0/3224641170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934022c60 con 0x7f994c06b0f0 2024-04-23T18:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.618+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c101940 con 0x7f994c06b0f0 2024-04-23T18:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.619+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f994c063150 con 0x7f994c06b0f0 2024-04-23T18:24:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.620+0000 7f99497fa700 1 -- 172.21.15.19:0/3224641170 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f993402a070 con 0x7f994c06b0f0 2024-04-23T18:24:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.621+0000 7f99497fa700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99380a3d50 0x7f99380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.621+0000 7f994bfff700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99380a3d50 0x7f99380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.621+0000 7f99497fa700 1 -- 172.21.15.19:0/3224641170 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9934022e00 con 0x7f994c06b0f0 2024-04-23T18:24:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.621+0000 7f99497fa700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] conn(0x7f99380ad530 0x7f99380af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.621+0000 7f99497fa700 1 -- 172.21.15.19:0/3224641170 --> [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99340056d0 con 0x7f99380ad530 2024-04-23T18:24:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.621+0000 7f99497fa700 1 -- 172.21.15.19:0/3224641170 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f99340d1ed0 con 0x7f994c06b0f0 2024-04-23T18:24:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.622+0000 7f994bfff700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99380a3d50 0x7f99380a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f993c005950 tx=0x7f993c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:31.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.622+0000 7f995097c700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] conn(0x7f99380ad530 0x7f99380af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.622+0000 7f995097c700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] conn(0x7f99380ad530 0x7f99380af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:31.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.623+0000 7f99497fa700 1 -- 172.21.15.19:0/3224641170 <== osd.8 v2:172.21.15.155:6816/3480056050 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99340056d0 con 0x7f99380ad530 2024-04-23T18:24:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.639+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 --> [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f994c063550 con 0x7f99380ad530 2024-04-23T18:24:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.640+0000 7f99497fa700 1 -- 172.21.15.19:0/3224641170 <== osd.8 v2:172.21.15.155:6816/3480056050 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f994c063550 con 0x7f99380ad530 2024-04-23T18:24:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 >> [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] conn(0x7f99380ad530 msgr2=0x7f99380af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f99523df700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] conn(0x7f99380ad530 0x7f99380af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99380a3d50 msgr2=0x7f99380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f99523df700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99380a3d50 0x7f99380a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f993c005950 tx=0x7f993c00b410 comp rx=0 tx=0).stop 2024-04-23T18:24:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994c06b0f0 msgr2=0x7f994c199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f99523df700 1 --2- 172.21.15.19:0/3224641170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f994c06b0f0 0x7f994c199770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9934005910 tx=0x7f99340047b0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f995097c700 1 -- 172.21.15.19:0/3224641170 reap_dead start 2024-04-23T18:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 shutdown_connections 2024-04-23T18:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 >> 172.21.15.19:0/3224641170 conn(0x7f994c0788f0 msgr2=0x7f994c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 shutdown_connections 2024-04-23T18:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.641+0000 7f99523df700 1 -- 172.21.15.19:0/3224641170 wait complete. 2024-04-23T18:24:31.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.497+0000 7f0d546cc700 1 -- 172.21.15.19:0/3716730563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c10f490 msgr2=0x7f0d4c111880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.497+0000 7f0d546cc700 1 --2- 172.21.15.19:0/3716730563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c10f490 0x7f0d4c111880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400b0a0 tx=0x7f0d4400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.497+0000 7f0d546cc700 1 -- 172.21.15.19:0/3716730563 shutdown_connections 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.497+0000 7f0d546cc700 1 --2- 172.21.15.19:0/3716730563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c10f490 0x7f0d4c111880 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.497+0000 7f0d546cc700 1 --2- 172.21.15.19:0/3716730563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c06b1f0 0x7f0d4c06b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.497+0000 7f0d546cc700 1 --2- 172.21.15.19:0/3716730563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06bb10 0x7f0d4c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.497+0000 7f0d546cc700 1 -- 172.21.15.19:0/3716730563 >> 172.21.15.19:0/3716730563 conn(0x7f0d4c0789e0 msgr2=0x7f0d4c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.497+0000 7f0d546cc700 1 -- 172.21.15.19:0/3716730563 shutdown_connections 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.497+0000 7f0d546cc700 1 -- 172.21.15.19:0/3716730563 wait complete. 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d546cc700 1 Processor -- start 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d546cc700 1 -- start start 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d546cc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c06b1f0 0x7f0d4c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d546cc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06bb10 0x7f0d4c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d546cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c10f490 0x7f0d4c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d546cc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c114e50 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d546cc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c114cd0 con 0x7f0d4c10f490 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d546cc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c114fd0 con 0x7f0d4c06bb10 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d52468700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c06b1f0 0x7f0d4c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d52468700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c06b1f0 0x7f0d4c19b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53944/0 (socket says 172.21.15.19:53944) 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d52468700 1 -- 172.21.15.19:0/3318551489 learned_addr learned my addr 172.21.15.19:0/3318551489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d51c67700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06bb10 0x7f0d4c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d52468700 1 -- 172.21.15.19:0/3318551489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06bb10 msgr2=0x7f0d4c19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d52468700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06bb10 0x7f0d4c19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d52468700 1 -- 172.21.15.19:0/3318551489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c10f490 msgr2=0x7f0d4c19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d52468700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d4c10f490 0x7f0d4c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d52468700 1 -- 172.21.15.19:0/3318551489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c1a0280 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.498+0000 7f0d51c67700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c06bb10 0x7f0d4c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:31.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.499+0000 7f0d52468700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c06b1f0 0x7f0d4c19b2c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000c9b0 tx=0x7f0d4000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:31.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.499+0000 7f0d3f7fe700 1 -- 172.21.15.19:0/3318551489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40007b00 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.499+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c1a0570 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.499+0000 7f0d3f7fe700 1 -- 172.21.15.19:0/3318551489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d40007ca0 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.499+0000 7f0d3f7fe700 1 -- 172.21.15.19:0/3318551489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40018730 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.499+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c1a7df0 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.500+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d4c063150 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.501+0000 7f0d3f7fe700 1 -- 172.21.15.19:0/3318551489 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0d4000f450 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.501+0000 7f0d3f7fe700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d380a3cd0 0x7f0d380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.502+0000 7f0d51c67700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d380a3cd0 0x7f0d380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.502+0000 7f0d3f7fe700 1 -- 172.21.15.19:0/3318551489 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0d400d08f0 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.502+0000 7f0d3f7fe700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] conn(0x7f0d380ad4b0 0x7f0d380af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.502+0000 7f0d3f7fe700 1 -- 172.21.15.19:0/3318551489 --> [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d40004500 con 0x7f0d380ad4b0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.502+0000 7f0d3f7fe700 1 -- 172.21.15.19:0/3318551489 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0d400d0dd0 con 0x7f0d4c06b1f0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.502+0000 7f0d51c67700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d380a3cd0 0x7f0d380a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c102e40 tx=0x7f0d48009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.502+0000 7f0d52c69700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] conn(0x7f0d380ad4b0 0x7f0d380af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.503+0000 7f0d52c69700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] conn(0x7f0d380ad4b0 0x7f0d380af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.504+0000 7f0d3f7fe700 1 -- 172.21.15.19:0/3318551489 <== osd.2 v2:172.21.15.155:6800/3680785490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d40004500 con 0x7f0d380ad4b0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.521+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 --> [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d4c063550 con 0x7f0d380ad4b0 2024-04-23T18:24:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.521+0000 7f0d3f7fe700 1 -- 172.21.15.19:0/3318551489 <== osd.2 v2:172.21.15.155:6800/3680785490 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0d4c063550 con 0x7f0d380ad4b0 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 >> [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] conn(0x7f0d380ad4b0 msgr2=0x7f0d380af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d546cc700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] conn(0x7f0d380ad4b0 0x7f0d380af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d380a3cd0 msgr2=0x7f0d380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d546cc700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0d380a3cd0 0x7f0d380a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c102e40 tx=0x7f0d48009550 comp rx=0 tx=0).stop 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c06b1f0 msgr2=0x7f0d4c19b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d546cc700 1 --2- 172.21.15.19:0/3318551489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c06b1f0 0x7f0d4c19b2c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000c9b0 tx=0x7f0d4000cd70 comp rx=0 tx=0).stop 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d52c69700 1 -- 172.21.15.19:0/3318551489 reap_dead start 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 shutdown_connections 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 >> 172.21.15.19:0/3318551489 conn(0x7f0d4c0789e0 msgr2=0x7f0d4c110840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 shutdown_connections 2024-04-23T18:24:31.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:31.522+0000 7f0d546cc700 1 -- 172.21.15.19:0/3318551489 wait complete. 2024-04-23T18:24:33.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:32 smithi019 ceph-mon[29215]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T18:24:33.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:32 smithi044 ceph-mon[32458]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T18:24:33.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:32 smithi155 ceph-mon[33468]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T18:24:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.057+0000 7f123b387700 1 -- 172.21.15.19:0/2899695801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f123406a750 msgr2=0x7f123406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.057+0000 7f123b387700 1 --2- 172.21.15.19:0/2899695801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f123406a750 0x7f123406abb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1224009990 tx=0x7f1224009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.057+0000 7f123b387700 1 -- 172.21.15.19:0/2899695801 shutdown_connections 2024-04-23T18:24:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.057+0000 7f123b387700 1 --2- 172.21.15.19:0/2899695801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f123406b0f0 0x7f123410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.057+0000 7f123b387700 1 --2- 172.21.15.19:0/2899695801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1234108a30 0x7f1234108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.057+0000 7f123b387700 1 --2- 172.21.15.19:0/2899695801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f123406a750 0x7f123406abb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.057+0000 7f123b387700 1 -- 172.21.15.19:0/2899695801 >> 172.21.15.19:0/2899695801 conn(0x7f12340788f0 msgr2=0x7f1234078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.057+0000 7f123b387700 1 -- 172.21.15.19:0/2899695801 shutdown_connections 2024-04-23T18:24:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.057+0000 7f123b387700 1 -- 172.21.15.19:0/2899695801 wait complete. 2024-04-23T18:24:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f123b387700 1 Processor -- start 2024-04-23T18:24:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f123b387700 1 -- start start 2024-04-23T18:24:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f123b387700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123406a750 0x7f12341992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f123b387700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f123406b0f0 0x7f12341997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f123b387700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1234108a30 0x7f123419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f123b387700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1234112cb0 con 0x7f123406b0f0 2024-04-23T18:24:34.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f123b387700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1234112b30 con 0x7f1234108a30 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f123b387700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1234112e30 con 0x7f123406a750 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f1238922700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f123406b0f0 0x7f12341997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f1238922700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f123406b0f0 0x7f12341997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53990/0 (socket says 172.21.15.19:53990) 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f1238922700 1 -- 172.21.15.19:0/2423792932 learned_addr learned my addr 172.21.15.19:0/2423792932 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.058+0000 7f1239123700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123406a750 0x7f12341992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f1239924700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1234108a30 0x7f123419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f1238922700 1 -- 172.21.15.19:0/2423792932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123406a750 msgr2=0x7f12341992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f1238922700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123406a750 0x7f12341992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f1238922700 1 -- 172.21.15.19:0/2423792932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1234108a30 msgr2=0x7f123419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f1238922700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1234108a30 0x7f123419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f1238922700 1 -- 172.21.15.19:0/2423792932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123419e2c0 con 0x7f123406b0f0 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f1238922700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f123406b0f0 0x7f12341997f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1224000c00 tx=0x7f12240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f122a7fc700 1 -- 172.21.15.19:0/2423792932 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224027070 con 0x7f123406b0f0 2024-04-23T18:24:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123419e550 con 0x7f123406b0f0 2024-04-23T18:24:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f122a7fc700 1 -- 172.21.15.19:0/2423792932 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12240040e0 con 0x7f123406b0f0 2024-04-23T18:24:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f122a7fc700 1 -- 172.21.15.19:0/2423792932 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122400f460 con 0x7f123406b0f0 2024-04-23T18:24:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.059+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12341a5e30 con 0x7f123406b0f0 2024-04-23T18:24:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.060+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1234063150 con 0x7f123406b0f0 2024-04-23T18:24:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.061+0000 7f122a7fc700 1 -- 172.21.15.19:0/2423792932 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f122402a070 con 0x7f123406b0f0 2024-04-23T18:24:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.062+0000 7f122a7fc700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f12200a3cd0 0x7f12200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.062+0000 7f1239123700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f12200a3cd0 0x7f12200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.062+0000 7f122a7fc700 1 -- 172.21.15.19:0/2423792932 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f12240d1a80 con 0x7f123406b0f0 2024-04-23T18:24:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.062+0000 7f122a7fc700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] conn(0x7f12200ad4b0 0x7f12200af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.062+0000 7f122a7fc700 1 -- 172.21.15.19:0/2423792932 --> [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1224011ec0 con 0x7f12200ad4b0 2024-04-23T18:24:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.062+0000 7f122a7fc700 1 -- 172.21.15.19:0/2423792932 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f12240d1f60 con 0x7f123406b0f0 2024-04-23T18:24:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.063+0000 7f1239924700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] conn(0x7f12200ad4b0 0x7f12200af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.063+0000 7f1239123700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f12200a3cd0 0x7f12200a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1230009a10 tx=0x7f1230009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.063+0000 7f1239924700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] conn(0x7f12200ad4b0 0x7f12200af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.064+0000 7f122a7fc700 1 -- 172.21.15.19:0/2423792932 <== osd.4 v2:172.21.15.44:6808/3405107831 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1224011ec0 con 0x7f12200ad4b0 2024-04-23T18:24:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.080+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 --> [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1234063550 con 0x7f12200ad4b0 2024-04-23T18:24:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.081+0000 7f122a7fc700 1 -- 172.21.15.19:0/2423792932 <== osd.4 v2:172.21.15.44:6808/3405107831 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1234063550 con 0x7f12200ad4b0 2024-04-23T18:24:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.081+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 >> [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] conn(0x7f12200ad4b0 msgr2=0x7f12200af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.081+0000 7f123b387700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] conn(0x7f12200ad4b0 0x7f12200af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.081+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f12200a3cd0 msgr2=0x7f12200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.081+0000 7f123b387700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f12200a3cd0 0x7f12200a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1230009a10 tx=0x7f1230009190 comp rx=0 tx=0).stop 2024-04-23T18:24:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.081+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f123406b0f0 msgr2=0x7f12341997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.081+0000 7f123b387700 1 --2- 172.21.15.19:0/2423792932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f123406b0f0 0x7f12341997f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1224000c00 tx=0x7f12240047b0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.081+0000 7f1239924700 1 -- 172.21.15.19:0/2423792932 reap_dead start 2024-04-23T18:24:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.082+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 shutdown_connections 2024-04-23T18:24:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.082+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 >> 172.21.15.19:0/2423792932 conn(0x7f12340788f0 msgr2=0x7f12340ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.082+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 shutdown_connections 2024-04-23T18:24:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:34.082+0000 7f123b387700 1 -- 172.21.15.19:0/2423792932 wait complete. 2024-04-23T18:24:35.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:34 smithi019 ceph-mon[29215]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T18:24:35.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:34 smithi044 ceph-mon[32458]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T18:24:35.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:34 smithi155 ceph-mon[33468]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T18:24:35.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-04-23T18:24:35.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-23T18:24:35.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-23T18:24:35.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-23T18:24:35.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-23T18:24:35.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-23T18:24:35.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-23T18:24:35.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:35 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-23T18:24:36.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-04-23T18:24:36.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-23T18:24:36.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-23T18:24:36.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-23T18:24:36.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-23T18:24:36.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-23T18:24:36.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-23T18:24:36.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:35 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-23T18:24:36.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-04-23T18:24:36.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-23T18:24:36.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-23T18:24:36.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-23T18:24:36.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-23T18:24:36.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-23T18:24:36.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-23T18:24:36.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:35 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-23T18:24:36.216 INFO:teuthology.orchestra.run.smithi019.stdout:257698037797 2024-04-23T18:24:36.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-23T18:24:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 -- 172.21.15.19:0/1092815423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed0107990 msgr2=0x7efed0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 --2- 172.21.15.19:0/1092815423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed0107990 0x7efed0107d70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efec4009990 tx=0x7efec4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 -- 172.21.15.19:0/1092815423 shutdown_connections 2024-04-23T18:24:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 --2- 172.21.15.19:0/1092815423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed0102300 0x7efed010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 --2- 172.21.15.19:0/1092815423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed0107990 0x7efed0107d70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 --2- 172.21.15.19:0/1092815423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed0101940 0x7efed0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 -- 172.21.15.19:0/1092815423 >> 172.21.15.19:0/1092815423 conn(0x7efed00788f0 msgr2=0x7efed0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 -- 172.21.15.19:0/1092815423 shutdown_connections 2024-04-23T18:24:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 -- 172.21.15.19:0/1092815423 wait complete. 2024-04-23T18:24:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 Processor -- start 2024-04-23T18:24:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.626+0000 7efed721e700 1 -- start start 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed721e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed0101940 0x7efed01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed721e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed0102300 0x7efed0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed721e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed0107990 0x7efed019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed721e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efed0111c20 con 0x7efed0101940 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed721e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efed0111aa0 con 0x7efed0102300 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed721e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efed0111da0 con 0x7efed0107990 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed4fba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed0101940 0x7efed01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed4fba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed0101940 0x7efed01992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54000/0 (socket says 172.21.15.19:54000) 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed4fba700 1 -- 172.21.15.19:0/2233407053 learned_addr learned my addr 172.21.15.19:0/2233407053 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efecffff700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed0102300 0x7efed0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed57bb700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed0107990 0x7efed019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed4fba700 1 -- 172.21.15.19:0/2233407053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed0107990 msgr2=0x7efed019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed4fba700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed0107990 0x7efed019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed4fba700 1 -- 172.21.15.19:0/2233407053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed0102300 msgr2=0x7efed0199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed4fba700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed0102300 0x7efed0199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed4fba700 1 -- 172.21.15.19:0/2233407053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed019e2a0 con 0x7efed0101940 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed57bb700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed0107990 0x7efed019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efecffff700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed0102300 0x7efed0199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efed4fba700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed0101940 0x7efed01992e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efec4005910 tx=0x7efec4003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.627+0000 7efecdffb700 1 -- 172.21.15.19:0/2233407053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4027070 con 0x7efed0101940 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.628+0000 7efecdffb700 1 -- 172.21.15.19:0/2233407053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec4004140 con 0x7efed0101940 2024-04-23T18:24:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.628+0000 7efecdffb700 1 -- 172.21.15.19:0/2233407053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec400f460 con 0x7efed0101940 2024-04-23T18:24:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.628+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed019e530 con 0x7efed0101940 2024-04-23T18:24:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.628+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed0103420 con 0x7efed0101940 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.629+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efed019e9c0 con 0x7efed0101940 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.630+0000 7efecdffb700 1 -- 172.21.15.19:0/2233407053 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efec402a070 con 0x7efed0101940 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.631+0000 7efecdffb700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efec00a3d50 0x7efec00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.631+0000 7efecffff700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efec00a3d50 0x7efec00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.631+0000 7efecdffb700 1 -- 172.21.15.19:0/2233407053 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7efec40d1fd0 con 0x7efed0101940 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.631+0000 7efecdffb700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] conn(0x7efec00ad890 0x7efec00afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.631+0000 7efecdffb700 1 -- 172.21.15.19:0/2233407053 --> [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efec00b0400 con 0x7efec00ad890 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.631+0000 7efecdffb700 1 -- 172.21.15.19:0/2233407053 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7efec40d2380 con 0x7efed0101940 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.631+0000 7efecffff700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efec00a3d50 0x7efec00a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efed0078f80 tx=0x7efebc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.631+0000 7efed57bb700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] conn(0x7efec00ad890 0x7efec00afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.632+0000 7efed57bb700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] conn(0x7efec00ad890 0x7efec00afd30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:36.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.633+0000 7efecdffb700 1 -- 172.21.15.19:0/2233407053 <== osd.5 v2:172.21.15.155:6808/2897505924 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efec00b0400 con 0x7efec00ad890 2024-04-23T18:24:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.649+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 --> [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efed0106c30 con 0x7efec00ad890 2024-04-23T18:24:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.650+0000 7efecdffb700 1 -- 172.21.15.19:0/2233407053 <== osd.5 v2:172.21.15.155:6808/2897505924 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efed0106c30 con 0x7efec00ad890 2024-04-23T18:24:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.650+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 >> [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] conn(0x7efec00ad890 msgr2=0x7efec00afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.650+0000 7efed721e700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] conn(0x7efec00ad890 0x7efec00afd30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.650+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efec00a3d50 msgr2=0x7efec00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.650+0000 7efed721e700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efec00a3d50 0x7efec00a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efed0078f80 tx=0x7efebc008040 comp rx=0 tx=0).stop 2024-04-23T18:24:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.650+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed0101940 msgr2=0x7efed01992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.650+0000 7efed721e700 1 --2- 172.21.15.19:0/2233407053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed0101940 0x7efed01992e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efec4005910 tx=0x7efec4003730 comp rx=0 tx=0).stop 2024-04-23T18:24:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.650+0000 7efed57bb700 1 -- 172.21.15.19:0/2233407053 reap_dead start 2024-04-23T18:24:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.650+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 shutdown_connections 2024-04-23T18:24:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.650+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 >> 172.21.15.19:0/2233407053 conn(0x7efed00788f0 msgr2=0x7efed006a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.651+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 shutdown_connections 2024-04-23T18:24:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:36.651+0000 7efed721e700 1 -- 172.21.15.19:0/2233407053 wait complete. 2024-04-23T18:24:36.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:36 smithi019 ceph-mon[29215]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T18:24:36.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-04-23T18:24:36.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-23T18:24:36.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-23T18:24:36.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-23T18:24:36.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-23T18:24:36.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-23T18:24:36.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-23T18:24:36.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:36 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-23T18:24:36.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:36 smithi019 ceph-mon[29215]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:24:37.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:36 smithi044 ceph-mon[32458]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T18:24:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-04-23T18:24:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-23T18:24:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-23T18:24:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-23T18:24:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-23T18:24:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-23T18:24:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-23T18:24:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:36 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-23T18:24:37.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:36 smithi044 ceph-mon[32458]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:24:37.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:36 smithi155 ceph-mon[33468]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T18:24:37.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-04-23T18:24:37.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-23T18:24:37.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-23T18:24:37.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-23T18:24:37.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-23T18:24:37.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-23T18:24:37.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-23T18:24:37.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:36 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-23T18:24:37.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:36 smithi155 ceph-mon[33468]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:24:37.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.247+0000 7f63f1a93700 1 -- 172.21.15.19:0/3097752037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec10f4f0 msgr2=0x7f63ec1118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.247+0000 7f63f1a93700 1 --2- 172.21.15.19:0/3097752037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec10f4f0 0x7f63ec1118e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b0a0 tx=0x7f63e400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.248+0000 7f63f1a93700 1 -- 172.21.15.19:0/3097752037 shutdown_connections 2024-04-23T18:24:37.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.248+0000 7f63f1a93700 1 --2- 172.21.15.19:0/3097752037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec10f4f0 0x7f63ec1118e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.248+0000 7f63f1a93700 1 --2- 172.21.15.19:0/3097752037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ec06a750 0x7f63ec06ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.248+0000 7f63f1a93700 1 --2- 172.21.15.19:0/3097752037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63ec06b070 0x7f63ec10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.248+0000 7f63f1a93700 1 -- 172.21.15.19:0/3097752037 >> 172.21.15.19:0/3097752037 conn(0x7f63ec0788f0 msgr2=0x7f63ec078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.248+0000 7f63f1a93700 1 -- 172.21.15.19:0/3097752037 shutdown_connections 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.248+0000 7f63f1a93700 1 -- 172.21.15.19:0/3097752037 wait complete. 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.248+0000 7f63f1a93700 1 Processor -- start 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.248+0000 7f63f1a93700 1 -- start start 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63f1a93700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ec06a750 0x7f63ec19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63f1a93700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63ec06b070 0x7f63ec19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63f1a93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec10f4f0 0x7f63ec19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63f1a93700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec114e50 con 0x7f63ec10f4f0 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63f1a93700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec114cd0 con 0x7f63ec06b070 2024-04-23T18:24:37.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63f1a93700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec114fd0 con 0x7f63ec06a750 2024-04-23T18:24:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63ebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec10f4f0 0x7f63ec19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63ebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec10f4f0 0x7f63ec19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54020/0 (socket says 172.21.15.19:54020) 2024-04-23T18:24:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63ebfff700 1 -- 172.21.15.19:0/3047775336 learned_addr learned my addr 172.21.15.19:0/3047775336 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63eb7fe700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ec06a750 0x7f63ec19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63eaffd700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63ec06b070 0x7f63ec19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63ebfff700 1 -- 172.21.15.19:0/3047775336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ec06a750 msgr2=0x7f63ec19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63ebfff700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ec06a750 0x7f63ec19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63ebfff700 1 -- 172.21.15.19:0/3047775336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63ec06b070 msgr2=0x7f63ec19b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63ebfff700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63ec06b070 0x7f63ec19b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63ebfff700 1 -- 172.21.15.19:0/3047775336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec1a03f0 con 0x7f63ec10f4f0 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63eb7fe700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ec06a750 0x7f63ec19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63eaffd700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63ec06b070 0x7f63ec19b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63ebfff700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec10f4f0 0x7f63ec19fcb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f63e4006070 tx=0x7f63e4004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63e8ff9700 1 -- 172.21.15.19:0/3047775336 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4021070 con 0x7f63ec10f4f0 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ec1a0680 con 0x7f63ec10f4f0 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63e8ff9700 1 -- 172.21.15.19:0/3047775336 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e4007d70 con 0x7f63ec10f4f0 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63e8ff9700 1 -- 172.21.15.19:0/3047775336 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e401d450 con 0x7f63ec10f4f0 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.249+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ec1a7f60 con 0x7f63ec10f4f0 2024-04-23T18:24:37.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.250+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63ec063110 con 0x7f63ec10f4f0 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.252+0000 7f63e8ff9700 1 -- 172.21.15.19:0/3047775336 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f63e4004000 con 0x7f63ec10f4f0 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.252+0000 7f63e8ff9700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63d40a3cd0 0x7f63d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.252+0000 7f63e8ff9700 1 -- 172.21.15.19:0/3047775336 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f63e400f020 con 0x7f63ec10f4f0 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.252+0000 7f63eb7fe700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63d40a3cd0 0x7f63d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.252+0000 7f63e8ff9700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] conn(0x7f63d40ad810 0x7f63d40afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.252+0000 7f63e8ff9700 1 -- 172.21.15.19:0/3047775336 --> [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63e402ee50 con 0x7f63d40ad810 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.253+0000 7f63e8ff9700 1 -- 172.21.15.19:0/3047775336 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f63e40d2250 con 0x7f63ec10f4f0 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.253+0000 7f63eaffd700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] conn(0x7f63d40ad810 0x7f63d40afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.253+0000 7f63eb7fe700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63d40a3cd0 0x7f63d40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0099d0 tx=0x7f63dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.253+0000 7f63eaffd700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] conn(0x7f63d40ad810 0x7f63d40afcb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.254+0000 7f63e8ff9700 1 -- 172.21.15.19:0/3047775336 <== osd.0 v2:172.21.15.19:6802/4279899748 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63e402ee50 con 0x7f63d40ad810 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.272+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 --> [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f63ec063540 con 0x7f63d40ad810 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.272+0000 7f63e8ff9700 1 -- 172.21.15.19:0/3047775336 <== osd.0 v2:172.21.15.19:6802/4279899748 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f63ec063540 con 0x7f63d40ad810 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.272+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] conn(0x7f63d40ad810 msgr2=0x7f63d40afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.272+0000 7f63f1a93700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] conn(0x7f63d40ad810 0x7f63d40afcb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.272+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63d40a3cd0 msgr2=0x7f63d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.272+0000 7f63f1a93700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f63d40a3cd0 0x7f63d40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0099d0 tx=0x7f63dc008040 comp rx=0 tx=0).stop 2024-04-23T18:24:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.272+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec10f4f0 msgr2=0x7f63ec19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.272+0000 7f63f1a93700 1 --2- 172.21.15.19:0/3047775336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec10f4f0 0x7f63ec19fcb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f63e4006070 tx=0x7f63e4004f10 comp rx=0 tx=0).stop 2024-04-23T18:24:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.272+0000 7f63ebfff700 1 -- 172.21.15.19:0/3047775336 reap_dead start 2024-04-23T18:24:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.273+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 shutdown_connections 2024-04-23T18:24:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.273+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 >> 172.21.15.19:0/3047775336 conn(0x7f63ec0788f0 msgr2=0x7f63ec110a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.273+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 shutdown_connections 2024-04-23T18:24:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:37.273+0000 7f63f1a93700 1 -- 172.21.15.19:0/3047775336 wait complete. 2024-04-23T18:24:37.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:37 smithi019 ceph-mon[29215]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:24:38.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:37 smithi044 ceph-mon[32458]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:24:38.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:37 smithi155 ceph-mon[33468]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:24:38.229 INFO:teuthology.orchestra.run.smithi019.stdout:141733920826 2024-04-23T18:24:38.229 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-23T18:24:38.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:38 smithi019 ceph-mon[29215]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:39.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:38 smithi044 ceph-mon[32458]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:39.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:38 smithi155 ceph-mon[33468]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:39.479 INFO:teuthology.orchestra.run.smithi019.stdout:167503724598 2024-04-23T18:24:39.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-23T18:24:40.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:40 smithi019 ceph-mon[29215]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 9 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:41.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:40 smithi044 ceph-mon[32458]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 9 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:41.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:40 smithi155 ceph-mon[33468]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 9 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.083+0000 7fa39c696700 1 -- 172.21.15.19:0/767181860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa394108a20 msgr2=0x7fa394108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.083+0000 7fa39c696700 1 --2- 172.21.15.19:0/767181860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa394108a20 0x7fa394108e00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa390009990 tx=0x7fa390009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.083+0000 7fa39c696700 1 -- 172.21.15.19:0/767181860 shutdown_connections 2024-04-23T18:24:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.083+0000 7fa39c696700 1 --2- 172.21.15.19:0/767181860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa39406b980 0x7fa39410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.083+0000 7fa39c696700 1 --2- 172.21.15.19:0/767181860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa394108a20 0x7fa394108e00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.083+0000 7fa39c696700 1 --2- 172.21.15.19:0/767181860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39406afe0 0x7fa39406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.083+0000 7fa39c696700 1 -- 172.21.15.19:0/767181860 >> 172.21.15.19:0/767181860 conn(0x7fa3940788b0 msgr2=0x7fa394078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:41.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.083+0000 7fa39c696700 1 -- 172.21.15.19:0/767181860 shutdown_connections 2024-04-23T18:24:41.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.083+0000 7fa39c696700 1 -- 172.21.15.19:0/767181860 wait complete. 2024-04-23T18:24:41.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39c696700 1 Processor -- start 2024-04-23T18:24:41.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39c696700 1 -- start start 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39c696700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa39406afe0 0x7fa394101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39c696700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa39406b980 0x7fa394101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39c696700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa394108a20 0x7fa394102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39c696700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa394112bc0 con 0x7fa39406b980 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39c696700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa394112a40 con 0x7fa394108a20 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39c696700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa394112d40 con 0x7fa39406afe0 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa399c31700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa39406b980 0x7fa394101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa399c31700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa39406b980 0x7fa394101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41880/0 (socket says 172.21.15.19:41880) 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa399c31700 1 -- 172.21.15.19:0/420533846 learned_addr learned my addr 172.21.15.19:0/420533846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39a432700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa39406afe0 0x7fa394101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39ac33700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa394108a20 0x7fa394102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa399c31700 1 -- 172.21.15.19:0/420533846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa39406afe0 msgr2=0x7fa394101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa399c31700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa39406afe0 0x7fa394101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa399c31700 1 -- 172.21.15.19:0/420533846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa394108a20 msgr2=0x7fa394102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa399c31700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa394108a20 0x7fa394102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa399c31700 1 -- 172.21.15.19:0/420533846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3941a7620 con 0x7fa39406b980 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa39ac33700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa394108a20 0x7fa394102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.084+0000 7fa399c31700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa39406b980 0x7fa394101f40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa38400e9a0 tx=0x7fa38400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.085+0000 7fa38b7fe700 1 -- 172.21.15.19:0/420533846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38400cbd0 con 0x7fa39406b980 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.085+0000 7fa38b7fe700 1 -- 172.21.15.19:0/420533846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38400cd70 con 0x7fa39406b980 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.085+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3941a7910 con 0x7fa39406b980 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.085+0000 7fa38b7fe700 1 -- 172.21.15.19:0/420533846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384018680 con 0x7fa39406b980 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.085+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3941a7e50 con 0x7fa39406b980 2024-04-23T18:24:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.086+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa394063110 con 0x7fa39406b980 2024-04-23T18:24:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.087+0000 7fa38b7fe700 1 -- 172.21.15.19:0/420533846 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa384010970 con 0x7fa39406b980 2024-04-23T18:24:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.088+0000 7fa38b7fe700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3800a3cd0 0x7fa3800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.088+0000 7fa39a432700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3800a3cd0 0x7fa3800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:41.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.088+0000 7fa38b7fe700 1 -- 172.21.15.19:0/420533846 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fa3840d1110 con 0x7fa39406b980 2024-04-23T18:24:41.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.088+0000 7fa38b7fe700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] conn(0x7fa3800ad810 0x7fa3800afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:41.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.088+0000 7fa38b7fe700 1 -- 172.21.15.19:0/420533846 --> [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa38400e1a0 con 0x7fa3800ad810 2024-04-23T18:24:41.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.088+0000 7fa38b7fe700 1 -- 172.21.15.19:0/420533846 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa3840d14c0 con 0x7fa39406b980 2024-04-23T18:24:41.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.088+0000 7fa39ac33700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] conn(0x7fa3800ad810 0x7fa3800afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:41.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.088+0000 7fa39a432700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3800a3cd0 0x7fa3800a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa390006010 tx=0x7fa39001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:41.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.088+0000 7fa39ac33700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] conn(0x7fa3800ad810 0x7fa3800afcb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:41.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.089+0000 7fa38b7fe700 1 -- 172.21.15.19:0/420533846 <== osd.3 v2:172.21.15.19:6810/3983142737 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa38400e1a0 con 0x7fa3800ad810 2024-04-23T18:24:41.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 --> [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa394063590 con 0x7fa3800ad810 2024-04-23T18:24:41.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa38b7fe700 1 -- 172.21.15.19:0/420533846 <== osd.3 v2:172.21.15.19:6810/3983142737 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa394063590 con 0x7fa3800ad810 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] conn(0x7fa3800ad810 msgr2=0x7fa3800afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa39c696700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] conn(0x7fa3800ad810 0x7fa3800afcb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3800a3cd0 msgr2=0x7fa3800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa39c696700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa3800a3cd0 0x7fa3800a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa390006010 tx=0x7fa39001a040 comp rx=0 tx=0).stop 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa39406b980 msgr2=0x7fa394101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa39c696700 1 --2- 172.21.15.19:0/420533846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa39406b980 0x7fa394101f40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa38400e9a0 tx=0x7fa38400ed60 comp rx=0 tx=0).stop 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa39ac33700 1 -- 172.21.15.19:0/420533846 reap_dead start 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 shutdown_connections 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.106+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 >> 172.21.15.19:0/420533846 conn(0x7fa3940788b0 msgr2=0x7fa3940ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.107+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 shutdown_connections 2024-04-23T18:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.107+0000 7fa39c696700 1 -- 172.21.15.19:0/420533846 wait complete. 2024-04-23T18:24:41.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.286+0000 7fb571e39700 1 -- 172.21.15.19:0/1956781570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c101930 msgr2=0x7fb56c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.286+0000 7fb571e39700 1 --2- 172.21.15.19:0/1956781570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c101930 0x7fb56c101db0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb554009990 tx=0x7fb554009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.286+0000 7fb571e39700 1 -- 172.21.15.19:0/1956781570 shutdown_connections 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.286+0000 7fb571e39700 1 --2- 172.21.15.19:0/1956781570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c1022f0 0x7fb56c10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.286+0000 7fb571e39700 1 --2- 172.21.15.19:0/1956781570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb56c107980 0x7fb56c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.286+0000 7fb571e39700 1 --2- 172.21.15.19:0/1956781570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c101930 0x7fb56c101db0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.286+0000 7fb571e39700 1 -- 172.21.15.19:0/1956781570 >> 172.21.15.19:0/1956781570 conn(0x7fb56c0788b0 msgr2=0x7fb56c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.286+0000 7fb571e39700 1 -- 172.21.15.19:0/1956781570 shutdown_connections 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.286+0000 7fb571e39700 1 -- 172.21.15.19:0/1956781570 wait complete. 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb571e39700 1 Processor -- start 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb571e39700 1 -- start start 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb571e39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb56c101930 0x7fb56c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb571e39700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c1022f0 0x7fb56c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb571e39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c107980 0x7fb56c1a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb571e39700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c111ae0 con 0x7fb56c107980 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb571e39700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c111960 con 0x7fb56c101930 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb571e39700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c111c60 con 0x7fb56c1022f0 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb56bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c107980 0x7fb56c1a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb56bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c107980 0x7fb56c1a5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41904/0 (socket says 172.21.15.19:41904) 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb56bfff700 1 -- 172.21.15.19:0/2333624211 learned_addr learned my addr 172.21.15.19:0/2333624211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb56affd700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c1022f0 0x7fb56c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.287+0000 7fb56b7fe700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb56c101930 0x7fb56c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb56bfff700 1 -- 172.21.15.19:0/2333624211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c1022f0 msgr2=0x7fb56c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb56bfff700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c1022f0 0x7fb56c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb56bfff700 1 -- 172.21.15.19:0/2333624211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb56c101930 msgr2=0x7fb56c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb56bfff700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb56c101930 0x7fb56c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb56bfff700 1 -- 172.21.15.19:0/2333624211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56c1a5560 con 0x7fb56c107980 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb56bfff700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c107980 0x7fb56c1a5020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb56400ada0 tx=0x7fb56400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb568ff9700 1 -- 172.21.15.19:0/2333624211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564017020 con 0x7fb56c107980 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56c1a5730 con 0x7fb56c107980 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb568ff9700 1 -- 172.21.15.19:0/2333624211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb56400ce30 con 0x7fb56c107980 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb56c1a5c90 con 0x7fb56c107980 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.288+0000 7fb568ff9700 1 -- 172.21.15.19:0/2333624211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564005350 con 0x7fb56c107980 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.289+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb56c063150 con 0x7fb56c107980 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.290+0000 7fb568ff9700 1 -- 172.21.15.19:0/2333624211 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb56400c980 con 0x7fb56c107980 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.291+0000 7fb568ff9700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb5580a3cd0 0x7fb5580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.291+0000 7fb56b7fe700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb5580a3cd0 0x7fb5580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.291+0000 7fb568ff9700 1 -- 172.21.15.19:0/2333624211 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb5640d09e0 con 0x7fb56c107980 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.291+0000 7fb568ff9700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] conn(0x7fb5580ad810 0x7fb5580afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.291+0000 7fb568ff9700 1 -- 172.21.15.19:0/2333624211 --> [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb56400fe50 con 0x7fb5580ad810 2024-04-23T18:24:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.291+0000 7fb568ff9700 1 -- 172.21.15.19:0/2333624211 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fb5640d0d90 con 0x7fb56c107980 2024-04-23T18:24:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.291+0000 7fb56affd700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] conn(0x7fb5580ad810 0x7fb5580afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.291+0000 7fb56b7fe700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb5580a3cd0 0x7fb5580a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb55c005fd0 tx=0x7fb55c005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.292+0000 7fb56affd700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] conn(0x7fb5580ad810 0x7fb5580afcb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.292+0000 7fb568ff9700 1 -- 172.21.15.19:0/2333624211 <== osd.9 v2:172.21.15.19:6826/2566199237 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb56400fe50 con 0x7fb5580ad810 2024-04-23T18:24:41.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.309+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 --> [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb56c063580 con 0x7fb5580ad810 2024-04-23T18:24:41.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.309+0000 7fb568ff9700 1 -- 172.21.15.19:0/2333624211 <== osd.9 v2:172.21.15.19:6826/2566199237 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb56c063580 con 0x7fb5580ad810 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.309+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] conn(0x7fb5580ad810 msgr2=0x7fb5580afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.309+0000 7fb571e39700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] conn(0x7fb5580ad810 0x7fb5580afcb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.309+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb5580a3cd0 msgr2=0x7fb5580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.309+0000 7fb571e39700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb5580a3cd0 0x7fb5580a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb55c005fd0 tx=0x7fb55c005db0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.309+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c107980 msgr2=0x7fb56c1a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.309+0000 7fb571e39700 1 --2- 172.21.15.19:0/2333624211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c107980 0x7fb56c1a5020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb56400ada0 tx=0x7fb56400c330 comp rx=0 tx=0).stop 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.310+0000 7fb56bfff700 1 -- 172.21.15.19:0/2333624211 reap_dead start 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.310+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 shutdown_connections 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.310+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 >> 172.21.15.19:0/2333624211 conn(0x7fb56c0788b0 msgr2=0x7fb56c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.310+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 shutdown_connections 2024-04-23T18:24:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.310+0000 7fb571e39700 1 -- 172.21.15.19:0/2333624211 wait complete. 2024-04-23T18:24:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.119+0000 7f68e56e9700 1 -- 172.21.15.19:0/1534541001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e01034c0 msgr2=0x7f68e010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.119+0000 7f68e56e9700 1 --2- 172.21.15.19:0/1534541001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e01034c0 0x7f68e010f8e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f68d800b0a0 tx=0x7f68d800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.119+0000 7f68e56e9700 1 -- 172.21.15.19:0/1534541001 shutdown_connections 2024-04-23T18:24:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.119+0000 7f68e56e9700 1 --2- 172.21.15.19:0/1534541001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e01034c0 0x7f68e010f8e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.119+0000 7f68e56e9700 1 --2- 172.21.15.19:0/1534541001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e0108a10 0x7f68e0108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.119+0000 7f68e56e9700 1 --2- 172.21.15.19:0/1534541001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e0102b00 0x7f68e0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.119+0000 7f68e56e9700 1 -- 172.21.15.19:0/1534541001 >> 172.21.15.19:0/1534541001 conn(0x7f68e0079c00 msgr2=0x7f68e007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.119+0000 7f68e56e9700 1 -- 172.21.15.19:0/1534541001 shutdown_connections 2024-04-23T18:24:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.119+0000 7f68e56e9700 1 -- 172.21.15.19:0/1534541001 wait complete. 2024-04-23T18:24:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.120+0000 7f68e56e9700 1 Processor -- start 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.120+0000 7f68e56e9700 1 -- start start 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.120+0000 7f68e56e9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e0102b00 0x7f68e0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.120+0000 7f68e56e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e01034c0 0x7f68e0075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.120+0000 7f68e56e9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e0108a10 0x7f68e00757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.120+0000 7f68e56e9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0112cb0 con 0x7f68e01034c0 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.120+0000 7f68e56e9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0112b30 con 0x7f68e0108a10 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.120+0000 7f68e56e9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0112e30 con 0x7f68e0102b00 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.120+0000 7f68de7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e01034c0 0x7f68e0075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68de7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e01034c0 0x7f68e0075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41926/0 (socket says 172.21.15.19:41926) 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68de7fc700 1 -- 172.21.15.19:0/317857600 learned_addr learned my addr 172.21.15.19:0/317857600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68df7fe700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e0108a10 0x7f68e00757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68deffd700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e0102b00 0x7f68e0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68de7fc700 1 -- 172.21.15.19:0/317857600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e0102b00 msgr2=0x7f68e0074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68de7fc700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e0102b00 0x7f68e0074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68de7fc700 1 -- 172.21.15.19:0/317857600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e0108a10 msgr2=0x7f68e00757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68de7fc700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e0108a10 0x7f68e00757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68de7fc700 1 -- 172.21.15.19:0/317857600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e01a7750 con 0x7f68e01034c0 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68de7fc700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e01034c0 0x7f68e0075280 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f68c800b530 tx=0x7f68c800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68d7fff700 1 -- 172.21.15.19:0/317857600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8015020 con 0x7f68e01034c0 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e01a7a40 con 0x7f68e01034c0 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68d7fff700 1 -- 172.21.15.19:0/317857600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c8011ce0 con 0x7f68e01034c0 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68d7fff700 1 -- 172.21.15.19:0/317857600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8010dd0 con 0x7f68e01034c0 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.121+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e01a7f80 con 0x7f68e01034c0 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.122+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68e00630d0 con 0x7f68e01034c0 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.123+0000 7f68d7fff700 1 -- 172.21.15.19:0/317857600 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f68c8018020 con 0x7f68e01034c0 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.124+0000 7f68d7fff700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f68cc0a3cd0 0x7f68cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.124+0000 7f68deffd700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f68cc0a3cd0 0x7f68cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.124+0000 7f68d7fff700 1 -- 172.21.15.19:0/317857600 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f68c80d0260 con 0x7f68e01034c0 2024-04-23T18:24:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.124+0000 7f68d7fff700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] conn(0x7f68cc0ad810 0x7f68cc0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.124+0000 7f68d7fff700 1 -- 172.21.15.19:0/317857600 --> [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68c8018ec0 con 0x7f68cc0ad810 2024-04-23T18:24:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.124+0000 7f68d7fff700 1 -- 172.21.15.19:0/317857600 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f68c80d06b0 con 0x7f68e01034c0 2024-04-23T18:24:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.124+0000 7f68df7fe700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] conn(0x7f68cc0ad810 0x7f68cc0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.124+0000 7f68deffd700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f68cc0a3cd0 0x7f68cc0a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009a10 tx=0x7f68d0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.125+0000 7f68df7fe700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] conn(0x7f68cc0ad810 0x7f68cc0afcb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.125+0000 7f68d7fff700 1 -- 172.21.15.19:0/317857600 <== osd.6 v2:172.21.15.19:6818/2995627219 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68c8018ec0 con 0x7f68cc0ad810 2024-04-23T18:24:42.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.142+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 --> [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f68e0063550 con 0x7f68cc0ad810 2024-04-23T18:24:42.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68d7fff700 1 -- 172.21.15.19:0/317857600 <== osd.6 v2:172.21.15.19:6818/2995627219 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f68e0063550 con 0x7f68cc0ad810 2024-04-23T18:24:42.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] conn(0x7f68cc0ad810 msgr2=0x7f68cc0afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68e56e9700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] conn(0x7f68cc0ad810 0x7f68cc0afcb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f68cc0a3cd0 msgr2=0x7f68cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68e56e9700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f68cc0a3cd0 0x7f68cc0a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009a10 tx=0x7f68d0009190 comp rx=0 tx=0).stop 2024-04-23T18:24:42.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e01034c0 msgr2=0x7f68e0075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68e56e9700 1 --2- 172.21.15.19:0/317857600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e01034c0 0x7f68e0075280 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f68c800b530 tx=0x7f68c800b840 comp rx=0 tx=0).stop 2024-04-23T18:24:42.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68df7fe700 1 -- 172.21.15.19:0/317857600 reap_dead start 2024-04-23T18:24:42.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 shutdown_connections 2024-04-23T18:24:42.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 >> 172.21.15.19:0/317857600 conn(0x7f68e0079c00 msgr2=0x7f68e0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:42.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 shutdown_connections 2024-04-23T18:24:42.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:42.143+0000 7f68e56e9700 1 -- 172.21.15.19:0/317857600 wait complete. 2024-04-23T18:24:43.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:42 smithi019 ceph-mon[29215]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:43.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:42 smithi044 ceph-mon[32458]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:43.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:42 smithi155 ceph-mon[33468]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:43.380 INFO:teuthology.orchestra.run.smithi019.stdout:292057776161 2024-04-23T18:24:43.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-23T18:24:43.847 INFO:teuthology.orchestra.run.smithi019.stdout:77309411396 2024-04-23T18:24:43.848 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-23T18:24:44.391 INFO:teuthology.orchestra.run.smithi019.stdout:34359738446 2024-04-23T18:24:44.392 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-23T18:24:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.980+0000 7fee7dc33700 1 -- 172.21.15.19:0/3988354323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee781022f0 msgr2=0x7fee7810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.980+0000 7fee7dc33700 1 --2- 172.21.15.19:0/3988354323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee781022f0 0x7fee7810e7e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fee7000b0a0 tx=0x7fee7000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.980+0000 7fee7dc33700 1 -- 172.21.15.19:0/3988354323 shutdown_connections 2024-04-23T18:24:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.980+0000 7fee7dc33700 1 --2- 172.21.15.19:0/3988354323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee781022f0 0x7fee7810e7e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.980+0000 7fee7dc33700 1 --2- 172.21.15.19:0/3988354323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78107980 0x7fee78107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.980+0000 7fee7dc33700 1 --2- 172.21.15.19:0/3988354323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78101930 0x7fee78101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.980+0000 7fee7dc33700 1 -- 172.21.15.19:0/3988354323 >> 172.21.15.19:0/3988354323 conn(0x7fee780788b0 msgr2=0x7fee78078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.980+0000 7fee7dc33700 1 -- 172.21.15.19:0/3988354323 shutdown_connections 2024-04-23T18:24:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.980+0000 7fee7dc33700 1 -- 172.21.15.19:0/3988354323 wait complete. 2024-04-23T18:24:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.980+0000 7fee7dc33700 1 Processor -- start 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee7dc33700 1 -- start start 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee7dc33700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78101930 0x7fee781034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee7dc33700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee781022f0 0x7fee78103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee7dc33700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78107980 0x7fee78103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee7dc33700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee78111bb0 con 0x7fee781022f0 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee7dc33700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fee78111a30 con 0x7fee78101930 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee7dc33700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee78111d30 con 0x7fee78107980 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee76ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee781022f0 0x7fee78103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee76ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee781022f0 0x7fee78103a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41860/0 (socket says 172.21.15.19:41860) 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee76ffd700 1 -- 172.21.15.19:0/1984895059 learned_addr learned my addr 172.21.15.19:0/1984895059 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee77fff700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78107980 0x7fee78103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee777fe700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78101930 0x7fee781034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee76ffd700 1 -- 172.21.15.19:0/1984895059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78107980 msgr2=0x7fee78103f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee76ffd700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78107980 0x7fee78103f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee76ffd700 1 -- 172.21.15.19:0/1984895059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78101930 msgr2=0x7fee781034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee76ffd700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78101930 0x7fee781034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee76ffd700 1 -- 172.21.15.19:0/1984895059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee781a7670 con 0x7fee781022f0 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee77fff700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78107980 0x7fee78103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.981+0000 7fee777fe700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78101930 0x7fee781034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.982+0000 7fee76ffd700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee781022f0 0x7fee78103a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00d980 tx=0x7fee6c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.982+0000 7fee74ff9700 1 -- 172.21.15.19:0/1984895059 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c017070 con 0x7fee781022f0 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.982+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee781a7960 con 0x7fee781022f0 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.982+0000 7fee74ff9700 1 -- 172.21.15.19:0/1984895059 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6c004830 con 0x7fee781022f0 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.982+0000 7fee74ff9700 1 -- 172.21.15.19:0/1984895059 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c005440 con 0x7fee781022f0 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.982+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee781a7ea0 con 0x7fee781022f0 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.983+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee78063110 con 0x7fee781022f0 2024-04-23T18:24:44.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.984+0000 7fee74ff9700 1 -- 172.21.15.19:0/1984895059 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fee6c0055e0 con 0x7fee781022f0 2024-04-23T18:24:44.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.985+0000 7fee74ff9700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee600a3cd0 0x7fee600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:44.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.985+0000 7fee777fe700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee600a3cd0 0x7fee600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:44.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.985+0000 7fee74ff9700 1 -- 172.21.15.19:0/1984895059 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fee6c0d0af0 con 0x7fee781022f0 2024-04-23T18:24:44.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.985+0000 7fee74ff9700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] conn(0x7fee600ad810 0x7fee600afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:44.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.985+0000 7fee74ff9700 1 -- 172.21.15.19:0/1984895059 --> [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee600b0380 con 0x7fee600ad810 2024-04-23T18:24:44.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.985+0000 7fee74ff9700 1 -- 172.21.15.19:0/1984895059 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fee6c0d0ea0 con 0x7fee781022f0 2024-04-23T18:24:44.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.985+0000 7fee777fe700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee600a3cd0 0x7fee600a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fee680099d0 tx=0x7fee68008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:44.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.985+0000 7fee77fff700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] conn(0x7fee600ad810 0x7fee600afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:44.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.986+0000 7fee77fff700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] conn(0x7fee600ad810 0x7fee600afcb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:44.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:40.987+0000 7fee74ff9700 1 -- 172.21.15.19:0/1984895059 <== osd.11 v2:172.21.15.155:6824/3982643864 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fee600b0380 con 0x7fee600ad810 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.004+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 --> [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee78063510 con 0x7fee600ad810 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.004+0000 7fee74ff9700 1 -- 172.21.15.19:0/1984895059 <== osd.11 v2:172.21.15.155:6824/3982643864 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fee78063510 con 0x7fee600ad810 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 >> [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] conn(0x7fee600ad810 msgr2=0x7fee600afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee7dc33700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] conn(0x7fee600ad810 0x7fee600afcb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee600a3cd0 msgr2=0x7fee600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee7dc33700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee600a3cd0 0x7fee600a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fee680099d0 tx=0x7fee68008040 comp rx=0 tx=0).stop 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee781022f0 msgr2=0x7fee78103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee7dc33700 1 --2- 172.21.15.19:0/1984895059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee781022f0 0x7fee78103a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00d980 tx=0x7fee6c00dd40 comp rx=0 tx=0).stop 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee77fff700 1 -- 172.21.15.19:0/1984895059 reap_dead start 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 shutdown_connections 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 >> 172.21.15.19:0/1984895059 conn(0x7fee780788b0 msgr2=0x7fee7806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 shutdown_connections 2024-04-23T18:24:44.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:41.005+0000 7fee7dc33700 1 -- 172.21.15.19:0/1984895059 wait complete. 2024-04-23T18:24:45.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:44 smithi019 ceph-mon[29215]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-04-23T18:24:45.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:44 smithi044 ceph-mon[32458]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-04-23T18:24:45.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:44 smithi155 ceph-mon[33468]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-04-23T18:24:45.122 INFO:teuthology.orchestra.run.smithi019.stdout:107374182463 2024-04-23T18:24:45.122 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-23T18:24:46.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:46 smithi019 ceph-mon[29215]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:47.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:46 smithi044 ceph-mon[32458]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:47.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:46 smithi155 ceph-mon[33468]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:24:47.230 INFO:teuthology.orchestra.run.smithi019.stdout:347892350999 2024-04-23T18:24:47.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-23T18:24:47.254 INFO:teuthology.orchestra.run.smithi019.stdout:201863462962 2024-04-23T18:24:47.254 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-23T18:24:47.553 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.601+0000 7fb89b1c4700 1 -- 172.21.15.19:0/4137191254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb894102300 msgr2=0x7fb89410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.601+0000 7fb89b1c4700 1 --2- 172.21.15.19:0/4137191254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb894102300 0x7fb89410e7f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00b0a0 tx=0x7fb88c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:24:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.602+0000 7fb89b1c4700 1 -- 172.21.15.19:0/4137191254 shutdown_connections 2024-04-23T18:24:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.602+0000 7fb89b1c4700 1 --2- 172.21.15.19:0/4137191254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb894102300 0x7fb89410e7f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.602+0000 7fb89b1c4700 1 --2- 172.21.15.19:0/4137191254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894107990 0x7fb894107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.602+0000 7fb89b1c4700 1 --2- 172.21.15.19:0/4137191254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb894101940 0x7fb894101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.602+0000 7fb89b1c4700 1 -- 172.21.15.19:0/4137191254 >> 172.21.15.19:0/4137191254 conn(0x7fb8940788f0 msgr2=0x7fb894078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:47.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.603+0000 7fb89b1c4700 1 -- 172.21.15.19:0/4137191254 shutdown_connections 2024-04-23T18:24:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.604+0000 7fb89b1c4700 1 -- 172.21.15.19:0/4137191254 wait complete. 2024-04-23T18:24:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.605+0000 7fb89b1c4700 1 Processor -- start 2024-04-23T18:24:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.605+0000 7fb89b1c4700 1 -- start start 2024-04-23T18:24:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.605+0000 7fb89b1c4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb894101940 0x7fb89410eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.605+0000 7fb89b1c4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894102300 0x7fb89410f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.605+0000 7fb89b1c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89410f930 0x7fb8940735c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.605+0000 7fb89b1c4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb894112d50 con 0x7fb89410f930 2024-04-23T18:24:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.605+0000 7fb89b1c4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb894112bd0 con 0x7fb894102300 2024-04-23T18:24:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.605+0000 7fb89b1c4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb894112ed0 con 0x7fb894101940 2024-04-23T18:24:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.606+0000 7fb898f60700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb894101940 0x7fb89410eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.607+0000 7fb898f60700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb894101940 0x7fb89410eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:45722/0 (socket says 172.21.15.19:45722) 2024-04-23T18:24:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.607+0000 7fb898f60700 1 -- 172.21.15.19:0/1106152279 learned_addr learned my addr 172.21.15.19:0/1106152279 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.607+0000 7fb893fff700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894102300 0x7fb89410f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.607+0000 7fb899761700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89410f930 0x7fb8940735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.607+0000 7fb898f60700 1 -- 172.21.15.19:0/1106152279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894102300 msgr2=0x7fb89410f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.607+0000 7fb898f60700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894102300 0x7fb89410f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.607+0000 7fb898f60700 1 -- 172.21.15.19:0/1106152279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89410f930 msgr2=0x7fb8940735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.607+0000 7fb898f60700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89410f930 0x7fb8940735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.607+0000 7fb898f60700 1 -- 172.21.15.19:0/1106152279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb894073bb0 con 0x7fb894101940 2024-04-23T18:24:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.607+0000 7fb899761700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89410f930 0x7fb8940735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.608+0000 7fb898f60700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb894101940 0x7fb89410eeb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00b500 tx=0x7fb87c00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.608+0000 7fb893fff700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894102300 0x7fb89410f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.608+0000 7fb891ffb700 1 -- 172.21.15.19:0/1106152279 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c015020 con 0x7fb894101940 2024-04-23T18:24:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.608+0000 7fb891ffb700 1 -- 172.21.15.19:0/1106152279 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb87c011ce0 con 0x7fb894101940 2024-04-23T18:24:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.608+0000 7fb891ffb700 1 -- 172.21.15.19:0/1106152279 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c010d30 con 0x7fb894101940 2024-04-23T18:24:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.610+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb894073ea0 con 0x7fb894101940 2024-04-23T18:24:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.611+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb894074360 con 0x7fb894101940 2024-04-23T18:24:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.613+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb894074950 con 0x7fb894101940 2024-04-23T18:24:47.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.614+0000 7fb891ffb700 1 -- 172.21.15.19:0/1106152279 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb87c018020 con 0x7fb894101940 2024-04-23T18:24:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.615+0000 7fb891ffb700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb8800a3e50 0x7fb8800a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.615+0000 7fb893fff700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb8800a3e50 0x7fb8800a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.615+0000 7fb891ffb700 1 -- 172.21.15.19:0/1106152279 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb87c0cf990 con 0x7fb894101940 2024-04-23T18:24:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.615+0000 7fb891ffb700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] conn(0x7fb8800ad990 0x7fb8800afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.615+0000 7fb891ffb700 1 -- 172.21.15.19:0/1106152279 --> [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb87c015e80 con 0x7fb8800ad990 2024-04-23T18:24:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.615+0000 7fb891ffb700 1 -- 172.21.15.19:0/1106152279 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fb87c0d42c0 con 0x7fb894101940 2024-04-23T18:24:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.616+0000 7fb899761700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] conn(0x7fb8800ad990 0x7fb8800afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.616+0000 7fb893fff700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb8800a3e50 0x7fb8800a6310 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb894079100 tx=0x7fb884008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.616+0000 7fb899761700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] conn(0x7fb8800ad990 0x7fb8800afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:47.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.617+0000 7fb891ffb700 1 -- 172.21.15.19:0/1106152279 <== osd.1 v2:172.21.15.44:6800/2943619157 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb87c015e80 con 0x7fb8800ad990 2024-04-23T18:24:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.635+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 --> [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb8940630d0 con 0x7fb8800ad990 2024-04-23T18:24:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.636+0000 7fb891ffb700 1 -- 172.21.15.19:0/1106152279 <== osd.1 v2:172.21.15.44:6800/2943619157 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb8940630d0 con 0x7fb8800ad990 2024-04-23T18:24:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.636+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 >> [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] conn(0x7fb8800ad990 msgr2=0x7fb8800afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.636+0000 7fb89b1c4700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] conn(0x7fb8800ad990 0x7fb8800afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.636+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb8800a3e50 msgr2=0x7fb8800a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.636+0000 7fb89b1c4700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb8800a3e50 0x7fb8800a6310 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb894079100 tx=0x7fb884008040 comp rx=0 tx=0).stop 2024-04-23T18:24:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.636+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb894101940 msgr2=0x7fb89410eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.636+0000 7fb89b1c4700 1 --2- 172.21.15.19:0/1106152279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb894101940 0x7fb89410eeb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00b500 tx=0x7fb87c00b8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.636+0000 7fb899761700 1 -- 172.21.15.19:0/1106152279 reap_dead start 2024-04-23T18:24:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.637+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 shutdown_connections 2024-04-23T18:24:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.637+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 >> 172.21.15.19:0/1106152279 conn(0x7fb8940788f0 msgr2=0x7fb8940ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.637+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 shutdown_connections 2024-04-23T18:24:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:47.637+0000 7fb89b1c4700 1 -- 172.21.15.19:0/1106152279 wait complete. 2024-04-23T18:24:48.495 INFO:teuthology.orchestra.run.smithi019.stdout:55834574924 2024-04-23T18:24:48.495 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-23T18:24:48.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.702+0000 7f9324487700 1 -- 172.21.15.19:0/3545470083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f931c101970 msgr2=0x7f931c101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:48.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.702+0000 7f9324487700 1 --2- 172.21.15.19:0/3545470083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f931c101970 0x7f931c101df0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f930c0099c0 tx=0x7f930c009cd0 comp rx=0 tx=0).stop 2024-04-23T18:24:48.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.702+0000 7f9324487700 1 -- 172.21.15.19:0/3545470083 shutdown_connections 2024-04-23T18:24:48.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.702+0000 7f9324487700 1 --2- 172.21.15.19:0/3545470083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f931c102330 0x7f931c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:48.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.702+0000 7f9324487700 1 --2- 172.21.15.19:0/3545470083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c1079c0 0x7f931c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:48.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.702+0000 7f9324487700 1 --2- 172.21.15.19:0/3545470083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f931c101970 0x7f931c101df0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:48.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.702+0000 7f9324487700 1 -- 172.21.15.19:0/3545470083 >> 172.21.15.19:0/3545470083 conn(0x7f931c078980 msgr2=0x7f931c078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:48.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.702+0000 7f9324487700 1 -- 172.21.15.19:0/3545470083 shutdown_connections 2024-04-23T18:24:48.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.702+0000 7f9324487700 1 -- 172.21.15.19:0/3545470083 wait complete. 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.703+0000 7f9324487700 1 Processor -- start 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.703+0000 7f9324487700 1 -- start start 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9324487700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f931c101970 0x7f931c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9324487700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f931c102330 0x7f931c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9324487700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c1079c0 0x7f931c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9324487700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f931c111b70 con 0x7f931c101970 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9324487700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f931c1119f0 con 0x7f931c102330 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9324487700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f931c111cf0 con 0x7f931c1079c0 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9322223700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f931c101970 0x7f931c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9322223700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f931c101970 0x7f931c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41948/0 (socket says 172.21.15.19:41948) 2024-04-23T18:24:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9322223700 1 -- 172.21.15.19:0/1856358462 learned_addr learned my addr 172.21.15.19:0/1856358462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9321a22700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f931c102330 0x7f931c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9322a24700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c1079c0 0x7f931c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9322223700 1 -- 172.21.15.19:0/1856358462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c1079c0 msgr2=0x7f931c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9322223700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c1079c0 0x7f931c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9322223700 1 -- 172.21.15.19:0/1856358462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f931c102330 msgr2=0x7f931c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9322223700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f931c102330 0x7f931c19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.704+0000 7f9322223700 1 -- 172.21.15.19:0/1856358462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c1a0390 con 0x7f931c101970 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.705+0000 7f9322223700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f931c101970 0x7f931c19b380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f931800c920 tx=0x7f931800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.705+0000 7f93137fe700 1 -- 172.21.15.19:0/1856358462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318007b00 con 0x7f931c101970 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.705+0000 7f93137fe700 1 -- 172.21.15.19:0/1856358462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9318004ce0 con 0x7f931c101970 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.705+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931c1a0680 con 0x7f931c101970 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.705+0000 7f93137fe700 1 -- 172.21.15.19:0/1856358462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318005550 con 0x7f931c101970 2024-04-23T18:24:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.705+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931c1a7f00 con 0x7f931c101970 2024-04-23T18:24:48.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.706+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f931c063150 con 0x7f931c101970 2024-04-23T18:24:48.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.707+0000 7f93137fe700 1 -- 172.21.15.19:0/1856358462 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9318007ca0 con 0x7f931c101970 2024-04-23T18:24:48.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.708+0000 7f93137fe700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f93080a3cd0 0x7f93080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:48.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.708+0000 7f93137fe700 1 -- 172.21.15.19:0/1856358462 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f93180d0660 con 0x7f931c101970 2024-04-23T18:24:48.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.708+0000 7f9321a22700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f93080a3cd0 0x7f93080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:48.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.708+0000 7f93137fe700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] conn(0x7f93080ad810 0x7f93080afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:48.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.708+0000 7f93137fe700 1 -- 172.21.15.19:0/1856358462 --> [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93080b0380 con 0x7f93080ad810 2024-04-23T18:24:48.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.708+0000 7f93137fe700 1 -- 172.21.15.19:0/1856358462 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f93180d0a10 con 0x7f931c101970 2024-04-23T18:24:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.709+0000 7f9322a24700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] conn(0x7f93080ad810 0x7f93080afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.709+0000 7f9321a22700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f93080a3cd0 0x7f93080a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f931c0792c0 tx=0x7f930c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.709+0000 7f9322a24700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] conn(0x7f93080ad810 0x7f93080afcb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:48.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.710+0000 7f93137fe700 1 -- 172.21.15.19:0/1856358462 <== osd.7 v2:172.21.15.44:6816/2128528085 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93080b0380 con 0x7f93080ad810 2024-04-23T18:24:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.728+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 --> [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f931c063550 con 0x7f93080ad810 2024-04-23T18:24:48.721 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:48 smithi019 ceph-mon[29215]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:24:48.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f93137fe700 1 -- 172.21.15.19:0/1856358462 <== osd.7 v2:172.21.15.44:6816/2128528085 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f931c063550 con 0x7f93080ad810 2024-04-23T18:24:48.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 >> [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] conn(0x7f93080ad810 msgr2=0x7f93080afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9324487700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] conn(0x7f93080ad810 0x7f93080afcb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f93080a3cd0 msgr2=0x7f93080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9324487700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f93080a3cd0 0x7f93080a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f931c0792c0 tx=0x7f930c005fb0 comp rx=0 tx=0).stop 2024-04-23T18:24:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f931c101970 msgr2=0x7f931c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9324487700 1 --2- 172.21.15.19:0/1856358462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f931c101970 0x7f931c19b380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f931800c920 tx=0x7f931800cc30 comp rx=0 tx=0).stop 2024-04-23T18:24:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9322a24700 1 -- 172.21.15.19:0/1856358462 reap_dead start 2024-04-23T18:24:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 shutdown_connections 2024-04-23T18:24:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 >> 172.21.15.19:0/1856358462 conn(0x7f931c078980 msgr2=0x7f931c0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 shutdown_connections 2024-04-23T18:24:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:48.729+0000 7f9324487700 1 -- 172.21.15.19:0/1856358462 wait complete. 2024-04-23T18:24:49.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:48 smithi044 ceph-mon[32458]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:24:49.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:48 smithi155 ceph-mon[33468]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:24:49.472 INFO:teuthology.orchestra.run.smithi019.stdout:223338299438 2024-04-23T18:24:49.472 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-23T18:24:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:50.683 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:50.850 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:50 smithi019 ceph-mon[29215]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-23T18:24:51.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:50 smithi044 ceph-mon[32458]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-23T18:24:51.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:50 smithi155 ceph-mon[33468]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-23T18:24:51.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.440+0000 7fbd1cf98700 1 -- 172.21.15.19:0/1698477931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd18103460 msgr2=0x7fbd1810f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.440+0000 7fbd1cf98700 1 --2- 172.21.15.19:0/1698477931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd18103460 0x7fbd1810f940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009990 tx=0x7fbd10009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.441+0000 7fbd1cf98700 1 -- 172.21.15.19:0/1698477931 shutdown_connections 2024-04-23T18:24:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.441+0000 7fbd1cf98700 1 --2- 172.21.15.19:0/1698477931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd18103460 0x7fbd1810f940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.441+0000 7fbd1cf98700 1 --2- 172.21.15.19:0/1698477931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd18108a70 0x7fbd18108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.441+0000 7fbd1cf98700 1 --2- 172.21.15.19:0/1698477931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd18102ac0 0x7fbd18102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.441+0000 7fbd1cf98700 1 -- 172.21.15.19:0/1698477931 >> 172.21.15.19:0/1698477931 conn(0x7fbd18079bf0 msgr2=0x7fbd1807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.441+0000 7fbd1cf98700 1 -- 172.21.15.19:0/1698477931 shutdown_connections 2024-04-23T18:24:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.441+0000 7fbd1cf98700 1 -- 172.21.15.19:0/1698477931 wait complete. 2024-04-23T18:24:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd1cf98700 1 Processor -- start 2024-04-23T18:24:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd1cf98700 1 -- start start 2024-04-23T18:24:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd1cf98700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd18102ac0 0x7fbd18199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd1cf98700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd18103460 0x7fbd18199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd1cf98700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd18108a70 0x7fbd1819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd1cf98700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18112cf0 con 0x7fbd18108a70 2024-04-23T18:24:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd1cf98700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18112b70 con 0x7fbd18102ac0 2024-04-23T18:24:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd1cf98700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18112e70 con 0x7fbd18103460 2024-04-23T18:24:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd16d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd18108a70 0x7fbd1819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd16d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd18108a70 0x7fbd1819dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44072/0 (socket says 172.21.15.19:44072) 2024-04-23T18:24:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd16d9d700 1 -- 172.21.15.19:0/4257922926 learned_addr learned my addr 172.21.15.19:0/4257922926 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd1659c700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd18102ac0 0x7fbd18199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd16d9d700 1 -- 172.21.15.19:0/4257922926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd18103460 msgr2=0x7fbd18199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd16d9d700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd18103460 0x7fbd18199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd16d9d700 1 -- 172.21.15.19:0/4257922926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd18102ac0 msgr2=0x7fbd18199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd16d9d700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd18102ac0 0x7fbd18199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd16d9d700 1 -- 172.21.15.19:0/4257922926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1819e210 con 0x7fbd18108a70 2024-04-23T18:24:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.442+0000 7fbd16d9d700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd18108a70 0x7fbd1819dad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbd100047b0 tx=0x7fbd10009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.443+0000 7fbd0f7fe700 1 -- 172.21.15.19:0/4257922926 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10027070 con 0x7fbd18108a70 2024-04-23T18:24:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.443+0000 7fbd1cf98700 1 -- 172.21.15.19:0/4257922926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1819e4a0 con 0x7fbd18108a70 2024-04-23T18:24:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.443+0000 7fbd0f7fe700 1 -- 172.21.15.19:0/4257922926 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd10004020 con 0x7fbd18108a70 2024-04-23T18:24:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.443+0000 7fbd0f7fe700 1 -- 172.21.15.19:0/4257922926 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10022d70 con 0x7fbd18108a70 2024-04-23T18:24:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.443+0000 7fbd1cf98700 1 -- 172.21.15.19:0/4257922926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181a5d80 con 0x7fbd18108a70 2024-04-23T18:24:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.444+0000 7fbd1cf98700 1 -- 172.21.15.19:0/4257922926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd18069730 con 0x7fbd18108a70 2024-04-23T18:24:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.445+0000 7fbd0f7fe700 1 -- 172.21.15.19:0/4257922926 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbd1002a070 con 0x7fbd18108a70 2024-04-23T18:24:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.445+0000 7fbd0f7fe700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbd040a3cd0 0x7fbd040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.445+0000 7fbd0f7fe700 1 -- 172.21.15.19:0/4257922926 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbd100d14c0 con 0x7fbd18108a70 2024-04-23T18:24:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.445+0000 7fbd1659c700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbd040a3cd0 0x7fbd040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.446+0000 7fbd1659c700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbd040a3cd0 0x7fbd040a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd00007dc0 tx=0x7fbd000072f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:51.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.450+0000 7fbd0f7fe700 1 -- 172.21.15.19:0/4257922926 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd1005eff0 con 0x7fbd18108a70 2024-04-23T18:24:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.662+0000 7fbd1cf98700 1 -- 172.21.15.19:0/4257922926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fbd1802df80 con 0x7fbd18108a70 2024-04-23T18:24:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.662+0000 7fbd0f7fe700 1 -- 172.21.15.19:0/4257922926 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fbd10013070 con 0x7fbd18108a70 2024-04-23T18:24:51.656 INFO:teuthology.orchestra.run.smithi019.stdout:313532612637 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 -- 172.21.15.19:0/4257922926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbd040a3cd0 msgr2=0x7fbd040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbd040a3cd0 0x7fbd040a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd00007dc0 tx=0x7fbd000072f0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 -- 172.21.15.19:0/4257922926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd18108a70 msgr2=0x7fbd1819dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd18108a70 0x7fbd1819dad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbd100047b0 tx=0x7fbd10009f90 comp rx=0 tx=0).stop 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 -- 172.21.15.19:0/4257922926 shutdown_connections 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbd040a3cd0 0x7fbd040a6190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd18108a70 0x7fbd1819dad0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd18102ac0 0x7fbd18199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 --2- 172.21.15.19:0/4257922926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd18103460 0x7fbd18199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 -- 172.21.15.19:0/4257922926 >> 172.21.15.19:0/4257922926 conn(0x7fbd18079bf0 msgr2=0x7fbd18100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 -- 172.21.15.19:0/4257922926 shutdown_connections 2024-04-23T18:24:51.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:51.665+0000 7fbd1cf98700 1 -- 172.21.15.19:0/4257922926 wait complete. 2024-04-23T18:24:52.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:51 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4257922926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:24:52.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:51 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4257922926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:24:52.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:51 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4257922926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:24:52.900 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:52 smithi019 ceph-mon[29215]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-23T18:24:53.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:52 smithi044 ceph-mon[32458]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-23T18:24:53.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:52 smithi155 ceph-mon[33468]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-23T18:24:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:53.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612632 got 313532612637 for osd.10 2024-04-23T18:24:53.987 DEBUG:teuthology.parallel:result is None 2024-04-23T18:24:54.950 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:54 smithi019 ceph-mon[29215]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-04-23T18:24:55.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:54 smithi044 ceph-mon[32458]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-04-23T18:24:55.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:54 smithi155 ceph-mon[33468]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-04-23T18:24:56.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:55 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:56.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:55 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:56.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:55 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:56.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:56 smithi019 ceph-mon[29215]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-23T18:24:57.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:56 smithi044 ceph-mon[32458]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-23T18:24:57.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:56 smithi155 ceph-mon[33468]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-23T18:24:58.230 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:57 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.230 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:57 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.230 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:57 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.230 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:57 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:57 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:57 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:57 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:57 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:57 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:57 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:57 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:57 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.597+0000 7f40fb6b3700 1 -- 172.21.15.19:0/3390876237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f4108a50 msgr2=0x7f40f4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.597+0000 7f40fb6b3700 1 --2- 172.21.15.19:0/3390876237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f4108a50 0x7f40f4108e30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009990 tx=0x7f40f0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.597+0000 7f40fb6b3700 1 -- 172.21.15.19:0/3390876237 shutdown_connections 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.597+0000 7f40fb6b3700 1 --2- 172.21.15.19:0/3390876237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40f4103450 0x7f40f410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.597+0000 7f40fb6b3700 1 --2- 172.21.15.19:0/3390876237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f4108a50 0x7f40f4108e30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.597+0000 7f40fb6b3700 1 --2- 172.21.15.19:0/3390876237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f4102a90 0x7f40f4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.597+0000 7f40fb6b3700 1 -- 172.21.15.19:0/3390876237 >> 172.21.15.19:0/3390876237 conn(0x7f40f4079ba0 msgr2=0x7f40f4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.598+0000 7f40fb6b3700 1 -- 172.21.15.19:0/3390876237 shutdown_connections 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.598+0000 7f40fb6b3700 1 -- 172.21.15.19:0/3390876237 wait complete. 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.598+0000 7f40fb6b3700 1 Processor -- start 2024-04-23T18:24:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.598+0000 7f40fb6b3700 1 -- start start 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.598+0000 7f40fb6b3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f4102a90 0x7f40f4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.598+0000 7f40fb6b3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40f4103450 0x7f40f4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.598+0000 7f40fb6b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f4108a50 0x7f40f40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.598+0000 7f40fb6b3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4112c60 con 0x7f40f4108a50 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.598+0000 7f40fb6b3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4112ae0 con 0x7f40f4103450 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.598+0000 7f40fb6b3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4112de0 con 0x7f40f4102a90 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f9c50700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f4108a50 0x7f40f40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f8c4e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40f4103450 0x7f40f4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f8c4e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40f4103450 0x7f40f4075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:36354/0 (socket says 172.21.15.19:36354) 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f9c50700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f4108a50 0x7f40f40757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44078/0 (socket says 172.21.15.19:44078) 2024-04-23T18:24:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f8c4e700 1 -- 172.21.15.19:0/2775900922 learned_addr learned my addr 172.21.15.19:0/2775900922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f944f700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f4102a90 0x7f40f4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f9c50700 1 -- 172.21.15.19:0/2775900922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f4102a90 msgr2=0x7f40f4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f9c50700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f4102a90 0x7f40f4074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f9c50700 1 -- 172.21.15.19:0/2775900922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40f4103450 msgr2=0x7f40f4075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f9c50700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40f4103450 0x7f40f4075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f9c50700 1 -- 172.21.15.19:0/2775900922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f41a7660 con 0x7f40f4108a50 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f8c4e700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40f4103450 0x7f40f4075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40f9c50700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f4108a50 0x7f40f40757c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00cc50 tx=0x7f40ec00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40ea7fc700 1 -- 172.21.15.19:0/2775900922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec016020 con 0x7f40f4108a50 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40fb6b3700 1 -- 172.21.15.19:0/2775900922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f41a7950 con 0x7f40f4108a50 2024-04-23T18:24:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40ea7fc700 1 -- 172.21.15.19:0/2775900922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40ec012420 con 0x7f40f4108a50 2024-04-23T18:24:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.599+0000 7f40ea7fc700 1 -- 172.21.15.19:0/2775900922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0115a0 con 0x7f40f4108a50 2024-04-23T18:24:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.600+0000 7f40fb6b3700 1 -- 172.21.15.19:0/2775900922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f41a7e40 con 0x7f40f4108a50 2024-04-23T18:24:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.600+0000 7f40fb6b3700 1 -- 172.21.15.19:0/2775900922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f4069730 con 0x7f40f4108a50 2024-04-23T18:24:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.602+0000 7f40ea7fc700 1 -- 172.21.15.19:0/2775900922 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f40ec019020 con 0x7f40f4108a50 2024-04-23T18:24:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.602+0000 7f40ea7fc700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40e00a3cd0 0x7f40e00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.602+0000 7f40f944f700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40e00a3cd0 0x7f40e00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.602+0000 7f40ea7fc700 1 -- 172.21.15.19:0/2775900922 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f40ec0d0740 con 0x7f40f4108a50 2024-04-23T18:24:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.603+0000 7f40f944f700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40e00a3cd0 0x7f40e00a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f40f0006010 tx=0x7f40f001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:58.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.606+0000 7f40ea7fc700 1 -- 172.21.15.19:0/2775900922 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40ec020020 con 0x7f40f4108a50 2024-04-23T18:24:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.817+0000 7f40fb6b3700 1 -- 172.21.15.19:0/2775900922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f40f402df80 con 0x7f40f4108a50 2024-04-23T18:24:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.817+0000 7f40ea7fc700 1 -- 172.21.15.19:0/2775900922 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f40ec09d0e0 con 0x7f40f4108a50 2024-04-23T18:24:58.811 INFO:teuthology.orchestra.run.smithi019.stdout:257698037802 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 -- 172.21.15.19:0/2775900922 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40e00a3cd0 msgr2=0x7f40e00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40e00a3cd0 0x7f40e00a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f40f0006010 tx=0x7f40f001a040 comp rx=0 tx=0).stop 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 -- 172.21.15.19:0/2775900922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f4108a50 msgr2=0x7f40f40757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f4108a50 0x7f40f40757c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00cc50 tx=0x7f40ec00cf60 comp rx=0 tx=0).stop 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 -- 172.21.15.19:0/2775900922 shutdown_connections 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40e00a3cd0 0x7f40e00a6190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f4108a50 0x7f40f40757c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f4102a90 0x7f40f4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 --2- 172.21.15.19:0/2775900922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40f4103450 0x7f40f4075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 -- 172.21.15.19:0/2775900922 >> 172.21.15.19:0/2775900922 conn(0x7f40f4079ba0 msgr2=0x7f40f41016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 -- 172.21.15.19:0/2775900922 shutdown_connections 2024-04-23T18:24:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:24:58.820+0000 7f40fb6b3700 1 -- 172.21.15.19:0/2775900922 wait complete. 2024-04-23T18:24:59.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:59 smithi019 ceph-mon[29215]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:24:59.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:24:59 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2775900922' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:24:59.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:59 smithi044 ceph-mon[32458]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:24:59.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:24:59 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2775900922' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:24:59.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:59 smithi155 ceph-mon[33468]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:24:59.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:24:59 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2775900922' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:25:01.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:01 smithi019 ceph-mon[29215]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-23T18:25:01.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:01 smithi044 ceph-mon[32458]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-23T18:25:01.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:01 smithi155 ceph-mon[33468]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-23T18:25:03.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.153+0000 7fbc4e824700 1 -- 172.21.15.19:0/630513432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc4806afe0 msgr2=0x7fbc4806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.153+0000 7fbc4e824700 1 --2- 172.21.15.19:0/630513432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc4806afe0 0x7fbc4806b440 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009990 tx=0x7fbc3c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.153+0000 7fbc4e824700 1 -- 172.21.15.19:0/630513432 shutdown_connections 2024-04-23T18:25:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.153+0000 7fbc4e824700 1 --2- 172.21.15.19:0/630513432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc4806b980 0x7fbc4810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.153+0000 7fbc4e824700 1 --2- 172.21.15.19:0/630513432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc48108a20 0x7fbc48108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.153+0000 7fbc4e824700 1 --2- 172.21.15.19:0/630513432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc4806afe0 0x7fbc4806b440 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.153+0000 7fbc4e824700 1 -- 172.21.15.19:0/630513432 >> 172.21.15.19:0/630513432 conn(0x7fbc480788b0 msgr2=0x7fbc48078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.153+0000 7fbc4e824700 1 -- 172.21.15.19:0/630513432 shutdown_connections 2024-04-23T18:25:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.153+0000 7fbc4e824700 1 -- 172.21.15.19:0/630513432 wait complete. 2024-04-23T18:25:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc4e824700 1 Processor -- start 2024-04-23T18:25:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc4e824700 1 -- start start 2024-04-23T18:25:03.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc4e824700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc4806afe0 0x7fbc481990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:03.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc4e824700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc4806b980 0x7fbc48199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:03.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc4e824700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc48108a20 0x7fbc4819d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:03.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc4e824700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48112c20 con 0x7fbc4806afe0 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc4e824700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48112aa0 con 0x7fbc4806b980 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc4e824700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48112da0 con 0x7fbc48108a20 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc4806afe0 0x7fbc481990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc4806afe0 0x7fbc481990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55186/0 (socket says 172.21.15.19:55186) 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc47fff700 1 -- 172.21.15.19:0/2483843082 learned_addr learned my addr 172.21.15.19:0/2483843082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc4cdc1700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc48108a20 0x7fbc4819d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.154+0000 7fbc477fe700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc4806b980 0x7fbc48199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc47fff700 1 -- 172.21.15.19:0/2483843082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc48108a20 msgr2=0x7fbc4819d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc47fff700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc48108a20 0x7fbc4819d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc47fff700 1 -- 172.21.15.19:0/2483843082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc4806b980 msgr2=0x7fbc48199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc47fff700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc4806b980 0x7fbc48199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc47fff700 1 -- 172.21.15.19:0/2483843082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4819e100 con 0x7fbc4806afe0 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc47fff700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc4806afe0 0x7fbc481990f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800b530 tx=0x7fbc3800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc457fa700 1 -- 172.21.15.19:0/2483843082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38015020 con 0x7fbc4806afe0 2024-04-23T18:25:03.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc457fa700 1 -- 172.21.15.19:0/2483843082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc38011ce0 con 0x7fbc4806afe0 2024-04-23T18:25:03.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc4e824700 1 -- 172.21.15.19:0/2483843082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4819e3f0 con 0x7fbc4806afe0 2024-04-23T18:25:03.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc457fa700 1 -- 172.21.15.19:0/2483843082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38010e40 con 0x7fbc4806afe0 2024-04-23T18:25:03.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.155+0000 7fbc4e824700 1 -- 172.21.15.19:0/2483843082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc481a5c70 con 0x7fbc4806afe0 2024-04-23T18:25:03.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.156+0000 7fbc4e824700 1 -- 172.21.15.19:0/2483843082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc34005330 con 0x7fbc4806afe0 2024-04-23T18:25:03.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.157+0000 7fbc457fa700 1 -- 172.21.15.19:0/2483843082 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbc38018020 con 0x7fbc4806afe0 2024-04-23T18:25:03.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.158+0000 7fbc457fa700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbc300a3cd0 0x7fbc300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:03.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.158+0000 7fbc477fe700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbc300a3cd0 0x7fbc300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:03.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.158+0000 7fbc457fa700 1 -- 172.21.15.19:0/2483843082 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbc380d0150 con 0x7fbc4806afe0 2024-04-23T18:25:03.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.159+0000 7fbc477fe700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbc300a3cd0 0x7fbc300a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c006010 tx=0x7fbc3c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:03.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.162+0000 7fbc457fa700 1 -- 172.21.15.19:0/2483843082 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc3801f020 con 0x7fbc4806afe0 2024-04-23T18:25:03.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.369+0000 7fbc4e824700 1 -- 172.21.15.19:0/2483843082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fbc340058f0 con 0x7fbc4806afe0 2024-04-23T18:25:03.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:03 smithi019 ceph-mon[29215]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-23T18:25:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.370+0000 7fbc457fa700 1 -- 172.21.15.19:0/2483843082 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbc3809caf0 con 0x7fbc4806afe0 2024-04-23T18:25:03.363 INFO:teuthology.orchestra.run.smithi019.stdout:167503724603 2024-04-23T18:25:03.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 -- 172.21.15.19:0/2483843082 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbc300a3cd0 msgr2=0x7fbc300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbc300a3cd0 0x7fbc300a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c006010 tx=0x7fbc3c01a040 comp rx=0 tx=0).stop 2024-04-23T18:25:03.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 -- 172.21.15.19:0/2483843082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc4806afe0 msgr2=0x7fbc481990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc4806afe0 0x7fbc481990f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800b530 tx=0x7fbc3800b840 comp rx=0 tx=0).stop 2024-04-23T18:25:03.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 -- 172.21.15.19:0/2483843082 shutdown_connections 2024-04-23T18:25:03.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbc300a3cd0 0x7fbc300a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc48108a20 0x7fbc4819d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc4806afe0 0x7fbc481990f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 --2- 172.21.15.19:0/2483843082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc4806b980 0x7fbc48199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 -- 172.21.15.19:0/2483843082 >> 172.21.15.19:0/2483843082 conn(0x7fbc480788b0 msgr2=0x7fbc480ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:03.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 -- 172.21.15.19:0/2483843082 shutdown_connections 2024-04-23T18:25:03.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.372+0000 7fbc4e824700 1 -- 172.21.15.19:0/2483843082 wait complete. 2024-04-23T18:25:03.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:03 smithi044 ceph-mon[32458]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-23T18:25:03.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:03 smithi155 ceph-mon[33468]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-23T18:25:04.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.806+0000 7f40f12f3700 1 -- 172.21.15.19:0/1371092683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ec075590 msgr2=0x7f40ec10ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:04.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.806+0000 7f40f12f3700 1 --2- 172.21.15.19:0/1371092683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ec075590 0x7f40ec10ef30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009990 tx=0x7f40d4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:04.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.806+0000 7f40f12f3700 1 -- 172.21.15.19:0/1371092683 shutdown_connections 2024-04-23T18:25:04.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.806+0000 7f40f12f3700 1 --2- 172.21.15.19:0/1371092683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40ec10f470 0x7f40ec111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:04.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.806+0000 7f40f12f3700 1 --2- 172.21.15.19:0/1371092683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40ec074c70 0x7f40ec075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:04.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.806+0000 7f40f12f3700 1 --2- 172.21.15.19:0/1371092683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ec075590 0x7f40ec10ef30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:04.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.806+0000 7f40f12f3700 1 -- 172.21.15.19:0/1371092683 >> 172.21.15.19:0/1371092683 conn(0x7f40ec079bb0 msgr2=0x7f40ec079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:04.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.806+0000 7f40f12f3700 1 -- 172.21.15.19:0/1371092683 shutdown_connections 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.806+0000 7f40f12f3700 1 -- 172.21.15.19:0/1371092683 wait complete. 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.807+0000 7f40f12f3700 1 Processor -- start 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.807+0000 7f40f12f3700 1 -- start start 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.807+0000 7f40f12f3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40ec074c70 0x7f40ec19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.807+0000 7f40f12f3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40ec075590 0x7f40ec19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.807+0000 7f40f12f3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ec10f470 0x7f40ec19fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.807+0000 7f40f12f3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec114e30 con 0x7f40ec10f470 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.807+0000 7f40f12f3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec114cb0 con 0x7f40ec075590 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.807+0000 7f40f12f3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec114fb0 con 0x7f40ec074c70 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40eb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ec10f470 0x7f40ec19fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40eb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ec10f470 0x7f40ec19fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55196/0 (socket says 172.21.15.19:55196) 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40eb7fe700 1 -- 172.21.15.19:0/3644321974 learned_addr learned my addr 172.21.15.19:0/3644321974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40ea7fc700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40ec075590 0x7f40ec19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:04.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40eaffd700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40ec074c70 0x7f40ec19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40eb7fe700 1 -- 172.21.15.19:0/3644321974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40ec074c70 msgr2=0x7f40ec19b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40eb7fe700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40ec074c70 0x7f40ec19b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40eb7fe700 1 -- 172.21.15.19:0/3644321974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40ec075590 msgr2=0x7f40ec19b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40eb7fe700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40ec075590 0x7f40ec19b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40eb7fe700 1 -- 172.21.15.19:0/3644321974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec1a0260 con 0x7f40ec10f470 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40eb7fe700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ec10f470 0x7f40ec19fb20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f40e400ada0 tx=0x7f40e400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40e3fff700 1 -- 172.21.15.19:0/3644321974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4017020 con 0x7f40ec10f470 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40e3fff700 1 -- 172.21.15.19:0/3644321974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e400ce30 con 0x7f40ec10f470 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40f12f3700 1 -- 172.21.15.19:0/3644321974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec1a0550 con 0x7f40ec10f470 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40e3fff700 1 -- 172.21.15.19:0/3644321974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4005310 con 0x7f40ec10f470 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.808+0000 7f40f12f3700 1 -- 172.21.15.19:0/3644321974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec1a7dd0 con 0x7f40ec10f470 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.809+0000 7f40f12f3700 1 -- 172.21.15.19:0/3644321974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ec069730 con 0x7f40ec10f470 2024-04-23T18:25:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.810+0000 7f40e3fff700 1 -- 172.21.15.19:0/3644321974 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f40e400c980 con 0x7f40ec10f470 2024-04-23T18:25:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.811+0000 7f40e3fff700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40d80a3cd0 0x7f40d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.811+0000 7f40eaffd700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40d80a3cd0 0x7f40d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.811+0000 7f40e3fff700 1 -- 172.21.15.19:0/3644321974 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f40e40cfc00 con 0x7f40ec10f470 2024-04-23T18:25:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.811+0000 7f40eaffd700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40d80a3cd0 0x7f40d80a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f40dc005ea0 tx=0x7f40dc005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:03.816+0000 7f40e3fff700 1 -- 172.21.15.19:0/3644321974 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40e4021020 con 0x7f40ec10f470 2024-04-23T18:25:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.024+0000 7f40f12f3700 1 -- 172.21.15.19:0/3644321974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f40ec1a7fa0 con 0x7f40ec10f470 2024-04-23T18:25:04.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.024+0000 7f40e3fff700 1 -- 172.21.15.19:0/3644321974 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f40e40192b0 con 0x7f40ec10f470 2024-04-23T18:25:04.462 INFO:teuthology.orchestra.run.smithi019.stdout:141733920831 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 -- 172.21.15.19:0/3644321974 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40d80a3cd0 msgr2=0x7f40d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40d80a3cd0 0x7f40d80a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f40dc005ea0 tx=0x7f40dc005e30 comp rx=0 tx=0).stop 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 -- 172.21.15.19:0/3644321974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ec10f470 msgr2=0x7f40ec19fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ec10f470 0x7f40ec19fb20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f40e400ada0 tx=0x7f40e400c330 comp rx=0 tx=0).stop 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 -- 172.21.15.19:0/3644321974 shutdown_connections 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f40d80a3cd0 0x7f40d80a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ec10f470 0x7f40ec19fb20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40ec074c70 0x7f40ec19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 --2- 172.21.15.19:0/3644321974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40ec075590 0x7f40ec19b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 -- 172.21.15.19:0/3644321974 >> 172.21.15.19:0/3644321974 conn(0x7f40ec079bb0 msgr2=0x7f40ec0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 -- 172.21.15.19:0/3644321974 shutdown_connections 2024-04-23T18:25:04.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:04.026+0000 7f40f12f3700 1 -- 172.21.15.19:0/3644321974 wait complete. 2024-04-23T18:25:04.464 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:04 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2483843082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:25:04.464 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:04 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3644321974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:25:04.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:04 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2483843082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:25:04.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:04 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3644321974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:25:04.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:04 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2483843082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:25:04.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:04 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3644321974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:25:05.080 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037797 got 257698037802 for osd.8 2024-04-23T18:25:05.081 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:05.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:05 smithi019 ceph-mon[29215]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:25:05.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:05 smithi044 ceph-mon[32458]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:25:05.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:05 smithi155 ceph-mon[33468]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:25:05.984 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920831 for osd.4 2024-04-23T18:25:05.985 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.194+0000 7fab2f2cb700 1 -- 172.21.15.19:0/1982773283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab28106990 msgr2=0x7fab28106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.194+0000 7fab2f2cb700 1 --2- 172.21.15.19:0/1982773283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab28106990 0x7fab28106d70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b0a0 tx=0x7fab2400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.194+0000 7fab2f2cb700 1 -- 172.21.15.19:0/1982773283 shutdown_connections 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.194+0000 7fab2f2cb700 1 --2- 172.21.15.19:0/1982773283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab281026e0 0x7fab2810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.194+0000 7fab2f2cb700 1 --2- 172.21.15.19:0/1982773283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab28106990 0x7fab28106d70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.194+0000 7fab2f2cb700 1 --2- 172.21.15.19:0/1982773283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab28101d20 0x7fab281021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.194+0000 7fab2f2cb700 1 -- 172.21.15.19:0/1982773283 >> 172.21.15.19:0/1982773283 conn(0x7fab280788f0 msgr2=0x7fab28078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.194+0000 7fab2f2cb700 1 -- 172.21.15.19:0/1982773283 shutdown_connections 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.195+0000 7fab2f2cb700 1 -- 172.21.15.19:0/1982773283 wait complete. 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.195+0000 7fab2f2cb700 1 Processor -- start 2024-04-23T18:25:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.195+0000 7fab2f2cb700 1 -- start start 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.195+0000 7fab2f2cb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab28101d20 0x7fab28199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.195+0000 7fab2f2cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab281026e0 0x7fab281996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.195+0000 7fab2f2cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab28106990 0x7fab2819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.195+0000 7fab2f2cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab28111ac0 con 0x7fab281026e0 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.195+0000 7fab2f2cb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fab28111940 con 0x7fab28101d20 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.195+0000 7fab2f2cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab28111c40 con 0x7fab28106990 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.195+0000 7fab2c866700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab281026e0 0x7fab281996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2c866700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab281026e0 0x7fab281996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55210/0 (socket says 172.21.15.19:55210) 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2c866700 1 -- 172.21.15.19:0/3881118999 learned_addr learned my addr 172.21.15.19:0/3881118999 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2d067700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab28101d20 0x7fab28199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2d868700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab28106990 0x7fab2819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2c866700 1 -- 172.21.15.19:0/3881118999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab28106990 msgr2=0x7fab2819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:07.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2c866700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab28106990 0x7fab2819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2c866700 1 -- 172.21.15.19:0/3881118999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab28101d20 msgr2=0x7fab28199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2c866700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab28101d20 0x7fab28199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2c866700 1 -- 172.21.15.19:0/3881118999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2819e140 con 0x7fab281026e0 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2d868700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab28106990 0x7fab2819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2c866700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab281026e0 0x7fab281996c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab1800d950 tx=0x7fab1800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab1e7fc700 1 -- 172.21.15.19:0/3881118999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18016070 con 0x7fab281026e0 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab1e7fc700 1 -- 172.21.15.19:0/3881118999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab18009df0 con 0x7fab281026e0 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab1e7fc700 1 -- 172.21.15.19:0/3881118999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1800f3c0 con 0x7fab281026e0 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2f2cb700 1 -- 172.21.15.19:0/3881118999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2819e430 con 0x7fab281026e0 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.196+0000 7fab2f2cb700 1 -- 172.21.15.19:0/3881118999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab281a5cb0 con 0x7fab281026e0 2024-04-23T18:25:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.197+0000 7fab2f2cb700 1 -- 172.21.15.19:0/3881118999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab28103080 con 0x7fab281026e0 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stdout:292057776166 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.198+0000 7fab1e7fc700 1 -- 172.21.15.19:0/3881118999 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fab18019070 con 0x7fab281026e0 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.199+0000 7fab1e7fc700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fab140a3cc0 0x7fab140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.199+0000 7fab2d067700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fab140a3cc0 0x7fab140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.199+0000 7fab1e7fc700 1 -- 172.21.15.19:0/3881118999 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fab180cff40 con 0x7fab281026e0 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.200+0000 7fab2d067700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fab140a3cc0 0x7fab140a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fab24009e60 tx=0x7fab2400b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.204+0000 7fab1e7fc700 1 -- 172.21.15.19:0/3881118999 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab1809c930 con 0x7fab281026e0 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.417+0000 7fab2f2cb700 1 -- 172.21.15.19:0/3881118999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fab281a5f10 con 0x7fab281026e0 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.417+0000 7fab1e7fc700 1 -- 172.21.15.19:0/3881118999 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fab1809c230 con 0x7fab281026e0 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 -- 172.21.15.19:0/3881118999 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fab140a3cc0 msgr2=0x7fab140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:07.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fab140a3cc0 0x7fab140a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fab24009e60 tx=0x7fab2400b7b0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 -- 172.21.15.19:0/3881118999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab281026e0 msgr2=0x7fab281996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab281026e0 0x7fab281996c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab1800d950 tx=0x7fab1800dc60 comp rx=0 tx=0).stop 2024-04-23T18:25:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 -- 172.21.15.19:0/3881118999 shutdown_connections 2024-04-23T18:25:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fab140a3cc0 0x7fab140a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab28106990 0x7fab2819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab28101d20 0x7fab28199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 --2- 172.21.15.19:0/3881118999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab281026e0 0x7fab281996c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 -- 172.21.15.19:0/3881118999 >> 172.21.15.19:0/3881118999 conn(0x7fab280788f0 msgr2=0x7fab280feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 -- 172.21.15.19:0/3881118999 shutdown_connections 2024-04-23T18:25:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:06.419+0000 7fab2f2cb700 1 -- 172.21.15.19:0/3881118999 wait complete. 2024-04-23T18:25:07.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:07 smithi019 ceph-mon[29215]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:07.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:07 smithi019 ceph-mon[29215]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:25:07.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:07 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3881118999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:25:07.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:07 smithi044 ceph-mon[32458]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:07.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:07 smithi044 ceph-mon[32458]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:25:07.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:07 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3881118999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:25:07.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:07 smithi155 ceph-mon[33468]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:07.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:07 smithi155 ceph-mon[33468]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:25:07.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:07 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3881118999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:25:07.920 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776166 for osd.9 2024-04-23T18:25:07.921 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:08.407 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724598 got 167503724603 for osd.5 2024-04-23T18:25:08.407 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:09.458 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:09 smithi019 ceph-mon[29215]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:25:09.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:09 smithi044 ceph-mon[32458]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:25:09.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:09 smithi155 ceph-mon[33468]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:25:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.511+0000 7fe1a3395700 1 -- 172.21.15.19:0/2866239757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe19c103480 msgr2=0x7fe19c10f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.511+0000 7fe1a3395700 1 --2- 172.21.15.19:0/2866239757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe19c103480 0x7fe19c10f980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe19400b0a0 tx=0x7fe19400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.511+0000 7fe1a3395700 1 -- 172.21.15.19:0/2866239757 shutdown_connections 2024-04-23T18:25:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.511+0000 7fe1a3395700 1 --2- 172.21.15.19:0/2866239757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe19c103480 0x7fe19c10f980 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.511+0000 7fe1a3395700 1 --2- 172.21.15.19:0/2866239757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe19c108ab0 0x7fe19c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.511+0000 7fe1a3395700 1 --2- 172.21.15.19:0/2866239757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c102ac0 0x7fe19c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.511+0000 7fe1a3395700 1 -- 172.21.15.19:0/2866239757 >> 172.21.15.19:0/2866239757 conn(0x7fe19c079bf0 msgr2=0x7fe19c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.511+0000 7fe1a3395700 1 -- 172.21.15.19:0/2866239757 shutdown_connections 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.511+0000 7fe1a3395700 1 -- 172.21.15.19:0/2866239757 wait complete. 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a3395700 1 Processor -- start 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a3395700 1 -- start start 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a3395700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c102ac0 0x7fe19c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a3395700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe19c103480 0x7fe19c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a3395700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe19c108ab0 0x7fe19c19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a3395700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c112cf0 con 0x7fe19c108ab0 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a3395700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c112b70 con 0x7fe19c102ac0 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a3395700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c112e70 con 0x7fe19c103480 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1932700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe19c108ab0 0x7fe19c19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1932700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe19c108ab0 0x7fe19c19d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54936/0 (socket says 172.21.15.19:54936) 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1932700 1 -- 172.21.15.19:0/2166671160 learned_addr learned my addr 172.21.15.19:0/2166671160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1131700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c102ac0 0x7fe19c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a0930700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe19c103480 0x7fe19c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1932700 1 -- 172.21.15.19:0/2166671160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe19c103480 msgr2=0x7fe19c1995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1932700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe19c103480 0x7fe19c1995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1932700 1 -- 172.21.15.19:0/2166671160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c102ac0 msgr2=0x7fe19c1990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1932700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c102ac0 0x7fe19c1990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1932700 1 -- 172.21.15.19:0/2166671160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c19e060 con 0x7fe19c108ab0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1131700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c102ac0 0x7fe19c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.512+0000 7fe1a1932700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe19c108ab0 0x7fe19c19d920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe19400b830 tx=0x7fe194004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.513+0000 7fe1927fc700 1 -- 172.21.15.19:0/2166671160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194021070 con 0x7fe19c108ab0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.513+0000 7fe1a3395700 1 -- 172.21.15.19:0/2166671160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c19e2f0 con 0x7fe19c108ab0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.513+0000 7fe1927fc700 1 -- 172.21.15.19:0/2166671160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe194003c00 con 0x7fe19c108ab0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.513+0000 7fe1927fc700 1 -- 172.21.15.19:0/2166671160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194004730 con 0x7fe19c108ab0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.513+0000 7fe1a3395700 1 -- 172.21.15.19:0/2166671160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c1a5bd0 con 0x7fe19c108ab0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.514+0000 7fe1a3395700 1 -- 172.21.15.19:0/2166671160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c069730 con 0x7fe19c108ab0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.515+0000 7fe1927fc700 1 -- 172.21.15.19:0/2166671160 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe194003db0 con 0x7fe19c108ab0 2024-04-23T18:25:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.516+0000 7fe1927fc700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe1880a3c40 0x7fe1880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.516+0000 7fe1a1131700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe1880a3c40 0x7fe1880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.516+0000 7fe1927fc700 1 -- 172.21.15.19:0/2166671160 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe19400f020 con 0x7fe19c108ab0 2024-04-23T18:25:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.516+0000 7fe1a1131700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe1880a3c40 0x7fe1880a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe198005e10 tx=0x7fe198005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.519+0000 7fe1927fc700 1 -- 172.21.15.19:0/2166671160 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe19402a020 con 0x7fe19c108ab0 2024-04-23T18:25:10.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.731+0000 7fe1a3395700 1 -- 172.21.15.19:0/2166671160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe19c02df80 con 0x7fe19c108ab0 2024-04-23T18:25:10.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.732+0000 7fe1927fc700 1 -- 172.21.15.19:0/2166671160 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe19409e960 con 0x7fe19c108ab0 2024-04-23T18:25:10.725 INFO:teuthology.orchestra.run.smithi019.stdout:223338299442 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 -- 172.21.15.19:0/2166671160 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe1880a3c40 msgr2=0x7fe1880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe1880a3c40 0x7fe1880a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe198005e10 tx=0x7fe198005d80 comp rx=0 tx=0).stop 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 -- 172.21.15.19:0/2166671160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe19c108ab0 msgr2=0x7fe19c19d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe19c108ab0 0x7fe19c19d920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe19400b830 tx=0x7fe194004d90 comp rx=0 tx=0).stop 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 -- 172.21.15.19:0/2166671160 shutdown_connections 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe1880a3c40 0x7fe1880a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe19c108ab0 0x7fe19c19d920 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c102ac0 0x7fe19c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 --2- 172.21.15.19:0/2166671160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe19c103480 0x7fe19c1995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 -- 172.21.15.19:0/2166671160 >> 172.21.15.19:0/2166671160 conn(0x7fe19c079bf0 msgr2=0x7fe19c100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:10.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 -- 172.21.15.19:0/2166671160 shutdown_connections 2024-04-23T18:25:10.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.734+0000 7fe1a3395700 1 -- 172.21.15.19:0/2166671160 wait complete. 2024-04-23T18:25:11.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:11 smithi019 ceph-mon[29215]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:11.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:11 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/232068280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:25:11.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:11 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2166671160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.434+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/2554798836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa8107990 msgr2=0x7f0aa8107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.434+0000 7f0aaf3b0700 1 --2- 172.21.15.19:0/2554798836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa8107990 0x7f0aa8107d70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009990 tx=0x7f0aa4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.434+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/2554798836 shutdown_connections 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.434+0000 7f0aaf3b0700 1 --2- 172.21.15.19:0/2554798836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa8102300 0x7f0aa810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.434+0000 7f0aaf3b0700 1 --2- 172.21.15.19:0/2554798836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa8107990 0x7f0aa8107d70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.434+0000 7f0aaf3b0700 1 --2- 172.21.15.19:0/2554798836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0aa8101940 0x7f0aa8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.434+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/2554798836 >> 172.21.15.19:0/2554798836 conn(0x7f0aa80788f0 msgr2=0x7f0aa8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.434+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/2554798836 shutdown_connections 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.434+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/2554798836 wait complete. 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aaf3b0700 1 Processor -- start 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aaf3b0700 1 -- start start 2024-04-23T18:25:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aaf3b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa8101940 0x7f0aa819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aaf3b0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0aa8102300 0x7f0aa819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aaf3b0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa8107990 0x7f0aa81a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aaf3b0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8111bb0 con 0x7f0aa8101940 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aaf3b0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8111a30 con 0x7f0aa8107990 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aaf3b0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8111d30 con 0x7f0aa8102300 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aad14c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa8101940 0x7f0aa819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aad14c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa8101940 0x7f0aa819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54920/0 (socket says 172.21.15.19:54920) 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aad14c700 1 -- 172.21.15.19:0/232068280 learned_addr learned my addr 172.21.15.19:0/232068280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aac94b700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0aa8102300 0x7f0aa819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.435+0000 7f0aad94d700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa8107990 0x7f0aa81a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0aad14c700 1 -- 172.21.15.19:0/232068280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0aa8102300 msgr2=0x7f0aa819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0aad14c700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0aa8102300 0x7f0aa819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0aad14c700 1 -- 172.21.15.19:0/232068280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa8107990 msgr2=0x7f0aa81a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0aad14c700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa8107990 0x7f0aa81a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0aad14c700 1 -- 172.21.15.19:0/232068280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa81a55f0 con 0x7f0aa8101940 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0aad14c700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa8101940 0x7f0aa819a030 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4005910 tx=0x7f0aa40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0a9e7fc700 1 -- 172.21.15.19:0/232068280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4027070 con 0x7f0aa8101940 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0a9e7fc700 1 -- 172.21.15.19:0/232068280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aa4003f20 con 0x7f0aa8101940 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/232068280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa81a57c0 con 0x7f0aa8101940 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0a9e7fc700 1 -- 172.21.15.19:0/232068280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4022c60 con 0x7f0aa8101940 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.436+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/232068280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa81a5c70 con 0x7f0aa8101940 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.437+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/232068280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa8109eb0 con 0x7f0aa8101940 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.438+0000 7f0a9e7fc700 1 -- 172.21.15.19:0/232068280 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0aa402a070 con 0x7f0aa8101940 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.439+0000 7f0a9e7fc700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0a940a3cc0 0x7f0a940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.439+0000 7f0aac94b700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0a940a3cc0 0x7f0a940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:11.265 INFO:teuthology.orchestra.run.smithi019.stdout:77309411404 2024-04-23T18:25:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.439+0000 7f0a9e7fc700 1 -- 172.21.15.19:0/232068280 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0aa4022e00 con 0x7f0aa8101940 2024-04-23T18:25:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.439+0000 7f0aac94b700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0a940a3cc0 0x7f0a940a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8078fd0 tx=0x7f0a9800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.444+0000 7f0a9e7fc700 1 -- 172.21.15.19:0/232068280 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0aa405eff0 con 0x7f0aa8101940 2024-04-23T18:25:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.659+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/232068280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0aa806ad10 con 0x7f0aa8101940 2024-04-23T18:25:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.659+0000 7f0a9e7fc700 1 -- 172.21.15.19:0/232068280 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0aa4013070 con 0x7f0aa8101940 2024-04-23T18:25:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/232068280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0a940a3cc0 msgr2=0x7f0a940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0a940a3cc0 0x7f0a940a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8078fd0 tx=0x7f0a9800b410 comp rx=0 tx=0).stop 2024-04-23T18:25:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/232068280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa8101940 msgr2=0x7f0aa819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa8101940 0x7f0aa819a030 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4005910 tx=0x7f0aa40047b0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/232068280 shutdown_connections 2024-04-23T18:25:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f0a940a3cc0 0x7f0a940a6180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa8107990 0x7f0aa81a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa8101940 0x7f0aa819a030 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 --2- 172.21.15.19:0/232068280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0aa8102300 0x7f0aa819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/232068280 >> 172.21.15.19:0/232068280 conn(0x7f0aa80788f0 msgr2=0x7f0aa81007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/232068280 shutdown_connections 2024-04-23T18:25:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:10.661+0000 7f0aaf3b0700 1 -- 172.21.15.19:0/232068280 wait complete. 2024-04-23T18:25:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:11 smithi044 ceph-mon[32458]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:11 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/232068280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:25:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:11 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2166671160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:25:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:11 smithi155 ceph-mon[33468]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:11 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/232068280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:25:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:11 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2166671160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:25:12.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.987+0000 7f4823c20700 1 -- 172.21.15.19:0/1539566713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f481c103480 msgr2=0x7f481c10f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.987+0000 7f4823c20700 1 --2- 172.21.15.19:0/1539566713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f481c103480 0x7f481c10f8d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f481400b0a0 tx=0x7f481400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.987+0000 7f4823c20700 1 -- 172.21.15.19:0/1539566713 shutdown_connections 2024-04-23T18:25:12.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.987+0000 7f4823c20700 1 --2- 172.21.15.19:0/1539566713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f481c103480 0x7f481c10f8d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.987+0000 7f4823c20700 1 --2- 172.21.15.19:0/1539566713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f481c108a70 0x7f481c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.987+0000 7f4823c20700 1 --2- 172.21.15.19:0/1539566713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f481c102ac0 0x7f481c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.987+0000 7f4823c20700 1 -- 172.21.15.19:0/1539566713 >> 172.21.15.19:0/1539566713 conn(0x7f481c079bb0 msgr2=0x7f481c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f4823c20700 1 -- 172.21.15.19:0/1539566713 shutdown_connections 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f4823c20700 1 -- 172.21.15.19:0/1539566713 wait complete. 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f4823c20700 1 Processor -- start 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f4823c20700 1 -- start start 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f4823c20700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f481c102ac0 0x7f481c199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f4823c20700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f481c103480 0x7f481c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f4823c20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f481c108a70 0x7f481c19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f4823c20700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f481c112c40 con 0x7f481c108a70 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f4823c20700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f481c112ac0 con 0x7f481c102ac0 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f4823c20700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f481c112dc0 con 0x7f481c103480 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.988+0000 7f48221bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f481c108a70 0x7f481c19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48221bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f481c108a70 0x7f481c19d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54944/0 (socket says 172.21.15.19:54944) 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48221bd700 1 -- 172.21.15.19:0/3795405334 learned_addr learned my addr 172.21.15.19:0/3795405334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48219bc700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f481c102ac0 0x7f481c199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48211bb700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f481c103480 0x7f481c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48221bd700 1 -- 172.21.15.19:0/3795405334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f481c103480 msgr2=0x7f481c1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48221bd700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f481c103480 0x7f481c1995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48221bd700 1 -- 172.21.15.19:0/3795405334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f481c102ac0 msgr2=0x7f481c199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48221bd700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f481c102ac0 0x7f481c199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48221bd700 1 -- 172.21.15.19:0/3795405334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c19e050 con 0x7f481c108a70 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48211bb700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f481c103480 0x7f481c1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f48221bd700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f481c108a70 0x7f481c19d910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f481400b830 tx=0x7f4814004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f4812ffd700 1 -- 172.21.15.19:0/3795405334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814021070 con 0x7f481c108a70 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f4823c20700 1 -- 172.21.15.19:0/3795405334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c19e2e0 con 0x7f481c108a70 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f4812ffd700 1 -- 172.21.15.19:0/3795405334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4814007e30 con 0x7f481c108a70 2024-04-23T18:25:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f4812ffd700 1 -- 172.21.15.19:0/3795405334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140047d0 con 0x7f481c108a70 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.989+0000 7f4823c20700 1 -- 172.21.15.19:0/3795405334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c1a5bc0 con 0x7f481c108a70 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.990+0000 7f4823c20700 1 -- 172.21.15.19:0/3795405334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481c069730 con 0x7f481c108a70 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.991+0000 7f4812ffd700 1 -- 172.21.15.19:0/3795405334 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4814007980 con 0x7f481c108a70 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.992+0000 7f4812ffd700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48080a3c40 0x7f48080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.992+0000 7f48219bc700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48080a3c40 0x7f48080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.992+0000 7f4812ffd700 1 -- 172.21.15.19:0/3795405334 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f481400f020 con 0x7f481c108a70 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.993+0000 7f48219bc700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48080a3c40 0x7f48080a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f481800abd0 tx=0x7f481800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:11.998+0000 7f4812ffd700 1 -- 172.21.15.19:0/3795405334 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f481402a020 con 0x7f481c108a70 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.208+0000 7f4823c20700 1 -- 172.21.15.19:0/3795405334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f481c02df80 con 0x7f481c108a70 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.208+0000 7f4812ffd700 1 -- 172.21.15.19:0/3795405334 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f481409e7e0 con 0x7f481c108a70 2024-04-23T18:25:12.233 INFO:teuthology.orchestra.run.smithi019.stdout:107374182469 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.210+0000 7f4823c20700 1 -- 172.21.15.19:0/3795405334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48080a3c40 msgr2=0x7f48080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.210+0000 7f4823c20700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48080a3c40 0x7f48080a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f481800abd0 tx=0x7f481800a380 comp rx=0 tx=0).stop 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.210+0000 7f4823c20700 1 -- 172.21.15.19:0/3795405334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f481c108a70 msgr2=0x7f481c19d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.210+0000 7f4823c20700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f481c108a70 0x7f481c19d910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f481400b830 tx=0x7f4814004ce0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.210+0000 7f4823c20700 1 -- 172.21.15.19:0/3795405334 shutdown_connections 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.210+0000 7f4823c20700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f48080a3c40 0x7f48080a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.211+0000 7f4823c20700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f481c108a70 0x7f481c19d910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.211+0000 7f4823c20700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f481c102ac0 0x7f481c199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.211+0000 7f4823c20700 1 --2- 172.21.15.19:0/3795405334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f481c103480 0x7f481c1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.211+0000 7f4823c20700 1 -- 172.21.15.19:0/3795405334 >> 172.21.15.19:0/3795405334 conn(0x7f481c079bb0 msgr2=0x7f481c1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.211+0000 7f4823c20700 1 -- 172.21.15.19:0/3795405334 shutdown_connections 2024-04-23T18:25:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.211+0000 7f4823c20700 1 -- 172.21.15.19:0/3795405334 wait complete. 2024-04-23T18:25:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.687+0000 7fcb70a16700 1 -- 172.21.15.19:0/347265526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6c103450 msgr2=0x7fcb6c10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.687+0000 7fcb70a16700 1 --2- 172.21.15.19:0/347265526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6c103450 0x7fcb6c10f8b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400b0a0 tx=0x7fcb6400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.688+0000 7fcb70a16700 1 -- 172.21.15.19:0/347265526 shutdown_connections 2024-04-23T18:25:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.688+0000 7fcb70a16700 1 --2- 172.21.15.19:0/347265526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6c103450 0x7fcb6c10f8b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.688+0000 7fcb70a16700 1 --2- 172.21.15.19:0/347265526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6c108a50 0x7fcb6c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.688+0000 7fcb70a16700 1 --2- 172.21.15.19:0/347265526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c102a90 0x7fcb6c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.688+0000 7fcb70a16700 1 -- 172.21.15.19:0/347265526 >> 172.21.15.19:0/347265526 conn(0x7fcb6c079ba0 msgr2=0x7fcb6c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.688+0000 7fcb70a16700 1 -- 172.21.15.19:0/347265526 shutdown_connections 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.688+0000 7fcb70a16700 1 -- 172.21.15.19:0/347265526 wait complete. 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.688+0000 7fcb70a16700 1 Processor -- start 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.688+0000 7fcb70a16700 1 -- start start 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb70a16700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c102a90 0x7fcb6c199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb70a16700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6c103450 0x7fcb6c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb70a16700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6c108a50 0x7fcb6c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb70a16700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c112be0 con 0x7fcb6c103450 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb70a16700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c112a60 con 0x7fcb6c108a50 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb70a16700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c112d60 con 0x7fcb6c102a90 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb69d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6c103450 0x7fcb6c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb69d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6c103450 0x7fcb6c199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54954/0 (socket says 172.21.15.19:54954) 2024-04-23T18:25:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb69d9b700 1 -- 172.21.15.19:0/2371690533 learned_addr learned my addr 172.21.15.19:0/2371690533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb6ad9d700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6c108a50 0x7fcb6c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb6a59c700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c102a90 0x7fcb6c199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb69d9b700 1 -- 172.21.15.19:0/2371690533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c102a90 msgr2=0x7fcb6c199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb69d9b700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c102a90 0x7fcb6c199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb69d9b700 1 -- 172.21.15.19:0/2371690533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6c108a50 msgr2=0x7fcb6c19d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb69d9b700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6c108a50 0x7fcb6c19d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb69d9b700 1 -- 172.21.15.19:0/2371690533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c19e040 con 0x7fcb6c103450 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb6a59c700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c102a90 0x7fcb6c199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.689+0000 7fcb69d9b700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6c103450 0x7fcb6c199570 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000c9b0 tx=0x7fcb6000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.690+0000 7fcb5b7fe700 1 -- 172.21.15.19:0/2371690533 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60007b00 con 0x7fcb6c103450 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.690+0000 7fcb5b7fe700 1 -- 172.21.15.19:0/2371690533 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb60007ca0 con 0x7fcb6c103450 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.690+0000 7fcb70a16700 1 -- 172.21.15.19:0/2371690533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6c19e330 con 0x7fcb6c103450 2024-04-23T18:25:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.690+0000 7fcb5b7fe700 1 -- 172.21.15.19:0/2371690533 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60018660 con 0x7fcb6c103450 2024-04-23T18:25:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.690+0000 7fcb70a16700 1 -- 172.21.15.19:0/2371690533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6c1a5bb0 con 0x7fcb6c103450 2024-04-23T18:25:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.691+0000 7fcb70a16700 1 -- 172.21.15.19:0/2371690533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6c069730 con 0x7fcb6c103450 2024-04-23T18:25:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.692+0000 7fcb5b7fe700 1 -- 172.21.15.19:0/2371690533 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcb6000f450 con 0x7fcb6c103450 2024-04-23T18:25:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.693+0000 7fcb5b7fe700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb540a3cc0 0x7fcb540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.693+0000 7fcb6a59c700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb540a3cc0 0x7fcb540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.693+0000 7fcb5b7fe700 1 -- 172.21.15.19:0/2371690533 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fcb600d0660 con 0x7fcb6c103450 2024-04-23T18:25:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.693+0000 7fcb6a59c700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb540a3cc0 0x7fcb540a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00baa0 tx=0x7fcb5c005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.697+0000 7fcb5b7fe700 1 -- 172.21.15.19:0/2371690533 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb60014030 con 0x7fcb6c103450 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.903+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2198863974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b8107940 msgr2=0x7f80b8107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.903+0000 7f80bfe5b700 1 --2- 172.21.15.19:0/2198863974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b8107940 0x7f80b8107d20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f80b40099c0 tx=0x7f80b4009cd0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.904+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2198863974 shutdown_connections 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.904+0000 7f80bfe5b700 1 --2- 172.21.15.19:0/2198863974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b81034b0 0x7f80b810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.904+0000 7f80bfe5b700 1 --2- 172.21.15.19:0/2198863974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b8107940 0x7f80b8107d20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.904+0000 7f80bfe5b700 1 --2- 172.21.15.19:0/2198863974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80b8102af0 0x7f80b8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.904+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2198863974 >> 172.21.15.19:0/2198863974 conn(0x7f80b8079ba0 msgr2=0x7f80b8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.904+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2198863974 shutdown_connections 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.904+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2198863974 wait complete. 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.904+0000 7f80bfe5b700 1 Processor -- start 2024-04-23T18:25:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.904+0000 7f80bfe5b700 1 -- start start 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bfe5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b8102af0 0x7f80b8198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bfe5b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b81034b0 0x7f80b8199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bfe5b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80b8107940 0x7f80b819d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bfe5b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8112c60 con 0x7f80b8102af0 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bfe5b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8112ae0 con 0x7f80b8107940 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bfe5b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8112de0 con 0x7f80b81034b0 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bdbf7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b8102af0 0x7f80b8198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bdbf7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b8102af0 0x7f80b8198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54980/0 (socket says 172.21.15.19:54980) 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bdbf7700 1 -- 172.21.15.19:0/2155406223 learned_addr learned my addr 172.21.15.19:0/2155406223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80be3f8700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80b8107940 0x7f80b819d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bd3f6700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b81034b0 0x7f80b8199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bdbf7700 1 -- 172.21.15.19:0/2155406223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b81034b0 msgr2=0x7f80b8199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bdbf7700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b81034b0 0x7f80b8199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bdbf7700 1 -- 172.21.15.19:0/2155406223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80b8107940 msgr2=0x7f80b819d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bdbf7700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80b8107940 0x7f80b819d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bdbf7700 1 -- 172.21.15.19:0/2155406223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b819e000 con 0x7f80b8102af0 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bd3f6700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b81034b0 0x7f80b8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.905+0000 7f80bdbf7700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b8102af0 0x7f80b8198ff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009fd0 tx=0x7f80b4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.906+0000 7f80aeffd700 1 -- 172.21.15.19:0/2155406223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4003780 con 0x7f80b8102af0 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.906+0000 7f80aeffd700 1 -- 172.21.15.19:0/2155406223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b4004ce0 con 0x7f80b8102af0 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.906+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2155406223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b819e290 con 0x7f80b8102af0 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.906+0000 7f80aeffd700 1 -- 172.21.15.19:0/2155406223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4003dd0 con 0x7f80b8102af0 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.906+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2155406223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b81a5b70 con 0x7f80b8102af0 2024-04-23T18:25:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.907+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2155406223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b8069730 con 0x7f80b8102af0 2024-04-23T18:25:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.909+0000 7f80aeffd700 1 -- 172.21.15.19:0/2155406223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f80b4029060 con 0x7f80b8102af0 2024-04-23T18:25:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.909+0000 7f80aeffd700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80a40a3de0 0x7f80a40a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.909+0000 7f80aeffd700 1 -- 172.21.15.19:0/2155406223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f80b40d0e60 con 0x7f80b8102af0 2024-04-23T18:25:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.910+0000 7f80bd3f6700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80a40a3de0 0x7f80a40a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.910+0000 7f80bd3f6700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80a40a3de0 0x7f80a40a62a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f80a8005fd0 tx=0x7f80a8005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:12.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.911+0000 7fcb70a16700 1 -- 172.21.15.19:0/2371690533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcb6c19a3c0 con 0x7fcb6c103450 2024-04-23T18:25:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.916+0000 7fcb5b7fe700 1 -- 172.21.15.19:0/2371690533 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcb6c19a3c0 con 0x7fcb6c103450 2024-04-23T18:25:12.909 INFO:teuthology.orchestra.run.smithi019.stdout:55834574929 2024-04-23T18:25:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.916+0000 7f80aeffd700 1 -- 172.21.15.19:0/2155406223 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80b4013070 con 0x7f80b8102af0 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.918+0000 7fcb70a16700 1 -- 172.21.15.19:0/2371690533 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb540a3cc0 msgr2=0x7fcb540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb540a3cc0 0x7fcb540a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00baa0 tx=0x7fcb5c005db0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 -- 172.21.15.19:0/2371690533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6c103450 msgr2=0x7fcb6c199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6c103450 0x7fcb6c199570 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000c9b0 tx=0x7fcb6000ccc0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 -- 172.21.15.19:0/2371690533 shutdown_connections 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcb540a3cc0 0x7fcb540a6180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6c108a50 0x7fcb6c19d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c102a90 0x7fcb6c199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 --2- 172.21.15.19:0/2371690533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6c103450 0x7fcb6c199570 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 -- 172.21.15.19:0/2371690533 >> 172.21.15.19:0/2371690533 conn(0x7fcb6c079ba0 msgr2=0x7fcb6c1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 -- 172.21.15.19:0/2371690533 shutdown_connections 2024-04-23T18:25:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:12.919+0000 7fcb70a16700 1 -- 172.21.15.19:0/2371690533 wait complete. 2024-04-23T18:25:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.126+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2155406223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f80b81a5d40 con 0x7f80b8102af0 2024-04-23T18:25:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.127+0000 7f80aeffd700 1 -- 172.21.15.19:0/2155406223 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f80b40163a0 con 0x7f80b8102af0 2024-04-23T18:25:13.120 INFO:teuthology.orchestra.run.smithi019.stdout:347892351005 2024-04-23T18:25:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2155406223 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80a40a3de0 msgr2=0x7f80a40a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80a40a3de0 0x7f80a40a62a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f80a8005fd0 tx=0x7f80a8005e30 comp rx=0 tx=0).stop 2024-04-23T18:25:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2155406223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b8102af0 msgr2=0x7f80b8198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b8102af0 0x7f80b8198ff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009fd0 tx=0x7f80b4005f10 comp rx=0 tx=0).stop 2024-04-23T18:25:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2155406223 shutdown_connections 2024-04-23T18:25:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f80a40a3de0 0x7f80a40a62a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80b8107940 0x7f80b819d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b8102af0 0x7f80b8198ff0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 --2- 172.21.15.19:0/2155406223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b81034b0 0x7f80b8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2155406223 >> 172.21.15.19:0/2155406223 conn(0x7f80b8079ba0 msgr2=0x7f80b8100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2155406223 shutdown_connections 2024-04-23T18:25:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.129+0000 7f80bfe5b700 1 -- 172.21.15.19:0/2155406223 wait complete. 2024-04-23T18:25:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.148+0000 7f534f016700 1 -- 172.21.15.19:0/1868458974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f534806afe0 msgr2=0x7f534806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.148+0000 7f534f016700 1 --2- 172.21.15.19:0/1868458974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f534806afe0 0x7f534806b440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5334009990 tx=0x7f5334009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.148+0000 7f534f016700 1 -- 172.21.15.19:0/1868458974 shutdown_connections 2024-04-23T18:25:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.148+0000 7f534f016700 1 --2- 172.21.15.19:0/1868458974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f534806b980 0x7f534810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.148+0000 7f534f016700 1 --2- 172.21.15.19:0/1868458974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5348108a20 0x7f5348108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.148+0000 7f534f016700 1 --2- 172.21.15.19:0/1868458974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f534806afe0 0x7f534806b440 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.148+0000 7f534f016700 1 -- 172.21.15.19:0/1868458974 >> 172.21.15.19:0/1868458974 conn(0x7f53480788b0 msgr2=0x7f5348078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.148+0000 7f534f016700 1 -- 172.21.15.19:0/1868458974 shutdown_connections 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.148+0000 7f534f016700 1 -- 172.21.15.19:0/1868458974 wait complete. 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534f016700 1 Processor -- start 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534f016700 1 -- start start 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534f016700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f534806afe0 0x7f53481990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534f016700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f534806b980 0x7f53481995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534f016700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5348108a20 0x7f534819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534f016700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5348112be0 con 0x7f5348108a20 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534f016700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5348112a60 con 0x7f534806afe0 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534f016700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5348112d60 con 0x7f534806b980 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534d5b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5348108a20 0x7f534819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534d5b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5348108a20 0x7f534819d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55002/0 (socket says 172.21.15.19:55002) 2024-04-23T18:25:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534d5b3700 1 -- 172.21.15.19:0/2571943252 learned_addr learned my addr 172.21.15.19:0/2571943252 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f534cdb2700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f534806afe0 0x7f53481990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.149+0000 7f5347fff700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f534806b980 0x7f53481995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f534d5b3700 1 -- 172.21.15.19:0/2571943252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f534806b980 msgr2=0x7f53481995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f534d5b3700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f534806b980 0x7f53481995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f534d5b3700 1 -- 172.21.15.19:0/2571943252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f534806afe0 msgr2=0x7f53481990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f534d5b3700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f534806afe0 0x7f53481990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f534d5b3700 1 -- 172.21.15.19:0/2571943252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534819e0c0 con 0x7f5348108a20 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f534cdb2700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f534806afe0 0x7f53481990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f534d5b3700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5348108a20 0x7f534819d980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5340007ee0 tx=0x7f534000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f5345ffb700 1 -- 172.21.15.19:0/2571943252 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340004bb0 con 0x7f5348108a20 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f5345ffb700 1 -- 172.21.15.19:0/2571943252 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5340004d50 con 0x7f5348108a20 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f534f016700 1 -- 172.21.15.19:0/2571943252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534819e3b0 con 0x7f5348108a20 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f5345ffb700 1 -- 172.21.15.19:0/2571943252 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340005920 con 0x7f5348108a20 2024-04-23T18:25:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.150+0000 7f534f016700 1 -- 172.21.15.19:0/2571943252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53481a5c30 con 0x7f5348108a20 2024-04-23T18:25:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.151+0000 7f534f016700 1 -- 172.21.15.19:0/2571943252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5348069730 con 0x7f5348108a20 2024-04-23T18:25:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.153+0000 7f5345ffb700 1 -- 172.21.15.19:0/2571943252 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f534001a020 con 0x7f5348108a20 2024-04-23T18:25:13.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.154+0000 7f5345ffb700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f53380a3cc0 0x7f53380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:13.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:13 smithi019 ceph-mon[29215]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:25:13.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:13 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3795405334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:25:13.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:13 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2371690533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:25:13.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:13 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2155406223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:25:13.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.154+0000 7f5345ffb700 1 -- 172.21.15.19:0/2571943252 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f53400cffc0 con 0x7f5348108a20 2024-04-23T18:25:13.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.154+0000 7f534cdb2700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f53380a3cc0 0x7f53380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:13.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.155+0000 7f534cdb2700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f53380a3cc0 0x7f53380a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f533c005950 tx=0x7f533c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:13.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.161+0000 7f5345ffb700 1 -- 172.21.15.19:0/2571943252 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5340010070 con 0x7f5348108a20 2024-04-23T18:25:13.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.373+0000 7f534f016700 1 -- 172.21.15.19:0/2571943252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f534802df80 con 0x7f5348108a20 2024-04-23T18:25:13.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.374+0000 7f5345ffb700 1 -- 172.21.15.19:0/2571943252 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f534009ca50 con 0x7f5348108a20 2024-04-23T18:25:13.367 INFO:teuthology.orchestra.run.smithi019.stdout:201863462968 2024-04-23T18:25:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.376+0000 7f534f016700 1 -- 172.21.15.19:0/2571943252 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f53380a3cc0 msgr2=0x7f53380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.376+0000 7f534f016700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f53380a3cc0 0x7f53380a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f533c005950 tx=0x7f533c00b410 comp rx=0 tx=0).stop 2024-04-23T18:25:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.376+0000 7f534f016700 1 -- 172.21.15.19:0/2571943252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5348108a20 msgr2=0x7f534819d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.376+0000 7f534f016700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5348108a20 0x7f534819d980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5340007ee0 tx=0x7f534000d3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.376+0000 7f534f016700 1 -- 172.21.15.19:0/2571943252 shutdown_connections 2024-04-23T18:25:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.376+0000 7f534f016700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f53380a3cc0 0x7f53380a6180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.376+0000 7f534f016700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5348108a20 0x7f534819d980 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.376+0000 7f534f016700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f534806afe0 0x7f53481990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.376+0000 7f534f016700 1 --2- 172.21.15.19:0/2571943252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f534806b980 0x7f53481995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.376+0000 7f534f016700 1 -- 172.21.15.19:0/2571943252 >> 172.21.15.19:0/2571943252 conn(0x7f53480788b0 msgr2=0x7f53480ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.377+0000 7f534f016700 1 -- 172.21.15.19:0/2571943252 shutdown_connections 2024-04-23T18:25:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:13.377+0000 7f534f016700 1 -- 172.21.15.19:0/2571943252 wait complete. 2024-04-23T18:25:13.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:13 smithi044 ceph-mon[32458]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:25:13.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:13 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3795405334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:25:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:13 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2371690533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:25:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:13 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2155406223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:25:13.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:13 smithi155 ceph-mon[33468]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:25:13.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:13 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3795405334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:25:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:13 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2371690533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:25:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:13 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2155406223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:25:14.046 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299438 got 223338299442 for osd.7 2024-04-23T18:25:14.047 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:14.479 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:14 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2571943252' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:25:14.514 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182463 got 107374182469 for osd.3 2024-04-23T18:25:14.514 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:14.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:14 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2571943252' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:25:14.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:14 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2571943252' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:25:15.208 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350999 got 347892351005 for osd.11 2024-04-23T18:25:15.208 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:15.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:15 smithi019 ceph-mon[29215]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:25:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:15 smithi044 ceph-mon[32458]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:25:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:15 smithi155 ceph-mon[33468]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:25:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.623+0000 7f34516d3700 1 -- 172.21.15.19:0/1384647805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f344c101930 msgr2=0x7f344c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.623+0000 7f34516d3700 1 --2- 172.21.15.19:0/1384647805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f344c101930 0x7f344c101db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3440009990 tx=0x7f3440009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.623+0000 7f34516d3700 1 -- 172.21.15.19:0/1384647805 shutdown_connections 2024-04-23T18:25:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.623+0000 7f34516d3700 1 --2- 172.21.15.19:0/1384647805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f344c1022f0 0x7f344c10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.623+0000 7f34516d3700 1 --2- 172.21.15.19:0/1384647805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344c107980 0x7f344c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.623+0000 7f34516d3700 1 --2- 172.21.15.19:0/1384647805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f344c101930 0x7f344c101db0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.623+0000 7f34516d3700 1 -- 172.21.15.19:0/1384647805 >> 172.21.15.19:0/1384647805 conn(0x7f344c0788b0 msgr2=0x7f344c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.623+0000 7f34516d3700 1 -- 172.21.15.19:0/1384647805 shutdown_connections 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.623+0000 7f34516d3700 1 -- 172.21.15.19:0/1384647805 wait complete. 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.624+0000 7f34516d3700 1 Processor -- start 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.624+0000 7f34516d3700 1 -- start start 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f34516d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f344c101930 0x7f344c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f34516d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f344c1022f0 0x7f344c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f34516d3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344c107980 0x7f344c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f34516d3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f344c111ad0 con 0x7f344c101930 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f34516d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f344c111950 con 0x7f344c1022f0 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f34516d3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f344c111c50 con 0x7f344c107980 2024-04-23T18:25:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f344c101930 0x7f344c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f344c101930 0x7f344c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55020/0 (socket says 172.21.15.19:55020) 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344affd700 1 -- 172.21.15.19:0/2425160443 learned_addr learned my addr 172.21.15.19:0/2425160443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344a7fc700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f344c1022f0 0x7f344c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344b7fe700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344c107980 0x7f344c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344affd700 1 -- 172.21.15.19:0/2425160443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344c107980 msgr2=0x7f344c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344affd700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344c107980 0x7f344c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344affd700 1 -- 172.21.15.19:0/2425160443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f344c1022f0 msgr2=0x7f344c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344affd700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f344c1022f0 0x7f344c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344affd700 1 -- 172.21.15.19:0/2425160443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344c19cf30 con 0x7f344c101930 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344a7fc700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f344c1022f0 0x7f344c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f344affd700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f344c101930 0x7f344c197f20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f343c00c9b0 tx=0x7f343c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f3433fff700 1 -- 172.21.15.19:0/2425160443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c007b00 con 0x7f344c101930 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f34516d3700 1 -- 172.21.15.19:0/2425160443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f344c19d220 con 0x7f344c101930 2024-04-23T18:25:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f3433fff700 1 -- 172.21.15.19:0/2425160443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f343c007ca0 con 0x7f344c101930 2024-04-23T18:25:15.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.626+0000 7f34516d3700 1 -- 172.21.15.19:0/2425160443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344c1a4aa0 con 0x7f344c101930 2024-04-23T18:25:15.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.625+0000 7f3433fff700 1 -- 172.21.15.19:0/2425160443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c018730 con 0x7f344c101930 2024-04-23T18:25:15.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.626+0000 7f34516d3700 1 -- 172.21.15.19:0/2425160443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f344c069730 con 0x7f344c101930 2024-04-23T18:25:15.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.628+0000 7f3433fff700 1 -- 172.21.15.19:0/2425160443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f343c00f450 con 0x7f344c101930 2024-04-23T18:25:15.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.628+0000 7f3433fff700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f34340a3c40 0x7f34340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:15.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.628+0000 7f344a7fc700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f34340a3c40 0x7f34340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:15.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.628+0000 7f3433fff700 1 -- 172.21.15.19:0/2425160443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f343c0d0710 con 0x7f344c101930 2024-04-23T18:25:15.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.629+0000 7f344a7fc700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f34340a3c40 0x7f34340a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3440006010 tx=0x7f344000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:15.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.632+0000 7f3433fff700 1 -- 172.21.15.19:0/2425160443 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f343c014030 con 0x7f344c101930 2024-04-23T18:25:15.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.845+0000 7f34516d3700 1 -- 172.21.15.19:0/2425160443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f344c199280 con 0x7f344c101930 2024-04-23T18:25:15.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.845+0000 7f3433fff700 1 -- 172.21.15.19:0/2425160443 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f343c09d190 con 0x7f344c101930 2024-04-23T18:25:15.838 INFO:teuthology.orchestra.run.smithi019.stdout:34359738453 2024-04-23T18:25:15.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 -- 172.21.15.19:0/2425160443 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f34340a3c40 msgr2=0x7f34340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:15.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f34340a3c40 0x7f34340a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3440006010 tx=0x7f344000b540 comp rx=0 tx=0).stop 2024-04-23T18:25:15.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 -- 172.21.15.19:0/2425160443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f344c101930 msgr2=0x7f344c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:15.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f344c101930 0x7f344c197f20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f343c00c9b0 tx=0x7f343c00cd70 comp rx=0 tx=0).stop 2024-04-23T18:25:15.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 -- 172.21.15.19:0/2425160443 shutdown_connections 2024-04-23T18:25:15.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f34340a3c40 0x7f34340a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:15.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344c107980 0x7f344c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:15.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f344c101930 0x7f344c197f20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:15.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 --2- 172.21.15.19:0/2425160443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f344c1022f0 0x7f344c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:15.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 -- 172.21.15.19:0/2425160443 >> 172.21.15.19:0/2425160443 conn(0x7f344c0788b0 msgr2=0x7f344c0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:15.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 -- 172.21.15.19:0/2425160443 shutdown_connections 2024-04-23T18:25:15.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:15.847+0000 7f34516d3700 1 -- 172.21.15.19:0/2425160443 wait complete. 2024-04-23T18:25:16.146 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574924 got 55834574929 for osd.1 2024-04-23T18:25:16.147 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:16.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:16 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2425160443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:25:16.474 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:16 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2425160443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:25:16.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:16 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2425160443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:25:17.011 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411404 for osd.2 2024-04-23T18:25:17.012 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:17.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:17 smithi019 ceph-mon[29215]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-23T18:25:17.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:17 smithi044 ceph-mon[32458]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-23T18:25:17.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:17 smithi155 ceph-mon[33468]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-23T18:25:17.731 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462962 got 201863462968 for osd.6 2024-04-23T18:25:17.731 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:18.882 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738453 for osd.0 2024-04-23T18:25:18.883 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:18.883 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-23T18:25:18.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T18:25:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:19.437 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:19 smithi019 ceph-mon[29215]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:25:19.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:19 smithi044 ceph-mon[32458]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:25:19.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:19 smithi155 ceph-mon[33468]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:25:20.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:25:20.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:25:20.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:20 smithi019 ceph-mon[29215]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-23T18:25:20.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:25:20.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:25:20.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:20 smithi019 ceph-mon[29215]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:25:20.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:25:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:25:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:20 smithi044 ceph-mon[32458]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-23T18:25:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:25:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:25:20.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:20 smithi044 ceph-mon[32458]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:25:20.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:25:20.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:25:20.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:20 smithi155 ceph-mon[33468]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-23T18:25:20.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:25:20.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:25:20.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:20 smithi155 ceph-mon[33468]: from='mgr.14150 172.21.15.19:0/2052686986' entity='mgr.x' 2024-04-23T18:25:20.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.934+0000 7f73e780c700 1 -- 172.21.15.19:0/3302391198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e006b0f0 msgr2=0x7f73e010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:20.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.934+0000 7f73e780c700 1 --2- 172.21.15.19:0/3302391198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e006b0f0 0x7f73e010f900 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f73d800b0a0 tx=0x7f73d800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.934+0000 7f73e780c700 1 -- 172.21.15.19:0/3302391198 shutdown_connections 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.934+0000 7f73e780c700 1 --2- 172.21.15.19:0/3302391198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e006b0f0 0x7f73e010f900 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.934+0000 7f73e780c700 1 --2- 172.21.15.19:0/3302391198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e0108a30 0x7f73e0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.934+0000 7f73e780c700 1 --2- 172.21.15.19:0/3302391198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e006a750 0x7f73e006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.934+0000 7f73e780c700 1 -- 172.21.15.19:0/3302391198 >> 172.21.15.19:0/3302391198 conn(0x7f73e00788f0 msgr2=0x7f73e0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.934+0000 7f73e780c700 1 -- 172.21.15.19:0/3302391198 shutdown_connections 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.934+0000 7f73e780c700 1 -- 172.21.15.19:0/3302391198 wait complete. 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e780c700 1 Processor -- start 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e780c700 1 -- start start 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e780c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e006a750 0x7f73e0101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e780c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e006b0f0 0x7f73e0101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e780c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e0108a30 0x7f73e0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e780c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0112c30 con 0x7f73e006b0f0 2024-04-23T18:25:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e780c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0112ab0 con 0x7f73e0108a30 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e780c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0112db0 con 0x7f73e006a750 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e4da7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e006b0f0 0x7f73e0101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e4da7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e006b0f0 0x7f73e0101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47028/0 (socket says 172.21.15.19:47028) 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e4da7700 1 -- 172.21.15.19:0/1814090979 learned_addr learned my addr 172.21.15.19:0/1814090979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e55a8700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e006a750 0x7f73e0101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e4da7700 1 -- 172.21.15.19:0/1814090979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e006a750 msgr2=0x7f73e0101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e4da7700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e006a750 0x7f73e0101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e4da7700 1 -- 172.21.15.19:0/1814090979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e0108a30 msgr2=0x7f73e0102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e5da9700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e0108a30 0x7f73e0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e4da7700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e0108a30 0x7f73e0102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.935+0000 7f73e4da7700 1 -- 172.21.15.19:0/1814090979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e01a7630 con 0x7f73e006b0f0 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.936+0000 7f73e4da7700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e006b0f0 0x7f73e0101f50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f73d000d950 tx=0x7f73d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.936+0000 7f73d67fc700 1 -- 172.21.15.19:0/1814090979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0016070 con 0x7f73e006b0f0 2024-04-23T18:25:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.936+0000 7f73d67fc700 1 -- 172.21.15.19:0/1814090979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73d0009df0 con 0x7f73e006b0f0 2024-04-23T18:25:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.936+0000 7f73e780c700 1 -- 172.21.15.19:0/1814090979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e01a7920 con 0x7f73e006b0f0 2024-04-23T18:25:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.936+0000 7f73d67fc700 1 -- 172.21.15.19:0/1814090979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d000f3c0 con 0x7f73e006b0f0 2024-04-23T18:25:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.936+0000 7f73e780c700 1 -- 172.21.15.19:0/1814090979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e01a7e60 con 0x7f73e006b0f0 2024-04-23T18:25:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.937+0000 7f73e780c700 1 -- 172.21.15.19:0/1814090979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e010b030 con 0x7f73e006b0f0 2024-04-23T18:25:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.938+0000 7f73d67fc700 1 -- 172.21.15.19:0/1814090979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f73d0019070 con 0x7f73e006b0f0 2024-04-23T18:25:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.939+0000 7f73d67fc700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73cc0a3c40 0x7f73cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.939+0000 7f73d67fc700 1 -- 172.21.15.19:0/1814090979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f73d00cffc0 con 0x7f73e006b0f0 2024-04-23T18:25:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.939+0000 7f73e55a8700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73cc0a3c40 0x7f73cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.939+0000 7f73e55a8700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73cc0a3c40 0x7f73cc0a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009a10 tx=0x7f73dc009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:20.944+0000 7f73d67fc700 1 -- 172.21.15.19:0/1814090979 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73d009c9b0 con 0x7f73e006b0f0 2024-04-23T18:25:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.148+0000 7f73e780c700 1 -- 172.21.15.19:0/1814090979 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73e0062e80 con 0x7f73cc0a3c40 2024-04-23T18:25:21.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.156+0000 7f73d67fc700 1 -- 172.21.15.19:0/1814090979 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259106 (secure 0 0 0) 0x7f73e0062e80 con 0x7f73cc0a3c40 2024-04-23T18:25:21.153 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.158+0000 7f73e780c700 1 -- 172.21.15.19:0/1814090979 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73cc0a3c40 msgr2=0x7f73cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.158+0000 7f73e780c700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73cc0a3c40 0x7f73cc0a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009a10 tx=0x7f73dc009190 comp rx=0 tx=0).stop 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.158+0000 7f73e780c700 1 -- 172.21.15.19:0/1814090979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e006b0f0 msgr2=0x7f73e0101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.158+0000 7f73e780c700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e006b0f0 0x7f73e0101f50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f73d000d950 tx=0x7f73d000dc60 comp rx=0 tx=0).stop 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.158+0000 7f73e780c700 1 -- 172.21.15.19:0/1814090979 shutdown_connections 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.158+0000 7f73e780c700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73cc0a3c40 0x7f73cc0a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.158+0000 7f73e780c700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e0108a30 0x7f73e0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.158+0000 7f73e780c700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e006a750 0x7f73e0101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.158+0000 7f73e780c700 1 --2- 172.21.15.19:0/1814090979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e006b0f0 0x7f73e0101f50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.158+0000 7f73e780c700 1 -- 172.21.15.19:0/1814090979 >> 172.21.15.19:0/1814090979 conn(0x7f73e00788f0 msgr2=0x7f73e00ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.159+0000 7f73e780c700 1 -- 172.21.15.19:0/1814090979 shutdown_connections 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:21.159+0000 7f73e780c700 1 -- 172.21.15.19:0/1814090979 wait complete. 2024-04-23T18:25:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-23T18:25:21.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:21 smithi019 ceph-mon[29215]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:21.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:21 smithi044 ceph-mon[32458]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:21.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:21 smithi155 ceph-mon[33468]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:22.125 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-04-23T18:25:19.373368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659383,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":916,"num_read_kb":581,"num_write":429,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":853,"ondisk_log_size":853,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381796,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560028,"statfs":{"total":1151940427776,"available":1151549468672,"internally_reserved":0,"allocated":12128256,"data_stored":7499553,"data_compressed":112350,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":58,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002896"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.383027+0000","last_change":"2024-04-23T18:23:56.098292+0000","last_active":"2024-04-23T18:25:05.383027+0000","last_peered":"2024-04-23T18:25:05.383027+0000","last_clean":"2024-04-23T18:25:05.383027+0000","last_became_active":"2024-04-23T18:23:56.096623+0000","last_became_peered":"2024-04-23T18:23:56.096623+0000","last_unstale":"2024-04-23T18:25:05.383027+0000","last_undegraded":"2024-04-23T18:25:05.383027+0000","last_fullsized":"2024-04-23T18:25:05.383027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:06.186581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.388004+0000","last_change":"2024-04-23T18:23:56.096807+0000","last_active":"2024-04-23T18:25:05.388004+0000","last_peered":"2024-04-23T18:25:05.388004+0000","last_clean":"2024-04-23T18:25:05.388004+0000","last_became_active":"2024-04-23T18:23:56.096327+0000","last_became_peered":"2024-04-23T18:23:56.096327+0000","last_unstale":"2024-04-23T18:25:05.388004+0000","last_undegraded":"2024-04-23T18:25:05.388004+0000","last_fullsized":"2024-04-23T18:25:05.388004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:56:16.391821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.394411+0000","last_change":"2024-04-23T18:23:56.101615+0000","last_active":"2024-04-23T18:25:05.394411+0000","last_peered":"2024-04-23T18:25:05.394411+0000","last_clean":"2024-04-23T18:25:05.394411+0000","last_became_active":"2024-04-23T18:23:56.100929+0000","last_became_peered":"2024-04-23T18:23:56.100929+0000","last_unstale":"2024-04-23T18:25:05.394411+0000","last_undegraded":"2024-04-23T18:25:05.394411+0000","last_fullsized":"2024-04-23T18:25:05.394411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:25.547029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.375061+0000","last_change":"2024-04-23T18:23:56.103627+0000","last_active":"2024-04-23T18:25:05.375061+0000","last_peered":"2024-04-23T18:25:05.375061+0000","last_clean":"2024-04-23T18:25:05.375061+0000","last_became_active":"2024-04-23T18:23:56.102752+0000","last_became_peered":"2024-04-23T18:23:56.102752+0000","last_unstale":"2024-04-23T18:25:05.375061+0000","last_undegraded":"2024-04-23T18:25:05.375061+0000","last_fullsized":"2024-04-23T18:25:05.375061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:28.460342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.369522+0000","last_change":"2024-04-23T18:23:56.094889+0000","last_active":"2024-04-23T18:25:05.369522+0000","last_peered":"2024-04-23T18:25:05.369522+0000","last_clean":"2024-04-23T18:25:05.369522+0000","last_became_active":"2024-04-23T18:23:56.094262+0000","last_became_peered":"2024-04-23T18:23:56.094262+0000","last_unstale":"2024-04-23T18:25:05.369522+0000","last_undegraded":"2024-04-23T18:25:05.369522+0000","last_fullsized":"2024-04-23T18:25:05.369522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:22.332114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.369406+0000","last_change":"2024-04-23T18:23:56.108893+0000","last_active":"2024-04-23T18:25:05.369406+0000","last_peered":"2024-04-23T18:25:05.369406+0000","last_clean":"2024-04-23T18:25:05.369406+0000","last_became_active":"2024-04-23T18:23:56.108229+0000","last_became_peered":"2024-04-23T18:23:56.108229+0000","last_unstale":"2024-04-23T18:25:05.369406+0000","last_undegraded":"2024-04-23T18:25:05.369406+0000","last_fullsized":"2024-04-23T18:25:05.369406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:50.930321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.384178+0000","last_change":"2024-04-23T18:23:56.098674+0000","last_active":"2024-04-23T18:25:05.384178+0000","last_peered":"2024-04-23T18:25:05.384178+0000","last_clean":"2024-04-23T18:25:05.384178+0000","last_became_active":"2024-04-23T18:23:56.098299+0000","last_became_peered":"2024-04-23T18:23:56.098299+0000","last_unstale":"2024-04-23T18:25:05.384178+0000","last_undegraded":"2024-04-23T18:25:05.384178+0000","last_fullsized":"2024-04-23T18:25:05.384178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:05.711410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.386974+0000","last_change":"2024-04-23T18:23:56.097437+0000","last_active":"2024-04-23T18:25:05.386974+0000","last_peered":"2024-04-23T18:25:05.386974+0000","last_clean":"2024-04-23T18:25:05.386974+0000","last_became_active":"2024-04-23T18:23:56.095702+0000","last_became_peered":"2024-04-23T18:23:56.095702+0000","last_unstale":"2024-04-23T18:25:05.386974+0000","last_undegraded":"2024-04-23T18:25:05.386974+0000","last_fullsized":"2024-04-23T18:25:05.386974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:35.754832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.384574+0000","last_change":"2024-04-23T18:23:56.096060+0000","last_active":"2024-04-23T18:25:05.384574+0000","last_peered":"2024-04-23T18:25:05.384574+0000","last_clean":"2024-04-23T18:25:05.384574+0000","last_became_active":"2024-04-23T18:23:56.095597+0000","last_became_peered":"2024-04-23T18:23:56.095597+0000","last_unstale":"2024-04-23T18:25:05.384574+0000","last_undegraded":"2024-04-23T18:25:05.384574+0000","last_fullsized":"2024-04-23T18:25:05.384574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:59.735347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.377984+0000","last_change":"2024-04-23T18:23:56.109033+0000","last_active":"2024-04-23T18:25:05.377984+0000","last_peered":"2024-04-23T18:25:05.377984+0000","last_clean":"2024-04-23T18:25:05.377984+0000","last_became_active":"2024-04-23T18:23:56.108472+0000","last_became_peered":"2024-04-23T18:23:56.108472+0000","last_unstale":"2024-04-23T18:25:05.377984+0000","last_undegraded":"2024-04-23T18:25:05.377984+0000","last_fullsized":"2024-04-23T18:25:05.377984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:32.491181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.398430+0000","last_change":"2024-04-23T18:23:56.097771+0000","last_active":"2024-04-23T18:25:05.398430+0000","last_peered":"2024-04-23T18:25:05.398430+0000","last_clean":"2024-04-23T18:25:05.398430+0000","last_became_active":"2024-04-23T18:23:56.097403+0000","last_became_peered":"2024-04-23T18:23:56.097403+0000","last_unstale":"2024-04-23T18:25:05.398430+0000","last_undegraded":"2024-04-23T18:25:05.398430+0000","last_fullsized":"2024-04-23T18:25:05.398430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:48:36.261543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.374752+0000","last_change":"2024-04-23T18:23:56.108550+0000","last_active":"2024-04-23T18:25:05.374752+0000","last_peered":"2024-04-23T18:25:05.374752+0000","last_clean":"2024-04-23T18:25:05.374752+0000","last_became_active":"2024-04-23T18:23:56.108191+0000","last_became_peered":"2024-04-23T18:23:56.108191+0000","last_unstale":"2024-04-23T18:25:05.374752+0000","last_undegraded":"2024-04-23T18:25:05.374752+0000","last_fullsized":"2024-04-23T18:25:05.374752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:25.142466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.375205+0000","last_change":"2024-04-23T18:23:56.092573+0000","last_active":"2024-04-23T18:25:05.375205+0000","last_peered":"2024-04-23T18:25:05.375205+0000","last_clean":"2024-04-23T18:25:05.375205+0000","last_became_active":"2024-04-23T18:23:56.092095+0000","last_became_peered":"2024-04-23T18:23:56.092095+0000","last_unstale":"2024-04-23T18:25:05.375205+0000","last_undegraded":"2024-04-23T18:25:05.375205+0000","last_fullsized":"2024-04-23T18:25:05.375205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:01.601206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.392359+0000","last_change":"2024-04-23T18:23:56.102073+0000","last_active":"2024-04-23T18:25:05.392359+0000","last_peered":"2024-04-23T18:25:05.392359+0000","last_clean":"2024-04-23T18:25:05.392359+0000","last_became_active":"2024-04-23T18:23:56.101059+0000","last_became_peered":"2024-04-23T18:23:56.101059+0000","last_unstale":"2024-04-23T18:25:05.392359+0000","last_undegraded":"2024-04-23T18:25:05.392359+0000","last_fullsized":"2024-04-23T18:25:05.392359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:37.904099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.395386+0000","last_change":"2024-04-23T18:23:56.108482+0000","last_active":"2024-04-23T18:25:05.395386+0000","last_peered":"2024-04-23T18:25:05.395386+0000","last_clean":"2024-04-23T18:25:05.395386+0000","last_became_active":"2024-04-23T18:23:56.108128+0000","last_became_peered":"2024-04-23T18:23:56.108128+0000","last_unstale":"2024-04-23T18:25:05.395386+0000","last_undegraded":"2024-04-23T18:25:05.395386+0000","last_fullsized":"2024-04-23T18:25:05.395386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:51.925881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.398957+0000","last_change":"2024-04-23T18:23:56.099691+0000","last_active":"2024-04-23T18:25:05.398957+0000","last_peered":"2024-04-23T18:25:05.398957+0000","last_clean":"2024-04-23T18:25:05.398957+0000","last_became_active":"2024-04-23T18:23:56.099272+0000","last_became_peered":"2024-04-23T18:23:56.099272+0000","last_unstale":"2024-04-23T18:25:05.398957+0000","last_undegraded":"2024-04-23T18:25:05.398957+0000","last_fullsized":"2024-04-23T18:25:05.398957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:41.834104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.383395+0000","last_change":"2024-04-23T18:23:56.097965+0000","last_active":"2024-04-23T18:25:05.383395+0000","last_peered":"2024-04-23T18:25:05.383395+0000","last_clean":"2024-04-23T18:25:05.383395+0000","last_became_active":"2024-04-23T18:23:56.096766+0000","last_became_peered":"2024-04-23T18:23:56.096766+0000","last_unstale":"2024-04-23T18:25:05.383395+0000","last_undegraded":"2024-04-23T18:25:05.383395+0000","last_fullsized":"2024-04-23T18:25:05.383395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:40:00.076939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.388729+0000","last_change":"2024-04-23T18:23:56.095158+0000","last_active":"2024-04-23T18:25:05.388729+0000","last_peered":"2024-04-23T18:25:05.388729+0000","last_clean":"2024-04-23T18:25:05.388729+0000","last_became_active":"2024-04-23T18:23:56.094812+0000","last_became_peered":"2024-04-23T18:23:56.094812+0000","last_unstale":"2024-04-23T18:25:05.388729+0000","last_undegraded":"2024-04-23T18:25:05.388729+0000","last_fullsized":"2024-04-23T18:25:05.388729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:40.874698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.378382+0000","last_change":"2024-04-23T18:23:56.099358+0000","last_active":"2024-04-23T18:25:05.378382+0000","last_peered":"2024-04-23T18:25:05.378382+0000","last_clean":"2024-04-23T18:25:05.378382+0000","last_became_active":"2024-04-23T18:23:56.099031+0000","last_became_peered":"2024-04-23T18:23:56.099031+0000","last_unstale":"2024-04-23T18:25:05.378382+0000","last_undegraded":"2024-04-23T18:25:05.378382+0000","last_fullsized":"2024-04-23T18:25:05.378382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:29.634932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:17.738565+0000","last_change":"2024-04-23T18:23:55.262939+0000","last_active":"2024-04-23T18:25:17.738565+0000","last_peered":"2024-04-23T18:25:17.738565+0000","last_clean":"2024-04-23T18:25:17.738565+0000","last_became_active":"2024-04-23T18:23:55.262609+0000","last_became_peered":"2024-04-23T18:23:55.262609+0000","last_unstale":"2024-04-23T18:25:17.738565+0000","last_undegraded":"2024-04-23T18:25:17.738565+0000","last_fullsized":"2024-04-23T18:25:17.738565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:12.418349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.372597+0000","last_change":"2024-04-23T18:23:56.092182+0000","last_active":"2024-04-23T18:25:05.372597+0000","last_peered":"2024-04-23T18:25:05.372597+0000","last_clean":"2024-04-23T18:25:05.372597+0000","last_became_active":"2024-04-23T18:23:56.091444+0000","last_became_peered":"2024-04-23T18:23:56.091444+0000","last_unstale":"2024-04-23T18:25:05.372597+0000","last_undegraded":"2024-04-23T18:25:05.372597+0000","last_fullsized":"2024-04-23T18:25:05.372597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:00.775498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696726+0000","last_change":"2024-04-23T18:23:55.259880+0000","last_active":"2024-04-23T18:24:36.696726+0000","last_peered":"2024-04-23T18:24:36.696726+0000","last_clean":"2024-04-23T18:24:36.696726+0000","last_became_active":"2024-04-23T18:23:55.259576+0000","last_became_peered":"2024-04-23T18:23:55.259576+0000","last_unstale":"2024-04-23T18:24:36.696726+0000","last_undegraded":"2024-04-23T18:24:36.696726+0000","last_fullsized":"2024-04-23T18:24:36.696726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:13.783776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.394342+0000","last_change":"2024-04-23T18:23:56.094307+0000","last_active":"2024-04-23T18:25:05.394342+0000","last_peered":"2024-04-23T18:25:05.394342+0000","last_clean":"2024-04-23T18:25:05.394342+0000","last_became_active":"2024-04-23T18:23:56.093893+0000","last_became_peered":"2024-04-23T18:23:56.093893+0000","last_unstale":"2024-04-23T18:25:05.394342+0000","last_undegraded":"2024-04-23T18:25:05.394342+0000","last_fullsized":"2024-04-23T18:25:05.394342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:12:35.385497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.697688+0000","last_change":"2024-04-23T18:23:55.263137+0000","last_active":"2024-04-23T18:24:36.697688+0000","last_peered":"2024-04-23T18:24:36.697688+0000","last_clean":"2024-04-23T18:24:36.697688+0000","last_became_active":"2024-04-23T18:23:55.262553+0000","last_became_peered":"2024-04-23T18:23:55.262553+0000","last_unstale":"2024-04-23T18:24:36.697688+0000","last_undegraded":"2024-04-23T18:24:36.697688+0000","last_fullsized":"2024-04-23T18:24:36.697688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:33:42.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.393759+0000","last_change":"2024-04-23T18:23:56.091098+0000","last_active":"2024-04-23T18:25:05.393759+0000","last_peered":"2024-04-23T18:25:05.393759+0000","last_clean":"2024-04-23T18:25:05.393759+0000","last_became_active":"2024-04-23T18:23:56.090635+0000","last_became_peered":"2024-04-23T18:23:56.090635+0000","last_unstale":"2024-04-23T18:25:05.393759+0000","last_undegraded":"2024-04-23T18:25:05.393759+0000","last_fullsized":"2024-04-23T18:25:05.393759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:17.787937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696489+0000","last_change":"2024-04-23T18:23:55.262146+0000","last_active":"2024-04-23T18:24:36.696489+0000","last_peered":"2024-04-23T18:24:36.696489+0000","last_clean":"2024-04-23T18:24:36.696489+0000","last_became_active":"2024-04-23T18:23:55.261838+0000","last_became_peered":"2024-04-23T18:23:55.261838+0000","last_unstale":"2024-04-23T18:24:36.696489+0000","last_undegraded":"2024-04-23T18:24:36.696489+0000","last_fullsized":"2024-04-23T18:24:36.696489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:42:39.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.387953+0000","last_change":"2024-04-23T18:24:36.915394+0000","last_active":"2024-04-23T18:25:05.387953+0000","last_peered":"2024-04-23T18:25:05.387953+0000","last_clean":"2024-04-23T18:25:05.387953+0000","last_became_active":"2024-04-23T18:24:36.915072+0000","last_became_peered":"2024-04-23T18:24:36.915072+0000","last_unstale":"2024-04-23T18:25:05.387953+0000","last_undegraded":"2024-04-23T18:25:05.387953+0000","last_fullsized":"2024-04-23T18:25:05.387953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.132410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":106,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:17.740249+0000","last_change":"2024-04-23T18:23:55.261399+0000","last_active":"2024-04-23T18:25:17.740249+0000","last_peered":"2024-04-23T18:25:17.740249+0000","last_clean":"2024-04-23T18:25:17.740249+0000","last_became_active":"2024-04-23T18:23:55.261012+0000","last_became_peered":"2024-04-23T18:23:55.261012+0000","last_unstale":"2024-04-23T18:25:17.740249+0000","last_undegraded":"2024-04-23T18:25:17.740249+0000","last_fullsized":"2024-04-23T18:25:17.740249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:31.377776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.377694+0000","last_change":"2024-04-23T18:23:56.096785+0000","last_active":"2024-04-23T18:25:05.377694+0000","last_peered":"2024-04-23T18:25:05.377694+0000","last_clean":"2024-04-23T18:25:05.377694+0000","last_became_active":"2024-04-23T18:23:56.096420+0000","last_became_peered":"2024-04-23T18:23:56.096420+0000","last_unstale":"2024-04-23T18:25:05.377694+0000","last_undegraded":"2024-04-23T18:25:05.377694+0000","last_fullsized":"2024-04-23T18:25:05.377694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:20:49.215071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.693029+0000","last_change":"2024-04-23T18:23:55.260645+0000","last_active":"2024-04-23T18:24:36.693029+0000","last_peered":"2024-04-23T18:24:36.693029+0000","last_clean":"2024-04-23T18:24:36.693029+0000","last_became_active":"2024-04-23T18:23:55.260118+0000","last_became_peered":"2024-04-23T18:23:55.260118+0000","last_unstale":"2024-04-23T18:24:36.693029+0000","last_undegraded":"2024-04-23T18:24:36.693029+0000","last_fullsized":"2024-04-23T18:24:36.693029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:14:12.066297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.370004+0000","last_change":"2024-04-23T18:24:36.702766+0000","last_active":"2024-04-23T18:25:05.370004+0000","last_peered":"2024-04-23T18:25:05.370004+0000","last_clean":"2024-04-23T18:25:05.370004+0000","last_became_active":"2024-04-23T18:24:36.702309+0000","last_became_peered":"2024-04-23T18:24:36.702309+0000","last_unstale":"2024-04-23T18:25:05.370004+0000","last_undegraded":"2024-04-23T18:25:05.370004+0000","last_fullsized":"2024-04-23T18:25:05.370004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:10.803924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.694625+0000","last_change":"2024-04-23T18:23:55.258636+0000","last_active":"2024-04-23T18:24:36.694625+0000","last_peered":"2024-04-23T18:24:36.694625+0000","last_clean":"2024-04-23T18:24:36.694625+0000","last_became_active":"2024-04-23T18:23:55.257744+0000","last_became_peered":"2024-04-23T18:23:55.257744+0000","last_unstale":"2024-04-23T18:24:36.694625+0000","last_undegraded":"2024-04-23T18:24:36.694625+0000","last_fullsized":"2024-04-23T18:24:36.694625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:46:07.454725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.391257+0000","last_change":"2024-04-23T18:23:56.094822+0000","last_active":"2024-04-23T18:25:05.391257+0000","last_peered":"2024-04-23T18:25:05.391257+0000","last_clean":"2024-04-23T18:25:05.391257+0000","last_became_active":"2024-04-23T18:23:56.094489+0000","last_became_peered":"2024-04-23T18:23:56.094489+0000","last_unstale":"2024-04-23T18:25:05.391257+0000","last_undegraded":"2024-04-23T18:25:05.391257+0000","last_fullsized":"2024-04-23T18:25:05.391257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:21.234363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.698040+0000","last_change":"2024-04-23T18:23:55.261322+0000","last_active":"2024-04-23T18:24:36.698040+0000","last_peered":"2024-04-23T18:24:36.698040+0000","last_clean":"2024-04-23T18:24:36.698040+0000","last_became_active":"2024-04-23T18:23:55.260990+0000","last_became_peered":"2024-04-23T18:23:55.260990+0000","last_unstale":"2024-04-23T18:24:36.698040+0000","last_undegraded":"2024-04-23T18:24:36.698040+0000","last_fullsized":"2024-04-23T18:24:36.698040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:31.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.391887+0000","last_change":"2024-04-23T18:23:56.093105+0000","last_active":"2024-04-23T18:25:05.391887+0000","last_peered":"2024-04-23T18:25:05.391887+0000","last_clean":"2024-04-23T18:25:05.391887+0000","last_became_active":"2024-04-23T18:23:56.092717+0000","last_became_peered":"2024-04-23T18:23:56.092717+0000","last_unstale":"2024-04-23T18:25:05.391887+0000","last_undegraded":"2024-04-23T18:25:05.391887+0000","last_fullsized":"2024-04-23T18:25:05.391887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:13:59.574057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":168,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:13.733651+0000","last_change":"2024-04-23T18:23:55.256307+0000","last_active":"2024-04-23T18:25:13.733651+0000","last_peered":"2024-04-23T18:25:13.733651+0000","last_clean":"2024-04-23T18:25:13.733651+0000","last_became_active":"2024-04-23T18:23:55.255959+0000","last_became_peered":"2024-04-23T18:23:55.255959+0000","last_unstale":"2024-04-23T18:25:13.733651+0000","last_undegraded":"2024-04-23T18:25:13.733651+0000","last_fullsized":"2024-04-23T18:25:13.733651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:04.945919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.396339+0000","last_change":"2024-04-23T18:23:56.095177+0000","last_active":"2024-04-23T18:25:05.396339+0000","last_peered":"2024-04-23T18:25:05.396339+0000","last_clean":"2024-04-23T18:25:05.396339+0000","last_became_active":"2024-04-23T18:23:56.094825+0000","last_became_peered":"2024-04-23T18:23:56.094825+0000","last_unstale":"2024-04-23T18:25:05.396339+0000","last_undegraded":"2024-04-23T18:25:05.396339+0000","last_fullsized":"2024-04-23T18:25:05.396339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:07:33.499817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696314+0000","last_change":"2024-04-23T18:23:55.264258+0000","last_active":"2024-04-23T18:24:36.696314+0000","last_peered":"2024-04-23T18:24:36.696314+0000","last_clean":"2024-04-23T18:24:36.696314+0000","last_became_active":"2024-04-23T18:23:55.263698+0000","last_became_peered":"2024-04-23T18:23:55.263698+0000","last_unstale":"2024-04-23T18:24:36.696314+0000","last_undegraded":"2024-04-23T18:24:36.696314+0000","last_fullsized":"2024-04-23T18:24:36.696314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:00.566948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.392441+0000","last_change":"2024-04-23T18:23:56.109085+0000","last_active":"2024-04-23T18:25:05.392441+0000","last_peered":"2024-04-23T18:25:05.392441+0000","last_clean":"2024-04-23T18:25:05.392441+0000","last_became_active":"2024-04-23T18:23:56.108033+0000","last_became_peered":"2024-04-23T18:23:56.108033+0000","last_unstale":"2024-04-23T18:25:05.392441+0000","last_undegraded":"2024-04-23T18:25:05.392441+0000","last_fullsized":"2024-04-23T18:25:05.392441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:59.980512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.700014+0000","last_change":"2024-04-23T18:23:55.256728+0000","last_active":"2024-04-23T18:24:36.700014+0000","last_peered":"2024-04-23T18:24:36.700014+0000","last_clean":"2024-04-23T18:24:36.700014+0000","last_became_active":"2024-04-23T18:23:55.256367+0000","last_became_peered":"2024-04-23T18:23:55.256367+0000","last_unstale":"2024-04-23T18:24:36.700014+0000","last_undegraded":"2024-04-23T18:24:36.700014+0000","last_fullsized":"2024-04-23T18:24:36.700014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:08.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.395768+0000","last_change":"2024-04-23T18:23:56.102898+0000","last_active":"2024-04-23T18:25:05.395768+0000","last_peered":"2024-04-23T18:25:05.395768+0000","last_clean":"2024-04-23T18:25:05.395768+0000","last_became_active":"2024-04-23T18:23:56.102373+0000","last_became_peered":"2024-04-23T18:23:56.102373+0000","last_unstale":"2024-04-23T18:25:05.395768+0000","last_undegraded":"2024-04-23T18:25:05.395768+0000","last_fullsized":"2024-04-23T18:25:05.395768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:49.316219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:13.733711+0000","last_change":"2024-04-23T18:23:55.255094+0000","last_active":"2024-04-23T18:25:13.733711+0000","last_peered":"2024-04-23T18:25:13.733711+0000","last_clean":"2024-04-23T18:25:13.733711+0000","last_became_active":"2024-04-23T18:23:55.254726+0000","last_became_peered":"2024-04-23T18:23:55.254726+0000","last_unstale":"2024-04-23T18:25:13.733711+0000","last_undegraded":"2024-04-23T18:25:13.733711+0000","last_fullsized":"2024-04-23T18:25:13.733711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:26.988184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":152,"num_read_kb":137,"num_write":35,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.398066+0000","last_change":"2024-04-23T18:23:56.099866+0000","last_active":"2024-04-23T18:25:05.398066+0000","last_peered":"2024-04-23T18:25:05.398066+0000","last_clean":"2024-04-23T18:25:05.398066+0000","last_became_active":"2024-04-23T18:23:56.098606+0000","last_became_peered":"2024-04-23T18:23:56.098606+0000","last_unstale":"2024-04-23T18:25:05.398066+0000","last_undegraded":"2024-04-23T18:25:05.398066+0000","last_fullsized":"2024-04-23T18:25:05.398066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:26.661857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.711203+0000","last_change":"2024-04-23T18:24:36.710791+0000","last_active":"2024-04-23T18:24:36.711203+0000","last_peered":"2024-04-23T18:24:36.711203+0000","last_clean":"2024-04-23T18:24:36.711203+0000","last_became_active":"2024-04-23T18:24:36.697731+0000","last_became_peered":"2024-04-23T18:24:36.697731+0000","last_unstale":"2024-04-23T18:24:36.711203+0000","last_undegraded":"2024-04-23T18:24:36.711203+0000","last_fullsized":"2024-04-23T18:24:36.711203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:16:46.722132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.366361+0000","last_change":"2024-04-23T18:23:56.099109+0000","last_active":"2024-04-23T18:25:05.366361+0000","last_peered":"2024-04-23T18:25:05.366361+0000","last_clean":"2024-04-23T18:25:05.366361+0000","last_became_active":"2024-04-23T18:23:56.098608+0000","last_became_peered":"2024-04-23T18:23:56.098608+0000","last_unstale":"2024-04-23T18:25:05.366361+0000","last_undegraded":"2024-04-23T18:25:05.366361+0000","last_fullsized":"2024-04-23T18:25:05.366361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:43.720160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.699083+0000","last_change":"2024-04-23T18:23:55.262523+0000","last_active":"2024-04-23T18:24:36.699083+0000","last_peered":"2024-04-23T18:24:36.699083+0000","last_clean":"2024-04-23T18:24:36.699083+0000","last_became_active":"2024-04-23T18:23:55.262216+0000","last_became_peered":"2024-04-23T18:23:55.262216+0000","last_unstale":"2024-04-23T18:24:36.699083+0000","last_undegraded":"2024-04-23T18:24:36.699083+0000","last_fullsized":"2024-04-23T18:24:36.699083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:35.268435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.399842+0000","last_change":"2024-04-23T18:23:56.106573+0000","last_active":"2024-04-23T18:25:05.399842+0000","last_peered":"2024-04-23T18:25:05.399842+0000","last_clean":"2024-04-23T18:25:05.399842+0000","last_became_active":"2024-04-23T18:23:56.106181+0000","last_became_peered":"2024-04-23T18:23:56.106181+0000","last_unstale":"2024-04-23T18:25:05.399842+0000","last_undegraded":"2024-04-23T18:25:05.399842+0000","last_fullsized":"2024-04-23T18:25:05.399842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:15.422647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:16.738644+0000","last_change":"2024-04-23T18:23:55.262813+0000","last_active":"2024-04-23T18:25:16.738644+0000","last_peered":"2024-04-23T18:25:16.738644+0000","last_clean":"2024-04-23T18:25:16.738644+0000","last_became_active":"2024-04-23T18:23:55.262365+0000","last_became_peered":"2024-04-23T18:23:55.262365+0000","last_unstale":"2024-04-23T18:25:16.738644+0000","last_undegraded":"2024-04-23T18:25:16.738644+0000","last_fullsized":"2024-04-23T18:25:16.738644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:15.234355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.387828+0000","last_change":"2024-04-23T18:23:56.101491+0000","last_active":"2024-04-23T18:25:05.387828+0000","last_peered":"2024-04-23T18:25:05.387828+0000","last_clean":"2024-04-23T18:25:05.387828+0000","last_became_active":"2024-04-23T18:23:56.101069+0000","last_became_peered":"2024-04-23T18:23:56.101069+0000","last_unstale":"2024-04-23T18:25:05.387828+0000","last_undegraded":"2024-04-23T18:25:05.387828+0000","last_fullsized":"2024-04-23T18:25:05.387828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:42.484653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.750105+0000","last_change":"2024-04-23T18:24:36.750105+0000","last_active":"2024-04-23T18:24:36.750105+0000","last_peered":"2024-04-23T18:24:36.750105+0000","last_clean":"2024-04-23T18:24:36.750105+0000","last_became_active":"2024-04-23T18:24:36.738326+0000","last_became_peered":"2024-04-23T18:24:36.738326+0000","last_unstale":"2024-04-23T18:24:36.750105+0000","last_undegraded":"2024-04-23T18:24:36.750105+0000","last_fullsized":"2024-04-23T18:24:36.750105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:07.414513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.379138+0000","last_change":"2024-04-23T18:23:56.105691+0000","last_active":"2024-04-23T18:25:05.379138+0000","last_peered":"2024-04-23T18:25:05.379138+0000","last_clean":"2024-04-23T18:25:05.379138+0000","last_became_active":"2024-04-23T18:23:56.105205+0000","last_became_peered":"2024-04-23T18:23:56.105205+0000","last_unstale":"2024-04-23T18:25:05.379138+0000","last_undegraded":"2024-04-23T18:25:05.379138+0000","last_fullsized":"2024-04-23T18:25:05.379138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:30.779821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696501+0000","last_change":"2024-04-23T18:23:55.261198+0000","last_active":"2024-04-23T18:24:36.696501+0000","last_peered":"2024-04-23T18:24:36.696501+0000","last_clean":"2024-04-23T18:24:36.696501+0000","last_became_active":"2024-04-23T18:23:55.260884+0000","last_became_peered":"2024-04-23T18:23:55.260884+0000","last_unstale":"2024-04-23T18:24:36.696501+0000","last_undegraded":"2024-04-23T18:24:36.696501+0000","last_fullsized":"2024-04-23T18:24:36.696501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:00.841671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.384470+0000","last_change":"2024-04-23T18:23:56.099450+0000","last_active":"2024-04-23T18:25:05.384470+0000","last_peered":"2024-04-23T18:25:05.384470+0000","last_clean":"2024-04-23T18:25:05.384470+0000","last_became_active":"2024-04-23T18:23:56.098723+0000","last_became_peered":"2024-04-23T18:23:56.098723+0000","last_unstale":"2024-04-23T18:25:05.384470+0000","last_undegraded":"2024-04-23T18:25:05.384470+0000","last_fullsized":"2024-04-23T18:25:05.384470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:53.027495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.692241+0000","last_change":"2024-04-23T18:23:55.262030+0000","last_active":"2024-04-23T18:24:36.692241+0000","last_peered":"2024-04-23T18:24:36.692241+0000","last_clean":"2024-04-23T18:24:36.692241+0000","last_became_active":"2024-04-23T18:23:55.261659+0000","last_became_peered":"2024-04-23T18:23:55.261659+0000","last_unstale":"2024-04-23T18:24:36.692241+0000","last_undegraded":"2024-04-23T18:24:36.692241+0000","last_fullsized":"2024-04-23T18:24:36.692241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:17.036709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.389538+0000","last_change":"2024-04-23T18:23:56.096511+0000","last_active":"2024-04-23T18:25:05.389538+0000","last_peered":"2024-04-23T18:25:05.389538+0000","last_clean":"2024-04-23T18:25:05.389538+0000","last_became_active":"2024-04-23T18:23:56.096154+0000","last_became_peered":"2024-04-23T18:23:56.096154+0000","last_unstale":"2024-04-23T18:25:05.389538+0000","last_undegraded":"2024-04-23T18:25:05.389538+0000","last_fullsized":"2024-04-23T18:25:05.389538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:03.926178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:15.736576+0000","last_change":"2024-04-23T18:23:55.263618+0000","last_active":"2024-04-23T18:25:15.736576+0000","last_peered":"2024-04-23T18:25:15.736576+0000","last_clean":"2024-04-23T18:25:15.736576+0000","last_became_active":"2024-04-23T18:23:55.261758+0000","last_became_peered":"2024-04-23T18:23:55.261758+0000","last_unstale":"2024-04-23T18:25:15.736576+0000","last_undegraded":"2024-04-23T18:25:15.736576+0000","last_fullsized":"2024-04-23T18:25:15.736576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:23:27.745668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.363984+0000","last_change":"2024-04-23T18:23:56.109292+0000","last_active":"2024-04-23T18:25:05.363984+0000","last_peered":"2024-04-23T18:25:05.363984+0000","last_clean":"2024-04-23T18:25:05.363984+0000","last_became_active":"2024-04-23T18:23:56.108414+0000","last_became_peered":"2024-04-23T18:23:56.108414+0000","last_unstale":"2024-04-23T18:25:05.363984+0000","last_undegraded":"2024-04-23T18:25:05.363984+0000","last_fullsized":"2024-04-23T18:25:05.363984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:13.428782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.378638+0000","last_change":"2024-04-23T18:23:56.097122+0000","last_active":"2024-04-23T18:25:05.378638+0000","last_peered":"2024-04-23T18:25:05.378638+0000","last_clean":"2024-04-23T18:25:05.378638+0000","last_became_active":"2024-04-23T18:23:56.096751+0000","last_became_peered":"2024-04-23T18:23:56.096751+0000","last_unstale":"2024-04-23T18:25:05.378638+0000","last_undegraded":"2024-04-23T18:25:05.378638+0000","last_fullsized":"2024-04-23T18:25:05.378638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:00:55.090370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:14.735221+0000","last_change":"2024-04-23T18:23:55.256458+0000","last_active":"2024-04-23T18:25:14.735221+0000","last_peered":"2024-04-23T18:25:14.735221+0000","last_clean":"2024-04-23T18:25:14.735221+0000","last_became_active":"2024-04-23T18:23:55.256127+0000","last_became_peered":"2024-04-23T18:23:55.256127+0000","last_unstale":"2024-04-23T18:25:14.735221+0000","last_undegraded":"2024-04-23T18:25:14.735221+0000","last_fullsized":"2024-04-23T18:25:14.735221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:03.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.394792+0000","last_change":"2024-04-23T18:23:56.103743+0000","last_active":"2024-04-23T18:25:05.394792+0000","last_peered":"2024-04-23T18:25:05.394792+0000","last_clean":"2024-04-23T18:25:05.394792+0000","last_became_active":"2024-04-23T18:23:56.103041+0000","last_became_peered":"2024-04-23T18:23:56.103041+0000","last_unstale":"2024-04-23T18:25:05.394792+0000","last_undegraded":"2024-04-23T18:25:05.394792+0000","last_fullsized":"2024-04-23T18:25:05.394792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:22.419555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.736937+0000","last_change":"2024-04-23T18:23:55.257008+0000","last_active":"2024-04-23T18:24:36.736937+0000","last_peered":"2024-04-23T18:24:36.736937+0000","last_clean":"2024-04-23T18:24:36.736937+0000","last_became_active":"2024-04-23T18:23:55.256648+0000","last_became_peered":"2024-04-23T18:23:55.256648+0000","last_unstale":"2024-04-23T18:24:36.736937+0000","last_undegraded":"2024-04-23T18:24:36.736937+0000","last_fullsized":"2024-04-23T18:24:36.736937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:00.867083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.695914+0000","last_change":"2024-04-23T18:22:38.610948+0000","last_active":"2024-04-23T18:24:36.695914+0000","last_peered":"2024-04-23T18:24:36.695914+0000","last_clean":"2024-04-23T18:24:36.695914+0000","last_became_active":"2024-04-23T18:22:38.610475+0000","last_became_peered":"2024-04-23T18:22:38.610475+0000","last_unstale":"2024-04-23T18:24:36.695914+0000","last_undegraded":"2024-04-23T18:24:36.695914+0000","last_fullsized":"2024-04-23T18:24:36.695914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_clean_scrub_stamp":"2024-04-23T18:19:07.980519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:08.209087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.379992+0000","last_change":"2024-04-23T18:24:36.699127+0000","last_active":"2024-04-23T18:25:05.379992+0000","last_peered":"2024-04-23T18:25:05.379992+0000","last_clean":"2024-04-23T18:25:05.379992+0000","last_became_active":"2024-04-23T18:24:36.698734+0000","last_became_peered":"2024-04-23T18:24:36.698734+0000","last_unstale":"2024-04-23T18:25:05.379992+0000","last_undegraded":"2024-04-23T18:25:05.379992+0000","last_fullsized":"2024-04-23T18:25:05.379992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:28.239466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.918722+0000","last_change":"2024-04-23T18:24:36.918722+0000","last_active":"2024-04-23T18:24:36.918722+0000","last_peered":"2024-04-23T18:24:36.918722+0000","last_clean":"2024-04-23T18:24:36.918722+0000","last_became_active":"2024-04-23T18:24:36.915658+0000","last_became_peered":"2024-04-23T18:24:36.915658+0000","last_unstale":"2024-04-23T18:24:36.918722+0000","last_undegraded":"2024-04-23T18:24:36.918722+0000","last_fullsized":"2024-04-23T18:24:36.918722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:34.706193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.697313+0000","last_change":"2024-04-23T18:23:55.265853+0000","last_active":"2024-04-23T18:24:36.697313+0000","last_peered":"2024-04-23T18:24:36.697313+0000","last_clean":"2024-04-23T18:24:36.697313+0000","last_became_active":"2024-04-23T18:23:55.265435+0000","last_became_peered":"2024-04-23T18:23:55.265435+0000","last_unstale":"2024-04-23T18:24:36.697313+0000","last_undegraded":"2024-04-23T18:24:36.697313+0000","last_fullsized":"2024-04-23T18:24:36.697313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.969720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.380863+0000","last_change":"2024-04-23T18:23:56.085609+0000","last_active":"2024-04-23T18:25:05.380863+0000","last_peered":"2024-04-23T18:25:05.380863+0000","last_clean":"2024-04-23T18:25:05.380863+0000","last_became_active":"2024-04-23T18:23:56.085244+0000","last_became_peered":"2024-04-23T18:23:56.085244+0000","last_unstale":"2024-04-23T18:25:05.380863+0000","last_undegraded":"2024-04-23T18:25:05.380863+0000","last_fullsized":"2024-04-23T18:25:05.380863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.211717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'13","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.692390+0000","last_change":"2024-04-23T18:23:55.260260+0000","last_active":"2024-04-23T18:24:36.692390+0000","last_peered":"2024-04-23T18:24:36.692390+0000","last_clean":"2024-04-23T18:24:36.692390+0000","last_became_active":"2024-04-23T18:23:55.259862+0000","last_became_peered":"2024-04-23T18:23:55.259862+0000","last_unstale":"2024-04-23T18:24:36.692390+0000","last_undegraded":"2024-04-23T18:24:36.692390+0000","last_fullsized":"2024-04-23T18:24:36.692390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:57.833293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.391043+0000","last_change":"2024-04-23T18:23:56.098826+0000","last_active":"2024-04-23T18:25:05.391043+0000","last_peered":"2024-04-23T18:25:05.391043+0000","last_clean":"2024-04-23T18:25:05.391043+0000","last_became_active":"2024-04-23T18:23:56.098494+0000","last_became_peered":"2024-04-23T18:23:56.098494+0000","last_unstale":"2024-04-23T18:25:05.391043+0000","last_undegraded":"2024-04-23T18:25:05.391043+0000","last_fullsized":"2024-04-23T18:25:05.391043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:12:24.953330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.737134+0000","last_change":"2024-04-23T18:23:55.258174+0000","last_active":"2024-04-23T18:24:36.737134+0000","last_peered":"2024-04-23T18:24:36.737134+0000","last_clean":"2024-04-23T18:24:36.737134+0000","last_became_active":"2024-04-23T18:23:55.257846+0000","last_became_peered":"2024-04-23T18:23:55.257846+0000","last_unstale":"2024-04-23T18:24:36.737134+0000","last_undegraded":"2024-04-23T18:24:36.737134+0000","last_fullsized":"2024-04-23T18:24:36.737134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:35.131566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.395955+0000","last_change":"2024-04-23T18:23:56.110758+0000","last_active":"2024-04-23T18:25:05.395955+0000","last_peered":"2024-04-23T18:25:05.395955+0000","last_clean":"2024-04-23T18:25:05.395955+0000","last_became_active":"2024-04-23T18:23:56.110372+0000","last_became_peered":"2024-04-23T18:23:56.110372+0000","last_unstale":"2024-04-23T18:25:05.395955+0000","last_undegraded":"2024-04-23T18:25:05.395955+0000","last_fullsized":"2024-04-23T18:25:05.395955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:40.193088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.914632+0000","last_change":"2024-04-23T18:23:55.265714+0000","last_active":"2024-04-23T18:24:36.914632+0000","last_peered":"2024-04-23T18:24:36.914632+0000","last_clean":"2024-04-23T18:24:36.914632+0000","last_became_active":"2024-04-23T18:23:55.265198+0000","last_became_peered":"2024-04-23T18:23:55.265198+0000","last_unstale":"2024-04-23T18:24:36.914632+0000","last_undegraded":"2024-04-23T18:24:36.914632+0000","last_fullsized":"2024-04-23T18:24:36.914632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:10.770748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.342678+0000","last_change":"2024-04-23T18:23:56.102347+0000","last_active":"2024-04-23T18:25:05.342678+0000","last_peered":"2024-04-23T18:25:05.342678+0000","last_clean":"2024-04-23T18:25:05.342678+0000","last_became_active":"2024-04-23T18:23:56.100687+0000","last_became_peered":"2024-04-23T18:23:56.100687+0000","last_unstale":"2024-04-23T18:25:05.342678+0000","last_undegraded":"2024-04-23T18:25:05.342678+0000","last_fullsized":"2024-04-23T18:25:05.342678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:20.378730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696205+0000","last_change":"2024-04-23T18:23:55.255943+0000","last_active":"2024-04-23T18:24:36.696205+0000","last_peered":"2024-04-23T18:24:36.696205+0000","last_clean":"2024-04-23T18:24:36.696205+0000","last_became_active":"2024-04-23T18:23:55.255618+0000","last_became_peered":"2024-04-23T18:23:55.255618+0000","last_unstale":"2024-04-23T18:24:36.696205+0000","last_undegraded":"2024-04-23T18:24:36.696205+0000","last_fullsized":"2024-04-23T18:24:36.696205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:01.408533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.388079+0000","last_change":"2024-04-23T18:23:56.091664+0000","last_active":"2024-04-23T18:25:05.388079+0000","last_peered":"2024-04-23T18:25:05.388079+0000","last_clean":"2024-04-23T18:25:05.388079+0000","last_became_active":"2024-04-23T18:23:56.091308+0000","last_became_peered":"2024-04-23T18:23:56.091308+0000","last_unstale":"2024-04-23T18:25:05.388079+0000","last_undegraded":"2024-04-23T18:25:05.388079+0000","last_fullsized":"2024-04-23T18:25:05.388079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:31.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696556+0000","last_change":"2024-04-23T18:23:55.260992+0000","last_active":"2024-04-23T18:24:36.696556+0000","last_peered":"2024-04-23T18:24:36.696556+0000","last_clean":"2024-04-23T18:24:36.696556+0000","last_became_active":"2024-04-23T18:23:55.260679+0000","last_became_peered":"2024-04-23T18:23:55.260679+0000","last_unstale":"2024-04-23T18:24:36.696556+0000","last_undegraded":"2024-04-23T18:24:36.696556+0000","last_fullsized":"2024-04-23T18:24:36.696556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:02:50.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.393515+0000","last_change":"2024-04-23T18:23:56.097956+0000","last_active":"2024-04-23T18:25:05.393515+0000","last_peered":"2024-04-23T18:25:05.393515+0000","last_clean":"2024-04-23T18:25:05.393515+0000","last_became_active":"2024-04-23T18:23:56.097231+0000","last_became_peered":"2024-04-23T18:23:56.097231+0000","last_unstale":"2024-04-23T18:25:05.393515+0000","last_undegraded":"2024-04-23T18:25:05.393515+0000","last_fullsized":"2024-04-23T18:25:05.393515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:10.945857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.714667+0000","last_change":"2024-04-23T18:24:36.714223+0000","last_active":"2024-04-23T18:24:36.714667+0000","last_peered":"2024-04-23T18:24:36.714667+0000","last_clean":"2024-04-23T18:24:36.714667+0000","last_became_active":"2024-04-23T18:24:36.700975+0000","last_became_peered":"2024-04-23T18:24:36.700975+0000","last_unstale":"2024-04-23T18:24:36.714667+0000","last_undegraded":"2024-04-23T18:24:36.714667+0000","last_fullsized":"2024-04-23T18:24:36.714667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:58:50.227699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.397687+0000","last_change":"2024-04-23T18:23:56.096889+0000","last_active":"2024-04-23T18:25:05.397687+0000","last_peered":"2024-04-23T18:25:05.397687+0000","last_clean":"2024-04-23T18:25:05.397687+0000","last_became_active":"2024-04-23T18:23:56.096520+0000","last_became_peered":"2024-04-23T18:23:56.096520+0000","last_unstale":"2024-04-23T18:25:05.397687+0000","last_undegraded":"2024-04-23T18:25:05.397687+0000","last_fullsized":"2024-04-23T18:25:05.397687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:34:53.785481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:15.737129+0000","last_change":"2024-04-23T18:23:55.258710+0000","last_active":"2024-04-23T18:25:15.737129+0000","last_peered":"2024-04-23T18:25:15.737129+0000","last_clean":"2024-04-23T18:25:15.737129+0000","last_became_active":"2024-04-23T18:23:55.256247+0000","last_became_peered":"2024-04-23T18:23:55.256247+0000","last_unstale":"2024-04-23T18:25:15.737129+0000","last_undegraded":"2024-04-23T18:25:15.737129+0000","last_fullsized":"2024-04-23T18:25:15.737129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:20.551614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.397705+0000","last_change":"2024-04-23T18:23:56.101862+0000","last_active":"2024-04-23T18:25:05.397705+0000","last_peered":"2024-04-23T18:25:05.397705+0000","last_clean":"2024-04-23T18:25:05.397705+0000","last_became_active":"2024-04-23T18:23:56.101458+0000","last_became_peered":"2024-04-23T18:23:56.101458+0000","last_unstale":"2024-04-23T18:25:05.397705+0000","last_undegraded":"2024-04-23T18:25:05.397705+0000","last_fullsized":"2024-04-23T18:25:05.397705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:59.292077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.699731+0000","last_change":"2024-04-23T18:23:55.258292+0000","last_active":"2024-04-23T18:24:36.699731+0000","last_peered":"2024-04-23T18:24:36.699731+0000","last_clean":"2024-04-23T18:24:36.699731+0000","last_became_active":"2024-04-23T18:23:55.257954+0000","last_became_peered":"2024-04-23T18:23:55.257954+0000","last_unstale":"2024-04-23T18:24:36.699731+0000","last_undegraded":"2024-04-23T18:24:36.699731+0000","last_fullsized":"2024-04-23T18:24:36.699731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:39.105967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.387905+0000","last_change":"2024-04-23T18:23:56.105977+0000","last_active":"2024-04-23T18:25:05.387905+0000","last_peered":"2024-04-23T18:25:05.387905+0000","last_clean":"2024-04-23T18:25:05.387905+0000","last_became_active":"2024-04-23T18:23:56.105582+0000","last_became_peered":"2024-04-23T18:23:56.105582+0000","last_unstale":"2024-04-23T18:25:05.387905+0000","last_undegraded":"2024-04-23T18:25:05.387905+0000","last_fullsized":"2024-04-23T18:25:05.387905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:28:10.700684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.694767+0000","last_change":"2024-04-23T18:23:55.252095+0000","last_active":"2024-04-23T18:24:36.694767+0000","last_peered":"2024-04-23T18:24:36.694767+0000","last_clean":"2024-04-23T18:24:36.694767+0000","last_became_active":"2024-04-23T18:23:55.251702+0000","last_became_peered":"2024-04-23T18:23:55.251702+0000","last_unstale":"2024-04-23T18:24:36.694767+0000","last_undegraded":"2024-04-23T18:24:36.694767+0000","last_fullsized":"2024-04-23T18:24:36.694767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:21.625905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.398619+0000","last_change":"2024-04-23T18:23:56.101997+0000","last_active":"2024-04-23T18:25:05.398619+0000","last_peered":"2024-04-23T18:25:05.398619+0000","last_clean":"2024-04-23T18:25:05.398619+0000","last_became_active":"2024-04-23T18:23:56.101629+0000","last_became_peered":"2024-04-23T18:23:56.101629+0000","last_unstale":"2024-04-23T18:25:05.398619+0000","last_undegraded":"2024-04-23T18:25:05.398619+0000","last_fullsized":"2024-04-23T18:25:05.398619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:42.484413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.388451+0000","last_change":"2024-04-23T18:23:56.090543+0000","last_active":"2024-04-23T18:25:05.388451+0000","last_peered":"2024-04-23T18:25:05.388451+0000","last_clean":"2024-04-23T18:25:05.388451+0000","last_became_active":"2024-04-23T18:23:56.090191+0000","last_became_peered":"2024-04-23T18:23:56.090191+0000","last_unstale":"2024-04-23T18:25:05.388451+0000","last_undegraded":"2024-04-23T18:25:05.388451+0000","last_fullsized":"2024-04-23T18:25:05.388451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:01.401822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.379225+0000","last_change":"2024-04-23T18:23:56.105537+0000","last_active":"2024-04-23T18:25:05.379225+0000","last_peered":"2024-04-23T18:25:05.379225+0000","last_clean":"2024-04-23T18:25:05.379225+0000","last_became_active":"2024-04-23T18:23:56.105130+0000","last_became_peered":"2024-04-23T18:23:56.105130+0000","last_unstale":"2024-04-23T18:25:05.379225+0000","last_undegraded":"2024-04-23T18:25:05.379225+0000","last_fullsized":"2024-04-23T18:25:05.379225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:59.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.377655+0000","last_change":"2024-04-23T18:23:56.097765+0000","last_active":"2024-04-23T18:25:05.377655+0000","last_peered":"2024-04-23T18:25:05.377655+0000","last_clean":"2024-04-23T18:25:05.377655+0000","last_became_active":"2024-04-23T18:23:56.097439+0000","last_became_peered":"2024-04-23T18:23:56.097439+0000","last_unstale":"2024-04-23T18:25:05.377655+0000","last_undegraded":"2024-04-23T18:25:05.377655+0000","last_fullsized":"2024-04-23T18:25:05.377655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:31.529751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.395414+0000","last_change":"2024-04-23T18:23:56.108548+0000","last_active":"2024-04-23T18:25:05.395414+0000","last_peered":"2024-04-23T18:25:05.395414+0000","last_clean":"2024-04-23T18:25:05.395414+0000","last_became_active":"2024-04-23T18:23:56.108214+0000","last_became_peered":"2024-04-23T18:23:56.108214+0000","last_unstale":"2024-04-23T18:25:05.395414+0000","last_undegraded":"2024-04-23T18:25:05.395414+0000","last_fullsized":"2024-04-23T18:25:05.395414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:41.738754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.380937+0000","last_change":"2024-04-23T18:24:36.701809+0000","last_active":"2024-04-23T18:25:05.380937+0000","last_peered":"2024-04-23T18:25:05.380937+0000","last_clean":"2024-04-23T18:25:05.380937+0000","last_became_active":"2024-04-23T18:24:36.701405+0000","last_became_peered":"2024-04-23T18:24:36.701405+0000","last_unstale":"2024-04-23T18:25:05.380937+0000","last_undegraded":"2024-04-23T18:25:05.380937+0000","last_fullsized":"2024-04-23T18:25:05.380937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:03.215622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.385607+0000","last_change":"2024-04-23T18:23:56.093969+0000","last_active":"2024-04-23T18:25:05.385607+0000","last_peered":"2024-04-23T18:25:05.385607+0000","last_clean":"2024-04-23T18:25:05.385607+0000","last_became_active":"2024-04-23T18:23:56.093568+0000","last_became_peered":"2024-04-23T18:23:56.093568+0000","last_unstale":"2024-04-23T18:25:05.385607+0000","last_undegraded":"2024-04-23T18:25:05.385607+0000","last_fullsized":"2024-04-23T18:25:05.385607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:08.361305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.392473+0000","last_change":"2024-04-23T18:23:56.096106+0000","last_active":"2024-04-23T18:25:05.392473+0000","last_peered":"2024-04-23T18:25:05.392473+0000","last_clean":"2024-04-23T18:25:05.392473+0000","last_became_active":"2024-04-23T18:23:56.095745+0000","last_became_peered":"2024-04-23T18:23:56.095745+0000","last_unstale":"2024-04-23T18:25:05.392473+0000","last_undegraded":"2024-04-23T18:25:05.392473+0000","last_fullsized":"2024-04-23T18:25:05.392473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:01.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.388934+0000","last_change":"2024-04-23T18:23:56.095474+0000","last_active":"2024-04-23T18:25:05.388934+0000","last_peered":"2024-04-23T18:25:05.388934+0000","last_clean":"2024-04-23T18:25:05.388934+0000","last_became_active":"2024-04-23T18:23:56.095116+0000","last_became_peered":"2024-04-23T18:23:56.095116+0000","last_unstale":"2024-04-23T18:25:05.388934+0000","last_undegraded":"2024-04-23T18:25:05.388934+0000","last_fullsized":"2024-04-23T18:25:05.388934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:50:00.339558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.384860+0000","last_change":"2024-04-23T18:23:56.096022+0000","last_active":"2024-04-23T18:25:05.384860+0000","last_peered":"2024-04-23T18:25:05.384860+0000","last_clean":"2024-04-23T18:25:05.384860+0000","last_became_active":"2024-04-23T18:23:56.095493+0000","last_became_peered":"2024-04-23T18:23:56.095493+0000","last_unstale":"2024-04-23T18:25:05.384860+0000","last_undegraded":"2024-04-23T18:25:05.384860+0000","last_fullsized":"2024-04-23T18:25:05.384860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:42.209072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.390148+0000","last_change":"2024-04-23T18:23:56.101697+0000","last_active":"2024-04-23T18:25:05.390148+0000","last_peered":"2024-04-23T18:25:05.390148+0000","last_clean":"2024-04-23T18:25:05.390148+0000","last_became_active":"2024-04-23T18:23:56.098708+0000","last_became_peered":"2024-04-23T18:23:56.098708+0000","last_unstale":"2024-04-23T18:25:05.390148+0000","last_undegraded":"2024-04-23T18:25:05.390148+0000","last_fullsized":"2024-04-23T18:25:05.390148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:14.170700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.399544+0000","last_change":"2024-04-23T18:23:56.096982+0000","last_active":"2024-04-23T18:25:05.399544+0000","last_peered":"2024-04-23T18:25:05.399544+0000","last_clean":"2024-04-23T18:25:05.399544+0000","last_became_active":"2024-04-23T18:23:56.096608+0000","last_became_peered":"2024-04-23T18:23:56.096608+0000","last_unstale":"2024-04-23T18:25:05.399544+0000","last_undegraded":"2024-04-23T18:25:05.399544+0000","last_fullsized":"2024-04-23T18:25:05.399544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:50.329337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.381320+0000","last_change":"2024-04-23T18:23:56.100345+0000","last_active":"2024-04-23T18:25:05.381320+0000","last_peered":"2024-04-23T18:25:05.381320+0000","last_clean":"2024-04-23T18:25:05.381320+0000","last_became_active":"2024-04-23T18:23:56.100043+0000","last_became_peered":"2024-04-23T18:23:56.100043+0000","last_unstale":"2024-04-23T18:25:05.381320+0000","last_undegraded":"2024-04-23T18:25:05.381320+0000","last_fullsized":"2024-04-23T18:25:05.381320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:36.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.387198+0000","last_change":"2024-04-23T18:23:56.101947+0000","last_active":"2024-04-23T18:25:05.387198+0000","last_peered":"2024-04-23T18:25:05.387198+0000","last_clean":"2024-04-23T18:25:05.387198+0000","last_became_active":"2024-04-23T18:23:56.101627+0000","last_became_peered":"2024-04-23T18:23:56.101627+0000","last_unstale":"2024-04-23T18:25:05.387198+0000","last_undegraded":"2024-04-23T18:25:05.387198+0000","last_fullsized":"2024-04-23T18:25:05.387198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:21:43.703653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279760,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68781,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":780,"num_read_kb":462,"num_write":246,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":206343,"data_compressed":5595,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351007,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":461377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":1114165,"data_compressed":37450,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":461239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037806,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":461617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1054808,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":460956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":523164,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":462295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":523736,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":460781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":461461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1053954,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63047,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63619,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63563,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:25:22.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T18:25:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:22.446 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:22 smithi019 ceph-mon[29215]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-23T18:25:22.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:22 smithi044 ceph-mon[32458]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-23T18:25:22.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:22 smithi155 ceph-mon[33468]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-23T18:25:24.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 -- 172.21.15.19:0/981517637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006b0f0 msgr2=0x7fdb4010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:24.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 --2- 172.21.15.19:0/981517637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006b0f0 0x7fdb4010f900 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800b0a0 tx=0x7fdb3800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 -- 172.21.15.19:0/981517637 shutdown_connections 2024-04-23T18:25:24.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 --2- 172.21.15.19:0/981517637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006b0f0 0x7fdb4010f900 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 --2- 172.21.15.19:0/981517637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40108a30 0x7fdb40108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 --2- 172.21.15.19:0/981517637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb4006a750 0x7fdb4006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 -- 172.21.15.19:0/981517637 >> 172.21.15.19:0/981517637 conn(0x7fdb400788f0 msgr2=0x7fdb40078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:24.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 -- 172.21.15.19:0/981517637 shutdown_connections 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 -- 172.21.15.19:0/981517637 wait complete. 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 Processor -- start 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.299+0000 7fdb4868e700 1 -- start start 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4868e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006a750 0x7fdb40199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4868e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb4006b0f0 0x7fdb40199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4868e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40108a30 0x7fdb4019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4868e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40112c80 con 0x7fdb4006a750 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4868e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40112b00 con 0x7fdb4006b0f0 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4868e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40112e00 con 0x7fdb40108a30 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4642a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006a750 0x7fdb40199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4642a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006a750 0x7fdb40199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47056/0 (socket says 172.21.15.19:47056) 2024-04-23T18:25:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4642a700 1 -- 172.21.15.19:0/1647158955 learned_addr learned my addr 172.21.15.19:0/1647158955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb45c29700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb4006b0f0 0x7fdb40199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb46c2b700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40108a30 0x7fdb4019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4642a700 1 -- 172.21.15.19:0/1647158955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40108a30 msgr2=0x7fdb4019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4642a700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40108a30 0x7fdb4019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4642a700 1 -- 172.21.15.19:0/1647158955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb4006b0f0 msgr2=0x7fdb40199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4642a700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb4006b0f0 0x7fdb40199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4642a700 1 -- 172.21.15.19:0/1647158955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4019e0d0 con 0x7fdb4006a750 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb45c29700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb4006b0f0 0x7fdb40199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4642a700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006a750 0x7fdb40199110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00b530 tx=0x7fdb3c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb377fe700 1 -- 172.21.15.19:0/1647158955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c015020 con 0x7fdb4006a750 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb4868e700 1 -- 172.21.15.19:0/1647158955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4019e3c0 con 0x7fdb4006a750 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb377fe700 1 -- 172.21.15.19:0/1647158955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb3c011ce0 con 0x7fdb4006a750 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.300+0000 7fdb377fe700 1 -- 172.21.15.19:0/1647158955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c010d20 con 0x7fdb4006a750 2024-04-23T18:25:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.301+0000 7fdb4868e700 1 -- 172.21.15.19:0/1647158955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb401a5c40 con 0x7fdb4006a750 2024-04-23T18:25:24.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.301+0000 7fdb4868e700 1 -- 172.21.15.19:0/1647158955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4010b030 con 0x7fdb4006a750 2024-04-23T18:25:24.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.303+0000 7fdb377fe700 1 -- 172.21.15.19:0/1647158955 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdb3c018020 con 0x7fdb4006a750 2024-04-23T18:25:24.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.303+0000 7fdb377fe700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdb2c0a3c40 0x7fdb2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:24.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.303+0000 7fdb45c29700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdb2c0a3c40 0x7fdb2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:24.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.303+0000 7fdb377fe700 1 -- 172.21.15.19:0/1647158955 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fdb3c0d01f0 con 0x7fdb4006a750 2024-04-23T18:25:24.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.304+0000 7fdb45c29700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdb2c0a3c40 0x7fdb2c0a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb30005fd0 tx=0x7fdb30005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:24.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.307+0000 7fdb377fe700 1 -- 172.21.15.19:0/1647158955 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb3c01f020 con 0x7fdb4006a750 2024-04-23T18:25:24.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.511+0000 7fdb4868e700 1 -- 172.21.15.19:0/1647158955 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb40062e80 con 0x7fdb2c0a3c40 2024-04-23T18:25:24.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.518+0000 7fdb377fe700 1 -- 172.21.15.19:0/1647158955 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259104 (secure 0 0 0) 0x7fdb40062e80 con 0x7fdb2c0a3c40 2024-04-23T18:25:24.514 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:24.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 -- 172.21.15.19:0/1647158955 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdb2c0a3c40 msgr2=0x7fdb2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:24.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdb2c0a3c40 0x7fdb2c0a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb30005fd0 tx=0x7fdb30005e30 comp rx=0 tx=0).stop 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 -- 172.21.15.19:0/1647158955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006a750 msgr2=0x7fdb40199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006a750 0x7fdb40199110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00b530 tx=0x7fdb3c00b8f0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 -- 172.21.15.19:0/1647158955 shutdown_connections 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fdb2c0a3c40 0x7fdb2c0a6100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb40108a30 0x7fdb4019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006a750 0x7fdb40199110 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 --2- 172.21.15.19:0/1647158955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb4006b0f0 0x7fdb40199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 -- 172.21.15.19:0/1647158955 >> 172.21.15.19:0/1647158955 conn(0x7fdb400788f0 msgr2=0x7fdb400ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 -- 172.21.15.19:0/1647158955 shutdown_connections 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:24.521+0000 7fdb4868e700 1 -- 172.21.15.19:0/1647158955 wait complete. 2024-04-23T18:25:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-23T18:25:24.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:24 smithi019 ceph-mon[29215]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:24.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:24 smithi044 ceph-mon[32458]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:24.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:24 smithi155 ceph-mon[33468]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:25.206 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-04-23T18:25:23.374287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659383,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":951,"num_read_kb":601,"num_write":429,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":853,"ondisk_log_size":853,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381796,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560028,"statfs":{"total":1151940427776,"available":1151549468672,"internally_reserved":0,"allocated":12128256,"data_stored":7499553,"data_compressed":112350,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002896"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.383027+0000","last_change":"2024-04-23T18:23:56.098292+0000","last_active":"2024-04-23T18:25:05.383027+0000","last_peered":"2024-04-23T18:25:05.383027+0000","last_clean":"2024-04-23T18:25:05.383027+0000","last_became_active":"2024-04-23T18:23:56.096623+0000","last_became_peered":"2024-04-23T18:23:56.096623+0000","last_unstale":"2024-04-23T18:25:05.383027+0000","last_undegraded":"2024-04-23T18:25:05.383027+0000","last_fullsized":"2024-04-23T18:25:05.383027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:06.186581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.388004+0000","last_change":"2024-04-23T18:23:56.096807+0000","last_active":"2024-04-23T18:25:05.388004+0000","last_peered":"2024-04-23T18:25:05.388004+0000","last_clean":"2024-04-23T18:25:05.388004+0000","last_became_active":"2024-04-23T18:23:56.096327+0000","last_became_peered":"2024-04-23T18:23:56.096327+0000","last_unstale":"2024-04-23T18:25:05.388004+0000","last_undegraded":"2024-04-23T18:25:05.388004+0000","last_fullsized":"2024-04-23T18:25:05.388004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:56:16.391821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.394411+0000","last_change":"2024-04-23T18:23:56.101615+0000","last_active":"2024-04-23T18:25:05.394411+0000","last_peered":"2024-04-23T18:25:05.394411+0000","last_clean":"2024-04-23T18:25:05.394411+0000","last_became_active":"2024-04-23T18:23:56.100929+0000","last_became_peered":"2024-04-23T18:23:56.100929+0000","last_unstale":"2024-04-23T18:25:05.394411+0000","last_undegraded":"2024-04-23T18:25:05.394411+0000","last_fullsized":"2024-04-23T18:25:05.394411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:25.547029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.375061+0000","last_change":"2024-04-23T18:23:56.103627+0000","last_active":"2024-04-23T18:25:05.375061+0000","last_peered":"2024-04-23T18:25:05.375061+0000","last_clean":"2024-04-23T18:25:05.375061+0000","last_became_active":"2024-04-23T18:23:56.102752+0000","last_became_peered":"2024-04-23T18:23:56.102752+0000","last_unstale":"2024-04-23T18:25:05.375061+0000","last_undegraded":"2024-04-23T18:25:05.375061+0000","last_fullsized":"2024-04-23T18:25:05.375061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:28.460342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.369522+0000","last_change":"2024-04-23T18:23:56.094889+0000","last_active":"2024-04-23T18:25:05.369522+0000","last_peered":"2024-04-23T18:25:05.369522+0000","last_clean":"2024-04-23T18:25:05.369522+0000","last_became_active":"2024-04-23T18:23:56.094262+0000","last_became_peered":"2024-04-23T18:23:56.094262+0000","last_unstale":"2024-04-23T18:25:05.369522+0000","last_undegraded":"2024-04-23T18:25:05.369522+0000","last_fullsized":"2024-04-23T18:25:05.369522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:22.332114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.369406+0000","last_change":"2024-04-23T18:23:56.108893+0000","last_active":"2024-04-23T18:25:05.369406+0000","last_peered":"2024-04-23T18:25:05.369406+0000","last_clean":"2024-04-23T18:25:05.369406+0000","last_became_active":"2024-04-23T18:23:56.108229+0000","last_became_peered":"2024-04-23T18:23:56.108229+0000","last_unstale":"2024-04-23T18:25:05.369406+0000","last_undegraded":"2024-04-23T18:25:05.369406+0000","last_fullsized":"2024-04-23T18:25:05.369406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:50.930321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.384178+0000","last_change":"2024-04-23T18:23:56.098674+0000","last_active":"2024-04-23T18:25:05.384178+0000","last_peered":"2024-04-23T18:25:05.384178+0000","last_clean":"2024-04-23T18:25:05.384178+0000","last_became_active":"2024-04-23T18:23:56.098299+0000","last_became_peered":"2024-04-23T18:23:56.098299+0000","last_unstale":"2024-04-23T18:25:05.384178+0000","last_undegraded":"2024-04-23T18:25:05.384178+0000","last_fullsized":"2024-04-23T18:25:05.384178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:05.711410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.386974+0000","last_change":"2024-04-23T18:23:56.097437+0000","last_active":"2024-04-23T18:25:05.386974+0000","last_peered":"2024-04-23T18:25:05.386974+0000","last_clean":"2024-04-23T18:25:05.386974+0000","last_became_active":"2024-04-23T18:23:56.095702+0000","last_became_peered":"2024-04-23T18:23:56.095702+0000","last_unstale":"2024-04-23T18:25:05.386974+0000","last_undegraded":"2024-04-23T18:25:05.386974+0000","last_fullsized":"2024-04-23T18:25:05.386974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:35.754832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.384574+0000","last_change":"2024-04-23T18:23:56.096060+0000","last_active":"2024-04-23T18:25:05.384574+0000","last_peered":"2024-04-23T18:25:05.384574+0000","last_clean":"2024-04-23T18:25:05.384574+0000","last_became_active":"2024-04-23T18:23:56.095597+0000","last_became_peered":"2024-04-23T18:23:56.095597+0000","last_unstale":"2024-04-23T18:25:05.384574+0000","last_undegraded":"2024-04-23T18:25:05.384574+0000","last_fullsized":"2024-04-23T18:25:05.384574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:59.735347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.377984+0000","last_change":"2024-04-23T18:23:56.109033+0000","last_active":"2024-04-23T18:25:05.377984+0000","last_peered":"2024-04-23T18:25:05.377984+0000","last_clean":"2024-04-23T18:25:05.377984+0000","last_became_active":"2024-04-23T18:23:56.108472+0000","last_became_peered":"2024-04-23T18:23:56.108472+0000","last_unstale":"2024-04-23T18:25:05.377984+0000","last_undegraded":"2024-04-23T18:25:05.377984+0000","last_fullsized":"2024-04-23T18:25:05.377984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:32.491181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.398430+0000","last_change":"2024-04-23T18:23:56.097771+0000","last_active":"2024-04-23T18:25:05.398430+0000","last_peered":"2024-04-23T18:25:05.398430+0000","last_clean":"2024-04-23T18:25:05.398430+0000","last_became_active":"2024-04-23T18:23:56.097403+0000","last_became_peered":"2024-04-23T18:23:56.097403+0000","last_unstale":"2024-04-23T18:25:05.398430+0000","last_undegraded":"2024-04-23T18:25:05.398430+0000","last_fullsized":"2024-04-23T18:25:05.398430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:48:36.261543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.374752+0000","last_change":"2024-04-23T18:23:56.108550+0000","last_active":"2024-04-23T18:25:05.374752+0000","last_peered":"2024-04-23T18:25:05.374752+0000","last_clean":"2024-04-23T18:25:05.374752+0000","last_became_active":"2024-04-23T18:23:56.108191+0000","last_became_peered":"2024-04-23T18:23:56.108191+0000","last_unstale":"2024-04-23T18:25:05.374752+0000","last_undegraded":"2024-04-23T18:25:05.374752+0000","last_fullsized":"2024-04-23T18:25:05.374752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:25.142466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.375205+0000","last_change":"2024-04-23T18:23:56.092573+0000","last_active":"2024-04-23T18:25:05.375205+0000","last_peered":"2024-04-23T18:25:05.375205+0000","last_clean":"2024-04-23T18:25:05.375205+0000","last_became_active":"2024-04-23T18:23:56.092095+0000","last_became_peered":"2024-04-23T18:23:56.092095+0000","last_unstale":"2024-04-23T18:25:05.375205+0000","last_undegraded":"2024-04-23T18:25:05.375205+0000","last_fullsized":"2024-04-23T18:25:05.375205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:01.601206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.392359+0000","last_change":"2024-04-23T18:23:56.102073+0000","last_active":"2024-04-23T18:25:05.392359+0000","last_peered":"2024-04-23T18:25:05.392359+0000","last_clean":"2024-04-23T18:25:05.392359+0000","last_became_active":"2024-04-23T18:23:56.101059+0000","last_became_peered":"2024-04-23T18:23:56.101059+0000","last_unstale":"2024-04-23T18:25:05.392359+0000","last_undegraded":"2024-04-23T18:25:05.392359+0000","last_fullsized":"2024-04-23T18:25:05.392359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:37.904099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.395386+0000","last_change":"2024-04-23T18:23:56.108482+0000","last_active":"2024-04-23T18:25:05.395386+0000","last_peered":"2024-04-23T18:25:05.395386+0000","last_clean":"2024-04-23T18:25:05.395386+0000","last_became_active":"2024-04-23T18:23:56.108128+0000","last_became_peered":"2024-04-23T18:23:56.108128+0000","last_unstale":"2024-04-23T18:25:05.395386+0000","last_undegraded":"2024-04-23T18:25:05.395386+0000","last_fullsized":"2024-04-23T18:25:05.395386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:51.925881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.398957+0000","last_change":"2024-04-23T18:23:56.099691+0000","last_active":"2024-04-23T18:25:05.398957+0000","last_peered":"2024-04-23T18:25:05.398957+0000","last_clean":"2024-04-23T18:25:05.398957+0000","last_became_active":"2024-04-23T18:23:56.099272+0000","last_became_peered":"2024-04-23T18:23:56.099272+0000","last_unstale":"2024-04-23T18:25:05.398957+0000","last_undegraded":"2024-04-23T18:25:05.398957+0000","last_fullsized":"2024-04-23T18:25:05.398957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:41.834104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.383395+0000","last_change":"2024-04-23T18:23:56.097965+0000","last_active":"2024-04-23T18:25:05.383395+0000","last_peered":"2024-04-23T18:25:05.383395+0000","last_clean":"2024-04-23T18:25:05.383395+0000","last_became_active":"2024-04-23T18:23:56.096766+0000","last_became_peered":"2024-04-23T18:23:56.096766+0000","last_unstale":"2024-04-23T18:25:05.383395+0000","last_undegraded":"2024-04-23T18:25:05.383395+0000","last_fullsized":"2024-04-23T18:25:05.383395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:40:00.076939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.388729+0000","last_change":"2024-04-23T18:23:56.095158+0000","last_active":"2024-04-23T18:25:05.388729+0000","last_peered":"2024-04-23T18:25:05.388729+0000","last_clean":"2024-04-23T18:25:05.388729+0000","last_became_active":"2024-04-23T18:23:56.094812+0000","last_became_peered":"2024-04-23T18:23:56.094812+0000","last_unstale":"2024-04-23T18:25:05.388729+0000","last_undegraded":"2024-04-23T18:25:05.388729+0000","last_fullsized":"2024-04-23T18:25:05.388729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:40.874698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.378382+0000","last_change":"2024-04-23T18:23:56.099358+0000","last_active":"2024-04-23T18:25:05.378382+0000","last_peered":"2024-04-23T18:25:05.378382+0000","last_clean":"2024-04-23T18:25:05.378382+0000","last_became_active":"2024-04-23T18:23:56.099031+0000","last_became_peered":"2024-04-23T18:23:56.099031+0000","last_unstale":"2024-04-23T18:25:05.378382+0000","last_undegraded":"2024-04-23T18:25:05.378382+0000","last_fullsized":"2024-04-23T18:25:05.378382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:29.634932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:17.738565+0000","last_change":"2024-04-23T18:23:55.262939+0000","last_active":"2024-04-23T18:25:17.738565+0000","last_peered":"2024-04-23T18:25:17.738565+0000","last_clean":"2024-04-23T18:25:17.738565+0000","last_became_active":"2024-04-23T18:23:55.262609+0000","last_became_peered":"2024-04-23T18:23:55.262609+0000","last_unstale":"2024-04-23T18:25:17.738565+0000","last_undegraded":"2024-04-23T18:25:17.738565+0000","last_fullsized":"2024-04-23T18:25:17.738565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:12.418349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.372597+0000","last_change":"2024-04-23T18:23:56.092182+0000","last_active":"2024-04-23T18:25:05.372597+0000","last_peered":"2024-04-23T18:25:05.372597+0000","last_clean":"2024-04-23T18:25:05.372597+0000","last_became_active":"2024-04-23T18:23:56.091444+0000","last_became_peered":"2024-04-23T18:23:56.091444+0000","last_unstale":"2024-04-23T18:25:05.372597+0000","last_undegraded":"2024-04-23T18:25:05.372597+0000","last_fullsized":"2024-04-23T18:25:05.372597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:00.775498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696726+0000","last_change":"2024-04-23T18:23:55.259880+0000","last_active":"2024-04-23T18:24:36.696726+0000","last_peered":"2024-04-23T18:24:36.696726+0000","last_clean":"2024-04-23T18:24:36.696726+0000","last_became_active":"2024-04-23T18:23:55.259576+0000","last_became_peered":"2024-04-23T18:23:55.259576+0000","last_unstale":"2024-04-23T18:24:36.696726+0000","last_undegraded":"2024-04-23T18:24:36.696726+0000","last_fullsized":"2024-04-23T18:24:36.696726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:13.783776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.394342+0000","last_change":"2024-04-23T18:23:56.094307+0000","last_active":"2024-04-23T18:25:05.394342+0000","last_peered":"2024-04-23T18:25:05.394342+0000","last_clean":"2024-04-23T18:25:05.394342+0000","last_became_active":"2024-04-23T18:23:56.093893+0000","last_became_peered":"2024-04-23T18:23:56.093893+0000","last_unstale":"2024-04-23T18:25:05.394342+0000","last_undegraded":"2024-04-23T18:25:05.394342+0000","last_fullsized":"2024-04-23T18:25:05.394342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:12:35.385497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.697688+0000","last_change":"2024-04-23T18:23:55.263137+0000","last_active":"2024-04-23T18:24:36.697688+0000","last_peered":"2024-04-23T18:24:36.697688+0000","last_clean":"2024-04-23T18:24:36.697688+0000","last_became_active":"2024-04-23T18:23:55.262553+0000","last_became_peered":"2024-04-23T18:23:55.262553+0000","last_unstale":"2024-04-23T18:24:36.697688+0000","last_undegraded":"2024-04-23T18:24:36.697688+0000","last_fullsized":"2024-04-23T18:24:36.697688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:33:42.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.393759+0000","last_change":"2024-04-23T18:23:56.091098+0000","last_active":"2024-04-23T18:25:05.393759+0000","last_peered":"2024-04-23T18:25:05.393759+0000","last_clean":"2024-04-23T18:25:05.393759+0000","last_became_active":"2024-04-23T18:23:56.090635+0000","last_became_peered":"2024-04-23T18:23:56.090635+0000","last_unstale":"2024-04-23T18:25:05.393759+0000","last_undegraded":"2024-04-23T18:25:05.393759+0000","last_fullsized":"2024-04-23T18:25:05.393759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:17.787937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696489+0000","last_change":"2024-04-23T18:23:55.262146+0000","last_active":"2024-04-23T18:24:36.696489+0000","last_peered":"2024-04-23T18:24:36.696489+0000","last_clean":"2024-04-23T18:24:36.696489+0000","last_became_active":"2024-04-23T18:23:55.261838+0000","last_became_peered":"2024-04-23T18:23:55.261838+0000","last_unstale":"2024-04-23T18:24:36.696489+0000","last_undegraded":"2024-04-23T18:24:36.696489+0000","last_fullsized":"2024-04-23T18:24:36.696489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:42:39.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.387953+0000","last_change":"2024-04-23T18:24:36.915394+0000","last_active":"2024-04-23T18:25:05.387953+0000","last_peered":"2024-04-23T18:25:05.387953+0000","last_clean":"2024-04-23T18:25:05.387953+0000","last_became_active":"2024-04-23T18:24:36.915072+0000","last_became_peered":"2024-04-23T18:24:36.915072+0000","last_unstale":"2024-04-23T18:25:05.387953+0000","last_undegraded":"2024-04-23T18:25:05.387953+0000","last_fullsized":"2024-04-23T18:25:05.387953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.132410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":106,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:17.740249+0000","last_change":"2024-04-23T18:23:55.261399+0000","last_active":"2024-04-23T18:25:17.740249+0000","last_peered":"2024-04-23T18:25:17.740249+0000","last_clean":"2024-04-23T18:25:17.740249+0000","last_became_active":"2024-04-23T18:23:55.261012+0000","last_became_peered":"2024-04-23T18:23:55.261012+0000","last_unstale":"2024-04-23T18:25:17.740249+0000","last_undegraded":"2024-04-23T18:25:17.740249+0000","last_fullsized":"2024-04-23T18:25:17.740249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:31.377776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.377694+0000","last_change":"2024-04-23T18:23:56.096785+0000","last_active":"2024-04-23T18:25:05.377694+0000","last_peered":"2024-04-23T18:25:05.377694+0000","last_clean":"2024-04-23T18:25:05.377694+0000","last_became_active":"2024-04-23T18:23:56.096420+0000","last_became_peered":"2024-04-23T18:23:56.096420+0000","last_unstale":"2024-04-23T18:25:05.377694+0000","last_undegraded":"2024-04-23T18:25:05.377694+0000","last_fullsized":"2024-04-23T18:25:05.377694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:20:49.215071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.693029+0000","last_change":"2024-04-23T18:23:55.260645+0000","last_active":"2024-04-23T18:24:36.693029+0000","last_peered":"2024-04-23T18:24:36.693029+0000","last_clean":"2024-04-23T18:24:36.693029+0000","last_became_active":"2024-04-23T18:23:55.260118+0000","last_became_peered":"2024-04-23T18:23:55.260118+0000","last_unstale":"2024-04-23T18:24:36.693029+0000","last_undegraded":"2024-04-23T18:24:36.693029+0000","last_fullsized":"2024-04-23T18:24:36.693029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:14:12.066297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.370004+0000","last_change":"2024-04-23T18:24:36.702766+0000","last_active":"2024-04-23T18:25:05.370004+0000","last_peered":"2024-04-23T18:25:05.370004+0000","last_clean":"2024-04-23T18:25:05.370004+0000","last_became_active":"2024-04-23T18:24:36.702309+0000","last_became_peered":"2024-04-23T18:24:36.702309+0000","last_unstale":"2024-04-23T18:25:05.370004+0000","last_undegraded":"2024-04-23T18:25:05.370004+0000","last_fullsized":"2024-04-23T18:25:05.370004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:10.803924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.694625+0000","last_change":"2024-04-23T18:23:55.258636+0000","last_active":"2024-04-23T18:24:36.694625+0000","last_peered":"2024-04-23T18:24:36.694625+0000","last_clean":"2024-04-23T18:24:36.694625+0000","last_became_active":"2024-04-23T18:23:55.257744+0000","last_became_peered":"2024-04-23T18:23:55.257744+0000","last_unstale":"2024-04-23T18:24:36.694625+0000","last_undegraded":"2024-04-23T18:24:36.694625+0000","last_fullsized":"2024-04-23T18:24:36.694625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:46:07.454725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.391257+0000","last_change":"2024-04-23T18:23:56.094822+0000","last_active":"2024-04-23T18:25:05.391257+0000","last_peered":"2024-04-23T18:25:05.391257+0000","last_clean":"2024-04-23T18:25:05.391257+0000","last_became_active":"2024-04-23T18:23:56.094489+0000","last_became_peered":"2024-04-23T18:23:56.094489+0000","last_unstale":"2024-04-23T18:25:05.391257+0000","last_undegraded":"2024-04-23T18:25:05.391257+0000","last_fullsized":"2024-04-23T18:25:05.391257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:21.234363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.698040+0000","last_change":"2024-04-23T18:23:55.261322+0000","last_active":"2024-04-23T18:24:36.698040+0000","last_peered":"2024-04-23T18:24:36.698040+0000","last_clean":"2024-04-23T18:24:36.698040+0000","last_became_active":"2024-04-23T18:23:55.260990+0000","last_became_peered":"2024-04-23T18:23:55.260990+0000","last_unstale":"2024-04-23T18:24:36.698040+0000","last_undegraded":"2024-04-23T18:24:36.698040+0000","last_fullsized":"2024-04-23T18:24:36.698040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:31.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.391887+0000","last_change":"2024-04-23T18:23:56.093105+0000","last_active":"2024-04-23T18:25:05.391887+0000","last_peered":"2024-04-23T18:25:05.391887+0000","last_clean":"2024-04-23T18:25:05.391887+0000","last_became_active":"2024-04-23T18:23:56.092717+0000","last_became_peered":"2024-04-23T18:23:56.092717+0000","last_unstale":"2024-04-23T18:25:05.391887+0000","last_undegraded":"2024-04-23T18:25:05.391887+0000","last_fullsized":"2024-04-23T18:25:05.391887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:13:59.574057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":178,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:18.740805+0000","last_change":"2024-04-23T18:23:55.256307+0000","last_active":"2024-04-23T18:25:18.740805+0000","last_peered":"2024-04-23T18:25:18.740805+0000","last_clean":"2024-04-23T18:25:18.740805+0000","last_became_active":"2024-04-23T18:23:55.255959+0000","last_became_peered":"2024-04-23T18:23:55.255959+0000","last_unstale":"2024-04-23T18:25:18.740805+0000","last_undegraded":"2024-04-23T18:25:18.740805+0000","last_fullsized":"2024-04-23T18:25:18.740805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:04.945919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.396339+0000","last_change":"2024-04-23T18:23:56.095177+0000","last_active":"2024-04-23T18:25:05.396339+0000","last_peered":"2024-04-23T18:25:05.396339+0000","last_clean":"2024-04-23T18:25:05.396339+0000","last_became_active":"2024-04-23T18:23:56.094825+0000","last_became_peered":"2024-04-23T18:23:56.094825+0000","last_unstale":"2024-04-23T18:25:05.396339+0000","last_undegraded":"2024-04-23T18:25:05.396339+0000","last_fullsized":"2024-04-23T18:25:05.396339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:07:33.499817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696314+0000","last_change":"2024-04-23T18:23:55.264258+0000","last_active":"2024-04-23T18:24:36.696314+0000","last_peered":"2024-04-23T18:24:36.696314+0000","last_clean":"2024-04-23T18:24:36.696314+0000","last_became_active":"2024-04-23T18:23:55.263698+0000","last_became_peered":"2024-04-23T18:23:55.263698+0000","last_unstale":"2024-04-23T18:24:36.696314+0000","last_undegraded":"2024-04-23T18:24:36.696314+0000","last_fullsized":"2024-04-23T18:24:36.696314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:00.566948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.392441+0000","last_change":"2024-04-23T18:23:56.109085+0000","last_active":"2024-04-23T18:25:05.392441+0000","last_peered":"2024-04-23T18:25:05.392441+0000","last_clean":"2024-04-23T18:25:05.392441+0000","last_became_active":"2024-04-23T18:23:56.108033+0000","last_became_peered":"2024-04-23T18:23:56.108033+0000","last_unstale":"2024-04-23T18:25:05.392441+0000","last_undegraded":"2024-04-23T18:25:05.392441+0000","last_fullsized":"2024-04-23T18:25:05.392441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:59.980512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.700014+0000","last_change":"2024-04-23T18:23:55.256728+0000","last_active":"2024-04-23T18:24:36.700014+0000","last_peered":"2024-04-23T18:24:36.700014+0000","last_clean":"2024-04-23T18:24:36.700014+0000","last_became_active":"2024-04-23T18:23:55.256367+0000","last_became_peered":"2024-04-23T18:23:55.256367+0000","last_unstale":"2024-04-23T18:24:36.700014+0000","last_undegraded":"2024-04-23T18:24:36.700014+0000","last_fullsized":"2024-04-23T18:24:36.700014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:08.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.395768+0000","last_change":"2024-04-23T18:23:56.102898+0000","last_active":"2024-04-23T18:25:05.395768+0000","last_peered":"2024-04-23T18:25:05.395768+0000","last_clean":"2024-04-23T18:25:05.395768+0000","last_became_active":"2024-04-23T18:23:56.102373+0000","last_became_peered":"2024-04-23T18:23:56.102373+0000","last_unstale":"2024-04-23T18:25:05.395768+0000","last_undegraded":"2024-04-23T18:25:05.395768+0000","last_fullsized":"2024-04-23T18:25:05.395768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:49.316219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":214,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:18.740849+0000","last_change":"2024-04-23T18:23:55.255094+0000","last_active":"2024-04-23T18:25:18.740849+0000","last_peered":"2024-04-23T18:25:18.740849+0000","last_clean":"2024-04-23T18:25:18.740849+0000","last_became_active":"2024-04-23T18:23:55.254726+0000","last_became_peered":"2024-04-23T18:23:55.254726+0000","last_unstale":"2024-04-23T18:25:18.740849+0000","last_undegraded":"2024-04-23T18:25:18.740849+0000","last_fullsized":"2024-04-23T18:25:18.740849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:26.988184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":142,"num_write":35,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.398066+0000","last_change":"2024-04-23T18:23:56.099866+0000","last_active":"2024-04-23T18:25:05.398066+0000","last_peered":"2024-04-23T18:25:05.398066+0000","last_clean":"2024-04-23T18:25:05.398066+0000","last_became_active":"2024-04-23T18:23:56.098606+0000","last_became_peered":"2024-04-23T18:23:56.098606+0000","last_unstale":"2024-04-23T18:25:05.398066+0000","last_undegraded":"2024-04-23T18:25:05.398066+0000","last_fullsized":"2024-04-23T18:25:05.398066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:26.661857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.711203+0000","last_change":"2024-04-23T18:24:36.710791+0000","last_active":"2024-04-23T18:24:36.711203+0000","last_peered":"2024-04-23T18:24:36.711203+0000","last_clean":"2024-04-23T18:24:36.711203+0000","last_became_active":"2024-04-23T18:24:36.697731+0000","last_became_peered":"2024-04-23T18:24:36.697731+0000","last_unstale":"2024-04-23T18:24:36.711203+0000","last_undegraded":"2024-04-23T18:24:36.711203+0000","last_fullsized":"2024-04-23T18:24:36.711203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:16:46.722132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.366361+0000","last_change":"2024-04-23T18:23:56.099109+0000","last_active":"2024-04-23T18:25:05.366361+0000","last_peered":"2024-04-23T18:25:05.366361+0000","last_clean":"2024-04-23T18:25:05.366361+0000","last_became_active":"2024-04-23T18:23:56.098608+0000","last_became_peered":"2024-04-23T18:23:56.098608+0000","last_unstale":"2024-04-23T18:25:05.366361+0000","last_undegraded":"2024-04-23T18:25:05.366361+0000","last_fullsized":"2024-04-23T18:25:05.366361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:43.720160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.699083+0000","last_change":"2024-04-23T18:23:55.262523+0000","last_active":"2024-04-23T18:24:36.699083+0000","last_peered":"2024-04-23T18:24:36.699083+0000","last_clean":"2024-04-23T18:24:36.699083+0000","last_became_active":"2024-04-23T18:23:55.262216+0000","last_became_peered":"2024-04-23T18:23:55.262216+0000","last_unstale":"2024-04-23T18:24:36.699083+0000","last_undegraded":"2024-04-23T18:24:36.699083+0000","last_fullsized":"2024-04-23T18:24:36.699083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:35.268435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.399842+0000","last_change":"2024-04-23T18:23:56.106573+0000","last_active":"2024-04-23T18:25:05.399842+0000","last_peered":"2024-04-23T18:25:05.399842+0000","last_clean":"2024-04-23T18:25:05.399842+0000","last_became_active":"2024-04-23T18:23:56.106181+0000","last_became_peered":"2024-04-23T18:23:56.106181+0000","last_unstale":"2024-04-23T18:25:05.399842+0000","last_undegraded":"2024-04-23T18:25:05.399842+0000","last_fullsized":"2024-04-23T18:25:05.399842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:15.422647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:16.738644+0000","last_change":"2024-04-23T18:23:55.262813+0000","last_active":"2024-04-23T18:25:16.738644+0000","last_peered":"2024-04-23T18:25:16.738644+0000","last_clean":"2024-04-23T18:25:16.738644+0000","last_became_active":"2024-04-23T18:23:55.262365+0000","last_became_peered":"2024-04-23T18:23:55.262365+0000","last_unstale":"2024-04-23T18:25:16.738644+0000","last_undegraded":"2024-04-23T18:25:16.738644+0000","last_fullsized":"2024-04-23T18:25:16.738644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:15.234355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.387828+0000","last_change":"2024-04-23T18:23:56.101491+0000","last_active":"2024-04-23T18:25:05.387828+0000","last_peered":"2024-04-23T18:25:05.387828+0000","last_clean":"2024-04-23T18:25:05.387828+0000","last_became_active":"2024-04-23T18:23:56.101069+0000","last_became_peered":"2024-04-23T18:23:56.101069+0000","last_unstale":"2024-04-23T18:25:05.387828+0000","last_undegraded":"2024-04-23T18:25:05.387828+0000","last_fullsized":"2024-04-23T18:25:05.387828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:42.484653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.750105+0000","last_change":"2024-04-23T18:24:36.750105+0000","last_active":"2024-04-23T18:24:36.750105+0000","last_peered":"2024-04-23T18:24:36.750105+0000","last_clean":"2024-04-23T18:24:36.750105+0000","last_became_active":"2024-04-23T18:24:36.738326+0000","last_became_peered":"2024-04-23T18:24:36.738326+0000","last_unstale":"2024-04-23T18:24:36.750105+0000","last_undegraded":"2024-04-23T18:24:36.750105+0000","last_fullsized":"2024-04-23T18:24:36.750105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:07.414513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.379138+0000","last_change":"2024-04-23T18:23:56.105691+0000","last_active":"2024-04-23T18:25:05.379138+0000","last_peered":"2024-04-23T18:25:05.379138+0000","last_clean":"2024-04-23T18:25:05.379138+0000","last_became_active":"2024-04-23T18:23:56.105205+0000","last_became_peered":"2024-04-23T18:23:56.105205+0000","last_unstale":"2024-04-23T18:25:05.379138+0000","last_undegraded":"2024-04-23T18:25:05.379138+0000","last_fullsized":"2024-04-23T18:25:05.379138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:30.779821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696501+0000","last_change":"2024-04-23T18:23:55.261198+0000","last_active":"2024-04-23T18:24:36.696501+0000","last_peered":"2024-04-23T18:24:36.696501+0000","last_clean":"2024-04-23T18:24:36.696501+0000","last_became_active":"2024-04-23T18:23:55.260884+0000","last_became_peered":"2024-04-23T18:23:55.260884+0000","last_unstale":"2024-04-23T18:24:36.696501+0000","last_undegraded":"2024-04-23T18:24:36.696501+0000","last_fullsized":"2024-04-23T18:24:36.696501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:00.841671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.384470+0000","last_change":"2024-04-23T18:23:56.099450+0000","last_active":"2024-04-23T18:25:05.384470+0000","last_peered":"2024-04-23T18:25:05.384470+0000","last_clean":"2024-04-23T18:25:05.384470+0000","last_became_active":"2024-04-23T18:23:56.098723+0000","last_became_peered":"2024-04-23T18:23:56.098723+0000","last_unstale":"2024-04-23T18:25:05.384470+0000","last_undegraded":"2024-04-23T18:25:05.384470+0000","last_fullsized":"2024-04-23T18:25:05.384470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:53.027495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.692241+0000","last_change":"2024-04-23T18:23:55.262030+0000","last_active":"2024-04-23T18:24:36.692241+0000","last_peered":"2024-04-23T18:24:36.692241+0000","last_clean":"2024-04-23T18:24:36.692241+0000","last_became_active":"2024-04-23T18:23:55.261659+0000","last_became_peered":"2024-04-23T18:23:55.261659+0000","last_unstale":"2024-04-23T18:24:36.692241+0000","last_undegraded":"2024-04-23T18:24:36.692241+0000","last_fullsized":"2024-04-23T18:24:36.692241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:17.036709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.389538+0000","last_change":"2024-04-23T18:23:56.096511+0000","last_active":"2024-04-23T18:25:05.389538+0000","last_peered":"2024-04-23T18:25:05.389538+0000","last_clean":"2024-04-23T18:25:05.389538+0000","last_became_active":"2024-04-23T18:23:56.096154+0000","last_became_peered":"2024-04-23T18:23:56.096154+0000","last_unstale":"2024-04-23T18:25:05.389538+0000","last_undegraded":"2024-04-23T18:25:05.389538+0000","last_fullsized":"2024-04-23T18:25:05.389538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:03.926178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:20.743257+0000","last_change":"2024-04-23T18:23:55.263618+0000","last_active":"2024-04-23T18:25:20.743257+0000","last_peered":"2024-04-23T18:25:20.743257+0000","last_clean":"2024-04-23T18:25:20.743257+0000","last_became_active":"2024-04-23T18:23:55.261758+0000","last_became_peered":"2024-04-23T18:23:55.261758+0000","last_unstale":"2024-04-23T18:25:20.743257+0000","last_undegraded":"2024-04-23T18:25:20.743257+0000","last_fullsized":"2024-04-23T18:25:20.743257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:23:27.745668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.363984+0000","last_change":"2024-04-23T18:23:56.109292+0000","last_active":"2024-04-23T18:25:05.363984+0000","last_peered":"2024-04-23T18:25:05.363984+0000","last_clean":"2024-04-23T18:25:05.363984+0000","last_became_active":"2024-04-23T18:23:56.108414+0000","last_became_peered":"2024-04-23T18:23:56.108414+0000","last_unstale":"2024-04-23T18:25:05.363984+0000","last_undegraded":"2024-04-23T18:25:05.363984+0000","last_fullsized":"2024-04-23T18:25:05.363984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:13.428782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.378638+0000","last_change":"2024-04-23T18:23:56.097122+0000","last_active":"2024-04-23T18:25:05.378638+0000","last_peered":"2024-04-23T18:25:05.378638+0000","last_clean":"2024-04-23T18:25:05.378638+0000","last_became_active":"2024-04-23T18:23:56.096751+0000","last_became_peered":"2024-04-23T18:23:56.096751+0000","last_unstale":"2024-04-23T18:25:05.378638+0000","last_undegraded":"2024-04-23T18:25:05.378638+0000","last_fullsized":"2024-04-23T18:25:05.378638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:00:55.090370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:19.742315+0000","last_change":"2024-04-23T18:23:55.256458+0000","last_active":"2024-04-23T18:25:19.742315+0000","last_peered":"2024-04-23T18:25:19.742315+0000","last_clean":"2024-04-23T18:25:19.742315+0000","last_became_active":"2024-04-23T18:23:55.256127+0000","last_became_peered":"2024-04-23T18:23:55.256127+0000","last_unstale":"2024-04-23T18:25:19.742315+0000","last_undegraded":"2024-04-23T18:25:19.742315+0000","last_fullsized":"2024-04-23T18:25:19.742315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:03.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.394792+0000","last_change":"2024-04-23T18:23:56.103743+0000","last_active":"2024-04-23T18:25:05.394792+0000","last_peered":"2024-04-23T18:25:05.394792+0000","last_clean":"2024-04-23T18:25:05.394792+0000","last_became_active":"2024-04-23T18:23:56.103041+0000","last_became_peered":"2024-04-23T18:23:56.103041+0000","last_unstale":"2024-04-23T18:25:05.394792+0000","last_undegraded":"2024-04-23T18:25:05.394792+0000","last_fullsized":"2024-04-23T18:25:05.394792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:22.419555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.736937+0000","last_change":"2024-04-23T18:23:55.257008+0000","last_active":"2024-04-23T18:24:36.736937+0000","last_peered":"2024-04-23T18:24:36.736937+0000","last_clean":"2024-04-23T18:24:36.736937+0000","last_became_active":"2024-04-23T18:23:55.256648+0000","last_became_peered":"2024-04-23T18:23:55.256648+0000","last_unstale":"2024-04-23T18:24:36.736937+0000","last_undegraded":"2024-04-23T18:24:36.736937+0000","last_fullsized":"2024-04-23T18:24:36.736937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:00.867083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.695974+0000","last_change":"2024-04-23T18:22:38.610948+0000","last_active":"2024-04-23T18:24:36.695974+0000","last_peered":"2024-04-23T18:24:36.695974+0000","last_clean":"2024-04-23T18:24:36.695974+0000","last_became_active":"2024-04-23T18:22:38.610475+0000","last_became_peered":"2024-04-23T18:22:38.610475+0000","last_unstale":"2024-04-23T18:24:36.695974+0000","last_undegraded":"2024-04-23T18:24:36.695974+0000","last_fullsized":"2024-04-23T18:24:36.695974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_clean_scrub_stamp":"2024-04-23T18:19:07.980519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:08.209087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.379992+0000","last_change":"2024-04-23T18:24:36.699127+0000","last_active":"2024-04-23T18:25:05.379992+0000","last_peered":"2024-04-23T18:25:05.379992+0000","last_clean":"2024-04-23T18:25:05.379992+0000","last_became_active":"2024-04-23T18:24:36.698734+0000","last_became_peered":"2024-04-23T18:24:36.698734+0000","last_unstale":"2024-04-23T18:25:05.379992+0000","last_undegraded":"2024-04-23T18:25:05.379992+0000","last_fullsized":"2024-04-23T18:25:05.379992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:28.239466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.918722+0000","last_change":"2024-04-23T18:24:36.918722+0000","last_active":"2024-04-23T18:24:36.918722+0000","last_peered":"2024-04-23T18:24:36.918722+0000","last_clean":"2024-04-23T18:24:36.918722+0000","last_became_active":"2024-04-23T18:24:36.915658+0000","last_became_peered":"2024-04-23T18:24:36.915658+0000","last_unstale":"2024-04-23T18:24:36.918722+0000","last_undegraded":"2024-04-23T18:24:36.918722+0000","last_fullsized":"2024-04-23T18:24:36.918722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:34.706193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.697313+0000","last_change":"2024-04-23T18:23:55.265853+0000","last_active":"2024-04-23T18:24:36.697313+0000","last_peered":"2024-04-23T18:24:36.697313+0000","last_clean":"2024-04-23T18:24:36.697313+0000","last_became_active":"2024-04-23T18:23:55.265435+0000","last_became_peered":"2024-04-23T18:23:55.265435+0000","last_unstale":"2024-04-23T18:24:36.697313+0000","last_undegraded":"2024-04-23T18:24:36.697313+0000","last_fullsized":"2024-04-23T18:24:36.697313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.969720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.380863+0000","last_change":"2024-04-23T18:23:56.085609+0000","last_active":"2024-04-23T18:25:05.380863+0000","last_peered":"2024-04-23T18:25:05.380863+0000","last_clean":"2024-04-23T18:25:05.380863+0000","last_became_active":"2024-04-23T18:23:56.085244+0000","last_became_peered":"2024-04-23T18:23:56.085244+0000","last_unstale":"2024-04-23T18:25:05.380863+0000","last_undegraded":"2024-04-23T18:25:05.380863+0000","last_fullsized":"2024-04-23T18:25:05.380863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.211717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'13","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.692390+0000","last_change":"2024-04-23T18:23:55.260260+0000","last_active":"2024-04-23T18:24:36.692390+0000","last_peered":"2024-04-23T18:24:36.692390+0000","last_clean":"2024-04-23T18:24:36.692390+0000","last_became_active":"2024-04-23T18:23:55.259862+0000","last_became_peered":"2024-04-23T18:23:55.259862+0000","last_unstale":"2024-04-23T18:24:36.692390+0000","last_undegraded":"2024-04-23T18:24:36.692390+0000","last_fullsized":"2024-04-23T18:24:36.692390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:57.833293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.391043+0000","last_change":"2024-04-23T18:23:56.098826+0000","last_active":"2024-04-23T18:25:05.391043+0000","last_peered":"2024-04-23T18:25:05.391043+0000","last_clean":"2024-04-23T18:25:05.391043+0000","last_became_active":"2024-04-23T18:23:56.098494+0000","last_became_peered":"2024-04-23T18:23:56.098494+0000","last_unstale":"2024-04-23T18:25:05.391043+0000","last_undegraded":"2024-04-23T18:25:05.391043+0000","last_fullsized":"2024-04-23T18:25:05.391043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:12:24.953330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.737134+0000","last_change":"2024-04-23T18:23:55.258174+0000","last_active":"2024-04-23T18:24:36.737134+0000","last_peered":"2024-04-23T18:24:36.737134+0000","last_clean":"2024-04-23T18:24:36.737134+0000","last_became_active":"2024-04-23T18:23:55.257846+0000","last_became_peered":"2024-04-23T18:23:55.257846+0000","last_unstale":"2024-04-23T18:24:36.737134+0000","last_undegraded":"2024-04-23T18:24:36.737134+0000","last_fullsized":"2024-04-23T18:24:36.737134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:35.131566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.395955+0000","last_change":"2024-04-23T18:23:56.110758+0000","last_active":"2024-04-23T18:25:05.395955+0000","last_peered":"2024-04-23T18:25:05.395955+0000","last_clean":"2024-04-23T18:25:05.395955+0000","last_became_active":"2024-04-23T18:23:56.110372+0000","last_became_peered":"2024-04-23T18:23:56.110372+0000","last_unstale":"2024-04-23T18:25:05.395955+0000","last_undegraded":"2024-04-23T18:25:05.395955+0000","last_fullsized":"2024-04-23T18:25:05.395955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:40.193088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.914632+0000","last_change":"2024-04-23T18:23:55.265714+0000","last_active":"2024-04-23T18:24:36.914632+0000","last_peered":"2024-04-23T18:24:36.914632+0000","last_clean":"2024-04-23T18:24:36.914632+0000","last_became_active":"2024-04-23T18:23:55.265198+0000","last_became_peered":"2024-04-23T18:23:55.265198+0000","last_unstale":"2024-04-23T18:24:36.914632+0000","last_undegraded":"2024-04-23T18:24:36.914632+0000","last_fullsized":"2024-04-23T18:24:36.914632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:10.770748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.342678+0000","last_change":"2024-04-23T18:23:56.102347+0000","last_active":"2024-04-23T18:25:05.342678+0000","last_peered":"2024-04-23T18:25:05.342678+0000","last_clean":"2024-04-23T18:25:05.342678+0000","last_became_active":"2024-04-23T18:23:56.100687+0000","last_became_peered":"2024-04-23T18:23:56.100687+0000","last_unstale":"2024-04-23T18:25:05.342678+0000","last_undegraded":"2024-04-23T18:25:05.342678+0000","last_fullsized":"2024-04-23T18:25:05.342678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:20.378730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696205+0000","last_change":"2024-04-23T18:23:55.255943+0000","last_active":"2024-04-23T18:24:36.696205+0000","last_peered":"2024-04-23T18:24:36.696205+0000","last_clean":"2024-04-23T18:24:36.696205+0000","last_became_active":"2024-04-23T18:23:55.255618+0000","last_became_peered":"2024-04-23T18:23:55.255618+0000","last_unstale":"2024-04-23T18:24:36.696205+0000","last_undegraded":"2024-04-23T18:24:36.696205+0000","last_fullsized":"2024-04-23T18:24:36.696205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:01.408533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.388079+0000","last_change":"2024-04-23T18:23:56.091664+0000","last_active":"2024-04-23T18:25:05.388079+0000","last_peered":"2024-04-23T18:25:05.388079+0000","last_clean":"2024-04-23T18:25:05.388079+0000","last_became_active":"2024-04-23T18:23:56.091308+0000","last_became_peered":"2024-04-23T18:23:56.091308+0000","last_unstale":"2024-04-23T18:25:05.388079+0000","last_undegraded":"2024-04-23T18:25:05.388079+0000","last_fullsized":"2024-04-23T18:25:05.388079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:31.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.696556+0000","last_change":"2024-04-23T18:23:55.260992+0000","last_active":"2024-04-23T18:24:36.696556+0000","last_peered":"2024-04-23T18:24:36.696556+0000","last_clean":"2024-04-23T18:24:36.696556+0000","last_became_active":"2024-04-23T18:23:55.260679+0000","last_became_peered":"2024-04-23T18:23:55.260679+0000","last_unstale":"2024-04-23T18:24:36.696556+0000","last_undegraded":"2024-04-23T18:24:36.696556+0000","last_fullsized":"2024-04-23T18:24:36.696556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:02:50.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.393515+0000","last_change":"2024-04-23T18:23:56.097956+0000","last_active":"2024-04-23T18:25:05.393515+0000","last_peered":"2024-04-23T18:25:05.393515+0000","last_clean":"2024-04-23T18:25:05.393515+0000","last_became_active":"2024-04-23T18:23:56.097231+0000","last_became_peered":"2024-04-23T18:23:56.097231+0000","last_unstale":"2024-04-23T18:25:05.393515+0000","last_undegraded":"2024-04-23T18:25:05.393515+0000","last_fullsized":"2024-04-23T18:25:05.393515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:10.945857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.714667+0000","last_change":"2024-04-23T18:24:36.714223+0000","last_active":"2024-04-23T18:24:36.714667+0000","last_peered":"2024-04-23T18:24:36.714667+0000","last_clean":"2024-04-23T18:24:36.714667+0000","last_became_active":"2024-04-23T18:24:36.700975+0000","last_became_peered":"2024-04-23T18:24:36.700975+0000","last_unstale":"2024-04-23T18:24:36.714667+0000","last_undegraded":"2024-04-23T18:24:36.714667+0000","last_fullsized":"2024-04-23T18:24:36.714667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:58:50.227699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.397687+0000","last_change":"2024-04-23T18:23:56.096889+0000","last_active":"2024-04-23T18:25:05.397687+0000","last_peered":"2024-04-23T18:25:05.397687+0000","last_clean":"2024-04-23T18:25:05.397687+0000","last_became_active":"2024-04-23T18:23:56.096520+0000","last_became_peered":"2024-04-23T18:23:56.096520+0000","last_unstale":"2024-04-23T18:25:05.397687+0000","last_undegraded":"2024-04-23T18:25:05.397687+0000","last_fullsized":"2024-04-23T18:25:05.397687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:34:53.785481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":106,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:20.743975+0000","last_change":"2024-04-23T18:23:55.258710+0000","last_active":"2024-04-23T18:25:20.743975+0000","last_peered":"2024-04-23T18:25:20.743975+0000","last_clean":"2024-04-23T18:25:20.743975+0000","last_became_active":"2024-04-23T18:23:55.256247+0000","last_became_peered":"2024-04-23T18:23:55.256247+0000","last_unstale":"2024-04-23T18:25:20.743975+0000","last_undegraded":"2024-04-23T18:25:20.743975+0000","last_fullsized":"2024-04-23T18:25:20.743975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:20.551614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.397705+0000","last_change":"2024-04-23T18:23:56.101862+0000","last_active":"2024-04-23T18:25:05.397705+0000","last_peered":"2024-04-23T18:25:05.397705+0000","last_clean":"2024-04-23T18:25:05.397705+0000","last_became_active":"2024-04-23T18:23:56.101458+0000","last_became_peered":"2024-04-23T18:23:56.101458+0000","last_unstale":"2024-04-23T18:25:05.397705+0000","last_undegraded":"2024-04-23T18:25:05.397705+0000","last_fullsized":"2024-04-23T18:25:05.397705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:59.292077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.699731+0000","last_change":"2024-04-23T18:23:55.258292+0000","last_active":"2024-04-23T18:24:36.699731+0000","last_peered":"2024-04-23T18:24:36.699731+0000","last_clean":"2024-04-23T18:24:36.699731+0000","last_became_active":"2024-04-23T18:23:55.257954+0000","last_became_peered":"2024-04-23T18:23:55.257954+0000","last_unstale":"2024-04-23T18:24:36.699731+0000","last_undegraded":"2024-04-23T18:24:36.699731+0000","last_fullsized":"2024-04-23T18:24:36.699731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:39.105967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.387905+0000","last_change":"2024-04-23T18:23:56.105977+0000","last_active":"2024-04-23T18:25:05.387905+0000","last_peered":"2024-04-23T18:25:05.387905+0000","last_clean":"2024-04-23T18:25:05.387905+0000","last_became_active":"2024-04-23T18:23:56.105582+0000","last_became_peered":"2024-04-23T18:23:56.105582+0000","last_unstale":"2024-04-23T18:25:05.387905+0000","last_undegraded":"2024-04-23T18:25:05.387905+0000","last_fullsized":"2024-04-23T18:25:05.387905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:28:10.700684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:24:36.694767+0000","last_change":"2024-04-23T18:23:55.252095+0000","last_active":"2024-04-23T18:24:36.694767+0000","last_peered":"2024-04-23T18:24:36.694767+0000","last_clean":"2024-04-23T18:24:36.694767+0000","last_became_active":"2024-04-23T18:23:55.251702+0000","last_became_peered":"2024-04-23T18:23:55.251702+0000","last_unstale":"2024-04-23T18:24:36.694767+0000","last_undegraded":"2024-04-23T18:24:36.694767+0000","last_fullsized":"2024-04-23T18:24:36.694767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:21.625905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.398619+0000","last_change":"2024-04-23T18:23:56.101997+0000","last_active":"2024-04-23T18:25:05.398619+0000","last_peered":"2024-04-23T18:25:05.398619+0000","last_clean":"2024-04-23T18:25:05.398619+0000","last_became_active":"2024-04-23T18:23:56.101629+0000","last_became_peered":"2024-04-23T18:23:56.101629+0000","last_unstale":"2024-04-23T18:25:05.398619+0000","last_undegraded":"2024-04-23T18:25:05.398619+0000","last_fullsized":"2024-04-23T18:25:05.398619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:42.484413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.388451+0000","last_change":"2024-04-23T18:23:56.090543+0000","last_active":"2024-04-23T18:25:05.388451+0000","last_peered":"2024-04-23T18:25:05.388451+0000","last_clean":"2024-04-23T18:25:05.388451+0000","last_became_active":"2024-04-23T18:23:56.090191+0000","last_became_peered":"2024-04-23T18:23:56.090191+0000","last_unstale":"2024-04-23T18:25:05.388451+0000","last_undegraded":"2024-04-23T18:25:05.388451+0000","last_fullsized":"2024-04-23T18:25:05.388451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:01.401822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.379225+0000","last_change":"2024-04-23T18:23:56.105537+0000","last_active":"2024-04-23T18:25:05.379225+0000","last_peered":"2024-04-23T18:25:05.379225+0000","last_clean":"2024-04-23T18:25:05.379225+0000","last_became_active":"2024-04-23T18:23:56.105130+0000","last_became_peered":"2024-04-23T18:23:56.105130+0000","last_unstale":"2024-04-23T18:25:05.379225+0000","last_undegraded":"2024-04-23T18:25:05.379225+0000","last_fullsized":"2024-04-23T18:25:05.379225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:59.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.377655+0000","last_change":"2024-04-23T18:23:56.097765+0000","last_active":"2024-04-23T18:25:05.377655+0000","last_peered":"2024-04-23T18:25:05.377655+0000","last_clean":"2024-04-23T18:25:05.377655+0000","last_became_active":"2024-04-23T18:23:56.097439+0000","last_became_peered":"2024-04-23T18:23:56.097439+0000","last_unstale":"2024-04-23T18:25:05.377655+0000","last_undegraded":"2024-04-23T18:25:05.377655+0000","last_fullsized":"2024-04-23T18:25:05.377655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:31.529751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.395414+0000","last_change":"2024-04-23T18:23:56.108548+0000","last_active":"2024-04-23T18:25:05.395414+0000","last_peered":"2024-04-23T18:25:05.395414+0000","last_clean":"2024-04-23T18:25:05.395414+0000","last_became_active":"2024-04-23T18:23:56.108214+0000","last_became_peered":"2024-04-23T18:23:56.108214+0000","last_unstale":"2024-04-23T18:25:05.395414+0000","last_undegraded":"2024-04-23T18:25:05.395414+0000","last_fullsized":"2024-04-23T18:25:05.395414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:41.738754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.380937+0000","last_change":"2024-04-23T18:24:36.701809+0000","last_active":"2024-04-23T18:25:05.380937+0000","last_peered":"2024-04-23T18:25:05.380937+0000","last_clean":"2024-04-23T18:25:05.380937+0000","last_became_active":"2024-04-23T18:24:36.701405+0000","last_became_peered":"2024-04-23T18:24:36.701405+0000","last_unstale":"2024-04-23T18:25:05.380937+0000","last_undegraded":"2024-04-23T18:25:05.380937+0000","last_fullsized":"2024-04-23T18:25:05.380937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:03.215622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.385607+0000","last_change":"2024-04-23T18:23:56.093969+0000","last_active":"2024-04-23T18:25:05.385607+0000","last_peered":"2024-04-23T18:25:05.385607+0000","last_clean":"2024-04-23T18:25:05.385607+0000","last_became_active":"2024-04-23T18:23:56.093568+0000","last_became_peered":"2024-04-23T18:23:56.093568+0000","last_unstale":"2024-04-23T18:25:05.385607+0000","last_undegraded":"2024-04-23T18:25:05.385607+0000","last_fullsized":"2024-04-23T18:25:05.385607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:08.361305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.392473+0000","last_change":"2024-04-23T18:23:56.096106+0000","last_active":"2024-04-23T18:25:05.392473+0000","last_peered":"2024-04-23T18:25:05.392473+0000","last_clean":"2024-04-23T18:25:05.392473+0000","last_became_active":"2024-04-23T18:23:56.095745+0000","last_became_peered":"2024-04-23T18:23:56.095745+0000","last_unstale":"2024-04-23T18:25:05.392473+0000","last_undegraded":"2024-04-23T18:25:05.392473+0000","last_fullsized":"2024-04-23T18:25:05.392473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:01.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.388934+0000","last_change":"2024-04-23T18:23:56.095474+0000","last_active":"2024-04-23T18:25:05.388934+0000","last_peered":"2024-04-23T18:25:05.388934+0000","last_clean":"2024-04-23T18:25:05.388934+0000","last_became_active":"2024-04-23T18:23:56.095116+0000","last_became_peered":"2024-04-23T18:23:56.095116+0000","last_unstale":"2024-04-23T18:25:05.388934+0000","last_undegraded":"2024-04-23T18:25:05.388934+0000","last_fullsized":"2024-04-23T18:25:05.388934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:50:00.339558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.384860+0000","last_change":"2024-04-23T18:23:56.096022+0000","last_active":"2024-04-23T18:25:05.384860+0000","last_peered":"2024-04-23T18:25:05.384860+0000","last_clean":"2024-04-23T18:25:05.384860+0000","last_became_active":"2024-04-23T18:23:56.095493+0000","last_became_peered":"2024-04-23T18:23:56.095493+0000","last_unstale":"2024-04-23T18:25:05.384860+0000","last_undegraded":"2024-04-23T18:25:05.384860+0000","last_fullsized":"2024-04-23T18:25:05.384860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:42.209072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.390148+0000","last_change":"2024-04-23T18:23:56.101697+0000","last_active":"2024-04-23T18:25:05.390148+0000","last_peered":"2024-04-23T18:25:05.390148+0000","last_clean":"2024-04-23T18:25:05.390148+0000","last_became_active":"2024-04-23T18:23:56.098708+0000","last_became_peered":"2024-04-23T18:23:56.098708+0000","last_unstale":"2024-04-23T18:25:05.390148+0000","last_undegraded":"2024-04-23T18:25:05.390148+0000","last_fullsized":"2024-04-23T18:25:05.390148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:14.170700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.399544+0000","last_change":"2024-04-23T18:23:56.096982+0000","last_active":"2024-04-23T18:25:05.399544+0000","last_peered":"2024-04-23T18:25:05.399544+0000","last_clean":"2024-04-23T18:25:05.399544+0000","last_became_active":"2024-04-23T18:23:56.096608+0000","last_became_peered":"2024-04-23T18:23:56.096608+0000","last_unstale":"2024-04-23T18:25:05.399544+0000","last_undegraded":"2024-04-23T18:25:05.399544+0000","last_fullsized":"2024-04-23T18:25:05.399544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:50.329337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.381320+0000","last_change":"2024-04-23T18:23:56.100345+0000","last_active":"2024-04-23T18:25:05.381320+0000","last_peered":"2024-04-23T18:25:05.381320+0000","last_clean":"2024-04-23T18:25:05.381320+0000","last_became_active":"2024-04-23T18:23:56.100043+0000","last_became_peered":"2024-04-23T18:23:56.100043+0000","last_unstale":"2024-04-23T18:25:05.381320+0000","last_undegraded":"2024-04-23T18:25:05.381320+0000","last_fullsized":"2024-04-23T18:25:05.381320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:36.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:25:05.387198+0000","last_change":"2024-04-23T18:23:56.101947+0000","last_active":"2024-04-23T18:25:05.387198+0000","last_peered":"2024-04-23T18:25:05.387198+0000","last_clean":"2024-04-23T18:25:05.387198+0000","last_became_active":"2024-04-23T18:23:56.101627+0000","last_became_peered":"2024-04-23T18:23:56.101627+0000","last_unstale":"2024-04-23T18:25:05.387198+0000","last_undegraded":"2024-04-23T18:25:05.387198+0000","last_fullsized":"2024-04-23T18:25:05.387198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:21:43.703653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279760,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68781,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":815,"num_read_kb":482,"num_write":246,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":206343,"data_compressed":5595,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":461377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612644,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":1114165,"data_compressed":37450,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":461239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037807,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":461617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1054808,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":460956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":523164,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":462295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":523736,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":460781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":461461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1053954,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33945,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63047,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63619,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63563,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:25:25.210 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-23T18:25:25.210 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T18:25:25.210 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-23T18:25:25.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-23T18:25:25.464 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:25.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:25 smithi019 ceph-mon[29215]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:25.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:25 smithi044 ceph-mon[32458]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:25.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:25 smithi155 ceph-mon[33468]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:26.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:26 smithi019 ceph-mon[29215]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:26.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:26 smithi044 ceph-mon[32458]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:26.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:26 smithi155 ceph-mon[33468]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.312+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/1608326581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4102a90 msgr2=0x7f6cb4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.312+0000 7f6cb8f6c700 1 --2- 172.21.15.19:0/1608326581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4102a90 0x7f6cb4102f10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009990 tx=0x7f6ca8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.313+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/1608326581 shutdown_connections 2024-04-23T18:25:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.313+0000 7f6cb8f6c700 1 --2- 172.21.15.19:0/1608326581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4103450 0x7f6cb410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.313+0000 7f6cb8f6c700 1 --2- 172.21.15.19:0/1608326581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4108a50 0x7f6cb4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.313+0000 7f6cb8f6c700 1 --2- 172.21.15.19:0/1608326581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4102a90 0x7f6cb4102f10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.313+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/1608326581 >> 172.21.15.19:0/1608326581 conn(0x7f6cb4079ba0 msgr2=0x7f6cb4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.313+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/1608326581 shutdown_connections 2024-04-23T18:25:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.313+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/1608326581 wait complete. 2024-04-23T18:25:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.313+0000 7f6cb8f6c700 1 Processor -- start 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.313+0000 7f6cb8f6c700 1 -- start start 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb8f6c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4102a90 0x7f6cb4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb8f6c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4103450 0x7f6cb4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb8f6c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4108a50 0x7f6cb40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb8f6c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4112c00 con 0x7f6cb4103450 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb8f6c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4112a80 con 0x7f6cb4108a50 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb8f6c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4112d80 con 0x7f6cb4102a90 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4103450 0x7f6cb4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4103450 0x7f6cb4075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47074/0 (socket says 172.21.15.19:47074) 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb1d9b700 1 -- 172.21.15.19:0/2356350861 learned_addr learned my addr 172.21.15.19:0/2356350861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb2d9d700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4108a50 0x7f6cb40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb259c700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4102a90 0x7f6cb4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb1d9b700 1 -- 172.21.15.19:0/2356350861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4102a90 msgr2=0x7f6cb4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb1d9b700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4102a90 0x7f6cb4074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb1d9b700 1 -- 172.21.15.19:0/2356350861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4108a50 msgr2=0x7f6cb40757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb1d9b700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4108a50 0x7f6cb40757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb1d9b700 1 -- 172.21.15.19:0/2356350861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb41a7680 con 0x7f6cb4103450 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.314+0000 7f6cb1d9b700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4103450 0x7f6cb4075280 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8000c00 tx=0x7f6ca80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.315+0000 7f6ca37fe700 1 -- 172.21.15.19:0/2356350861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8027070 con 0x7f6cb4103450 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.315+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/2356350861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb41a7910 con 0x7f6cb4103450 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.315+0000 7f6ca37fe700 1 -- 172.21.15.19:0/2356350861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca8003f20 con 0x7f6cb4103450 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.315+0000 7f6ca37fe700 1 -- 172.21.15.19:0/2356350861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8022cf0 con 0x7f6cb4103450 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.315+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/2356350861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb41a7e40 con 0x7f6cb4103450 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.316+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/2356350861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb4069730 con 0x7f6cb4103450 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.317+0000 7f6ca37fe700 1 -- 172.21.15.19:0/2356350861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6ca802a070 con 0x7f6cb4103450 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.317+0000 7f6ca37fe700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6c9c0a3c40 0x7f6c9c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.318+0000 7f6cb259c700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6c9c0a3c40 0x7f6c9c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.318+0000 7f6ca37fe700 1 -- 172.21.15.19:0/2356350861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f6ca80d1540 con 0x7f6cb4103450 2024-04-23T18:25:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.318+0000 7f6cb259c700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6c9c0a3c40 0x7f6c9c0a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400bbb0 tx=0x7f6ca400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.321+0000 7f6ca37fe700 1 -- 172.21.15.19:0/2356350861 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ca8013070 con 0x7f6cb4103450 2024-04-23T18:25:27.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.575+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/2356350861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6cb402df80 con 0x7f6cb4103450 2024-04-23T18:25:27.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.576+0000 7f6ca37fe700 1 -- 172.21.15.19:0/2356350861 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6ca809dfc0 con 0x7f6cb4103450 2024-04-23T18:25:27.570 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:27.571 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T18:25:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/2356350861 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6c9c0a3c40 msgr2=0x7f6c9c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6c9c0a3c40 0x7f6c9c0a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400bbb0 tx=0x7f6ca400b590 comp rx=0 tx=0).stop 2024-04-23T18:25:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/2356350861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4103450 msgr2=0x7f6cb4075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4103450 0x7f6cb4075280 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8000c00 tx=0x7f6ca80047b0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/2356350861 shutdown_connections 2024-04-23T18:25:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f6c9c0a3c40 0x7f6c9c0a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4108a50 0x7f6cb40757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4102a90 0x7f6cb4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 --2- 172.21.15.19:0/2356350861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4103450 0x7f6cb4075280 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/2356350861 >> 172.21.15.19:0/2356350861 conn(0x7f6cb4079ba0 msgr2=0x7f6cb41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/2356350861 shutdown_connections 2024-04-23T18:25:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:27.578+0000 7f6cb8f6c700 1 -- 172.21.15.19:0/2356350861 wait complete. 2024-04-23T18:25:28.435 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-23T18:25:28.436 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.556+0000 7fd30d067640 1 -- 172.21.15.19:0/3067849849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd308100e90 msgr2=0x7fd3081012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.556+0000 7fd30d067640 1 --2- 172.21.15.19:0/3067849849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd308100e90 0x7fd3081012b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009900 tx=0x7fd2f802a410 comp rx=0 tx=0).stop 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.556+0000 7fd30d067640 1 -- 172.21.15.19:0/3067849849 shutdown_connections 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.556+0000 7fd30d067640 1 --2- 172.21.15.19:0/3067849849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3081029a0 0x7fd3081092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.556+0000 7fd30d067640 1 --2- 172.21.15.19:0/3067849849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308101fe0 0x7fd308102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.556+0000 7fd30d067640 1 --2- 172.21.15.19:0/3067849849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd308100e90 0x7fd3081012b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.556+0000 7fd30d067640 1 -- 172.21.15.19:0/3067849849 >> 172.21.15.19:0/3067849849 conn(0x7fd3080fc400 msgr2=0x7fd3080fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.556+0000 7fd30d067640 1 -- 172.21.15.19:0/3067849849 shutdown_connections 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.556+0000 7fd30d067640 1 -- 172.21.15.19:0/3067849849 wait complete. 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30d067640 1 Processor -- start 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30d067640 1 -- start start 2024-04-23T18:25:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30d067640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd308100e90 0x7fd308069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30d067640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308101fe0 0x7fd30819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30d067640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3081029a0 0x7fd3081a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30d067640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd30810c9e0 con 0x7fd3081029a0 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30d067640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd30810c860 con 0x7fd308100e90 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30d067640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd30810cb60 con 0x7fd308101fe0 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3081029a0 0x7fd3081a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3081029a0 0x7fd3081a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47088/0 (socket says 172.21.15.19:47088) 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30759e640 1 -- 172.21.15.19:0/1239544197 learned_addr learned my addr 172.21.15.19:0/1239544197 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30659c640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308101fe0 0x7fd30819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd306d9d640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd308100e90 0x7fd308069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30759e640 1 -- 172.21.15.19:0/1239544197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308101fe0 msgr2=0x7fd30819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30759e640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308101fe0 0x7fd30819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30759e640 1 -- 172.21.15.19:0/1239544197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd308100e90 msgr2=0x7fd308069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30759e640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd308100e90 0x7fd308069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd30759e640 1 -- 172.21.15.19:0/1239544197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3081a2090 con 0x7fd3081029a0 2024-04-23T18:25:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.557+0000 7fd306d9d640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd308100e90 0x7fd308069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.558+0000 7fd30759e640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3081029a0 0x7fd3081a18a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd300009fd0 tx=0x7fd30000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.558+0000 7fd2effff640 1 -- 172.21.15.19:0/1239544197 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300017070 con 0x7fd3081029a0 2024-04-23T18:25:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.558+0000 7fd30d067640 1 -- 172.21.15.19:0/1239544197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3081a2320 con 0x7fd3081029a0 2024-04-23T18:25:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.558+0000 7fd2effff640 1 -- 172.21.15.19:0/1239544197 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd300003e20 con 0x7fd3081029a0 2024-04-23T18:25:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.558+0000 7fd2effff640 1 -- 172.21.15.19:0/1239544197 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300010460 con 0x7fd3081029a0 2024-04-23T18:25:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.558+0000 7fd30d067640 1 -- 172.21.15.19:0/1239544197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3081a2870 con 0x7fd3081029a0 2024-04-23T18:25:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.559+0000 7fd30d067640 1 -- 172.21.15.19:0/1239544197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd308068d70 con 0x7fd3081029a0 2024-04-23T18:25:28.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.560+0000 7fd2effff640 1 -- 172.21.15.19:0/1239544197 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd300003970 con 0x7fd3081029a0 2024-04-23T18:25:28.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.561+0000 7fd2effff640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd2f00a3be0 0x7fd2f00a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:28.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.561+0000 7fd306d9d640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd2f00a3be0 0x7fd2f00a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:28.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.561+0000 7fd2effff640 1 -- 172.21.15.19:0/1239544197 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd3000d0170 con 0x7fd3081029a0 2024-04-23T18:25:28.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.561+0000 7fd306d9d640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd2f00a3be0 0x7fd2f00a60a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8000c00 tx=0x7fd2f80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.566+0000 7fd2effff640 1 -- 172.21.15.19:0/1239544197 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd30009cb00 con 0x7fd3081029a0 2024-04-23T18:25:28.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.759+0000 7fd30d067640 1 -- 172.21.15.19:0/1239544197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd308062b50 con 0x7fd3081029a0 2024-04-23T18:25:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.760+0000 7fd2effff640 1 -- 172.21.15.19:0/1239544197 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22642 (secure 0 0 0) 0x7fd30009c400 con 0x7fd3081029a0 2024-04-23T18:25:28.753 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:28.753 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:24:36.689178+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:25:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 -- 172.21.15.19:0/1239544197 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd2f00a3be0 msgr2=0x7fd2f00a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd2f00a3be0 0x7fd2f00a60a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8000c00 tx=0x7fd2f80314a0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 -- 172.21.15.19:0/1239544197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3081029a0 msgr2=0x7fd3081a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3081029a0 0x7fd3081a18a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd300009fd0 tx=0x7fd30000b260 comp rx=0 tx=0).stop 2024-04-23T18:25:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 -- 172.21.15.19:0/1239544197 shutdown_connections 2024-04-23T18:25:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fd2f00a3be0 0x7fd2f00a60a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3081029a0 0x7fd3081a18a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308101fe0 0x7fd30819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 --2- 172.21.15.19:0/1239544197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd308100e90 0x7fd308069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 -- 172.21.15.19:0/1239544197 >> 172.21.15.19:0/1239544197 conn(0x7fd3080fc400 msgr2=0x7fd308105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:28.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 -- 172.21.15.19:0/1239544197 shutdown_connections 2024-04-23T18:25:28.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.762+0000 7fd30d067640 1 -- 172.21.15.19:0/1239544197 wait complete. 2024-04-23T18:25:28.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:28 smithi019 ceph-mon[29215]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:28.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:28 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2356350861' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:25:28.767 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:23:53.296504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:23:54.573484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:25:28.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:25:28.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:28 smithi044 ceph-mon[32458]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:28.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:28 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2356350861' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:25:28.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:28 smithi155 ceph-mon[33468]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:28.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:28 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2356350861' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:25:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 -- 172.21.15.19:0/3538195891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49a4102a00 msgr2=0x7f49a4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 --2- 172.21.15.19:0/3538195891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49a4102a00 0x7f49a4109320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f499c009ca0 tx=0x7f499c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 -- 172.21.15.19:0/3538195891 shutdown_connections 2024-04-23T18:25:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 --2- 172.21.15.19:0/3538195891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4102040 0x7f49a41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 --2- 172.21.15.19:0/3538195891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49a4102a00 0x7f49a4109320 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 --2- 172.21.15.19:0/3538195891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49a4100e60 0x7f49a4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 -- 172.21.15.19:0/3538195891 >> 172.21.15.19:0/3538195891 conn(0x7f49a40fc440 msgr2=0x7f49a40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 -- 172.21.15.19:0/3538195891 shutdown_connections 2024-04-23T18:25:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 -- 172.21.15.19:0/3538195891 wait complete. 2024-04-23T18:25:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 Processor -- start 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.879+0000 7f49abfcd640 1 -- start start 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49abfcd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49a4100e60 0x7f49a4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49abfcd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4102040 0x7f49a41987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49abfcd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49a4102a00 0x7f49a419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49abfcd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49a410cac0 con 0x7f49a4100e60 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49abfcd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49a410c940 con 0x7f49a4102040 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49abfcd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f49a410cc40 con 0x7f49a4102a00 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49a9d69640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49a4100e60 0x7f49a4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49a9d69640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49a4100e60 0x7f49a4198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47108/0 (socket says 172.21.15.19:47108) 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49a9d69640 1 -- 172.21.15.19:0/4120936103 learned_addr learned my addr 172.21.15.19:0/4120936103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49a9568640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4102040 0x7f49a41987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49a9d69640 1 -- 172.21.15.19:0/4120936103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49a4102a00 msgr2=0x7f49a419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49a9d69640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49a4102a00 0x7f49a419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49a9d69640 1 -- 172.21.15.19:0/4120936103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4102040 msgr2=0x7f49a41987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49a9d69640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4102040 0x7f49a41987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49a9d69640 1 -- 172.21.15.19:0/4120936103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a419fea0 con 0x7f49a4100e60 2024-04-23T18:25:28.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49a9d69640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49a4100e60 0x7f49a4198280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f49a000b450 tx=0x7f49a000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:28.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f499affd640 1 -- 172.21.15.19:0/4120936103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0015020 con 0x7f49a4100e60 2024-04-23T18:25:28.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f499affd640 1 -- 172.21.15.19:0/4120936103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a0011d40 con 0x7f49a4100e60 2024-04-23T18:25:28.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49abfcd640 1 -- 172.21.15.19:0/4120936103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a41a0190 con 0x7f49a4100e60 2024-04-23T18:25:28.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f499affd640 1 -- 172.21.15.19:0/4120936103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a000f340 con 0x7f49a4100e60 2024-04-23T18:25:28.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.880+0000 7f49abfcd640 1 -- 172.21.15.19:0/4120936103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a41a06e0 con 0x7f49a4100e60 2024-04-23T18:25:28.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.881+0000 7f49abfcd640 1 -- 172.21.15.19:0/4120936103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a41a09a0 con 0x7f49a4100e60 2024-04-23T18:25:28.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.883+0000 7f499affd640 1 -- 172.21.15.19:0/4120936103 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f49a0018020 con 0x7f49a4100e60 2024-04-23T18:25:28.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.883+0000 7f499affd640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f49900a3bb0 0x7f49900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:28.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.884+0000 7f49a9568640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f49900a3bb0 0x7f49900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:28.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.884+0000 7f499affd640 1 -- 172.21.15.19:0/4120936103 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f49a00d0570 con 0x7f49a4100e60 2024-04-23T18:25:28.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.884+0000 7f49a9568640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f49900a3bb0 0x7f49900a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4994005d00 tx=0x7f4994005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:28.887+0000 7f499affd640 1 -- 172.21.15.19:0/4120936103 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49a001f020 con 0x7f49a4100e60 2024-04-23T18:25:29.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.075+0000 7f49abfcd640 1 -- 172.21.15.19:0/4120936103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49a41a0b70 con 0x7f49a4100e60 2024-04-23T18:25:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.076+0000 7f499affd640 1 -- 172.21.15.19:0/4120936103 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f49a000d040 con 0x7f49a4100e60 2024-04-23T18:25:29.069 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:25:29.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 -- 172.21.15.19:0/4120936103 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f49900a3bb0 msgr2=0x7f49900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f49900a3bb0 0x7f49900a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4994005d00 tx=0x7f4994005c50 comp rx=0 tx=0).stop 2024-04-23T18:25:29.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 -- 172.21.15.19:0/4120936103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49a4100e60 msgr2=0x7f49a4198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49a4100e60 0x7f49a4198280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f49a000b450 tx=0x7f49a000b910 comp rx=0 tx=0).stop 2024-04-23T18:25:29.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 -- 172.21.15.19:0/4120936103 shutdown_connections 2024-04-23T18:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f49900a3bb0 0x7f49900a6070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4102040 0x7f49a41987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49a4102a00 0x7f49a419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 --2- 172.21.15.19:0/4120936103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49a4100e60 0x7f49a4198280 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 -- 172.21.15.19:0/4120936103 >> 172.21.15.19:0/4120936103 conn(0x7f49a40fc440 msgr2=0x7f49a406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 -- 172.21.15.19:0/4120936103 shutdown_connections 2024-04-23T18:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.078+0000 7f49abfcd640 1 -- 172.21.15.19:0/4120936103 wait complete. 2024-04-23T18:25:29.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:25:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 -- 172.21.15.19:0/188685204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f949c101fc0 msgr2=0x7f949c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 --2- 172.21.15.19:0/188685204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f949c101fc0 0x7f949c102440 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9490009900 tx=0x7f949002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 -- 172.21.15.19:0/188685204 shutdown_connections 2024-04-23T18:25:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 --2- 172.21.15.19:0/188685204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f949c102980 0x7f949c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 --2- 172.21.15.19:0/188685204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f949c101fc0 0x7f949c102440 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 --2- 172.21.15.19:0/188685204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949c100e70 0x7f949c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 -- 172.21.15.19:0/188685204 >> 172.21.15.19:0/188685204 conn(0x7f949c0fc3e0 msgr2=0x7f949c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 -- 172.21.15.19:0/188685204 shutdown_connections 2024-04-23T18:25:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 -- 172.21.15.19:0/188685204 wait complete. 2024-04-23T18:25:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 Processor -- start 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.195+0000 7f94a1376640 1 -- start start 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f94a1376640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f949c100e70 0x7f949c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f94a1376640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f949c101fc0 0x7f949c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f94a1376640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949c102980 0x7f949c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f94a1376640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f949c10db10 con 0x7f949c101fc0 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f94a1376640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f949c10d990 con 0x7f949c102980 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f94a1376640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f949c10dc90 con 0x7f949c100e70 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f949c101fc0 0x7f949c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f949c101fc0 0x7f949c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47122/0 (socket says 172.21.15.19:47122) 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949a7fc640 1 -- 172.21.15.19:0/2039373954 learned_addr learned my addr 172.21.15.19:0/2039373954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949affd640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f949c100e70 0x7f949c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949b7fe640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949c102980 0x7f949c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949a7fc640 1 -- 172.21.15.19:0/2039373954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f949c100e70 msgr2=0x7f949c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949a7fc640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f949c100e70 0x7f949c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949a7fc640 1 -- 172.21.15.19:0/2039373954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949c102980 msgr2=0x7f949c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949a7fc640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949c102980 0x7f949c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949a7fc640 1 -- 172.21.15.19:0/2039373954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949c19fdf0 con 0x7f949c101fc0 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949b7fe640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949c102980 0x7f949c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f949a7fc640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f949c101fc0 0x7f949c198710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9490006010 tx=0x7f94900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f9483fff640 1 -- 172.21.15.19:0/2039373954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949003f030 con 0x7f949c101fc0 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f9483fff640 1 -- 172.21.15.19:0/2039373954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9490003cb0 con 0x7f949c101fc0 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f94a1376640 1 -- 172.21.15.19:0/2039373954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949c1a0080 con 0x7f949c101fc0 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f9483fff640 1 -- 172.21.15.19:0/2039373954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490035070 con 0x7f949c101fc0 2024-04-23T18:25:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.196+0000 7f94a1376640 1 -- 172.21.15.19:0/2039373954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949c1a0570 con 0x7f949c101fc0 2024-04-23T18:25:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.197+0000 7f94a1376640 1 -- 172.21.15.19:0/2039373954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f949c1081d0 con 0x7f949c101fc0 2024-04-23T18:25:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.198+0000 7f9483fff640 1 -- 172.21.15.19:0/2039373954 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9490042030 con 0x7f949c101fc0 2024-04-23T18:25:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.199+0000 7f9483fff640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f94840a3bb0 0x7f94840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.199+0000 7f949affd640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f94840a3bb0 0x7f94840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.199+0000 7f9483fff640 1 -- 172.21.15.19:0/2039373954 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f94900f0a90 con 0x7f949c101fc0 2024-04-23T18:25:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.200+0000 7f949affd640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f94840a3bb0 0x7f94840a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f948c00bbe0 tx=0x7f948c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:29.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.204+0000 7f9483fff640 1 -- 172.21.15.19:0/2039373954 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94900bd420 con 0x7f949c101fc0 2024-04-23T18:25:29.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.388+0000 7f94a1376640 1 -- 172.21.15.19:0/2039373954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f949c062b50 con 0x7f949c101fc0 2024-04-23T18:25:29.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.389+0000 7f9483fff640 1 -- 172.21.15.19:0/2039373954 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f94900bcd20 con 0x7f949c101fc0 2024-04-23T18:25:29.382 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-23T18:25:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 -- 172.21.15.19:0/2039373954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f94840a3bb0 msgr2=0x7f94840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f94840a3bb0 0x7f94840a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f948c00bbe0 tx=0x7f948c00b390 comp rx=0 tx=0).stop 2024-04-23T18:25:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 -- 172.21.15.19:0/2039373954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f949c101fc0 msgr2=0x7f949c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f949c101fc0 0x7f949c198710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9490006010 tx=0x7f94900042f0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 -- 172.21.15.19:0/2039373954 shutdown_connections 2024-04-23T18:25:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f94840a3bb0 0x7f94840a6070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949c102980 0x7f949c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f949c101fc0 0x7f949c198710 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 --2- 172.21.15.19:0/2039373954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f949c100e70 0x7f949c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 -- 172.21.15.19:0/2039373954 >> 172.21.15.19:0/2039373954 conn(0x7f949c0fc3e0 msgr2=0x7f949c1053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:29.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 -- 172.21.15.19:0/2039373954 shutdown_connections 2024-04-23T18:25:29.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.391+0000 7f94a1376640 1 -- 172.21.15.19:0/2039373954 wait complete. 2024-04-23T18:25:29.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:25:29.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.509+0000 7f9cee376640 1 -- 172.21.15.19:0/280615908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ce8103230 msgr2=0x7f9ce81036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.509+0000 7f9cee376640 1 --2- 172.21.15.19:0/280615908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ce8103230 0x7f9ce81036b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009930 tx=0x7f9cdc02a420 comp rx=0 tx=0).stop 2024-04-23T18:25:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.509+0000 7f9cee376640 1 -- 172.21.15.19:0/280615908 shutdown_connections 2024-04-23T18:25:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.509+0000 7f9cee376640 1 --2- 172.21.15.19:0/280615908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce8103bf0 0x7f9ce810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.509+0000 7f9cee376640 1 --2- 172.21.15.19:0/280615908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ce8103230 0x7f9ce81036b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.509+0000 7f9cee376640 1 --2- 172.21.15.19:0/280615908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce8100260 0x7f9ce8100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.509+0000 7f9cee376640 1 -- 172.21.15.19:0/280615908 >> 172.21.15.19:0/280615908 conn(0x7f9ce80fbb90 msgr2=0x7f9ce80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.509+0000 7f9cee376640 1 -- 172.21.15.19:0/280615908 shutdown_connections 2024-04-23T18:25:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.509+0000 7f9cee376640 1 -- 172.21.15.19:0/280615908 wait complete. 2024-04-23T18:25:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9cee376640 1 Processor -- start 2024-04-23T18:25:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9cee376640 1 -- start start 2024-04-23T18:25:29.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9cee376640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce8100260 0x7f9ce8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9cee376640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ce8103230 0x7f9ce8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9cee376640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce8103bf0 0x7f9ce819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9cee376640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce810db50 con 0x7f9ce8103230 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9cee376640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce810d9d0 con 0x7f9ce8100260 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9cee376640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce810dcd0 con 0x7f9ce8103bf0 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ce8103230 0x7f9ce8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ce8103230 0x7f9ce8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47138/0 (socket says 172.21.15.19:47138) 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9ce77fe640 1 -- 172.21.15.19:0/1716623691 learned_addr learned my addr 172.21.15.19:0/1716623691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9cec913640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce8103bf0 0x7f9ce819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.510+0000 7f9ce7fff640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce8100260 0x7f9ce8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9ce77fe640 1 -- 172.21.15.19:0/1716623691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce8103bf0 msgr2=0x7f9ce819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9ce77fe640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce8103bf0 0x7f9ce819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9ce77fe640 1 -- 172.21.15.19:0/1716623691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce8100260 msgr2=0x7f9ce8198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9ce77fe640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce8100260 0x7f9ce8198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9ce77fe640 1 -- 172.21.15.19:0/1716623691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce819fe30 con 0x7f9ce8103230 2024-04-23T18:25:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9cec913640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce8103bf0 0x7f9ce819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9ce77fe640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ce8103230 0x7f9ce8198750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc031500 tx=0x7f9cdc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9ce57fa640 1 -- 172.21.15.19:0/1716623691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc03f070 con 0x7f9ce8103230 2024-04-23T18:25:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9ce57fa640 1 -- 172.21.15.19:0/1716623691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cdc031c50 con 0x7f9ce8103230 2024-04-23T18:25:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9cee376640 1 -- 172.21.15.19:0/1716623691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce81a00c0 con 0x7f9ce8103230 2024-04-23T18:25:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9ce57fa640 1 -- 172.21.15.19:0/1716623691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc035030 con 0x7f9ce8103230 2024-04-23T18:25:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.511+0000 7f9cee376640 1 -- 172.21.15.19:0/1716623691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce81a05b0 con 0x7f9ce8103230 2024-04-23T18:25:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.512+0000 7f9cee376640 1 -- 172.21.15.19:0/1716623691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce81093e0 con 0x7f9ce8103230 2024-04-23T18:25:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.513+0000 7f9ce57fa640 1 -- 172.21.15.19:0/1716623691 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9cdc042070 con 0x7f9ce8103230 2024-04-23T18:25:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.514+0000 7f9ce57fa640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9cd00a3bb0 0x7f9cd00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.514+0000 7f9ce7fff640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9cd00a3bb0 0x7f9cd00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.514+0000 7f9ce57fa640 1 -- 172.21.15.19:0/1716623691 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9cdc0f0d10 con 0x7f9ce8103230 2024-04-23T18:25:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.514+0000 7f9ce7fff640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9cd00a3bb0 0x7f9cd00a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8005fd0 tx=0x7f9cd8005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:29.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.519+0000 7f9ce57fa640 1 -- 172.21.15.19:0/1716623691 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cdc0f6020 con 0x7f9ce8103230 2024-04-23T18:25:29.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.706+0000 7f9cee376640 1 -- 172.21.15.19:0/1716623691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ce8062b50 con 0x7f9ce8103230 2024-04-23T18:25:29.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.707+0000 7f9ce57fa640 1 -- 172.21.15.19:0/1716623691 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f9cdc0bd6a0 con 0x7f9ce8103230 2024-04-23T18:25:29.700 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-23T18:25:29.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 -- 172.21.15.19:0/1716623691 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9cd00a3bb0 msgr2=0x7f9cd00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9cd00a3bb0 0x7f9cd00a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8005fd0 tx=0x7f9cd8005c70 comp rx=0 tx=0).stop 2024-04-23T18:25:29.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 -- 172.21.15.19:0/1716623691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ce8103230 msgr2=0x7f9ce8198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ce8103230 0x7f9ce8198750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc031500 tx=0x7f9cdc003c10 comp rx=0 tx=0).stop 2024-04-23T18:25:29.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 -- 172.21.15.19:0/1716623691 shutdown_connections 2024-04-23T18:25:29.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f9cd00a3bb0 0x7f9cd00a6070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce8103bf0 0x7f9ce819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ce8103230 0x7f9ce8198750 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 --2- 172.21.15.19:0/1716623691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce8100260 0x7f9ce8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 -- 172.21.15.19:0/1716623691 >> 172.21.15.19:0/1716623691 conn(0x7f9ce80fbb90 msgr2=0x7f9ce81065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 -- 172.21.15.19:0/1716623691 shutdown_connections 2024-04-23T18:25:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.709+0000 7f9cee376640 1 -- 172.21.15.19:0/1716623691 wait complete. 2024-04-23T18:25:29.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:29 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1239544197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:29.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:29 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4120936103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:29.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:29 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2039373954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:29.713 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:25:29.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:29 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1239544197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:29.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:29 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4120936103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:29.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:29 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2039373954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.823+0000 7f93889c6640 1 -- 172.21.15.19:0/2035907639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9384102ad0 msgr2=0x7f9384104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.823+0000 7f93889c6640 1 --2- 172.21.15.19:0/2035907639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9384102ad0 0x7f9384104ec0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9374009900 tx=0x7f937402a410 comp rx=0 tx=0).stop 2024-04-23T18:25:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.823+0000 7f93889c6640 1 -- 172.21.15.19:0/2035907639 shutdown_connections 2024-04-23T18:25:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.823+0000 7f93889c6640 1 --2- 172.21.15.19:0/2035907639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384105400 0x7f93841077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.823+0000 7f93889c6640 1 --2- 172.21.15.19:0/2035907639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9384107d30 0x7f938410a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.823+0000 7f93889c6640 1 --2- 172.21.15.19:0/2035907639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9384102ad0 0x7f9384104ec0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.823+0000 7f93889c6640 1 -- 172.21.15.19:0/2035907639 >> 172.21.15.19:0/2035907639 conn(0x7f93840fc400 msgr2=0x7f93840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.823+0000 7f93889c6640 1 -- 172.21.15.19:0/2035907639 shutdown_connections 2024-04-23T18:25:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.823+0000 7f93889c6640 1 -- 172.21.15.19:0/2035907639 wait complete. 2024-04-23T18:25:29.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.823+0000 7f93889c6640 1 Processor -- start 2024-04-23T18:25:29.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f93889c6640 1 -- start start 2024-04-23T18:25:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f93889c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9384102ad0 0x7f93841981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f93889c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384105400 0x7f9384198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f93889c6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9384107d30 0x7f938419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f93889c6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f938410dae0 con 0x7f9384102ad0 2024-04-23T18:25:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f93889c6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f938410d960 con 0x7f9384105400 2024-04-23T18:25:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f93889c6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f938410dc60 con 0x7f9384107d30 2024-04-23T18:25:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f938259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9384102ad0 0x7f93841981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f938259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9384102ad0 0x7f93841981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38132/0 (socket says 172.21.15.19:38132) 2024-04-23T18:25:29.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f938259c640 1 -- 172.21.15.19:0/411983070 learned_addr learned my addr 172.21.15.19:0/411983070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:29.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f9381d9b640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384105400 0x7f9384198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f9382d9d640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9384107d30 0x7f938419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f938259c640 1 -- 172.21.15.19:0/411983070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9384107d30 msgr2=0x7f938419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f938259c640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9384107d30 0x7f938419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f938259c640 1 -- 172.21.15.19:0/411983070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384105400 msgr2=0x7f9384198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f938259c640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384105400 0x7f9384198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f938259c640 1 -- 172.21.15.19:0/411983070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938419fe10 con 0x7f9384102ad0 2024-04-23T18:25:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f9382d9d640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9384107d30 0x7f938419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.824+0000 7f938259c640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9384102ad0 0x7f93841981f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9374005940 tx=0x7f93740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.825+0000 7f93737fe640 1 -- 172.21.15.19:0/411983070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937403f030 con 0x7f9384102ad0 2024-04-23T18:25:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.825+0000 7f93889c6640 1 -- 172.21.15.19:0/411983070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93841a00a0 con 0x7f9384102ad0 2024-04-23T18:25:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.825+0000 7f93737fe640 1 -- 172.21.15.19:0/411983070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9374003cb0 con 0x7f9384102ad0 2024-04-23T18:25:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.825+0000 7f93737fe640 1 -- 172.21.15.19:0/411983070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374035070 con 0x7f9384102ad0 2024-04-23T18:25:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.825+0000 7f93889c6640 1 -- 172.21.15.19:0/411983070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93841a05e0 con 0x7f9384102ad0 2024-04-23T18:25:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.826+0000 7f93889c6640 1 -- 172.21.15.19:0/411983070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93840fdef0 con 0x7f9384102ad0 2024-04-23T18:25:29.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:29 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1239544197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:29.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:29 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4120936103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:29.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:29 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2039373954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.827+0000 7f93737fe640 1 -- 172.21.15.19:0/411983070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9374042030 con 0x7f9384102ad0 2024-04-23T18:25:29.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.828+0000 7f93737fe640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f936c0a3bb0 0x7f936c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.828+0000 7f9381d9b640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f936c0a3bb0 0x7f936c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.828+0000 7f93737fe640 1 -- 172.21.15.19:0/411983070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f93740f0850 con 0x7f9384102ad0 2024-04-23T18:25:29.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.828+0000 7f9381d9b640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f936c0a3bb0 0x7f936c0a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9384199700 tx=0x7f9378005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:29.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:29.832+0000 7f93737fe640 1 -- 172.21.15.19:0/411983070 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93740bd240 con 0x7f9384102ad0 2024-04-23T18:25:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.058+0000 7f93889c6640 1 -- 172.21.15.19:0/411983070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9384062b50 con 0x7f9384102ad0 2024-04-23T18:25:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.059+0000 7f93737fe640 1 -- 172.21.15.19:0/411983070 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7f9374003e60 con 0x7f9384102ad0 2024-04-23T18:25:30.053 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:30.053 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:24:02.624724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24508,24469,24521,24491]},"id":1}]} 2024-04-23T18:25:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-23T18:25:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 -- 172.21.15.19:0/411983070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f936c0a3bb0 msgr2=0x7f936c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f936c0a3bb0 0x7f936c0a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9384199700 tx=0x7f9378005f90 comp rx=0 tx=0).stop 2024-04-23T18:25:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 -- 172.21.15.19:0/411983070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9384102ad0 msgr2=0x7f93841981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9384102ad0 0x7f93841981f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9374005940 tx=0x7f93740042f0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 -- 172.21.15.19:0/411983070 shutdown_connections 2024-04-23T18:25:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f936c0a3bb0 0x7f936c0a6070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384105400 0x7f9384198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9384107d30 0x7f938419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 --2- 172.21.15.19:0/411983070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9384102ad0 0x7f93841981f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 -- 172.21.15.19:0/411983070 >> 172.21.15.19:0/411983070 conn(0x7f93840fc400 msgr2=0x7f9384103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 -- 172.21.15.19:0/411983070 shutdown_connections 2024-04-23T18:25:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.061+0000 7f93889c6640 1 -- 172.21.15.19:0/411983070 wait complete. 2024-04-23T18:25:30.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:25:30.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.175+0000 7f21a5a60640 1 -- 172.21.15.19:0/1986974783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21a0101fa0 msgr2=0x7f21a01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.175+0000 7f21a5a60640 1 --2- 172.21.15.19:0/1986974783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21a0101fa0 0x7f21a01023c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2190009900 tx=0x7f219002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:30.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.175+0000 7f21a5a60640 1 -- 172.21.15.19:0/1986974783 shutdown_connections 2024-04-23T18:25:30.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.175+0000 7f21a5a60640 1 --2- 172.21.15.19:0/1986974783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a0103b50 0x7f21a010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.175+0000 7f21a5a60640 1 --2- 172.21.15.19:0/1986974783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21a0103190 0x7f21a0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.175+0000 7f21a5a60640 1 --2- 172.21.15.19:0/1986974783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21a0101fa0 0x7f21a01023c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.175+0000 7f21a5a60640 1 -- 172.21.15.19:0/1986974783 >> 172.21.15.19:0/1986974783 conn(0x7f21a00fd530 msgr2=0x7f21a00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.175+0000 7f21a5a60640 1 -- 172.21.15.19:0/1986974783 shutdown_connections 2024-04-23T18:25:30.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.175+0000 7f21a5a60640 1 -- 172.21.15.19:0/1986974783 wait complete. 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f21a5a60640 1 Processor -- start 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f21a5a60640 1 -- start start 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f21a5a60640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21a0101fa0 0x7f21a01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f21a5a60640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21a0103190 0x7f21a0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f21a5a60640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a0103b50 0x7f21a019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f21a5a60640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21a010dae0 con 0x7f21a0101fa0 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f21a5a60640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f21a010d960 con 0x7f21a0103b50 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f21a5a60640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f21a010dc60 con 0x7f21a0103190 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f219effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21a0101fa0 0x7f21a01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f219effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21a0101fa0 0x7f21a01981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38150/0 (socket says 172.21.15.19:38150) 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f219effd640 1 -- 172.21.15.19:0/3844989343 learned_addr learned my addr 172.21.15.19:0/3844989343 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f21967fc640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21a0103190 0x7f21a0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f219effd640 1 -- 172.21.15.19:0/3844989343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21a0103190 msgr2=0x7f21a0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f219effd640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21a0103190 0x7f21a0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f219effd640 1 -- 172.21.15.19:0/3844989343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a0103b50 msgr2=0x7f21a019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f219effd640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a0103b50 0x7f21a019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f219effd640 1 -- 172.21.15.19:0/3844989343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a019fe10 con 0x7f21a0101fa0 2024-04-23T18:25:30.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.176+0000 7f219effd640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21a0101fa0 0x7f21a01981f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2190006010 tx=0x7f21900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.177+0000 7f21a4a5e640 1 -- 172.21.15.19:0/3844989343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219003f030 con 0x7f21a0101fa0 2024-04-23T18:25:30.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.177+0000 7f21a5a60640 1 -- 172.21.15.19:0/3844989343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a01a00a0 con 0x7f21a0101fa0 2024-04-23T18:25:30.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.177+0000 7f21a4a5e640 1 -- 172.21.15.19:0/3844989343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2190003cb0 con 0x7f21a0101fa0 2024-04-23T18:25:30.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.177+0000 7f21a4a5e640 1 -- 172.21.15.19:0/3844989343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190035070 con 0x7f21a0101fa0 2024-04-23T18:25:30.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.177+0000 7f21a5a60640 1 -- 172.21.15.19:0/3844989343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a01a0590 con 0x7f21a0101fa0 2024-04-23T18:25:30.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.178+0000 7f21a5a60640 1 -- 172.21.15.19:0/3844989343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a0109390 con 0x7f21a0101fa0 2024-04-23T18:25:30.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.179+0000 7f21a4a5e640 1 -- 172.21.15.19:0/3844989343 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2190042030 con 0x7f21a0101fa0 2024-04-23T18:25:30.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.179+0000 7f21a4a5e640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f218c0a3bb0 0x7f218c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.180+0000 7f21967fc640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f218c0a3bb0 0x7f218c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.180+0000 7f21a4a5e640 1 -- 172.21.15.19:0/3844989343 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f21900f0ed0 con 0x7f21a0101fa0 2024-04-23T18:25:30.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.180+0000 7f21967fc640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f218c0a3bb0 0x7f218c0a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2188005fd0 tx=0x7f2188005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.183+0000 7f21a4a5e640 1 -- 172.21.15.19:0/3844989343 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21900bd8c0 con 0x7f21a0101fa0 2024-04-23T18:25:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.372+0000 7f21a5a60640 1 -- 172.21.15.19:0/3844989343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21a0062b50 con 0x7f21a0101fa0 2024-04-23T18:25:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.373+0000 7f21a4a5e640 1 -- 172.21.15.19:0/3844989343 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22642 (secure 0 0 0) 0x7f21900bd1c0 con 0x7f21a0101fa0 2024-04-23T18:25:30.366 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:30.367 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:24:36.689178+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:25:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.375+0000 7f21a5a60640 1 -- 172.21.15.19:0/3844989343 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f218c0a3bb0 msgr2=0x7f218c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.375+0000 7f21a5a60640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f218c0a3bb0 0x7f218c0a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2188005fd0 tx=0x7f2188005850 comp rx=0 tx=0).stop 2024-04-23T18:25:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.375+0000 7f21a5a60640 1 -- 172.21.15.19:0/3844989343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21a0101fa0 msgr2=0x7f21a01981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.375+0000 7f21a5a60640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21a0101fa0 0x7f21a01981f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2190006010 tx=0x7f21900042f0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.375+0000 7f21a5a60640 1 -- 172.21.15.19:0/3844989343 shutdown_connections 2024-04-23T18:25:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.375+0000 7f21a5a60640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f218c0a3bb0 0x7f218c0a6070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.375+0000 7f21a5a60640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21a0103b50 0x7f21a019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.375+0000 7f21a5a60640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21a0103190 0x7f21a0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.375+0000 7f21a5a60640 1 --2- 172.21.15.19:0/3844989343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21a0101fa0 0x7f21a01981f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.375+0000 7f21a5a60640 1 -- 172.21.15.19:0/3844989343 >> 172.21.15.19:0/3844989343 conn(0x7f21a00fd530 msgr2=0x7f21a0106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.376+0000 7f21a5a60640 1 -- 172.21.15.19:0/3844989343 shutdown_connections 2024-04-23T18:25:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.376+0000 7f21a5a60640 1 -- 172.21.15.19:0/3844989343 wait complete. 2024-04-23T18:25:30.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:23:53.296504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:23:54.573484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:25:30.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.492+0000 7f3b17b7d640 1 -- 172.21.15.19:0/3666878530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b101029a0 msgr2=0x7f3b101092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.492+0000 7f3b17b7d640 1 --2- 172.21.15.19:0/3666878530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b101029a0 0x7f3b101092c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009ca0 tx=0x7f3b0c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.492+0000 7f3b17b7d640 1 -- 172.21.15.19:0/3666878530 shutdown_connections 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.492+0000 7f3b17b7d640 1 --2- 172.21.15.19:0/3666878530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b10101fe0 0x7f3b10102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.492+0000 7f3b17b7d640 1 --2- 172.21.15.19:0/3666878530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b101029a0 0x7f3b101092c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.492+0000 7f3b17b7d640 1 --2- 172.21.15.19:0/3666878530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b10100e90 0x7f3b101012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.492+0000 7f3b17b7d640 1 -- 172.21.15.19:0/3666878530 >> 172.21.15.19:0/3666878530 conn(0x7f3b100fc400 msgr2=0x7f3b100fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.492+0000 7f3b17b7d640 1 -- 172.21.15.19:0/3666878530 shutdown_connections 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b17b7d640 1 -- 172.21.15.19:0/3666878530 wait complete. 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b17b7d640 1 Processor -- start 2024-04-23T18:25:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b17b7d640 1 -- start start 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b17b7d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b10100e90 0x7f3b101970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b17b7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b10101fe0 0x7f3b10197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b17b7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b101029a0 0x7f3b1019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b17b7d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1010c9d0 con 0x7f3b10101fe0 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b17b7d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1010c850 con 0x7f3b101029a0 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b17b7d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1010cb50 con 0x7f3b10100e90 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b15118640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b10101fe0 0x7f3b10197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b15118640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b10101fe0 0x7f3b10197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38168/0 (socket says 172.21.15.19:38168) 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b15118640 1 -- 172.21.15.19:0/1715928070 learned_addr learned my addr 172.21.15.19:0/1715928070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.493+0000 7f3b15919640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b10100e90 0x7f3b101970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b1611a640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b101029a0 0x7f3b1019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b15118640 1 -- 172.21.15.19:0/1715928070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b10100e90 msgr2=0x7f3b101970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b15118640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b10100e90 0x7f3b101970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b15118640 1 -- 172.21.15.19:0/1715928070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b101029a0 msgr2=0x7f3b1019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b15118640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b101029a0 0x7f3b1019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b15118640 1 -- 172.21.15.19:0/1715928070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1019ec60 con 0x7f3b10101fe0 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b15919640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b10100e90 0x7f3b101970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b1611a640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b101029a0 0x7f3b1019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b15118640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b10101fe0 0x7f3b10197620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000c970 tx=0x7f3b0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b06ffd640 1 -- 172.21.15.19:0/1715928070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00007c10 con 0x7f3b10101fe0 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b06ffd640 1 -- 172.21.15.19:0/1715928070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b00007db0 con 0x7f3b10101fe0 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b17b7d640 1 -- 172.21.15.19:0/1715928070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1019ef50 con 0x7f3b10101fe0 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b06ffd640 1 -- 172.21.15.19:0/1715928070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0000f870 con 0x7f3b10101fe0 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.494+0000 7f3b17b7d640 1 -- 172.21.15.19:0/1715928070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1019f460 con 0x7f3b10101fe0 2024-04-23T18:25:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.495+0000 7f3b17b7d640 1 -- 172.21.15.19:0/1715928070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b10108140 con 0x7f3b10101fe0 2024-04-23T18:25:30.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.496+0000 7f3b06ffd640 1 -- 172.21.15.19:0/1715928070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3b0000fa10 con 0x7f3b10101fe0 2024-04-23T18:25:30.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.497+0000 7f3b06ffd640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3afc0a3c30 0x7f3afc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.497+0000 7f3b15919640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3afc0a3c30 0x7f3afc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.497+0000 7f3b06ffd640 1 -- 172.21.15.19:0/1715928070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3b000d0a70 con 0x7f3b10101fe0 2024-04-23T18:25:30.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.497+0000 7f3b15919640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3afc0a3c30 0x7f3afc0a60f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3b080058c0 tx=0x7f3b08005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.500+0000 7f3b06ffd640 1 -- 172.21.15.19:0/1715928070 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b0009d520 con 0x7f3b10101fe0 2024-04-23T18:25:30.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.685+0000 7f3b17b7d640 1 -- 172.21.15.19:0/1715928070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b1002d810 con 0x7f3b10101fe0 2024-04-23T18:25:30.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.686+0000 7f3b06ffd640 1 -- 172.21.15.19:0/1715928070 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f3b0009ce20 con 0x7f3b10101fe0 2024-04-23T18:25:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:25:30.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.687+0000 7f3b17b7d640 1 -- 172.21.15.19:0/1715928070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3afc0a3c30 msgr2=0x7f3afc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.687+0000 7f3b17b7d640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3afc0a3c30 0x7f3afc0a60f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3b080058c0 tx=0x7f3b08005ec0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.688+0000 7f3b17b7d640 1 -- 172.21.15.19:0/1715928070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b10101fe0 msgr2=0x7f3b10197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.688+0000 7f3b17b7d640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b10101fe0 0x7f3b10197620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000c970 tx=0x7f3b0000ce30 comp rx=0 tx=0).stop 2024-04-23T18:25:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.688+0000 7f3b17b7d640 1 -- 172.21.15.19:0/1715928070 shutdown_connections 2024-04-23T18:25:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.688+0000 7f3b17b7d640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3afc0a3c30 0x7f3afc0a60f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.688+0000 7f3b17b7d640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b101029a0 0x7f3b1019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.688+0000 7f3b17b7d640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b10101fe0 0x7f3b10197620 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.688+0000 7f3b17b7d640 1 --2- 172.21.15.19:0/1715928070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b10100e90 0x7f3b101970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.688+0000 7f3b17b7d640 1 -- 172.21.15.19:0/1715928070 >> 172.21.15.19:0/1715928070 conn(0x7f3b100fc400 msgr2=0x7f3b10105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.688+0000 7f3b17b7d640 1 -- 172.21.15.19:0/1715928070 shutdown_connections 2024-04-23T18:25:30.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.688+0000 7f3b17b7d640 1 -- 172.21.15.19:0/1715928070 wait complete. 2024-04-23T18:25:30.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:30 smithi019 ceph-mon[29215]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:30 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1716623691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:30 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/411983070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:30 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3844989343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:30.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:25:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.804+0000 7f9973dbb640 1 -- 172.21.15.19:0/2311092684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f996c107c90 msgr2=0x7f996c10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.804+0000 7f9973dbb640 1 --2- 172.21.15.19:0/2311092684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f996c107c90 0x7f996c10a0e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9964009ca0 tx=0x7f996402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.804+0000 7f9973dbb640 1 -- 172.21.15.19:0/2311092684 shutdown_connections 2024-04-23T18:25:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.804+0000 7f9973dbb640 1 --2- 172.21.15.19:0/2311092684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f996c105360 0x7f996c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.804+0000 7f9973dbb640 1 --2- 172.21.15.19:0/2311092684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f996c107c90 0x7f996c10a0e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.804+0000 7f9973dbb640 1 --2- 172.21.15.19:0/2311092684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f996c102a30 0x7f996c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.804+0000 7f9973dbb640 1 -- 172.21.15.19:0/2311092684 >> 172.21.15.19:0/2311092684 conn(0x7f996c0fc440 msgr2=0x7f996c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.804+0000 7f9973dbb640 1 -- 172.21.15.19:0/2311092684 shutdown_connections 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.804+0000 7f9973dbb640 1 -- 172.21.15.19:0/2311092684 wait complete. 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9973dbb640 1 Processor -- start 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9973dbb640 1 -- start start 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9973dbb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f996c102a30 0x7f996c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9973dbb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f996c105360 0x7f996c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9973dbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f996c107c90 0x7f996c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9973dbb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f996c10daf0 con 0x7f996c107c90 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9973dbb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f996c10d970 con 0x7f996c102a30 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9973dbb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f996c10dc70 con 0x7f996c105360 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9972358640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f996c107c90 0x7f996c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9972358640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f996c107c90 0x7f996c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38188/0 (socket says 172.21.15.19:38188) 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9972358640 1 -- 172.21.15.19:0/3923810958 learned_addr learned my addr 172.21.15.19:0/3923810958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9971b57640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f996c102a30 0x7f996c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9971356640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f996c105360 0x7f996c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9972358640 1 -- 172.21.15.19:0/3923810958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f996c105360 msgr2=0x7f996c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9972358640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f996c105360 0x7f996c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9972358640 1 -- 172.21.15.19:0/3923810958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f996c102a30 msgr2=0x7f996c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9972358640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f996c102a30 0x7f996c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9972358640 1 -- 172.21.15.19:0/3923810958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c19fd90 con 0x7f996c107c90 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9971356640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f996c105360 0x7f996c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.805+0000 7f9972358640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f996c107c90 0x7f996c19f720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f99640097b0 tx=0x7f9964003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.806+0000 7f9962ffd640 1 -- 172.21.15.19:0/3923810958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964004240 con 0x7f996c107c90 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.806+0000 7f9962ffd640 1 -- 172.21.15.19:0/3923810958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99640317c0 con 0x7f996c107c90 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.806+0000 7f9973dbb640 1 -- 172.21.15.19:0/3923810958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c1a0020 con 0x7f996c107c90 2024-04-23T18:25:30.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.806+0000 7f9962ffd640 1 -- 172.21.15.19:0/3923810958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996402fd80 con 0x7f996c107c90 2024-04-23T18:25:30.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.806+0000 7f9973dbb640 1 -- 172.21.15.19:0/3923810958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c1a05c0 con 0x7f996c107c90 2024-04-23T18:25:30.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.807+0000 7f9973dbb640 1 -- 172.21.15.19:0/3923810958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f996c0fdf30 con 0x7f996c107c90 2024-04-23T18:25:30.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.808+0000 7f9962ffd640 1 -- 172.21.15.19:0/3923810958 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f99640043e0 con 0x7f996c107c90 2024-04-23T18:25:30.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.808+0000 7f9962ffd640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99580a3c30 0x7f99580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.808+0000 7f9971b57640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99580a3c30 0x7f99580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.808+0000 7f9962ffd640 1 -- 172.21.15.19:0/3923810958 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f99640f1bf0 con 0x7f996c107c90 2024-04-23T18:25:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.809+0000 7f9971b57640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99580a3c30 0x7f99580a60f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9968005fd0 tx=0x7f996800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:30.813+0000 7f9962ffd640 1 -- 172.21.15.19:0/3923810958 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99640be5e0 con 0x7f996c107c90 2024-04-23T18:25:30.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:30 smithi044 ceph-mon[32458]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:30.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:30 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1716623691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:30.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:30 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/411983070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:30.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:30 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3844989343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:30.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:30 smithi155 ceph-mon[33468]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:30.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:30 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1716623691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:30.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:30 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/411983070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:30.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:30 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3844989343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.001+0000 7f9973dbb640 1 -- 172.21.15.19:0/3923810958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f996c062b50 con 0x7f996c107c90 2024-04-23T18:25:30.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.001+0000 7f9962ffd640 1 -- 172.21.15.19:0/3923810958 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f99640098b0 con 0x7f996c107c90 2024-04-23T18:25:30.994 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 -- 172.21.15.19:0/3923810958 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99580a3c30 msgr2=0x7f99580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99580a3c30 0x7f99580a60f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9968005fd0 tx=0x7f996800b390 comp rx=0 tx=0).stop 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 -- 172.21.15.19:0/3923810958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f996c107c90 msgr2=0x7f996c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f996c107c90 0x7f996c19f720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f99640097b0 tx=0x7f9964003c20 comp rx=0 tx=0).stop 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 -- 172.21.15.19:0/3923810958 shutdown_connections 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f99580a3c30 0x7f99580a60f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f996c107c90 0x7f996c19f720 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f996c105360 0x7f996c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 --2- 172.21.15.19:0/3923810958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f996c102a30 0x7f996c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 -- 172.21.15.19:0/3923810958 >> 172.21.15.19:0/3923810958 conn(0x7f996c0fc440 msgr2=0x7f996c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 -- 172.21.15.19:0/3923810958 shutdown_connections 2024-04-23T18:25:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.003+0000 7f9973dbb640 1 -- 172.21.15.19:0/3923810958 wait complete. 2024-04-23T18:25:31.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:25:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 -- 172.21.15.19:0/3790858419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52f0102040 msgr2=0x7f52f01024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 --2- 172.21.15.19:0/3790858419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52f0102040 0x7f52f01024c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009900 tx=0x7f52e402a3f0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 -- 172.21.15.19:0/3790858419 shutdown_connections 2024-04-23T18:25:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 --2- 172.21.15.19:0/3790858419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0102a00 0x7f52f0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 --2- 172.21.15.19:0/3790858419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52f0102040 0x7f52f01024c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 --2- 172.21.15.19:0/3790858419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f0100e60 0x7f52f0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 -- 172.21.15.19:0/3790858419 >> 172.21.15.19:0/3790858419 conn(0x7f52f00fc420 msgr2=0x7f52f00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 -- 172.21.15.19:0/3790858419 shutdown_connections 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 -- 172.21.15.19:0/3790858419 wait complete. 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 Processor -- start 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.119+0000 7f52f4dad640 1 -- start start 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52f4dad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f0100e60 0x7f52f0197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52f4dad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0102040 0x7f52f0197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52f4dad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52f0102a00 0x7f52f019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52f4dad640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52f010c9f0 con 0x7f52f0102a00 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52f4dad640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52f010c870 con 0x7f52f0100e60 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52f4dad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52f010cb70 con 0x7f52f0102040 2024-04-23T18:25:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52f0102a00 0x7f52f019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52f0102a00 0x7f52f019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38200/0 (socket says 172.21.15.19:38200) 2024-04-23T18:25:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52eed9d640 1 -- 172.21.15.19:0/1659125950 learned_addr learned my addr 172.21.15.19:0/1659125950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52ee59c640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f0100e60 0x7f52f0197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52edd9b640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0102040 0x7f52f0197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52eed9d640 1 -- 172.21.15.19:0/1659125950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0102040 msgr2=0x7f52f0197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52eed9d640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0102040 0x7f52f0197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52eed9d640 1 -- 172.21.15.19:0/1659125950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f0100e60 msgr2=0x7f52f0197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52eed9d640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f0100e60 0x7f52f0197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52eed9d640 1 -- 172.21.15.19:0/1659125950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f019ec20 con 0x7f52f0102a00 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52edd9b640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0102040 0x7f52f0197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52eed9d640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52f0102a00 0x7f52f019e5b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c970 tx=0x7f52e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52df7fe640 1 -- 172.21.15.19:0/1659125950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8007bb0 con 0x7f52f0102a00 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.120+0000 7f52df7fe640 1 -- 172.21.15.19:0/1659125950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e8007d50 con 0x7f52f0102a00 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.121+0000 7f52f4dad640 1 -- 172.21.15.19:0/1659125950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f019ef10 con 0x7f52f0102a00 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.121+0000 7f52df7fe640 1 -- 172.21.15.19:0/1659125950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8018770 con 0x7f52f0102a00 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.121+0000 7f52f4dad640 1 -- 172.21.15.19:0/1659125950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f019f3d0 con 0x7f52f0102a00 2024-04-23T18:25:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.121+0000 7f52f4dad640 1 -- 172.21.15.19:0/1659125950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f0108200 con 0x7f52f0102a00 2024-04-23T18:25:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.123+0000 7f52df7fe640 1 -- 172.21.15.19:0/1659125950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f52e800f990 con 0x7f52f0102a00 2024-04-23T18:25:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.123+0000 7f52df7fe640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f52d80a3c30 0x7f52d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.123+0000 7f52ee59c640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f52d80a3c30 0x7f52d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.123+0000 7f52df7fe640 1 -- 172.21.15.19:0/1659125950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f52e80d0430 con 0x7f52f0102a00 2024-04-23T18:25:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.124+0000 7f52ee59c640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f52d80a3c30 0x7f52d80a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52f0101bb0 tx=0x7f52e0007600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.128+0000 7f52df7fe640 1 -- 172.21.15.19:0/1659125950 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52e8014030 con 0x7f52f0102a00 2024-04-23T18:25:31.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.317+0000 7f52f4dad640 1 -- 172.21.15.19:0/1659125950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f52f0062bf0 con 0x7f52f0102a00 2024-04-23T18:25:31.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.318+0000 7f52df7fe640 1 -- 172.21.15.19:0/1659125950 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f52e809ce20 con 0x7f52f0102a00 2024-04-23T18:25:31.311 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-23T18:25:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 -- 172.21.15.19:0/1659125950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f52d80a3c30 msgr2=0x7f52d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f52d80a3c30 0x7f52d80a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52f0101bb0 tx=0x7f52e0007600 comp rx=0 tx=0).stop 2024-04-23T18:25:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 -- 172.21.15.19:0/1659125950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52f0102a00 msgr2=0x7f52f019e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52f0102a00 0x7f52f019e5b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c970 tx=0x7f52e800ce30 comp rx=0 tx=0).stop 2024-04-23T18:25:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 -- 172.21.15.19:0/1659125950 shutdown_connections 2024-04-23T18:25:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f52d80a3c30 0x7f52d80a60f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52f0102a00 0x7f52f019e5b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0102040 0x7f52f0197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 --2- 172.21.15.19:0/1659125950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52f0100e60 0x7f52f0197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 -- 172.21.15.19:0/1659125950 >> 172.21.15.19:0/1659125950 conn(0x7f52f00fc420 msgr2=0x7f52f01053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:31.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 -- 172.21.15.19:0/1659125950 shutdown_connections 2024-04-23T18:25:31.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.320+0000 7f52f4dad640 1 -- 172.21.15.19:0/1659125950 wait complete. 2024-04-23T18:25:31.324 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:25:31.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.433+0000 7fc70b9cd640 1 -- 172.21.15.19:0/3799698755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc704107c90 msgr2=0x7fc70410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.433+0000 7fc70b9cd640 1 --2- 172.21.15.19:0/3799698755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc704107c90 0x7fc70410a0e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009ca0 tx=0x7fc6fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.433+0000 7fc70b9cd640 1 -- 172.21.15.19:0/3799698755 shutdown_connections 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.433+0000 7fc70b9cd640 1 --2- 172.21.15.19:0/3799698755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc704105360 0x7fc704107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.433+0000 7fc70b9cd640 1 --2- 172.21.15.19:0/3799698755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc704107c90 0x7fc70410a0e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.433+0000 7fc70b9cd640 1 --2- 172.21.15.19:0/3799698755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc704102a30 0x7fc704104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.433+0000 7fc70b9cd640 1 -- 172.21.15.19:0/3799698755 >> 172.21.15.19:0/3799698755 conn(0x7fc7040fc440 msgr2=0x7fc7040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.433+0000 7fc70b9cd640 1 -- 172.21.15.19:0/3799698755 shutdown_connections 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.433+0000 7fc70b9cd640 1 -- 172.21.15.19:0/3799698755 wait complete. 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc70b9cd640 1 Processor -- start 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc70b9cd640 1 -- start start 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc70b9cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc704102a30 0x7fc7041981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc70b9cd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc704105360 0x7fc704198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc70b9cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc704107c90 0x7fc70419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc70b9cd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410daf0 con 0x7fc704107c90 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc70b9cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410d970 con 0x7fc704102a30 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc70b9cd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410dc70 con 0x7fc704105360 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc709f6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc704107c90 0x7fc70419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc709f6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc704107c90 0x7fc70419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38212/0 (socket says 172.21.15.19:38212) 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc709f6a640 1 -- 172.21.15.19:0/1464072296 learned_addr learned my addr 172.21.15.19:0/1464072296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc709769640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc704102a30 0x7fc7041981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc709f6a640 1 -- 172.21.15.19:0/1464072296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc704105360 msgr2=0x7fc704198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc708f68640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc704105360 0x7fc704198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc709f6a640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc704105360 0x7fc704198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc709f6a640 1 -- 172.21.15.19:0/1464072296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc704102a30 msgr2=0x7fc7041981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.434+0000 7fc709f6a640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc704102a30 0x7fc7041981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.435+0000 7fc709f6a640 1 -- 172.21.15.19:0/1464072296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70419fde0 con 0x7fc704107c90 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.435+0000 7fc709769640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc704102a30 0x7fc7041981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.435+0000 7fc709f6a640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc704107c90 0x7fc70419f770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0097b0 tx=0x7fc6fc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.435+0000 7fc6fa7fc640 1 -- 172.21.15.19:0/1464072296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc004240 con 0x7fc704107c90 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.435+0000 7fc6fa7fc640 1 -- 172.21.15.19:0/1464072296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6fc0317c0 con 0x7fc704107c90 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.435+0000 7fc70b9cd640 1 -- 172.21.15.19:0/1464072296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7041a0070 con 0x7fc704107c90 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.435+0000 7fc6fa7fc640 1 -- 172.21.15.19:0/1464072296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc02fd80 con 0x7fc704107c90 2024-04-23T18:25:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.435+0000 7fc70b9cd640 1 -- 172.21.15.19:0/1464072296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7041a05c0 con 0x7fc704107c90 2024-04-23T18:25:31.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.436+0000 7fc70b9cd640 1 -- 172.21.15.19:0/1464072296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7040fdf30 con 0x7fc704107c90 2024-04-23T18:25:31.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.437+0000 7fc6fa7fc640 1 -- 172.21.15.19:0/1464072296 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc6fc0043e0 con 0x7fc704107c90 2024-04-23T18:25:31.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.438+0000 7fc6fa7fc640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc6f00a3bb0 0x7fc6f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.438+0000 7fc709769640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc6f00a3bb0 0x7fc6f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.438+0000 7fc6fa7fc640 1 -- 172.21.15.19:0/1464072296 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc6fc0f1d00 con 0x7fc704107c90 2024-04-23T18:25:31.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.438+0000 7fc709769640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc6f00a3bb0 0x7fc6f00a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc7000058c0 tx=0x7fc700005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:31.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.442+0000 7fc6fa7fc640 1 -- 172.21.15.19:0/1464072296 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6fc0be6f0 con 0x7fc704107c90 2024-04-23T18:25:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.669+0000 7fc70b9cd640 1 -- 172.21.15.19:0/1464072296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc704062b50 con 0x7fc704107c90 2024-04-23T18:25:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.670+0000 7fc6fa7fc640 1 -- 172.21.15.19:0/1464072296 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7fc6fc0de070 con 0x7fc704107c90 2024-04-23T18:25:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:24:02.624724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24508,24469,24521,24491]},"id":1}]} 2024-04-23T18:25:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-23T18:25:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 -- 172.21.15.19:0/1464072296 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc6f00a3bb0 msgr2=0x7fc6f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc6f00a3bb0 0x7fc6f00a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc7000058c0 tx=0x7fc700005850 comp rx=0 tx=0).stop 2024-04-23T18:25:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 -- 172.21.15.19:0/1464072296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc704107c90 msgr2=0x7fc70419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc704107c90 0x7fc70419f770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0097b0 tx=0x7fc6fc003c20 comp rx=0 tx=0).stop 2024-04-23T18:25:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 -- 172.21.15.19:0/1464072296 shutdown_connections 2024-04-23T18:25:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fc6f00a3bb0 0x7fc6f00a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc704107c90 0x7fc70419f770 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc704105360 0x7fc704198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 --2- 172.21.15.19:0/1464072296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc704102a30 0x7fc7041981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 -- 172.21.15.19:0/1464072296 >> 172.21.15.19:0/1464072296 conn(0x7fc7040fc440 msgr2=0x7fc704103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 -- 172.21.15.19:0/1464072296 shutdown_connections 2024-04-23T18:25:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.672+0000 7fc70b9cd640 1 -- 172.21.15.19:0/1464072296 wait complete. 2024-04-23T18:25:31.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:31 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1715928070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:31.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:31 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3923810958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:31.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:31 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1659125950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:31.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:25:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.785+0000 7fbd097d8640 1 -- 172.21.15.19:0/1481072385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd04102a00 msgr2=0x7fbd04109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.785+0000 7fbd097d8640 1 --2- 172.21.15.19:0/1481072385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd04102a00 0x7fbd04109320 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009ca0 tx=0x7fbcfc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.785+0000 7fbd097d8640 1 -- 172.21.15.19:0/1481072385 shutdown_connections 2024-04-23T18:25:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.785+0000 7fbd097d8640 1 --2- 172.21.15.19:0/1481072385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd04102a00 0x7fbd04109320 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.785+0000 7fbd097d8640 1 --2- 172.21.15.19:0/1481072385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd04102040 0x7fbd041024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.785+0000 7fbd097d8640 1 --2- 172.21.15.19:0/1481072385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd04100e60 0x7fbd04101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.785+0000 7fbd097d8640 1 -- 172.21.15.19:0/1481072385 >> 172.21.15.19:0/1481072385 conn(0x7fbd040fc440 msgr2=0x7fbd040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.785+0000 7fbd097d8640 1 -- 172.21.15.19:0/1481072385 shutdown_connections 2024-04-23T18:25:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.785+0000 7fbd097d8640 1 -- 172.21.15.19:0/1481072385 wait complete. 2024-04-23T18:25:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.785+0000 7fbd097d8640 1 Processor -- start 2024-04-23T18:25:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd097d8640 1 -- start start 2024-04-23T18:25:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd097d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd04100e60 0x7fbd04198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd097d8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd04102040 0x7fbd04198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd097d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd04102a00 0x7fbd0419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd097d8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0410ca90 con 0x7fbd04102a00 2024-04-23T18:25:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd097d8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0410c910 con 0x7fbd04100e60 2024-04-23T18:25:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd097d8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0410cc10 con 0x7fbd04102040 2024-04-23T18:25:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd04102a00 0x7fbd0419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd04102a00 0x7fbd0419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38226/0 (socket says 172.21.15.19:38226) 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd037fe640 1 -- 172.21.15.19:0/2471497396 learned_addr learned my addr 172.21.15.19:0/2471497396 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd02ffd640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd04100e60 0x7fbd04198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd027fc640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd04102040 0x7fbd04198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd037fe640 1 -- 172.21.15.19:0/2471497396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd04102040 msgr2=0x7fbd04198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd037fe640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd04102040 0x7fbd04198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd037fe640 1 -- 172.21.15.19:0/2471497396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd04100e60 msgr2=0x7fbd04198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd037fe640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd04100e60 0x7fbd04198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd037fe640 1 -- 172.21.15.19:0/2471497396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0419fe50 con 0x7fbd04102a00 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbd037fe640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd04102a00 0x7fbd0419f7e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc031a20 tx=0x7fbcfc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.786+0000 7fbcebfff640 1 -- 172.21.15.19:0/2471497396 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc004240 con 0x7fbd04102a00 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.787+0000 7fbcebfff640 1 -- 172.21.15.19:0/2471497396 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcfc003e10 con 0x7fbd04102a00 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.787+0000 7fbd097d8640 1 -- 172.21.15.19:0/2471497396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd041a00e0 con 0x7fbd04102a00 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.787+0000 7fbcebfff640 1 -- 172.21.15.19:0/2471497396 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc02fd90 con 0x7fbd04102a00 2024-04-23T18:25:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.787+0000 7fbd097d8640 1 -- 172.21.15.19:0/2471497396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd041a06b0 con 0x7fbd04102a00 2024-04-23T18:25:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.788+0000 7fbd097d8640 1 -- 172.21.15.19:0/2471497396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd041a0910 con 0x7fbd04102a00 2024-04-23T18:25:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.789+0000 7fbcebfff640 1 -- 172.21.15.19:0/2471497396 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbcfc0043e0 con 0x7fbd04102a00 2024-04-23T18:25:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.789+0000 7fbcebfff640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbcec0a3bb0 0x7fbcec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:31.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.789+0000 7fbd02ffd640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbcec0a3bb0 0x7fbcec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:31.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.790+0000 7fbcebfff640 1 -- 172.21.15.19:0/2471497396 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbcfc0f1c70 con 0x7fbd04102a00 2024-04-23T18:25:31.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.790+0000 7fbd02ffd640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbcec0a3bb0 0x7fbcec0a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbd04101bb0 tx=0x7fbcf4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:31.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.794+0000 7fbcebfff640 1 -- 172.21.15.19:0/2471497396 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcfc0be600 con 0x7fbd04102a00 2024-04-23T18:25:31.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:31 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1715928070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:31.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:31 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3923810958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:31.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:31 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1659125950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:31.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:31 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1715928070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:31.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:31 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3923810958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:31.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:31 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1659125950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.983+0000 7fbd097d8640 1 -- 172.21.15.19:0/2471497396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd041a0ca0 con 0x7fbd04102a00 2024-04-23T18:25:31.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.984+0000 7fbcebfff640 1 -- 172.21.15.19:0/2471497396 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22642 (secure 0 0 0) 0x7fbd041a0ca0 con 0x7fbd04102a00 2024-04-23T18:25:31.977 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:31.977 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:24:36.689178+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:25:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 -- 172.21.15.19:0/2471497396 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbcec0a3bb0 msgr2=0x7fbcec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbcec0a3bb0 0x7fbcec0a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbd04101bb0 tx=0x7fbcf4009110 comp rx=0 tx=0).stop 2024-04-23T18:25:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 -- 172.21.15.19:0/2471497396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd04102a00 msgr2=0x7fbd0419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd04102a00 0x7fbd0419f7e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc031a20 tx=0x7fbcfc003c20 comp rx=0 tx=0).stop 2024-04-23T18:25:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 -- 172.21.15.19:0/2471497396 shutdown_connections 2024-04-23T18:25:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fbcec0a3bb0 0x7fbcec0a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd04102a00 0x7fbd0419f7e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd04102040 0x7fbd04198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 --2- 172.21.15.19:0/2471497396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd04100e60 0x7fbd04198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 -- 172.21.15.19:0/2471497396 >> 172.21.15.19:0/2471497396 conn(0x7fbd040fc440 msgr2=0x7fbd040fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 -- 172.21.15.19:0/2471497396 shutdown_connections 2024-04-23T18:25:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:31.986+0000 7fbd097d8640 1 -- 172.21.15.19:0/2471497396 wait complete. 2024-04-23T18:25:31.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:25:32.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 -- 172.21.15.19:0/2825274027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288107cd0 msgr2=0x7f428810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 --2- 172.21.15.19:0/2825274027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288107cd0 0x7f428810a0c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4280009ca0 tx=0x7f428002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 -- 172.21.15.19:0/2825274027 shutdown_connections 2024-04-23T18:25:32.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 --2- 172.21.15.19:0/2825274027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4288107cd0 0x7f428810a0c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 --2- 172.21.15.19:0/2825274027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42881053a0 0x7f4288107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 --2- 172.21.15.19:0/2825274027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4288102a70 0x7f4288104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 -- 172.21.15.19:0/2825274027 >> 172.21.15.19:0/2825274027 conn(0x7f42880fc400 msgr2=0x7f42880fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 -- 172.21.15.19:0/2825274027 shutdown_connections 2024-04-23T18:25:32.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 -- 172.21.15.19:0/2825274027 wait complete. 2024-04-23T18:25:32.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 Processor -- start 2024-04-23T18:25:32.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.103+0000 7f428dc31640 1 -- start start 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f428dc31640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4288102a70 0x7f4288102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f428dc31640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42881053a0 0x7f4288100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f428dc31640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4288107cd0 0x7f4288100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f428dc31640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f428810dad0 con 0x7f42881053a0 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f428dc31640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f428810d950 con 0x7f4288107cd0 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f428dc31640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f428810dc50 con 0x7f4288102a70 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42881053a0 0x7f4288100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4286ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42881053a0 0x7f4288100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38242/0 (socket says 172.21.15.19:38242) 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4286ffd640 1 -- 172.21.15.19:0/2944004340 learned_addr learned my addr 172.21.15.19:0/2944004340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f42877fe640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4288102a70 0x7f4288102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4287fff640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4288107cd0 0x7f4288100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4286ffd640 1 -- 172.21.15.19:0/2944004340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4288102a70 msgr2=0x7f4288102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4286ffd640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4288102a70 0x7f4288102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4286ffd640 1 -- 172.21.15.19:0/2944004340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4288107cd0 msgr2=0x7f4288100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4286ffd640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4288107cd0 0x7f4288100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4286ffd640 1 -- 172.21.15.19:0/2944004340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4288101820 con 0x7f42881053a0 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4286ffd640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42881053a0 0x7f4288100a20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f427800e9f0 tx=0x7f427800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4284ff9640 1 -- 172.21.15.19:0/2944004340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427800ccc0 con 0x7f42881053a0 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4284ff9640 1 -- 172.21.15.19:0/2944004340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42780054d0 con 0x7f42881053a0 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f428dc31640 1 -- 172.21.15.19:0/2944004340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42881a2300 con 0x7f42881053a0 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.104+0000 7f4284ff9640 1 -- 172.21.15.19:0/2944004340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278010430 con 0x7f42881053a0 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.105+0000 7f428dc31640 1 -- 172.21.15.19:0/2944004340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42881a27d0 con 0x7f42881053a0 2024-04-23T18:25:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.105+0000 7f428dc31640 1 -- 172.21.15.19:0/2944004340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42880fdef0 con 0x7f42881053a0 2024-04-23T18:25:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.107+0000 7f4284ff9640 1 -- 172.21.15.19:0/2944004340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f42780046c0 con 0x7f42881053a0 2024-04-23T18:25:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.107+0000 7f4284ff9640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f42700a3bb0 0x7f42700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.107+0000 7f4284ff9640 1 -- 172.21.15.19:0/2944004340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f42780d0710 con 0x7f42881053a0 2024-04-23T18:25:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.107+0000 7f42877fe640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f42700a3bb0 0x7f42700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.108+0000 7f42877fe640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f42700a3bb0 0x7f42700a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f427c0095f0 tx=0x7f427c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.112+0000 7f4284ff9640 1 -- 172.21.15.19:0/2944004340 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f427809d100 con 0x7f42881053a0 2024-04-23T18:25:32.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.338+0000 7f428dc31640 1 -- 172.21.15.19:0/2944004340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4288062b50 con 0x7f42881053a0 2024-04-23T18:25:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.339+0000 7f4284ff9640 1 -- 172.21.15.19:0/2944004340 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7f427809ca00 con 0x7f42881053a0 2024-04-23T18:25:32.332 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:32.332 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:24:02.624724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24508,24469,24521,24491]},"id":1}]} 2024-04-23T18:25:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 -- 172.21.15.19:0/2944004340 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f42700a3bb0 msgr2=0x7f42700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f42700a3bb0 0x7f42700a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f427c0095f0 tx=0x7f427c005f90 comp rx=0 tx=0).stop 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 -- 172.21.15.19:0/2944004340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42881053a0 msgr2=0x7f4288100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42881053a0 0x7f4288100a20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f427800e9f0 tx=0x7f427800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 -- 172.21.15.19:0/2944004340 shutdown_connections 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f42700a3bb0 0x7f42700a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4288107cd0 0x7f4288100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42881053a0 0x7f4288100a20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 --2- 172.21.15.19:0/2944004340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4288102a70 0x7f4288102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 -- 172.21.15.19:0/2944004340 >> 172.21.15.19:0/2944004340 conn(0x7f42880fc400 msgr2=0x7f42881033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.341+0000 7f428dc31640 1 -- 172.21.15.19:0/2944004340 shutdown_connections 2024-04-23T18:25:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.342+0000 7f428dc31640 1 -- 172.21.15.19:0/2944004340 wait complete. 2024-04-23T18:25:32.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:23:55.611471+0000', 'modified': '2024-04-23T18:24:02.624724+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24521, 'mds_1': 24508, 'mds_2': 24491, 'mds_3': 24487, 'mds_4': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/716138605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 716138605}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 716138605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14604': {'gid': 14604, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/2593314342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 2593314342}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 2593314342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6833/3128313210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 3128313210}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 3128313210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6833/3545031610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 3545031610}, {'type': 'v1', 'addr': '172.21.15.44:6833', 'nonce': 3545031610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6835/659103876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 659103876}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 659103876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.155:6835/698922157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 698922157}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 698922157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.155:6837/980782961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 980782961}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 980782961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.44:6837/728759484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 728759484}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 728759484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.155:6839/3934638415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 3934638415}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 3934638415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6839/2221834529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 2221834529}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 2221834529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24487, 24508, 24469, 24521, 24491]} 2024-04-23T18:25:32.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T18:25:32.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.457+0000 7f9488605640 1 -- 172.21.15.19:0/3406594878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480102a00 msgr2=0x7f9480109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.457+0000 7f9488605640 1 --2- 172.21.15.19:0/3406594878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480102a00 0x7f9480109320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9478009ca0 tx=0x7f947802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.457+0000 7f9488605640 1 -- 172.21.15.19:0/3406594878 shutdown_connections 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.457+0000 7f9488605640 1 --2- 172.21.15.19:0/3406594878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9480102040 0x7f94801024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.457+0000 7f9488605640 1 --2- 172.21.15.19:0/3406594878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480102a00 0x7f9480109320 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.457+0000 7f9488605640 1 --2- 172.21.15.19:0/3406594878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9480100e60 0x7f9480101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.457+0000 7f9488605640 1 -- 172.21.15.19:0/3406594878 >> 172.21.15.19:0/3406594878 conn(0x7f94800fc440 msgr2=0x7f94800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.457+0000 7f9488605640 1 -- 172.21.15.19:0/3406594878 shutdown_connections 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.457+0000 7f9488605640 1 -- 172.21.15.19:0/3406594878 wait complete. 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9488605640 1 Processor -- start 2024-04-23T18:25:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9488605640 1 -- start start 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9488605640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9480100e60 0x7f948006b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9488605640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480102040 0x7f948019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9488605640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9480102a00 0x7f948019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9488605640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f948010c9f0 con 0x7f9480102040 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9488605640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f948010c870 con 0x7f9480102a00 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9488605640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f948010cb70 con 0x7f9480100e60 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9485ba0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480102040 0x7f948019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9485ba0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480102040 0x7f948019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38254/0 (socket says 172.21.15.19:38254) 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9485ba0640 1 -- 172.21.15.19:0/7380246 learned_addr learned my addr 172.21.15.19:0/7380246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f94863a1640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9480100e60 0x7f948006b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9486ba2640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9480102a00 0x7f948019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9485ba0640 1 -- 172.21.15.19:0/7380246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9480100e60 msgr2=0x7f948006b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9485ba0640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9480100e60 0x7f948006b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9485ba0640 1 -- 172.21.15.19:0/7380246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9480102a00 msgr2=0x7f948019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9485ba0640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9480102a00 0x7f948019f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9485ba0640 1 -- 172.21.15.19:0/7380246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948019fd90 con 0x7f9480102040 2024-04-23T18:25:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.458+0000 7f9485ba0640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480102040 0x7f948019ce60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f947000e9f0 tx=0x7f947000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.459+0000 7f94777fe640 1 -- 172.21.15.19:0/7380246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947000cd80 con 0x7f9480102040 2024-04-23T18:25:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.459+0000 7f94777fe640 1 -- 172.21.15.19:0/7380246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94700108e0 con 0x7f9480102040 2024-04-23T18:25:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.459+0000 7f9488605640 1 -- 172.21.15.19:0/7380246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948019ffc0 con 0x7f9480102040 2024-04-23T18:25:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.459+0000 7f94777fe640 1 -- 172.21.15.19:0/7380246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470018980 con 0x7f9480102040 2024-04-23T18:25:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.459+0000 7f9488605640 1 -- 172.21.15.19:0/7380246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801a04e0 con 0x7f9480102040 2024-04-23T18:25:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f94777fe640 1 -- 172.21.15.19:0/7380246 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9470010430 con 0x7f9480102040 2024-04-23T18:25:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f948b3fa740 1 Processor -- start 2024-04-23T18:25:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f948b3fa740 1 -- start start 2024-04-23T18:25:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d4d50 0x55b38a6d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d5710 0x55b38a6dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f948b3fa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b38a638c50 con 0x55b38a6d3b80 2024-04-23T18:25:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f948b3fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b38a5d5830 con 0x55b38a6d5710 2024-04-23T18:25:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f948b3fa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b38a634c10 con 0x55b38a6d4d50 2024-04-23T18:25:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f9486ba2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f9486ba2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38270/0 (socket says 172.21.15.19:38270) 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f9486ba2640 1 -- 172.21.15.19:0/3351354054 learned_addr learned my addr 172.21.15.19:0/3351354054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f94863a1640 1 --2- 172.21.15.19:0/3351354054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d4d50 0x55b38a6d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.462+0000 7f9485ba0640 1 --2- 172.21.15.19:0/3351354054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d5710 0x55b38a6dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f9486ba2640 1 -- 172.21.15.19:0/3351354054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d4d50 msgr2=0x55b38a6d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f9486ba2640 1 --2- 172.21.15.19:0/3351354054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d4d50 0x55b38a6d51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f9486ba2640 1 -- 172.21.15.19:0/3351354054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d5710 msgr2=0x55b38a6dbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f9486ba2640 1 --2- 172.21.15.19:0/3351354054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d5710 0x55b38a6dbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f9486ba2640 1 -- 172.21.15.19:0/3351354054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b38a34b2d0 con 0x55b38a6d3b80 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f94777fe640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f946c0a3bb0 0x7f946c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f9486ba2640 1 --2- 172.21.15.19:0/3351354054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d3fa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f947802ade0 tx=0x7f94780093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2450cf9259945ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f94863a1640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f946c0a3bb0 0x7f946c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f94777fe640 1 -- 172.21.15.19:0/7380246 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f94700d07c0 con 0x7f9480102040 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f9474bfe640 1 -- 172.21.15.19:0/3351354054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478031650 con 0x55b38a6d3b80 2024-04-23T18:25:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f9474bfe640 1 -- 172.21.15.19:0/3351354054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9478036070 con 0x55b38a6d3b80 2024-04-23T18:25:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.463+0000 7f9474bfe640 1 -- 172.21.15.19:0/3351354054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94780045e0 con 0x55b38a6d3b80 2024-04-23T18:25:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f94863a1640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f946c0a3bb0 0x7f946c0a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f947c005d00 tx=0x7f947c005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 -- 172.21.15.19:0/3351354054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 msgr2=0x55b38a6d3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 --2- 172.21.15.19:0/3351354054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d3fa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f947802ade0 tx=0x7f94780093e0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 -- 172.21.15.19:0/3351354054 shutdown_connections 2024-04-23T18:25:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 --2- 172.21.15.19:0/3351354054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d5710 0x55b38a6dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 --2- 172.21.15.19:0/3351354054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d4d50 0x55b38a6d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 --2- 172.21.15.19:0/3351354054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d3fa0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 -- 172.21.15.19:0/3351354054 >> 172.21.15.19:0/3351354054 conn(0x55b38a6cf070 msgr2=0x55b38a6cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 -- 172.21.15.19:0/3351354054 shutdown_connections 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 -- 172.21.15.19:0/3351354054 wait complete. 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 -- start start 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 Processor -- start 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6ddf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d4d50 0x55b38a6de4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.464+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d5710 0x55b38a6e5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f948b3fa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b38a6dc6c0 con 0x55b38a6d3b80 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f948b3fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b38a5d3470 con 0x55b38a6d4d50 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f948b3fa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b38a6dc840 con 0x55b38a6d5710 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f94863a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6ddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f94863a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6ddf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38278/0 (socket says 172.21.15.19:38278) 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f948b3fa740 10 client.0 _ll_register_callbacks cb 0x55b38a5f8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f94863a1640 1 -- 172.21.15.19:0/967746856 learned_addr learned my addr 172.21.15.19:0/967746856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f9485ba0640 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d4d50 0x55b38a6de4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f9486ba2640 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d5710 0x55b38a6e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f94863a1640 1 -- 172.21.15.19:0/967746856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d5710 msgr2=0x55b38a6e5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f94863a1640 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d5710 0x55b38a6e5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f94863a1640 1 -- 172.21.15.19:0/967746856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d4d50 msgr2=0x55b38a6de4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f94863a1640 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d4d50 0x55b38a6de4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f94863a1640 1 -- 172.21.15.19:0/967746856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b38a4156b0 con 0x55b38a6d3b80 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f94863a1640 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6ddf80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f947c00d010 tx=0x7f947c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6694c31fca7b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f9469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f9469ffb640 1 -- 172.21.15.19:0/967746856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c023070 con 0x55b38a6d3b80 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f9469ffb640 1 -- 172.21.15.19:0/967746856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f947c01f860 con 0x55b38a6d3b80 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f948b3fa740 1 -- 172.21.15.19:0/967746856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9478036070 con 0x55b38a6d3b80 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f9469ffb640 1 -- 172.21.15.19:0/967746856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f947c02f050 con 0x55b38a6d3b80 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f948b3fa740 10 client.15015 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.465+0000 7f948b3fa740 1 -- 172.21.15.19:0/967746856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b38a6e8940 con 0x55b38a6d3b80 2024-04-23T18:25:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.466+0000 7f9469ffb640 1 -- 172.21.15.19:0/967746856 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9478036070 con 0x55b38a6d3b80 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.466+0000 7f9469ffb640 1 -- 172.21.15.19:0/967746856 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f947c00f040 con 0x55b38a6d3b80 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.466+0000 7f948b3fa740 10 client.15015 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.466+0000 7f948b3fa740 10 client.15015 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.466+0000 7f948b3fa740 20 client.15015 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.466+0000 7f948b3fa740 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55b38a6eb060 0x55b38a6ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.466+0000 7f948b3fa740 4 client.15015 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.466+0000 7f948b3fa740 1 -- 172.21.15.19:0/967746856 --> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b38a646670 con 0x55b38a6eb060 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.466+0000 7f9485ba0640 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55b38a6eb060 0x55b38a6ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.467+0000 7f9485ba0640 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55b38a6eb060 0x55b38a6ec4d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3aa69f6037b932db server_cookie=d6d952f4d048c6b1 in_seq=0 out_seq=0 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.467+0000 7f9469ffb640 10 client.15015 ms_handle_connect on v2:172.21.15.155:6834/698922157 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f9469ffb640 1 -- 172.21.15.19:0/967746856 <== mds.4 v2:172.21.15.155:6834/698922157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b38a646670 con 0x55b38a6eb060 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f9469ffb640 10 client.15015 handle_command_reply: tid=0 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 client.15015 shutdown 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 -- 172.21.15.19:0/967746856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 msgr2=0x55b38a6ddf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6ddf80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f947c00d010 tx=0x7f947c004730 comp rx=0 tx=0).stop 2024-04-23T18:25:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 -- 172.21.15.19:0/967746856 shutdown_connections 2024-04-23T18:25:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55b38a6eb060 0x55b38a6ec4d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d4d50 0x55b38a6de4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d5710 0x55b38a6e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f9469ffb640 0 client.15015 ms_handle_reset on v2:172.21.15.155:6834/698922157 2024-04-23T18:25:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 --2- 172.21.15.19:0/967746856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6ddf80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 -- 172.21.15.19:0/967746856 >> 172.21.15.19:0/967746856 conn(0x55b38a6cf070 msgr2=0x55b38a6ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 -- 172.21.15.19:0/967746856 shutdown_connections 2024-04-23T18:25:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 1 -- 172.21.15.19:0/967746856 wait complete. 2024-04-23T18:25:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.468+0000 7f948b3fa740 20 client.15015 trim_cache size 0 max 16384 2024-04-23T18:25:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f948b3fa740 1 Processor -- start 2024-04-23T18:25:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f948b3fa740 1 -- start start 2024-04-23T18:25:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d4d50 0x55b38a6d8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d5710 0x55b38a6e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f948b3fa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b38a6ea810 con 0x55b38a6d3b80 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f948b3fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b38a6eaee0 con 0x55b38a6d4d50 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f948b3fa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b38a6eb5b0 con 0x55b38a6d5710 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f94863a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f94863a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38288/0 (socket says 172.21.15.19:38288) 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.486+0000 7f94863a1640 1 -- 172.21.15.19:0/2904763508 learned_addr learned my addr 172.21.15.19:0/2904763508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.487+0000 7f94863a1640 1 -- 172.21.15.19:0/2904763508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d5710 msgr2=0x55b38a6e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.487+0000 7f94863a1640 1 --2- 172.21.15.19:0/2904763508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d5710 0x55b38a6e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.487+0000 7f94863a1640 1 -- 172.21.15.19:0/2904763508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d4d50 msgr2=0x55b38a6d8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.487+0000 7f94863a1640 1 --2- 172.21.15.19:0/2904763508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d4d50 0x55b38a6d8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.487+0000 7f94863a1640 1 -- 172.21.15.19:0/2904763508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b38a6d47d0 con 0x55b38a6d3b80 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.487+0000 7f94863a1640 1 --2- 172.21.15.19:0/2904763508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d8790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c480 tx=0x7f947c005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55c1c444f1aa4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.487+0000 7f946bfff640 1 -- 172.21.15.19:0/2904763508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c023040 con 0x55b38a6d3b80 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.487+0000 7f946bfff640 1 -- 172.21.15.19:0/2904763508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947c03a070 con 0x55b38a6d3b80 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.487+0000 7f946bfff640 1 -- 172.21.15.19:0/2904763508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c027870 con 0x55b38a6d3b80 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 -- 172.21.15.19:0/2904763508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 msgr2=0x55b38a6d8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 --2- 172.21.15.19:0/2904763508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d8790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c480 tx=0x7f947c005930 comp rx=0 tx=0).stop 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 -- 172.21.15.19:0/2904763508 shutdown_connections 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 --2- 172.21.15.19:0/2904763508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d4d50 0x55b38a6d8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 --2- 172.21.15.19:0/2904763508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d5710 0x55b38a6e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 --2- 172.21.15.19:0/2904763508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6d8790 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 -- 172.21.15.19:0/2904763508 >> 172.21.15.19:0/2904763508 conn(0x55b38a6cf070 msgr2=0x55b38a6cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 -- 172.21.15.19:0/2904763508 shutdown_connections 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 -- 172.21.15.19:0/2904763508 wait complete. 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 -- start start 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 Processor -- start 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d4d50 0x55b38a6e1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d5710 0x55b38a6e4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b38a6ec350 con 0x55b38a6d3b80 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b38a6ebc80 con 0x55b38a6d5710 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b38a6eca20 con 0x55b38a6d4d50 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f948b3fa740 10 client.0 _ll_register_callbacks cb 0x7f947c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f94863a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f94863a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6e0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38298/0 (socket says 172.21.15.19:38298) 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f94863a1640 1 -- 172.21.15.19:0/1921009595 learned_addr learned my addr 172.21.15.19:0/1921009595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f9486ba2640 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d5710 0x55b38a6e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.488+0000 7f9485ba0640 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d4d50 0x55b38a6e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f94863a1640 1 -- 172.21.15.19:0/1921009595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d4d50 msgr2=0x55b38a6e1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f94863a1640 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d4d50 0x55b38a6e1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f94863a1640 1 -- 172.21.15.19:0/1921009595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d5710 msgr2=0x55b38a6e4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f94863a1640 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d5710 0x55b38a6e4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f94863a1640 1 -- 172.21.15.19:0/1921009595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b38a6e4660 con 0x55b38a6d3b80 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f94863a1640 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6e0d30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f947c005740 tx=0x7f947c0273b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427d4399d1ce61ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f9469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f9469ffb640 1 -- 172.21.15.19:0/1921009595 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c042020 con 0x55b38a6d3b80 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f9469ffb640 1 -- 172.21.15.19:0/1921009595 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f947c01fa70 con 0x55b38a6d3b80 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f948b3fa740 1 -- 172.21.15.19:0/1921009595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f947c03a070 con 0x55b38a6d3b80 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f9469ffb640 1 -- 172.21.15.19:0/1921009595 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f947c023040 con 0x55b38a6d3b80 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f948b3fa740 10 client.15021 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f948b3fa740 1 -- 172.21.15.19:0/1921009595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b38a751010 con 0x55b38a6d3b80 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f9469ffb640 1 -- 172.21.15.19:0/1921009595 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947c03a070 con 0x55b38a6d3b80 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f9469ffb640 1 -- 172.21.15.19:0/1921009595 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f947c0110d0 con 0x55b38a6d3b80 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f948b3fa740 10 client.15021 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f948b3fa740 10 client.15021 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.489+0000 7f948b3fa740 20 client.15021 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f948b3fa740 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55b38a7556c0 0x55b38a7521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f948b3fa740 4 client.15021 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f948b3fa740 1 -- 172.21.15.19:0/1921009595 --> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] -- command(tid 0: {"prefix": "status"}) -- 0x55b38a6404e0 con 0x55b38a7556c0 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f9485ba0640 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55b38a7556c0 0x55b38a7521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f9485ba0640 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55b38a7556c0 0x55b38a7521d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=55563c974bf0686d server_cookie=550498f8ed9d00c8 in_seq=0 out_seq=0 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f9469ffb640 10 client.15021 ms_handle_connect on v2:172.21.15.155:6834/698922157 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f9469ffb640 1 -- 172.21.15.19:0/1921009595 <== mds.4 v2:172.21.15.155:6834/698922157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b38a6404e0 con 0x55b38a7556c0 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f9469ffb640 10 client.15021 handle_command_reply: tid=0 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f948b3fa740 1 client.15021 shutdown 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f948b3fa740 1 -- 172.21.15.19:0/1921009595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 msgr2=0x55b38a6e0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f948b3fa740 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6e0d30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f947c005740 tx=0x7f947c0273b0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f948b3fa740 1 -- 172.21.15.19:0/1921009595 shutdown_connections 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f948b3fa740 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55b38a7556c0 0x55b38a7521d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.490+0000 7f948b3fa740 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b38a6d5710 0x55b38a6e4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f948b3fa740 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b38a6d4d50 0x55b38a6e1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f948b3fa740 1 --2- 172.21.15.19:0/1921009595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a6d3b80 0x55b38a6e0d30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9469ffb640 0 client.15021 ms_handle_reset on v2:172.21.15.155:6834/698922157 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f948b3fa740 1 -- 172.21.15.19:0/1921009595 >> 172.21.15.19:0/1921009595 conn(0x55b38a6cf070 msgr2=0x55b38a6d8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f948b3fa740 1 -- 172.21.15.19:0/1921009595 shutdown_connections 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f948b3fa740 1 -- 172.21.15.19:0/1921009595 wait complete. 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f948b3fa740 20 client.15021 trim_cache size 0 max 16384 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24469, 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:25:32.490 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:25:32.491 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:25:32.491 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 90.102234385000003, 2024-04-23T18:25:32.491 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-23T18:25:32.491 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 88, 2024-04-23T18:25:32.491 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:25:32.491 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 119.955167366 2024-04-23T18:25:32.491 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:25:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 -- 172.21.15.19:0/7380246 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f946c0a3bb0 msgr2=0x7f946c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f946c0a3bb0 0x7f946c0a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f947c005d00 tx=0x7f947c005c70 comp rx=0 tx=0).stop 2024-04-23T18:25:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 -- 172.21.15.19:0/7380246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480102040 msgr2=0x7f948019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480102040 0x7f948019ce60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f947000e9f0 tx=0x7f947000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 -- 172.21.15.19:0/7380246 shutdown_connections 2024-04-23T18:25:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f946c0a3bb0 0x7f946c0a6070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9480102a00 0x7f948019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9480102040 0x7f948019ce60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 --2- 172.21.15.19:0/7380246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9480100e60 0x7f948006b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 -- 172.21.15.19:0/7380246 >> 172.21.15.19:0/7380246 conn(0x7f94800fc440 msgr2=0x7f9480105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 -- 172.21.15.19:0/7380246 shutdown_connections 2024-04-23T18:25:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.491+0000 7f9488605640 1 -- 172.21.15.19:0/7380246 wait complete. 2024-04-23T18:25:32.495 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T18:25:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.603+0000 7f352156f640 1 -- 172.21.15.19:0/4174804904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f351c102040 msgr2=0x7f351c1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.603+0000 7f352156f640 1 --2- 172.21.15.19:0/4174804904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f351c102040 0x7f351c1024c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f350c009900 tx=0x7f350c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 -- 172.21.15.19:0/4174804904 shutdown_connections 2024-04-23T18:25:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 --2- 172.21.15.19:0/4174804904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c102a00 0x7f351c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 --2- 172.21.15.19:0/4174804904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f351c102040 0x7f351c1024c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 --2- 172.21.15.19:0/4174804904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f351c100e60 0x7f351c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 -- 172.21.15.19:0/4174804904 >> 172.21.15.19:0/4174804904 conn(0x7f351c0fc440 msgr2=0x7f351c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 -- 172.21.15.19:0/4174804904 shutdown_connections 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 -- 172.21.15.19:0/4174804904 wait complete. 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 Processor -- start 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 -- start start 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c100e60 0x7f351c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f351c102040 0x7f351c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.604+0000 7f352156f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f351c102a00 0x7f351c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f352156f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f351c10ca10 con 0x7f351c102040 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f352156f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f351c10c890 con 0x7f351c100e60 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f352156f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f351c10cb90 con 0x7f351c102a00 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f351c102040 0x7f351c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f351c102040 0x7f351c19cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38318/0 (socket says 172.21.15.19:38318) 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351a7fc640 1 -- 172.21.15.19:0/690434889 learned_addr learned my addr 172.21.15.19:0/690434889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351affd640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c100e60 0x7f351c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351b7fe640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f351c102a00 0x7f351c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351a7fc640 1 -- 172.21.15.19:0/690434889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f351c102a00 msgr2=0x7f351c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351a7fc640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f351c102a00 0x7f351c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351a7fc640 1 -- 172.21.15.19:0/690434889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c100e60 msgr2=0x7f351c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351a7fc640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c100e60 0x7f351c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351a7fc640 1 -- 172.21.15.19:0/690434889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c19fde0 con 0x7f351c102040 2024-04-23T18:25:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f351a7fc640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f351c102040 0x7f351c19cee0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f350c005940 tx=0x7f350c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f3503fff640 1 -- 172.21.15.19:0/690434889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c03f030 con 0x7f351c102040 2024-04-23T18:25:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f352156f640 1 -- 172.21.15.19:0/690434889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c1a0070 con 0x7f351c102040 2024-04-23T18:25:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f3503fff640 1 -- 172.21.15.19:0/690434889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350c003cb0 con 0x7f351c102040 2024-04-23T18:25:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f3503fff640 1 -- 172.21.15.19:0/690434889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c035070 con 0x7f351c102040 2024-04-23T18:25:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.605+0000 7f352156f640 1 -- 172.21.15.19:0/690434889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c1a05b0 con 0x7f351c102040 2024-04-23T18:25:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.608+0000 7f3503fff640 1 -- 172.21.15.19:0/690434889 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f350c042030 con 0x7f351c102040 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.608+0000 7f3524366740 1 Processor -- start 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.608+0000 7f3524366740 1 -- start start 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.608+0000 7f3524366740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.608+0000 7f3524366740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.608+0000 7f3524366740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80297b20 0x555a8029e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f3524366740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555a801f6c50 con 0x555a80295f90 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f3524366740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555a80193830 con 0x555a80297160 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f3524366740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a801f2c10 con 0x555a80297b20 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38334/0 (socket says 172.21.15.19:38334) 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351b7fe640 1 -- 172.21.15.19:0/2686502333 learned_addr learned my addr 172.21.15.19:0/2686502333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351affd640 1 --2- 172.21.15.19:0/2686502333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f3503fff640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f35040a3bb0 0x7f35040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351b7fe640 1 -- 172.21.15.19:0/2686502333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80297b20 msgr2=0x555a8029e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351b7fe640 1 --2- 172.21.15.19:0/2686502333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80297b20 0x555a8029e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351b7fe640 1 -- 172.21.15.19:0/2686502333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 msgr2=0x555a802975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351b7fe640 1 --2- 172.21.15.19:0/2686502333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351b7fe640 1 -- 172.21.15.19:0/2686502333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a7ff092d0 con 0x555a80295f90 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f3503fff640 1 -- 172.21.15.19:0/690434889 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f350c0f0850 con 0x7f351c102040 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351b7fe640 1 --2- 172.21.15.19:0/2686502333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802963b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3514009fc0 tx=0x7f3514010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7accdd085615c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f351b7fe640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f35040a3bb0 0x7f35040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f35017fa640 1 -- 172.21.15.19:0/2686502333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514003cd0 con 0x555a80295f90 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f35017fa640 1 -- 172.21.15.19:0/2686502333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35140045a0 con 0x555a80295f90 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.609+0000 7f35017fa640 1 -- 172.21.15.19:0/2686502333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351401b420 con 0x555a80295f90 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f3524366740 1 -- 172.21.15.19:0/2686502333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 msgr2=0x555a802963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f351b7fe640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f35040a3bb0 0x7f35040a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f351401a000 tx=0x7f351401bb40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f3524366740 1 --2- 172.21.15.19:0/2686502333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802963b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3514009fc0 tx=0x7f3514010860 comp rx=0 tx=0).stop 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f3524366740 1 -- 172.21.15.19:0/2686502333 shutdown_connections 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f3524366740 1 --2- 172.21.15.19:0/2686502333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f3524366740 1 --2- 172.21.15.19:0/2686502333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80297b20 0x555a8029e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f3524366740 1 --2- 172.21.15.19:0/2686502333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802963b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f3524366740 1 -- 172.21.15.19:0/2686502333 >> 172.21.15.19:0/2686502333 conn(0x555a8028d070 msgr2=0x555a8028d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f3524366740 1 -- 172.21.15.19:0/2686502333 shutdown_connections 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f3524366740 1 -- 172.21.15.19:0/2686502333 wait complete. 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.610+0000 7f3524366740 1 -- start start 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f3524366740 1 Processor -- start 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f3524366740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802a0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f3524366740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f3524366740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80297b20 0x555a802a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f3524366740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555a8029eb30 con 0x555a80295f90 2024-04-23T18:25:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f3524366740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555a80191470 con 0x555a80297160 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f3524366740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a8029ecb0 con 0x555a80297b20 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f351affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f351affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38342/0 (socket says 172.21.15.19:38342) 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f3524366740 10 client.0 _ll_register_callbacks cb 0x555a801b6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f351affd640 1 -- 172.21.15.19:0/3468049072 learned_addr learned my addr 172.21.15.19:0/3468049072 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f351a7fc640 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f351affd640 1 -- 172.21.15.19:0/3468049072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80297b20 msgr2=0x555a802a79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f351affd640 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80297b20 0x555a802a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f351affd640 1 -- 172.21.15.19:0/3468049072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 msgr2=0x555a802a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f351affd640 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f351affd640 1 -- 172.21.15.19:0/3468049072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a7ffd36b0 con 0x555a80295f90 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f351affd640 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802a0470 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3510012720 tx=0x7f3510012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86fe109d0829964 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f34feff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f34feff5640 1 -- 172.21.15.19:0/3468049072 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510004050 con 0x555a80295f90 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f34feff5640 1 -- 172.21.15.19:0/3468049072 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f35100041f0 con 0x555a80295f90 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.611+0000 7f3524366740 1 -- 172.21.15.19:0/3468049072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35140045a0 con 0x555a80295f90 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f34feff5640 1 -- 172.21.15.19:0/3468049072 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f35100043f0 con 0x555a80295f90 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f3524366740 10 client.15033 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f3524366740 1 -- 172.21.15.19:0/3468049072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a802aae30 con 0x555a80295f90 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f34feff5640 1 -- 172.21.15.19:0/3468049072 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35140045a0 con 0x555a80295f90 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f34feff5640 1 -- 172.21.15.19:0/3468049072 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3510003b20 con 0x555a80295f90 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f3524366740 10 client.15033 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f3524366740 10 client.15033 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f3524366740 20 client.15033 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f3524366740 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x555a802ad540 0x555a802ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f3524366740 4 client.15033 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f3524366740 1 -- 172.21.15.19:0/3468049072 --> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a80204670 con 0x555a802ad540 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.612+0000 7f351a7fc640 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x555a802ad540 0x555a802ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.613+0000 7f351a7fc640 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x555a802ad540 0x555a802ae9b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fcc3b8f6b83db1cf server_cookie=93c9c5ce788e3c54 in_seq=0 out_seq=0 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.613+0000 7f34feff5640 10 client.15033 ms_handle_connect on v2:172.21.15.155:6836/980782961 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.613+0000 7f34feff5640 1 -- 172.21.15.19:0/3468049072 <== mds.3 v2:172.21.15.155:6836/980782961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a80204670 con 0x555a802ad540 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.613+0000 7f34feff5640 10 client.15033 handle_command_reply: tid=0 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 client.15033 shutdown 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 -- 172.21.15.19:0/3468049072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 msgr2=0x555a802a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802a0470 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3510012720 tx=0x7f3510012be0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 -- 172.21.15.19:0/3468049072 shutdown_connections 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x555a802ad540 0x555a802ae9b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80297b20 0x555a802a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f34feff5640 0 client.15033 ms_handle_reset on v2:172.21.15.155:6836/980782961 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 --2- 172.21.15.19:0/3468049072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80295f90 0x555a802a0470 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 -- 172.21.15.19:0/3468049072 >> 172.21.15.19:0/3468049072 conn(0x555a8028d070 msgr2=0x555a8028c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 -- 172.21.15.19:0/3468049072 shutdown_connections 2024-04-23T18:25:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 1 -- 172.21.15.19:0/3468049072 wait complete. 2024-04-23T18:25:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.614+0000 7f3524366740 20 client.15033 trim_cache size 0 max 16384 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f3524366740 1 Processor -- start 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f3524366740 1 -- start start 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f3524366740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 0x555a8028ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f3524366740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297160 0x555a802a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f3524366740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297b20 0x555a80290490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f3524366740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555a802accf0 con 0x555a80297160 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f3524366740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555a802ad3c0 con 0x555a80297b20 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f3524366740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a802ada90 con 0x555a80295f90 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f351a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297160 0x555a802a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f351a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297160 0x555a802a79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38346/0 (socket says 172.21.15.19:38346) 2024-04-23T18:25:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f351a7fc640 1 -- 172.21.15.19:0/2940940469 learned_addr learned my addr 172.21.15.19:0/2940940469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f351b7fe640 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297b20 0x555a80290490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f351affd640 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 0x555a8028ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f351a7fc640 1 -- 172.21.15.19:0/2940940469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 msgr2=0x555a8028ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f351a7fc640 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 0x555a8028ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f351a7fc640 1 -- 172.21.15.19:0/2940940469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297b20 msgr2=0x555a80290490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.632+0000 7f351a7fc640 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297b20 0x555a80290490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f351a7fc640 1 -- 172.21.15.19:0/2940940469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8029b320 con 0x555a80297160 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f351b7fe640 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297b20 0x555a80290490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f351affd640 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 0x555a8028ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f351a7fc640 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297160 0x555a802a79b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f350c0b8fa0 tx=0x7f350c031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2e3f0145a0521 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3500ff9640 1 -- 172.21.15.19:0/2940940469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c049040 con 0x555a80297160 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3500ff9640 1 -- 172.21.15.19:0/2940940469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350c0bcb40 con 0x555a80297160 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3500ff9640 1 -- 172.21.15.19:0/2940940469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c0dd070 con 0x555a80297160 2024-04-23T18:25:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3524366740 1 -- 172.21.15.19:0/2940940469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297160 msgr2=0x555a802a79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3524366740 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297160 0x555a802a79b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f350c0b8fa0 tx=0x7f350c031980 comp rx=0 tx=0).stop 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3524366740 1 -- 172.21.15.19:0/2940940469 shutdown_connections 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3524366740 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297b20 0x555a80290490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3524366740 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297160 0x555a802a79b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3524366740 1 --2- 172.21.15.19:0/2940940469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 0x555a8028ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3524366740 1 -- 172.21.15.19:0/2940940469 >> 172.21.15.19:0/2940940469 conn(0x555a8028d070 msgr2=0x555a8028d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.633+0000 7f3524366740 1 -- 172.21.15.19:0/2940940469 shutdown_connections 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f3524366740 1 -- 172.21.15.19:0/2940940469 wait complete. 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f3524366740 1 -- start start 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f3524366740 1 Processor -- start 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f3524366740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 0x555a802a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f3524366740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802a3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f3524366740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297b20 0x555a802a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f3524366740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555a802ae830 con 0x555a80297b20 2024-04-23T18:25:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f3524366740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555a802ae160 con 0x555a80297160 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f3524366740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a802aef00 con 0x555a80295f90 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f3524366740 10 client.0 _ll_register_callbacks cb 0x7f3510009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f351b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297b20 0x555a802a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f351b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297b20 0x555a802a62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38356/0 (socket says 172.21.15.19:38356) 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f351b7fe640 1 -- 172.21.15.19:0/2483060297 learned_addr learned my addr 172.21.15.19:0/2483060297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f351b7fe640 1 -- 172.21.15.19:0/2483060297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 msgr2=0x555a802a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f351affd640 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 0x555a802a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f351b7fe640 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 0x555a802a2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f351b7fe640 1 -- 172.21.15.19:0/2483060297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 msgr2=0x555a802a3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f351b7fe640 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.634+0000 7f351b7fe640 1 -- 172.21.15.19:0/2483060297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a802a68a0 con 0x555a80297b20 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f351b7fe640 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297b20 0x555a802a62b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3514006010 tx=0x7f3514018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64acdb398e277aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f34feff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f34feff5640 1 -- 172.21.15.19:0/2483060297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514037060 con 0x555a80297b20 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f34feff5640 1 -- 172.21.15.19:0/2483060297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3514003970 con 0x555a80297b20 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f3524366740 1 -- 172.21.15.19:0/2483060297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f350c0bcb40 con 0x555a80297b20 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f34feff5640 1 -- 172.21.15.19:0/2483060297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3514009c50 con 0x555a80297b20 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f3524366740 10 client.15039 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f3524366740 1 -- 172.21.15.19:0/2483060297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a802a7bb0 con 0x555a80297b20 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f34feff5640 1 -- 172.21.15.19:0/2483060297 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350c0bcb40 con 0x555a80297b20 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f34feff5640 1 -- 172.21.15.19:0/2483060297 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3514011500 con 0x555a80297b20 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f3524366740 10 client.15039 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f3524366740 10 client.15039 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f3524366740 20 client.15039 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f3524366740 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x555a80317ae0 0x555a803167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f3524366740 4 client.15039 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.635+0000 7f3524366740 1 -- 172.21.15.19:0/2483060297 --> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] -- command(tid 0: {"prefix": "status"}) -- 0x555a801fe4e0 con 0x555a80317ae0 2024-04-23T18:25:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f351affd640 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x555a80317ae0 0x555a803167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f34feff5640 10 client.15039 ms_handle_connect on v2:172.21.15.155:6836/980782961 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f351affd640 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x555a80317ae0 0x555a803167f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=74f56bc36518f7e server_cookie=efbb2d639458d14d in_seq=0 out_seq=0 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f34feff5640 1 -- 172.21.15.19:0/2483060297 <== mds.3 v2:172.21.15.155:6836/980782961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555a801fe4e0 con 0x555a80317ae0 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f34feff5640 10 client.15039 handle_command_reply: tid=0 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 client.15039 shutdown 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 -- 172.21.15.19:0/2483060297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297b20 msgr2=0x555a802a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297b20 0x555a802a62b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3514006010 tx=0x7f3514018000 comp rx=0 tx=0).stop 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 -- 172.21.15.19:0/2483060297 shutdown_connections 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x555a80317ae0 0x555a803167f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a80297160 0x555a802a3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f34feff5640 0 client.15039 ms_handle_reset on v2:172.21.15.155:6836/980782961 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a80297b20 0x555a802a62b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 --2- 172.21.15.19:0/2483060297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a80295f90 0x555a802a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 -- 172.21.15.19:0/2483060297 >> 172.21.15.19:0/2483060297 conn(0x555a8028d070 msgr2=0x555a8028b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 -- 172.21.15.19:0/2483060297 shutdown_connections 2024-04-23T18:25:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 1 -- 172.21.15.19:0/2483060297 wait complete. 2024-04-23T18:25:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.636+0000 7f3524366740 20 client.15039 trim_cache size 0 max 16384 2024-04-23T18:25:32.637 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:25:32.637 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:32.637 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-04-23T18:25:32.637 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24487, 2024-04-23T18:25:32.637 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:25:32.637 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:25:32.637 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:25:32.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 91.253403156000005, 2024-04-23T18:25:32.638 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-23T18:25:32.638 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 88, 2024-04-23T18:25:32.638 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:25:32.638 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 117.154406012 2024-04-23T18:25:32.638 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 -- 172.21.15.19:0/690434889 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f35040a3bb0 msgr2=0x7f35040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f35040a3bb0 0x7f35040a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f351401a000 tx=0x7f351401bb40 comp rx=0 tx=0).stop 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 -- 172.21.15.19:0/690434889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f351c102040 msgr2=0x7f351c19cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f351c102040 0x7f351c19cee0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f350c005940 tx=0x7f350c0042f0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 -- 172.21.15.19:0/690434889 shutdown_connections 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f35040a3bb0 0x7f35040a6070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f351c102a00 0x7f351c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f351c102040 0x7f351c19cee0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 --2- 172.21.15.19:0/690434889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f351c100e60 0x7f351c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 -- 172.21.15.19:0/690434889 >> 172.21.15.19:0/690434889 conn(0x7f351c0fc440 msgr2=0x7f351c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 -- 172.21.15.19:0/690434889 shutdown_connections 2024-04-23T18:25:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.637+0000 7f352156f640 1 -- 172.21.15.19:0/690434889 wait complete. 2024-04-23T18:25:32.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:32 smithi019 ceph-mon[29215]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-23T18:25:32.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:32 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1464072296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:32.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:32 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2471497396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:32.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:32 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2944004340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:32.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:25:32.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.752+0000 7f23f6e05640 1 -- 172.21.15.19:0/1154147232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23f0101fe0 msgr2=0x7f23f0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.752+0000 7f23f6e05640 1 --2- 172.21.15.19:0/1154147232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23f0101fe0 0x7f23f0102460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009900 tx=0x7f23dc02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.752+0000 7f23f6e05640 1 -- 172.21.15.19:0/1154147232 shutdown_connections 2024-04-23T18:25:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.752+0000 7f23f6e05640 1 --2- 172.21.15.19:0/1154147232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f01029a0 0x7f23f01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.752+0000 7f23f6e05640 1 --2- 172.21.15.19:0/1154147232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23f0101fe0 0x7f23f0102460 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.752+0000 7f23f6e05640 1 --2- 172.21.15.19:0/1154147232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f0100e90 0x7f23f01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.752+0000 7f23f6e05640 1 -- 172.21.15.19:0/1154147232 >> 172.21.15.19:0/1154147232 conn(0x7f23f00fc400 msgr2=0x7f23f00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.752+0000 7f23f6e05640 1 -- 172.21.15.19:0/1154147232 shutdown_connections 2024-04-23T18:25:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.752+0000 7f23f6e05640 1 -- 172.21.15.19:0/1154147232 wait complete. 2024-04-23T18:25:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f6e05640 1 Processor -- start 2024-04-23T18:25:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f6e05640 1 -- start start 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f6e05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23f0100e90 0x7f23f0104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f6e05640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f0101fe0 0x7f23f0103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f6e05640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f01029a0 0x7f23f0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f6e05640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23f010c9d0 con 0x7f23f0100e90 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f6e05640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23f010c850 con 0x7f23f01029a0 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f6e05640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23f010cb50 con 0x7f23f0101fe0 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f4ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23f0100e90 0x7f23f0104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f4ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23f0100e90 0x7f23f0104dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38380/0 (socket says 172.21.15.19:38380) 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23f4ba1640 1 -- 172.21.15.19:0/3258572909 learned_addr learned my addr 172.21.15.19:0/3258572909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.753+0000 7f23effff640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f0101fe0 0x7f23f0103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23f4ba1640 1 -- 172.21.15.19:0/3258572909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f0101fe0 msgr2=0x7f23f0103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f0101fe0 0x7f23f0103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23f4ba1640 1 -- 172.21.15.19:0/3258572909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f01029a0 msgr2=0x7f23f0103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f01029a0 0x7f23f0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23f4ba1640 1 -- 172.21.15.19:0/3258572909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f0104250 con 0x7f23f0100e90 2024-04-23T18:25:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23f0100e90 0x7f23f0104dc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f23e400c920 tx=0x7f23e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23edffb640 1 -- 172.21.15.19:0/3258572909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4007c70 con 0x7f23f0100e90 2024-04-23T18:25:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23edffb640 1 -- 172.21.15.19:0/3258572909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e4007e10 con 0x7f23f0100e90 2024-04-23T18:25:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23f6e05640 1 -- 172.21.15.19:0/3258572909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f01a22f0 con 0x7f23f0100e90 2024-04-23T18:25:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23edffb640 1 -- 172.21.15.19:0/3258572909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4018820 con 0x7f23f0100e90 2024-04-23T18:25:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.754+0000 7f23f6e05640 1 -- 172.21.15.19:0/3258572909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f01a2810 con 0x7f23f0100e90 2024-04-23T18:25:32.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.756+0000 7f23edffb640 1 -- 172.21.15.19:0/3258572909 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f23e400f450 con 0x7f23f0100e90 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f9bf9740 1 Processor -- start 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f9bf9740 1 -- start start 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23edffb640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23e00a3bb0 0x7f23e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511ab80 0x55f53511afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511bd50 0x55f53511c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f535122fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f53507fc50 con 0x55f53511c710 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f53501c830 con 0x55f53511ab80 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f53507bc10 con 0x55f53511bd50 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23edffb640 1 -- 172.21.15.19:0/3258572909 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f23e40d0510 con 0x7f23f0100e90 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f53a2640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23e00a3bb0 0x7f23e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f53a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f535122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f53a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f535122fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38392/0 (socket says 172.21.15.19:38392) 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23effff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511bd50 0x55f53511c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f53a2640 1 -- 172.21.15.19:0/1620934448 learned_addr learned my addr 172.21.15.19:0/1620934448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.757+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/1620934448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511ab80 0x55f53511afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23f53a2640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23e00a3bb0 0x7f23e00a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f23e8013510 tx=0x7f23e8013910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23effff640 1 -- 172.21.15.19:0/1620934448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511ab80 msgr2=0x55f53511afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23effff640 1 --2- 172.21.15.19:0/1620934448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511ab80 0x55f53511afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23effff640 1 -- 172.21.15.19:0/1620934448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 msgr2=0x55f535122fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23effff640 1 --2- 172.21.15.19:0/1620934448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f535122fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23effff640 1 -- 172.21.15.19:0/1620934448 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f534d922d0 con 0x55f53511bd50 2024-04-23T18:25:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23f53a2640 1 --2- 172.21.15.19:0/1620934448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f535122fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23effff640 1 --2- 172.21.15.19:0/1620934448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511bd50 0x55f53511c1d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23dc006010 tx=0x7f23dc031de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=119b24d4fab1f848 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23daffd640 1 -- 172.21.15.19:0/1620934448 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc035030 con 0x55f53511bd50 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23daffd640 1 -- 172.21.15.19:0/1620934448 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23dc040070 con 0x55f53511bd50 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.758+0000 7f23daffd640 1 -- 172.21.15.19:0/1620934448 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc003ac0 con 0x55f53511bd50 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 -- 172.21.15.19:0/1620934448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511bd50 msgr2=0x55f53511c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/1620934448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511bd50 0x55f53511c1d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23dc006010 tx=0x7f23dc031de0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 -- 172.21.15.19:0/1620934448 shutdown_connections 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/1620934448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f535122fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/1620934448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511bd50 0x55f53511c1d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/1620934448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511ab80 0x55f53511afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 -- 172.21.15.19:0/1620934448 >> 172.21.15.19:0/1620934448 conn(0x55f535116070 msgr2=0x55f535116470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 -- 172.21.15.19:0/1620934448 shutdown_connections 2024-04-23T18:25:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 -- 172.21.15.19:0/1620934448 wait complete. 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 -- start start 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.759+0000 7f23f9bf9740 1 Processor -- start 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f535124fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f535125510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 0x55f53512c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f5351236c0 con 0x55f53511ab80 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f53501a470 con 0x55f53511bd50 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f535123840 con 0x55f53511c710 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f9bf9740 10 client.0 _ll_register_callbacks cb 0x55f53503ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f4ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f535124fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f4ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f535124fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38408/0 (socket says 172.21.15.19:38408) 2024-04-23T18:25:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f4ba1640 1 -- 172.21.15.19:0/2753106489 learned_addr learned my addr 172.21.15.19:0/2753106489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23effff640 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f535125510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f53a2640 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 0x55f53512c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f4ba1640 1 -- 172.21.15.19:0/2753106489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 msgr2=0x55f53512c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 0x55f53512c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f4ba1640 1 -- 172.21.15.19:0/2753106489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 msgr2=0x55f535125510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f535125510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f4ba1640 1 -- 172.21.15.19:0/2753106489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f534e5c6b0 con 0x55f53511ab80 2024-04-23T18:25:32.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f535124fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f23e40991d0 tx=0x7f23e40082a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39010111fdd742f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23cffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23cffff640 1 -- 172.21.15.19:0/2753106489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4008980 con 0x55f53511ab80 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23cffff640 1 -- 172.21.15.19:0/2753106489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f23e400fa90 con 0x55f53511ab80 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.760+0000 7f23f9bf9740 1 -- 172.21.15.19:0/2753106489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23dc040070 con 0x55f53511ab80 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23cffff640 1 -- 172.21.15.19:0/2753106489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f23e409c800 con 0x55f53511ab80 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23f9bf9740 10 client.15051 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23f9bf9740 1 -- 172.21.15.19:0/2753106489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f53512f990 con 0x55f53511ab80 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23cffff640 1 -- 172.21.15.19:0/2753106489 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23dc040070 con 0x55f53511ab80 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23cffff640 1 -- 172.21.15.19:0/2753106489 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23e40b3030 con 0x55f53511ab80 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23f9bf9740 10 client.15051 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23f9bf9740 10 client.15051 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:25:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23f9bf9740 20 client.15051 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x55f535132000 0x55f535133470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23f9bf9740 4 client.15051 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23f9bf9740 1 -- 172.21.15.19:0/2753106489 --> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f53508d670 con 0x55f535132000 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.761+0000 7f23effff640 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x55f535132000 0x55f535133470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.762+0000 7f23effff640 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x55f535132000 0x55f535133470 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=996e773360f03850 server_cookie=f607f50d30b68ce8 in_seq=0 out_seq=0 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.762+0000 7f23cffff640 10 client.15051 ms_handle_connect on v2:172.21.15.44:6836/728759484 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.762+0000 7f23cffff640 1 -- 172.21.15.19:0/2753106489 <== mds.2 v2:172.21.15.44:6836/728759484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f53508d670 con 0x55f535132000 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23cffff640 10 client.15051 handle_command_reply: tid=0 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 client.15051 shutdown 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 -- 172.21.15.19:0/2753106489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 msgr2=0x55f535124fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f535124fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f23e40991d0 tx=0x7f23e40082a0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 -- 172.21.15.19:0/2753106489 shutdown_connections 2024-04-23T18:25:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x55f535132000 0x55f535133470 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f535125510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 0x55f53512c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/2753106489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f535124fd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 -- 172.21.15.19:0/2753106489 >> 172.21.15.19:0/2753106489 conn(0x55f535116070 msgr2=0x55f5351151d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23cffff640 0 client.15051 ms_handle_reset on v2:172.21.15.44:6836/728759484 2024-04-23T18:25:32.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 -- 172.21.15.19:0/2753106489 shutdown_connections 2024-04-23T18:25:32.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 1 -- 172.21.15.19:0/2753106489 wait complete. 2024-04-23T18:25:32.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.763+0000 7f23f9bf9740 20 client.15051 trim_cache size 0 max 16384 2024-04-23T18:25:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.781+0000 7f23f9bf9740 1 Processor -- start 2024-04-23T18:25:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.781+0000 7f23f9bf9740 1 -- start start 2024-04-23T18:25:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.781+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f53511c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.781+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f53512c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.781+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 0x55f5351110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.781+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f5351317b0 con 0x55f53511ab80 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.781+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f535131e80 con 0x55f53511bd50 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.781+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f535132550 con 0x55f53511c710 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.781+0000 7f23f4ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f53511c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.781+0000 7f23f4ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f53511c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38424/0 (socket says 172.21.15.19:38424) 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23f4ba1640 1 -- 172.21.15.19:0/1365070346 learned_addr learned my addr 172.21.15.19:0/1365070346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23f4ba1640 1 -- 172.21.15.19:0/1365070346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 msgr2=0x55f5351110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23f53a2640 1 --2- 172.21.15.19:0/1365070346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 0x55f5351110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/1365070346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 0x55f5351110b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23f4ba1640 1 -- 172.21.15.19:0/1365070346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 msgr2=0x55f53512c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23effff640 1 --2- 172.21.15.19:0/1365070346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f53512c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/1365070346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f53512c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23f4ba1640 1 -- 172.21.15.19:0/1365070346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f535116d40 con 0x55f53511ab80 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23f53a2640 1 --2- 172.21.15.19:0/1365070346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 0x55f5351110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/1365070346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f53511c130 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f23e40ce070 tx=0x7f23e409c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa04a8e987cc53e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23da7fc640 1 -- 172.21.15.19:0/1365070346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40a0bb0 con 0x55f53511ab80 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23da7fc640 1 -- 172.21.15.19:0/1365070346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e400fa90 con 0x55f53511ab80 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.782+0000 7f23da7fc640 1 -- 172.21.15.19:0/1365070346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40a4fe0 con 0x55f53511ab80 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 -- 172.21.15.19:0/1365070346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 msgr2=0x55f53511c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/1365070346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f53511c130 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f23e40ce070 tx=0x7f23e409c5d0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 -- 172.21.15.19:0/1365070346 shutdown_connections 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/1365070346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f53512c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/1365070346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511c710 0x55f5351110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/1365070346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511ab80 0x55f53511c130 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 -- 172.21.15.19:0/1365070346 >> 172.21.15.19:0/1365070346 conn(0x55f535116070 msgr2=0x55f535116450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 -- 172.21.15.19:0/1365070346 shutdown_connections 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 -- 172.21.15.19:0/1365070346 wait complete. 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 -- start start 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.783+0000 7f23f9bf9740 1 Processor -- start 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511ab80 0x55f535127c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f5351281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f9bf9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f53512b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f5351332f0 con 0x55f53511c710 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f535132c20 con 0x55f53511bd50 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f9bf9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f5351339c0 con 0x55f53511ab80 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f9bf9740 10 client.0 _ll_register_callbacks cb 0x7f23e409c1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f53a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f53512b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f53a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f53512b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38432/0 (socket says 172.21.15.19:38432) 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f53a2640 1 -- 172.21.15.19:0/3930300689 learned_addr learned my addr 172.21.15.19:0/3930300689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23effff640 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f5351281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511ab80 0x55f535127c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f53a2640 1 -- 172.21.15.19:0/3930300689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511ab80 msgr2=0x55f535127c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f53a2640 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511ab80 0x55f535127c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f53a2640 1 -- 172.21.15.19:0/3930300689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 msgr2=0x55f5351281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f53a2640 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f5351281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f53a2640 1 -- 172.21.15.19:0/3930300689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f53512b720 con 0x55f53511c710 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f53a2640 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f53512b130 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f23e8010fc0 tx=0x7f23e8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fe20ff905997c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23cffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23cffff640 1 -- 172.21.15.19:0/3930300689 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e802d060 con 0x55f53511c710 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23cffff640 1 -- 172.21.15.19:0/3930300689 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f23e801e870 con 0x55f53511c710 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23f9bf9740 1 -- 172.21.15.19:0/3930300689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e400fa90 con 0x55f53511c710 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.784+0000 7f23cffff640 1 -- 172.21.15.19:0/3930300689 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f23e801ea70 con 0x55f53511c710 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23f9bf9740 10 client.15057 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23f9bf9740 1 -- 172.21.15.19:0/3930300689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f535197fb0 con 0x55f53511c710 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23cffff640 1 -- 172.21.15.19:0/3930300689 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e400fa90 con 0x55f53511c710 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23cffff640 1 -- 172.21.15.19:0/3930300689 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23e8027030 con 0x55f53511c710 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23f9bf9740 10 client.15057 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23f9bf9740 10 client.15057 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23f9bf9740 20 client.15057 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x55f53519cbf0 0x55f53519b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23f9bf9740 4 client.15057 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23f9bf9740 1 -- 172.21.15.19:0/3930300689 --> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] -- command(tid 0: {"prefix": "status"}) -- 0x55f5350874e0 con 0x55f53519cbf0 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.785+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x55f53519cbf0 0x55f53519b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23cffff640 10 client.15057 ms_handle_connect on v2:172.21.15.44:6836/728759484 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f4ba1640 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x55f53519cbf0 0x55f53519b900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6837bea326ac3743 server_cookie=6fe3848344955387 in_seq=0 out_seq=0 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23cffff640 1 -- 172.21.15.19:0/3930300689 <== mds.2 v2:172.21.15.44:6836/728759484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f5350874e0 con 0x55f53519cbf0 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23cffff640 10 client.15057 handle_command_reply: tid=0 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 client.15057 shutdown 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 -- 172.21.15.19:0/3930300689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 msgr2=0x55f53512b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f53512b130 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f23e8010fc0 tx=0x7f23e8015040 comp rx=0 tx=0).stop 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 -- 172.21.15.19:0/3930300689 shutdown_connections 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x55f53519cbf0 0x55f53519b900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f53511bd50 0x55f5351281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23cffff640 0 client.15057 ms_handle_reset on v2:172.21.15.44:6836/728759484 2024-04-23T18:25:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f53511c710 0x55f53512b130 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 --2- 172.21.15.19:0/3930300689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f53511ab80 0x55f535127c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 -- 172.21.15.19:0/3930300689 >> 172.21.15.19:0/3930300689 conn(0x55f535116070 msgr2=0x55f535116450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 -- 172.21.15.19:0/3930300689 shutdown_connections 2024-04-23T18:25:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 1 -- 172.21.15.19:0/3930300689 wait complete. 2024-04-23T18:25:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.786+0000 7f23f9bf9740 20 client.15057 trim_cache size 0 max 16384 2024-04-23T18:25:32.787 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:25:32.787 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:32.787 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-04-23T18:25:32.787 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24491, 2024-04-23T18:25:32.788 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:25:32.788 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:25:32.788 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:25:32.788 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 92.412772614999994, 2024-04-23T18:25:32.788 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-23T18:25:32.788 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 88, 2024-04-23T18:25:32.788 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:25:32.788 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 114.35404164400001 2024-04-23T18:25:32.788 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:25:32.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 -- 172.21.15.19:0/3258572909 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23e00a3bb0 msgr2=0x7f23e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23e00a3bb0 0x7f23e00a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f23e8013510 tx=0x7f23e8013910 comp rx=0 tx=0).stop 2024-04-23T18:25:32.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 -- 172.21.15.19:0/3258572909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23f0100e90 msgr2=0x7f23f0104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23f0100e90 0x7f23f0104dc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f23e400c920 tx=0x7f23e400cde0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 -- 172.21.15.19:0/3258572909 shutdown_connections 2024-04-23T18:25:32.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f23e00a3bb0 0x7f23e00a6070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f01029a0 0x7f23f0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f0101fe0 0x7f23f0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 --2- 172.21.15.19:0/3258572909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23f0100e90 0x7f23f0104dc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 -- 172.21.15.19:0/3258572909 >> 172.21.15.19:0/3258572909 conn(0x7f23f00fc400 msgr2=0x7f23f00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 -- 172.21.15.19:0/3258572909 shutdown_connections 2024-04-23T18:25:32.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.787+0000 7f23f6e05640 1 -- 172.21.15.19:0/3258572909 wait complete. 2024-04-23T18:25:32.791 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:25:32.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:32 smithi044 ceph-mon[32458]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-23T18:25:32.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:32 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1464072296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:32.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:32 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2471497396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:32.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:32 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2944004340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:32.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:32 smithi155 ceph-mon[33468]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-23T18:25:32.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:32 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1464072296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:32.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:32 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2471497396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:32.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:32 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2944004340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:32.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 -- 172.21.15.19:0/2046241174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb660102ab0 msgr2=0x7fb660104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 --2- 172.21.15.19:0/2046241174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb660102ab0 0x7fb660104ea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009900 tx=0x7fb65c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:32.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 -- 172.21.15.19:0/2046241174 shutdown_connections 2024-04-23T18:25:32.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 --2- 172.21.15.19:0/2046241174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6601053e0 0x7fb6601077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 --2- 172.21.15.19:0/2046241174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb660107d10 0x7fb66010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 --2- 172.21.15.19:0/2046241174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb660102ab0 0x7fb660104ea0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 -- 172.21.15.19:0/2046241174 >> 172.21.15.19:0/2046241174 conn(0x7fb6600fc400 msgr2=0x7fb6600fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 -- 172.21.15.19:0/2046241174 shutdown_connections 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 -- 172.21.15.19:0/2046241174 wait complete. 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 Processor -- start 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 -- start start 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb660102ab0 0x7fb6601981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6601053e0 0x7fb660198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb660107d10 0x7fb66019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb66010daf0 con 0x7fb6601053e0 2024-04-23T18:25:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb66010d970 con 0x7fb660102ab0 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.901+0000 7fb66817d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb66010dc70 con 0x7fb660107d10 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb665718640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6601053e0 0x7fb660198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb665718640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6601053e0 0x7fb660198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38452/0 (socket says 172.21.15.19:38452) 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb665718640 1 -- 172.21.15.19:0/2971681892 learned_addr learned my addr 172.21.15.19:0/2971681892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb665f19640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb660102ab0 0x7fb6601981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb665718640 1 -- 172.21.15.19:0/2971681892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb660107d10 msgr2=0x7fb66019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb665718640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb660107d10 0x7fb66019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb665718640 1 -- 172.21.15.19:0/2971681892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb660102ab0 msgr2=0x7fb6601981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb665718640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb660102ab0 0x7fb6601981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb665718640 1 -- 172.21.15.19:0/2971681892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66019fdf0 con 0x7fb6601053e0 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb665718640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6601053e0 0x7fb660198710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e9c0 tx=0x7fb65000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb656ffd640 1 -- 172.21.15.19:0/2971681892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65000cd80 con 0x7fb6601053e0 2024-04-23T18:25:32.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb656ffd640 1 -- 172.21.15.19:0/2971681892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6500054d0 con 0x7fb6601053e0 2024-04-23T18:25:32.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb66817d640 1 -- 172.21.15.19:0/2971681892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6601a00e0 con 0x7fb6601053e0 2024-04-23T18:25:32.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb656ffd640 1 -- 172.21.15.19:0/2971681892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650010430 con 0x7fb6601053e0 2024-04-23T18:25:32.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.902+0000 7fb66817d640 1 -- 172.21.15.19:0/2971681892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6601a0630 con 0x7fb6601053e0 2024-04-23T18:25:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.904+0000 7fb656ffd640 1 -- 172.21.15.19:0/2971681892 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb650004fd0 con 0x7fb6601053e0 2024-04-23T18:25:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb66af75740 1 Processor -- start 2024-04-23T18:25:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb66af75740 1 -- start start 2024-04-23T18:25:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb66af75740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb1b80 0x5567feeb1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb66af75740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feeb31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb66af75740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb3710 0x5567feeb9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb66af75740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567fee16c50 con 0x5567feeb3710 2024-04-23T18:25:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb66af75740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5567fedb3830 con 0x5567feeb2d50 2024-04-23T18:25:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb66af75740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567fee12c10 con 0x5567feeb1b80 2024-04-23T18:25:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb665718640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb3710 0x5567feeb9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb665718640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb3710 0x5567feeb9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38460/0 (socket says 172.21.15.19:38460) 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb665718640 1 -- 172.21.15.19:0/3181100315 learned_addr learned my addr 172.21.15.19:0/3181100315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb656ffd640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb64c0a3bb0 0x7fb64c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb66671a640 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb1b80 0x5567feeb1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb665f19640 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feeb31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb665718640 1 -- 172.21.15.19:0/3181100315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb1b80 msgr2=0x5567feeb1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb665718640 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb1b80 0x5567feeb1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb665718640 1 -- 172.21.15.19:0/3181100315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 msgr2=0x5567feeb31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.905+0000 7fb665718640 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feeb31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.906+0000 7fb665718640 1 -- 172.21.15.19:0/3181100315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567feb292d0 con 0x5567feeb3710 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.906+0000 7fb665f19640 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feeb31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.906+0000 7fb66671a640 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb1b80 0x5567feeb1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.906+0000 7fb66671a640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb64c0a3bb0 0x7fb64c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.906+0000 7fb656ffd640 1 -- 172.21.15.19:0/2971681892 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb6500d08f0 con 0x7fb6601053e0 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.906+0000 7fb665718640 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb3710 0x5567feeb9fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb650085a00 tx=0x7fb650067230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14280c2e51cfa745 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.906+0000 7fb64bfff640 1 -- 172.21.15.19:0/3181100315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650087070 con 0x5567feeb3710 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.906+0000 7fb66671a640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb64c0a3bb0 0x7fb64c0a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x5567feeb2c50 tx=0x7fb658004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.906+0000 7fb64bfff640 1 -- 172.21.15.19:0/3181100315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb650084320 con 0x5567feeb3710 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.906+0000 7fb64bfff640 1 -- 172.21.15.19:0/3181100315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500046c0 con 0x5567feeb3710 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 -- 172.21.15.19:0/3181100315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb3710 msgr2=0x5567feeb9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb3710 0x5567feeb9fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb650085a00 tx=0x7fb650067230 comp rx=0 tx=0).stop 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 -- 172.21.15.19:0/3181100315 shutdown_connections 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feeb31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb3710 0x5567feeb9fa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 --2- 172.21.15.19:0/3181100315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb1b80 0x5567feeb1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 -- 172.21.15.19:0/3181100315 >> 172.21.15.19:0/3181100315 conn(0x5567feead070 msgr2=0x5567feead470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 -- 172.21.15.19:0/3181100315 shutdown_connections 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 -- 172.21.15.19:0/3181100315 wait complete. 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 -- start start 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.907+0000 7fb66af75740 1 Processor -- start 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb66af75740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb1b80 0x5567feebbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb66af75740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feebc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb66af75740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb3710 0x5567feec3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb66af75740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567feeba6c0 con 0x5567feeb1b80 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb66af75740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5567fedb1470 con 0x5567feeb2d50 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb66af75740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567feeba840 con 0x5567feeb3710 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb66af75740 10 client.0 _ll_register_callbacks cb 0x5567fedd6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb665f19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb1b80 0x5567feebbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb665f19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb1b80 0x5567feebbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38476/0 (socket says 172.21.15.19:38476) 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb665f19640 1 -- 172.21.15.19:0/2016671933 learned_addr learned my addr 172.21.15.19:0/2016671933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb66671a640 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb3710 0x5567feec3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb665718640 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feebc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb665f19640 1 -- 172.21.15.19:0/2016671933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb3710 msgr2=0x5567feec3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb665f19640 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb3710 0x5567feec3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb665f19640 1 -- 172.21.15.19:0/2016671933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 msgr2=0x5567feebc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb665f19640 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feebc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb665f19640 1 -- 172.21.15.19:0/2016671933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567febf36b0 con 0x5567feeb1b80 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb665f19640 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb1b80 0x5567feebbeb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb65c02a930 tx=0x7fb65c031d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ac5475710a3e14 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb6497fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb6497fa640 1 -- 172.21.15.19:0/2016671933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c035030 con 0x5567feeb1b80 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb6497fa640 1 -- 172.21.15.19:0/2016671933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb65c004040 con 0x5567feeb1b80 2024-04-23T18:25:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.908+0000 7fb66af75740 1 -- 172.21.15.19:0/2016671933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5567fee24670 con 0x5567feeb1b80 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb6497fa640 1 -- 172.21.15.19:0/2016671933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb65c004240 con 0x5567feeb1b80 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb66af75740 10 client.15069 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb66af75740 1 -- 172.21.15.19:0/2016671933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567feec6780 con 0x5567feeb1b80 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb6497fa640 1 -- 172.21.15.19:0/2016671933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5567fee24670 con 0x5567feeb1b80 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb6497fa640 1 -- 172.21.15.19:0/2016671933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb65c004960 con 0x5567feeb1b80 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb66af75740 10 client.15069 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb66af75740 10 client.15069 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb66af75740 20 client.15069 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb66af75740 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x5567feec8e60 0x5567feeca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb66af75740 4 client.15069 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb66af75740 1 -- 172.21.15.19:0/2016671933 --> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567fee1e4e0 con 0x5567feec8e60 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.909+0000 7fb665718640 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x5567feec8e60 0x5567feeca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.910+0000 7fb665718640 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x5567feec8e60 0x5567feeca2d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2f5dd6ceb6441cd server_cookie=de6e98a5641844fd in_seq=0 out_seq=0 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.910+0000 7fb6497fa640 10 client.15069 ms_handle_connect on v2:172.21.15.155:6838/3934638415 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb6497fa640 1 -- 172.21.15.19:0/2016671933 <== mds.1 v2:172.21.15.155:6838/3934638415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567fee1e4e0 con 0x5567feec8e60 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb6497fa640 10 client.15069 handle_command_reply: tid=0 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 client.15069 shutdown 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 -- 172.21.15.19:0/2016671933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb1b80 msgr2=0x5567feebbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb1b80 0x5567feebbeb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb65c02a930 tx=0x7fb65c031d40 comp rx=0 tx=0).stop 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 -- 172.21.15.19:0/2016671933 shutdown_connections 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x5567feec8e60 0x5567feeca2d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feebc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb3710 0x5567feec3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb6497fa640 0 client.15069 ms_handle_reset on v2:172.21.15.155:6838/3934638415 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 --2- 172.21.15.19:0/2016671933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb1b80 0x5567feebbeb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 -- 172.21.15.19:0/2016671933 >> 172.21.15.19:0/2016671933 conn(0x5567feead070 msgr2=0x5567feeac1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 -- 172.21.15.19:0/2016671933 shutdown_connections 2024-04-23T18:25:32.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 1 -- 172.21.15.19:0/2016671933 wait complete. 2024-04-23T18:25:32.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.911+0000 7fb66af75740 20 client.15069 trim_cache size 0 max 16384 2024-04-23T18:25:32.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.929+0000 7fb66af75740 1 Processor -- start 2024-04-23T18:25:32.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.929+0000 7fb66af75740 1 -- start start 2024-04-23T18:25:32.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.929+0000 7fb66af75740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feead070 0x5567feebabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.929+0000 7fb66af75740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb1b80 0x5567feec33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.929+0000 7fb66af75740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feeaff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.929+0000 7fb66af75740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567feec8610 con 0x5567feead070 2024-04-23T18:25:32.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.929+0000 7fb66af75740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5567feec8ce0 con 0x5567feeb2d50 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.929+0000 7fb66af75740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567feec93b0 con 0x5567feeb1b80 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb665f19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feead070 0x5567feebabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb665f19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feead070 0x5567feebabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38478/0 (socket says 172.21.15.19:38478) 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb665f19640 1 -- 172.21.15.19:0/675310160 learned_addr learned my addr 172.21.15.19:0/675310160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb665f19640 1 -- 172.21.15.19:0/675310160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb1b80 msgr2=0x5567feec33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb665f19640 1 --2- 172.21.15.19:0/675310160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb1b80 0x5567feec33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb665f19640 1 -- 172.21.15.19:0/675310160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 msgr2=0x5567feeaff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb665f19640 1 --2- 172.21.15.19:0/675310160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feeaff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb665f19640 1 -- 172.21.15.19:0/675310160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567feeb1560 con 0x5567feead070 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb665f19640 1 --2- 172.21.15.19:0/675310160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feead070 0x5567feebabe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb65c004dd0 tx=0x7fb65c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905b1c4759d122c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb64b7fe640 1 -- 172.21.15.19:0/675310160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c035070 con 0x5567feead070 2024-04-23T18:25:32.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb64b7fe640 1 -- 172.21.15.19:0/675310160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb65c0317b0 con 0x5567feead070 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.930+0000 7fb64b7fe640 1 -- 172.21.15.19:0/675310160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c004a40 con 0x5567feead070 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 -- 172.21.15.19:0/675310160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feead070 msgr2=0x5567feebabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 --2- 172.21.15.19:0/675310160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feead070 0x5567feebabe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb65c004dd0 tx=0x7fb65c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 -- 172.21.15.19:0/675310160 shutdown_connections 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 --2- 172.21.15.19:0/675310160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb2d50 0x5567feeaff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 --2- 172.21.15.19:0/675310160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feeb1b80 0x5567feec33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 --2- 172.21.15.19:0/675310160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feead070 0x5567feebabe0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 -- 172.21.15.19:0/675310160 >> 172.21.15.19:0/675310160 conn(0x5567feeba100 msgr2=0x5567feeb6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 -- 172.21.15.19:0/675310160 shutdown_connections 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 -- 172.21.15.19:0/675310160 wait complete. 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 -- start start 2024-04-23T18:25:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 Processor -- start 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feead070 0x5567feebe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.931+0000 7fb66af75740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb1b80 0x5567feebead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66af75740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb2d50 0x5567feec1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66af75740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567feeca150 con 0x5567feeb2d50 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66af75740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5567feec9a80 con 0x5567feeb1b80 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66af75740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567feeca820 con 0x5567feead070 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66af75740 10 client.0 _ll_register_callbacks cb 0x7fb650005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66671a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb2d50 0x5567feec1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66671a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb2d50 0x5567feec1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38482/0 (socket says 172.21.15.19:38482) 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66671a640 1 -- 172.21.15.19:0/3734779408 learned_addr learned my addr 172.21.15.19:0/3734779408 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb665f19640 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feead070 0x5567feebe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb665718640 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb1b80 0x5567feebead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66671a640 1 -- 172.21.15.19:0/3734779408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feead070 msgr2=0x5567feebe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66671a640 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feead070 0x5567feebe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66671a640 1 -- 172.21.15.19:0/3734779408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb1b80 msgr2=0x5567feebead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66671a640 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb1b80 0x5567feebead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66671a640 1 -- 172.21.15.19:0/3734779408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567feec2090 con 0x5567feeb2d50 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb665718640 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb1b80 0x5567feebead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66671a640 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb2d50 0x5567feec1aa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb65801cc30 tx=0x7fb658005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3072086fdb48755e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb6497fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb6497fa640 1 -- 172.21.15.19:0/3734779408 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65802a060 con 0x5567feeb2d50 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb6497fa640 1 -- 172.21.15.19:0/3734779408 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb658010430 con 0x5567feeb2d50 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb66af75740 1 -- 172.21.15.19:0/3734779408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb65c0317b0 con 0x5567feeb2d50 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.932+0000 7fb6497fa640 1 -- 172.21.15.19:0/3734779408 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb658010630 con 0x5567feeb2d50 2024-04-23T18:25:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb66af75740 10 client.15075 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb66af75740 1 -- 172.21.15.19:0/3734779408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567feec3400 con 0x5567feeb2d50 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb6497fa640 1 -- 172.21.15.19:0/3734779408 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb65c0317b0 con 0x5567feeb2d50 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb6497fa640 1 -- 172.21.15.19:0/3734779408 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb6580307b0 con 0x5567feeb2d50 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb66af75740 10 client.15075 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb66af75740 10 client.15075 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb66af75740 20 client.15075 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb66af75740 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x5567feeb3710 0x5567fef32290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb66af75740 4 client.15075 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb66af75740 1 -- 172.21.15.19:0/3734779408 --> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] -- command(tid 0: {"prefix": "status"}) -- 0x5567fed9a9a0 con 0x5567feeb3710 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.933+0000 7fb665f19640 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x5567feeb3710 0x5567fef32290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb665f19640 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x5567feeb3710 0x5567fef32290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbd25c6cb01df07b server_cookie=54459780f8a62b54 in_seq=0 out_seq=0 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb6497fa640 10 client.15075 ms_handle_connect on v2:172.21.15.155:6838/3934638415 2024-04-23T18:25:32.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb6497fa640 1 -- 172.21.15.19:0/3734779408 <== mds.1 v2:172.21.15.155:6838/3934638415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5567fed9a9a0 con 0x5567feeb3710 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb6497fa640 10 client.15075 handle_command_reply: tid=0 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 client.15075 shutdown 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 -- 172.21.15.19:0/3734779408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb2d50 msgr2=0x5567feec1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb2d50 0x5567feec1aa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb65801cc30 tx=0x7fb658005bd0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 -- 172.21.15.19:0/3734779408 shutdown_connections 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x5567feeb3710 0x5567fef32290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567feeb1b80 0x5567feebead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567feeb2d50 0x5567feec1aa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 --2- 172.21.15.19:0/3734779408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567feead070 0x5567feebe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb6497fa640 0 client.15075 ms_handle_reset on v2:172.21.15.155:6838/3934638415 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 -- 172.21.15.19:0/3734779408 >> 172.21.15.19:0/3734779408 conn(0x5567feeba100 msgr2=0x5567feeb6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 -- 172.21.15.19:0/3734779408 shutdown_connections 2024-04-23T18:25:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 1 -- 172.21.15.19:0/3734779408 wait complete. 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.934+0000 7fb66af75740 20 client.15075 trim_cache size 0 max 16384 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24508, 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 93.568222112000001, 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-23T18:25:32.935 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 88, 2024-04-23T18:25:32.936 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:25:32.936 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 111.56936359700001 2024-04-23T18:25:32.936 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:25:32.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 -- 172.21.15.19:0/2971681892 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb64c0a3bb0 msgr2=0x7fb64c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb64c0a3bb0 0x7fb64c0a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x5567feeb2c50 tx=0x7fb658004a40 comp rx=0 tx=0).stop 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 -- 172.21.15.19:0/2971681892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6601053e0 msgr2=0x7fb660198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6601053e0 0x7fb660198710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e9c0 tx=0x7fb65000ee80 comp rx=0 tx=0).stop 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 -- 172.21.15.19:0/2971681892 shutdown_connections 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb64c0a3bb0 0x7fb64c0a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb660107d10 0x7fb66019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6601053e0 0x7fb660198710 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 --2- 172.21.15.19:0/2971681892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb660102ab0 0x7fb6601981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 -- 172.21.15.19:0/2971681892 >> 172.21.15.19:0/2971681892 conn(0x7fb6600fc400 msgr2=0x7fb6601033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 -- 172.21.15.19:0/2971681892 shutdown_connections 2024-04-23T18:25:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:32.935+0000 7fb66817d640 1 -- 172.21.15.19:0/2971681892 wait complete. 2024-04-23T18:25:32.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:25:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.048+0000 7f73e628a640 1 -- 172.21.15.19:0/1997413249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e0100e90 msgr2=0x7f73e01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.048+0000 7f73e628a640 1 --2- 172.21.15.19:0/1997413249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e0100e90 0x7f73e01012b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009900 tx=0x7f73d002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.049+0000 7f73e628a640 1 -- 172.21.15.19:0/1997413249 shutdown_connections 2024-04-23T18:25:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.049+0000 7f73e628a640 1 --2- 172.21.15.19:0/1997413249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e01029a0 0x7f73e01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.049+0000 7f73e628a640 1 --2- 172.21.15.19:0/1997413249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e0101fe0 0x7f73e0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.049+0000 7f73e628a640 1 --2- 172.21.15.19:0/1997413249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e0100e90 0x7f73e01012b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.049+0000 7f73e628a640 1 -- 172.21.15.19:0/1997413249 >> 172.21.15.19:0/1997413249 conn(0x7f73e00fc400 msgr2=0x7f73e00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.049+0000 7f73e628a640 1 -- 172.21.15.19:0/1997413249 shutdown_connections 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.049+0000 7f73e628a640 1 -- 172.21.15.19:0/1997413249 wait complete. 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.049+0000 7f73e628a640 1 Processor -- start 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.049+0000 7f73e628a640 1 -- start start 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73e628a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e0100e90 0x7f73e0104da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73e628a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e0101fe0 0x7f73e0103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73e628a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e01029a0 0x7f73e0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73e628a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73e010c9e0 con 0x7f73e0100e90 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73e628a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73e010c860 con 0x7f73e01029a0 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73e628a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73e010cb60 con 0x7f73e0101fe0 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e0100e90 0x7f73e0104da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e0100e90 0x7f73e0104da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38508/0 (socket says 172.21.15.19:38508) 2024-04-23T18:25:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dffff640 1 -- 172.21.15.19:0/2219727581 learned_addr learned my addr 172.21.15.19:0/2219727581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73e4827640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e01029a0 0x7f73e0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73df7fe640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e0101fe0 0x7f73e0103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dffff640 1 -- 172.21.15.19:0/2219727581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e0101fe0 msgr2=0x7f73e0103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dffff640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e0101fe0 0x7f73e0103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dffff640 1 -- 172.21.15.19:0/2219727581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e01029a0 msgr2=0x7f73e0103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dffff640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e01029a0 0x7f73e0103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dffff640 1 -- 172.21.15.19:0/2219727581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e01041b0 con 0x7f73e0100e90 2024-04-23T18:25:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73df7fe640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e0101fe0 0x7f73e0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dffff640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e0100e90 0x7f73e0104da0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006010 tx=0x7f73d00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dd7fa640 1 -- 172.21.15.19:0/2219727581 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d003f030 con 0x7f73e0100e90 2024-04-23T18:25:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dd7fa640 1 -- 172.21.15.19:0/2219727581 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73d0003cb0 con 0x7f73e0100e90 2024-04-23T18:25:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73e628a640 1 -- 172.21.15.19:0/2219727581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e01a22f0 con 0x7f73e0100e90 2024-04-23T18:25:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73dd7fa640 1 -- 172.21.15.19:0/2219727581 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0035070 con 0x7f73e0100e90 2024-04-23T18:25:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.050+0000 7f73e628a640 1 -- 172.21.15.19:0/2219727581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e01a27e0 con 0x7f73e0100e90 2024-04-23T18:25:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73dd7fa640 1 -- 172.21.15.19:0/2219727581 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f73d0042030 con 0x7f73e0100e90 2024-04-23T18:25:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73e8fa7740 1 Processor -- start 2024-04-23T18:25:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73e8fa7740 1 -- start start 2024-04-23T18:25:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a50b80 0x559167a50fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 0x559167a58fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5591679b5c50 con 0x559167a51d50 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559167952830 con 0x559167a50b80 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5591679b1c10 con 0x559167a52710 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38514/0 (socket says 172.21.15.19:38514) 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73dffff640 1 -- 172.21.15.19:0/2482239154 learned_addr learned my addr 172.21.15.19:0/2482239154 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73dd7fa640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73c80a3bb0 0x7f73c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73df7fe640 1 --2- 172.21.15.19:0/2482239154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 0x559167a58fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73e4827640 1 --2- 172.21.15.19:0/2482239154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a50b80 0x559167a50fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73dffff640 1 -- 172.21.15.19:0/2482239154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 msgr2=0x559167a58fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73dffff640 1 --2- 172.21.15.19:0/2482239154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 0x559167a58fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73dffff640 1 -- 172.21.15.19:0/2482239154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a50b80 msgr2=0x559167a50fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73dffff640 1 --2- 172.21.15.19:0/2482239154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a50b80 0x559167a50fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73e4827640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73c80a3bb0 0x7f73c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.053+0000 7f73dffff640 1 -- 172.21.15.19:0/2482239154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591676c82d0 con 0x559167a51d50 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.054+0000 7f73df7fe640 1 --2- 172.21.15.19:0/2482239154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 0x559167a58fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.054+0000 7f73dd7fa640 1 -- 172.21.15.19:0/2219727581 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f73d00f0b20 con 0x7f73e0100e90 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.054+0000 7f73dffff640 1 --2- 172.21.15.19:0/2482239154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a521d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f73d00b97e0 tx=0x7f73d002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5a5d9b2407193e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.054+0000 7f73e4827640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73c80a3bb0 0x7f73c80a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f73d8005c40 tx=0x7f73d8007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.054+0000 7f73c67fc640 1 -- 172.21.15.19:0/2482239154 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0049030 con 0x559167a51d50 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.054+0000 7f73c67fc640 1 -- 172.21.15.19:0/2482239154 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73d0003e60 con 0x559167a51d50 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.054+0000 7f73c67fc640 1 -- 172.21.15.19:0/2482239154 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d00c11c0 con 0x559167a51d50 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2482239154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 msgr2=0x559167a521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/2482239154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a521d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f73d00b97e0 tx=0x7f73d002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2482239154 shutdown_connections 2024-04-23T18:25:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/2482239154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 0x559167a58fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/2482239154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a521d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/2482239154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a50b80 0x559167a50fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2482239154 >> 172.21.15.19:0/2482239154 conn(0x559167a4c070 msgr2=0x559167a4c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2482239154 shutdown_connections 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2482239154 wait complete. 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 -- start start 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 Processor -- start 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a50b80 0x559167a5ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a5b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 0x559167a621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559167a594e0 con 0x559167a51d50 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559167950470 con 0x559167a50b80 2024-04-23T18:25:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559167a59660 con 0x559167a52710 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.055+0000 7f73e8fa7740 10 client.0 _ll_register_callbacks cb 0x559167975f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73df7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a5b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73df7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a5b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38524/0 (socket says 172.21.15.19:38524) 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73df7fe640 1 -- 172.21.15.19:0/359152638 learned_addr learned my addr 172.21.15.19:0/359152638 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73dffff640 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a50b80 0x559167a5ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73e4827640 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 0x559167a621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73df7fe640 1 -- 172.21.15.19:0/359152638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 msgr2=0x559167a621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73df7fe640 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 0x559167a621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73df7fe640 1 -- 172.21.15.19:0/359152638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a50b80 msgr2=0x559167a5ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73df7fe640 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a50b80 0x559167a5ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73df7fe640 1 -- 172.21.15.19:0/359152638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591677926b0 con 0x559167a51d50 2024-04-23T18:25:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73df7fe640 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a5b1b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f73d4010700 tx=0x7f73d4010bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d81cf3354b79d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73bf7fe640 1 -- 172.21.15.19:0/359152638 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d400c910 con 0x559167a51d50 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73bf7fe640 1 -- 172.21.15.19:0/359152638 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f73d400cab0 con 0x559167a51d50 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73e8fa7740 1 -- 172.21.15.19:0/359152638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5591679c3670 con 0x559167a51d50 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73bf7fe640 1 -- 172.21.15.19:0/359152638 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73d400ccb0 con 0x559167a51d50 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73e8fa7740 10 client.15087 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.056+0000 7f73e8fa7740 1 -- 172.21.15.19:0/359152638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559167a65630 con 0x559167a51d50 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.057+0000 7f73bf7fe640 1 -- 172.21.15.19:0/359152638 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5591679c3670 con 0x559167a51d50 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.057+0000 7f73bf7fe640 1 -- 172.21.15.19:0/359152638 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f73d4011440 con 0x559167a51d50 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.057+0000 7f73e8fa7740 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.057+0000 7f73e8fa7740 10 client.15087 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.057+0000 7f73e8fa7740 20 client.15087 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.057+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559167a67cf0 0x559167a69160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.057+0000 7f73e8fa7740 4 client.15087 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.057+0000 7f73e8fa7740 1 -- 172.21.15.19:0/359152638 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591679bd4e0 con 0x559167a67cf0 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.057+0000 7f73dffff640 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559167a67cf0 0x559167a69160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.058+0000 7f73dffff640 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559167a67cf0 0x559167a69160 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae2bef5bd4320471 server_cookie=d20efbec63423fd2 in_seq=0 out_seq=0 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.058+0000 7f73bf7fe640 10 client.15087 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.058+0000 7f73bf7fe640 1 -- 172.21.15.19:0/359152638 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591679bd4e0 con 0x559167a67cf0 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.058+0000 7f73bf7fe640 10 client.15087 handle_command_reply: tid=0 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.058+0000 7f73e8fa7740 1 client.15087 shutdown 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 1 -- 172.21.15.19:0/359152638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 msgr2=0x559167a5b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a5b1b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f73d4010700 tx=0x7f73d4010bc0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 1 -- 172.21.15.19:0/359152638 shutdown_connections 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559167a67cf0 0x559167a69160 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a52710 0x559167a621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a51d50 0x559167a5b1b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73bf7fe640 0 client.15087 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:25:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/359152638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a50b80 0x559167a5ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 1 -- 172.21.15.19:0/359152638 >> 172.21.15.19:0/359152638 conn(0x559167a4c070 msgr2=0x559167a4b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 1 -- 172.21.15.19:0/359152638 shutdown_connections 2024-04-23T18:25:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 1 -- 172.21.15.19:0/359152638 wait complete. 2024-04-23T18:25:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.059+0000 7f73e8fa7740 20 client.15087 trim_cache size 0 max 16384 2024-04-23T18:25:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73e8fa7740 1 Processor -- start 2024-04-23T18:25:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73e8fa7740 1 -- start start 2024-04-23T18:25:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a50b80 0x559167a5a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a51d50 0x559167a621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a52710 0x559167a5adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559167a674a0 con 0x559167a50b80 2024-04-23T18:25:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559167a67b70 con 0x559167a52710 2024-04-23T18:25:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559167a68240 con 0x559167a51d50 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a50b80 0x559167a5a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a50b80 0x559167a5a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38526/0 (socket says 172.21.15.19:38526) 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73dffff640 1 -- 172.21.15.19:0/519638421 learned_addr learned my addr 172.21.15.19:0/519638421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73e4827640 1 --2- 172.21.15.19:0/519638421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a52710 0x559167a5adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73dffff640 1 -- 172.21.15.19:0/519638421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a51d50 msgr2=0x559167a621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73dffff640 1 --2- 172.21.15.19:0/519638421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a51d50 0x559167a621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73dffff640 1 -- 172.21.15.19:0/519638421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a52710 msgr2=0x559167a5adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73dffff640 1 --2- 172.21.15.19:0/519638421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a52710 0x559167a5adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.077+0000 7f73dffff640 1 -- 172.21.15.19:0/519638421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559167a4bd40 con 0x559167a50b80 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73dffff640 1 --2- 172.21.15.19:0/519638421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a50b80 0x559167a5a880 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f73d00bcda0 tx=0x7f73d00f4600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8db31d979ee423 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73c5ffb640 1 -- 172.21.15.19:0/519638421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d00ac020 con 0x559167a50b80 2024-04-23T18:25:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73c5ffb640 1 -- 172.21.15.19:0/519638421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73d00f6040 con 0x559167a50b80 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73c5ffb640 1 -- 172.21.15.19:0/519638421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0049040 con 0x559167a50b80 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73e8fa7740 1 -- 172.21.15.19:0/519638421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a50b80 msgr2=0x559167a5a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/519638421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a50b80 0x559167a5a880 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f73d00bcda0 tx=0x7f73d00f4600 comp rx=0 tx=0).stop 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73e8fa7740 1 -- 172.21.15.19:0/519638421 shutdown_connections 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/519638421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a52710 0x559167a5adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/519638421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a51d50 0x559167a621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/519638421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a50b80 0x559167a5a880 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73e8fa7740 1 -- 172.21.15.19:0/519638421 >> 172.21.15.19:0/519638421 conn(0x559167a4c070 msgr2=0x559167a470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73e8fa7740 1 -- 172.21.15.19:0/519638421 shutdown_connections 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.078+0000 7f73e8fa7740 1 -- 172.21.15.19:0/519638421 wait complete. 2024-04-23T18:25:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e8fa7740 1 -- start start 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e8fa7740 1 Processor -- start 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a50b80 0x559167a5d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a51d50 0x559167a5dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e8fa7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a52710 0x559167a60d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559167a68fe0 con 0x559167a52710 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559167a68910 con 0x559167a51d50 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e8fa7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559167a696b0 con 0x559167a50b80 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e8fa7740 10 client.0 _ll_register_callbacks cb 0x7f73d4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e4827640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a52710 0x559167a60d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e4827640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a52710 0x559167a60d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38538/0 (socket says 172.21.15.19:38538) 2024-04-23T18:25:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e4827640 1 -- 172.21.15.19:0/2031839102 learned_addr learned my addr 172.21.15.19:0/2031839102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73dffff640 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a50b80 0x559167a5d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73df7fe640 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a51d50 0x559167a5dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e4827640 1 -- 172.21.15.19:0/2031839102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a50b80 msgr2=0x559167a5d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e4827640 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a50b80 0x559167a5d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e4827640 1 -- 172.21.15.19:0/2031839102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a51d50 msgr2=0x559167a5dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e4827640 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a51d50 0x559167a5dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.079+0000 7f73e4827640 1 -- 172.21.15.19:0/2031839102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559167a61370 con 0x559167a52710 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73e4827640 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a52710 0x559167a60d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f73d8012d00 tx=0x7f73d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a30ff7cc4e73e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73bf7fe640 1 -- 172.21.15.19:0/2031839102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d802b060 con 0x559167a52710 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73bf7fe640 1 -- 172.21.15.19:0/2031839102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f73d8026410 con 0x559167a52710 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2031839102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d00f6040 con 0x559167a52710 2024-04-23T18:25:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73bf7fe640 1 -- 172.21.15.19:0/2031839102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73d8026610 con 0x559167a52710 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73e8fa7740 10 client.15093 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2031839102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559167acdca0 con 0x559167a52710 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73bf7fe640 1 -- 172.21.15.19:0/2031839102 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73d00f6040 con 0x559167a52710 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73bf7fe640 1 -- 172.21.15.19:0/2031839102 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f73d8019040 con 0x559167a52710 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73e8fa7740 10 client.15093 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73e8fa7740 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73e8fa7740 20 client.15093 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559167ad28f0 0x559167ad1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73e8fa7740 4 client.15093 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.080+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2031839102 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "status"}) -- 0x5591679399a0 con 0x559167ad28f0 2024-04-23T18:25:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73dffff640 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559167ad28f0 0x559167ad1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73dffff640 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559167ad28f0 0x559167ad1600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e2b2a68af6427c server_cookie=b0d89fd9f86d859b in_seq=0 out_seq=0 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73bf7fe640 10 client.15093 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73bf7fe640 1 -- 172.21.15.19:0/2031839102 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5591679399a0 con 0x559167ad28f0 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73bf7fe640 10 client.15093 handle_command_reply: tid=0 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73e8fa7740 1 client.15093 shutdown 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2031839102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a52710 msgr2=0x559167a60d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a52710 0x559167a60d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f73d8012d00 tx=0x7f73d8004ab0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2031839102 shutdown_connections 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559167ad28f0 0x559167ad1600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559167a51d50 0x559167a5dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559167a52710 0x559167a60d80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73e8fa7740 1 --2- 172.21.15.19:0/2031839102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559167a50b80 0x559167a5d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73bf7fe640 0 client.15093 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.081+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2031839102 >> 172.21.15.19:0/2031839102 conn(0x559167a4c070 msgr2=0x559167a58a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2031839102 shutdown_connections 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e8fa7740 1 -- 172.21.15.19:0/2031839102 wait complete. 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e8fa7740 20 client.15093 trim_cache size 0 max 16384 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24521, 2024-04-23T18:25:33.083 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:25:33.084 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:25:33.084 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:25:33.084 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 96.734036814000007, 2024-04-23T18:25:33.084 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-23T18:25:33.084 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 88, 2024-04-23T18:25:33.084 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:25:33.084 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 102.801362286 2024-04-23T18:25:33.084 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:25:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 -- 172.21.15.19:0/2219727581 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73c80a3bb0 msgr2=0x7f73c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73c80a3bb0 0x7f73c80a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f73d8005c40 tx=0x7f73d8007520 comp rx=0 tx=0).stop 2024-04-23T18:25:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 -- 172.21.15.19:0/2219727581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e0100e90 msgr2=0x7f73e0104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e0100e90 0x7f73e0104da0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006010 tx=0x7f73d00043b0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 -- 172.21.15.19:0/2219727581 shutdown_connections 2024-04-23T18:25:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f73c80a3bb0 0x7f73c80a6070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e01029a0 0x7f73e0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73e0101fe0 0x7f73e0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 --2- 172.21.15.19:0/2219727581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e0100e90 0x7f73e0104da0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 -- 172.21.15.19:0/2219727581 >> 172.21.15.19:0/2219727581 conn(0x7f73e00fc400 msgr2=0x7f73e00fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 -- 172.21.15.19:0/2219727581 shutdown_connections 2024-04-23T18:25:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:33.082+0000 7f73e628a640 1 -- 172.21.15.19:0/2219727581 wait complete. 2024-04-23T18:25:33.087 INFO:tasks.cephadm:Setup complete, yielding 2024-04-23T18:25:33.087 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T18:25:33.100 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:25:33.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-23T18:25:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:34.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:34 smithi019 ceph-mon[29215]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:35.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:34 smithi044 ceph-mon[32458]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:35.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:34 smithi155 ceph-mon[33468]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.408+0000 7feca7454700 1 -- 172.21.15.19:0/3203926589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca0105e50 msgr2=0x7feca0108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.408+0000 7feca7454700 1 --2- 172.21.15.19:0/3203926589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca0105e50 0x7feca0108240 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec94009990 tx=0x7fec94009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.408+0000 7feca7454700 1 -- 172.21.15.19:0/3203926589 shutdown_connections 2024-04-23T18:25:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.408+0000 7feca7454700 1 --2- 172.21.15.19:0/3203926589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca0108780 0x7feca010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.408+0000 7feca7454700 1 --2- 172.21.15.19:0/3203926589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca0103520 0x7feca0105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.408+0000 7feca7454700 1 --2- 172.21.15.19:0/3203926589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca0105e50 0x7feca0108240 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.408+0000 7feca7454700 1 -- 172.21.15.19:0/3203926589 >> 172.21.15.19:0/3203926589 conn(0x7feca00fcf30 msgr2=0x7feca00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.408+0000 7feca7454700 1 -- 172.21.15.19:0/3203926589 shutdown_connections 2024-04-23T18:25:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.409+0000 7feca7454700 1 -- 172.21.15.19:0/3203926589 wait complete. 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.409+0000 7feca7454700 1 Processor -- start 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.409+0000 7feca7454700 1 -- start start 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.409+0000 7feca7454700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca0103520 0x7feca0198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.409+0000 7feca7454700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca0105e50 0x7feca01990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.409+0000 7feca7454700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca0108780 0x7feca01a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.409+0000 7feca7454700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feca010e520 con 0x7feca0103520 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.409+0000 7feca7454700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feca010e3a0 con 0x7feca0105e50 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.409+0000 7feca7454700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feca010e6a0 con 0x7feca0108780 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca51f0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca0103520 0x7feca0198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca51f0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca0103520 0x7feca0198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38554/0 (socket says 172.21.15.19:38554) 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca51f0700 1 -- 172.21.15.19:0/1129009942 learned_addr learned my addr 172.21.15.19:0/1129009942 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca49ef700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca0105e50 0x7feca01990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca59f1700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca0108780 0x7feca01a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca51f0700 1 -- 172.21.15.19:0/1129009942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca0108780 msgr2=0x7feca01a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca51f0700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca0108780 0x7feca01a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca51f0700 1 -- 172.21.15.19:0/1129009942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca0105e50 msgr2=0x7feca01990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca51f0700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca0105e50 0x7feca01990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca51f0700 1 -- 172.21.15.19:0/1129009942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca01a07c0 con 0x7feca0103520 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca59f1700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca0108780 0x7feca01a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca51f0700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca0103520 0x7feca0198b60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fec98007e50 tx=0x7fec9800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7fec9e7fc700 1 -- 172.21.15.19:0/1129009942 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98016020 con 0x7feca0103520 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca7454700 1 -- 172.21.15.19:0/1129009942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca01a0ab0 con 0x7feca0103520 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7fec9e7fc700 1 -- 172.21.15.19:0/1129009942 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec9800adc0 con 0x7feca0103520 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7fec9e7fc700 1 -- 172.21.15.19:0/1129009942 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98011700 con 0x7feca0103520 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.410+0000 7feca7454700 1 -- 172.21.15.19:0/1129009942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca01a1040 con 0x7feca0103520 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.412+0000 7feca7454700 1 -- 172.21.15.19:0/1129009942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca0069730 con 0x7feca0103520 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.412+0000 7fec9e7fc700 1 -- 172.21.15.19:0/1129009942 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fec98019020 con 0x7feca0103520 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.413+0000 7fec9e7fc700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec880a3c40 0x7fec880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.413+0000 7feca49ef700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec880a3c40 0x7fec880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.413+0000 7fec9e7fc700 1 -- 172.21.15.19:0/1129009942 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fec980cfee0 con 0x7feca0103520 2024-04-23T18:25:35.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.414+0000 7feca49ef700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec880a3c40 0x7fec880a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fec9400b5c0 tx=0x7fec94005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.418+0000 7fec9e7fc700 1 -- 172.21.15.19:0/1129009942 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec98020070 con 0x7feca0103520 2024-04-23T18:25:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.646+0000 7feca7454700 1 -- 172.21.15.19:0/1129009942 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7feca0062e80 con 0x7fec880a3c40 2024-04-23T18:25:35.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.647+0000 7fec9e7fc700 1 -- 172.21.15.19:0/1129009942 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7feca0062e80 con 0x7fec880a3c40 2024-04-23T18:25:35.640 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-04-23T18:25:35.640 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-04-23T18:25:35.640 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-04-23T18:25:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 -- 172.21.15.19:0/1129009942 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec880a3c40 msgr2=0x7fec880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec880a3c40 0x7fec880a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fec9400b5c0 tx=0x7fec94005fd0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 -- 172.21.15.19:0/1129009942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca0103520 msgr2=0x7feca0198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca0103520 0x7feca0198b60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fec98007e50 tx=0x7fec9800a340 comp rx=0 tx=0).stop 2024-04-23T18:25:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 -- 172.21.15.19:0/1129009942 shutdown_connections 2024-04-23T18:25:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fec880a3c40 0x7fec880a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca0108780 0x7feca01a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca0103520 0x7feca0198b60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 --2- 172.21.15.19:0/1129009942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca0105e50 0x7feca01990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 -- 172.21.15.19:0/1129009942 >> 172.21.15.19:0/1129009942 conn(0x7feca00fcf30 msgr2=0x7feca0103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 -- 172.21.15.19:0/1129009942 shutdown_connections 2024-04-23T18:25:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:35.649+0000 7feca7454700 1 -- 172.21.15.19:0/1129009942 wait complete. 2024-04-23T18:25:36.538 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-23T18:25:36.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:36 smithi019 ceph-mon[29215]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:36.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:36 smithi019 ceph-mon[29215]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:37.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:36 smithi044 ceph-mon[32458]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:37.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:36 smithi044 ceph-mon[32458]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:37.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:36 smithi155 ceph-mon[33468]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:37.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:36 smithi155 ceph-mon[33468]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:38.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.606+0000 7f58f81c9700 1 -- 172.21.15.19:0/1732824366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f0103560 msgr2=0x7f58f0109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:38.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.606+0000 7f58f81c9700 1 --2- 172.21.15.19:0/1732824366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f0103560 0x7f58f0109e10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b0d0 tx=0x7f58e800b3e0 comp rx=0 tx=0).stop 2024-04-23T18:25:38.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.607+0000 7f58f81c9700 1 -- 172.21.15.19:0/1732824366 shutdown_connections 2024-04-23T18:25:38.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.607+0000 7f58f81c9700 1 --2- 172.21.15.19:0/1732824366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f0103560 0x7f58f0109e10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:38.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.607+0000 7f58f81c9700 1 --2- 172.21.15.19:0/1732824366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f01019c0 0x7f58f0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:38.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.607+0000 7f58f81c9700 1 --2- 172.21.15.19:0/1732824366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f0102bc0 0x7f58f0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:38.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.607+0000 7f58f81c9700 1 -- 172.21.15.19:0/1732824366 >> 172.21.15.19:0/1732824366 conn(0x7f58f00fcf60 msgr2=0x7f58f00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:38.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.607+0000 7f58f81c9700 1 -- 172.21.15.19:0/1732824366 shutdown_connections 2024-04-23T18:25:38.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.607+0000 7f58f81c9700 1 -- 172.21.15.19:0/1732824366 wait complete. 2024-04-23T18:25:38.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.607+0000 7f58f81c9700 1 Processor -- start 2024-04-23T18:25:38.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.607+0000 7f58f81c9700 1 -- start start 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f81c9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f01019c0 0x7f58f006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f81c9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f0102bc0 0x7f58f019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f81c9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f0103560 0x7f58f019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f81c9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58f010d4b0 con 0x7f58f01019c0 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f81c9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f58f010d330 con 0x7f58f0102bc0 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f81c9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58f010d630 con 0x7f58f0103560 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f5f65700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f01019c0 0x7f58f006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f5f65700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f01019c0 0x7f58f006bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38570/0 (socket says 172.21.15.19:38570) 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f5f65700 1 -- 172.21.15.19:0/2414730251 learned_addr learned my addr 172.21.15.19:0/2414730251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f5764700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f0102bc0 0x7f58f019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:38.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f6766700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f0103560 0x7f58f019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f5f65700 1 -- 172.21.15.19:0/2414730251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f0103560 msgr2=0x7f58f019ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f5f65700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f0103560 0x7f58f019ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f5f65700 1 -- 172.21.15.19:0/2414730251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f0102bc0 msgr2=0x7f58f019d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f5f65700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f0102bc0 0x7f58f019d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f5f65700 1 -- 172.21.15.19:0/2414730251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f01a0830 con 0x7f58f01019c0 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f6766700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f0103560 0x7f58f019ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f5f65700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f01019c0 0x7f58f006bcf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f58e000d980 tx=0x7f58e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58e6ffd700 1 -- 172.21.15.19:0/2414730251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0017030 con 0x7f58f01019c0 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58e6ffd700 1 -- 172.21.15.19:0/2414730251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e00044d0 con 0x7f58f01019c0 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f81c9700 1 -- 172.21.15.19:0/2414730251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f01a0b20 con 0x7f58f01019c0 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58e6ffd700 1 -- 172.21.15.19:0/2414730251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00039e0 con 0x7f58f01019c0 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.608+0000 7f58f81c9700 1 -- 172.21.15.19:0/2414730251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01a1060 con 0x7f58f01019c0 2024-04-23T18:25:38.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.609+0000 7f58f81c9700 1 -- 172.21.15.19:0/2414730251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f0069770 con 0x7f58f01019c0 2024-04-23T18:25:38.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.610+0000 7f58e6ffd700 1 -- 172.21.15.19:0/2414730251 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f58e001a070 con 0x7f58f01019c0 2024-04-23T18:25:38.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.611+0000 7f58e6ffd700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f58dc0a3c40 0x7f58dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:38.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.611+0000 7f58f5764700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f58dc0a3c40 0x7f58dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:38.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.611+0000 7f58e6ffd700 1 -- 172.21.15.19:0/2414730251 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f58e00cfd70 con 0x7f58f01019c0 2024-04-23T18:25:38.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.612+0000 7f58f5764700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f58dc0a3c40 0x7f58dc0a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f58f006c250 tx=0x7f58ec009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:38.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.616+0000 7f58e6ffd700 1 -- 172.21.15.19:0/2414730251 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58e00232f0 con 0x7f58f01019c0 2024-04-23T18:25:38.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:38 smithi019 ceph-mon[29215]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.839+0000 7f58f81c9700 1 -- 172.21.15.19:0/2414730251 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f58f0062e80 con 0x7f58dc0a3c40 2024-04-23T18:25:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.853+0000 7f58e6ffd700 1 -- 172.21.15.19:0/2414730251 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f58f0062e80 con 0x7f58dc0a3c40 2024-04-23T18:25:38.846 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T18:25:38.846 INFO:teuthology.orchestra.run.smithi019.stdout:mds.a smithi019 running (111s) 19s ago 111s 19.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 14eec03debed 2024-04-23T18:25:38.846 INFO:teuthology.orchestra.run.smithi019.stdout:mds.b smithi044 running (108s) 41s ago 108s 28.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 bb88eed0142d 2024-04-23T18:25:38.846 INFO:teuthology.orchestra.run.smithi019.stdout:mds.c smithi155 running (2m) 41s ago 2m 20.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ef0ea28b882b 2024-04-23T18:25:38.846 INFO:teuthology.orchestra.run.smithi019.stdout:mds.d smithi019 running (2m) 19s ago 2m 15.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6ed95b660d4e 2024-04-23T18:25:38.846 INFO:teuthology.orchestra.run.smithi019.stdout:mds.e smithi044 running (2m) 41s ago 2m 16.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 17343ea90f62 2024-04-23T18:25:38.846 INFO:teuthology.orchestra.run.smithi019.stdout:mds.f smithi155 running (2m) 41s ago 2m 17.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c9f647b73407 2024-04-23T18:25:38.846 INFO:teuthology.orchestra.run.smithi019.stdout:mds.g smithi019 running (114s) 19s ago 114s 16.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 dbbf0ce9b4e9 2024-04-23T18:25:38.846 INFO:teuthology.orchestra.run.smithi019.stdout:mds.h smithi044 running (2m) 41s ago 2m 17.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f624123f2dae 2024-04-23T18:25:38.846 INFO:teuthology.orchestra.run.smithi019.stdout:mds.i smithi155 running (117s) 41s ago 117s 19.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9d59faab4de6 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:mds.j smithi019 running (2m) 19s ago 2m 13.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ae775ed8da7e 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:mds.k smithi044 running (2m) 41s ago 2m 16.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3fbb3b37d5de 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:mds.l smithi155 running (2m) 41s ago 2m 16.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c234d349ecda 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.x smithi019 *:9283,8765 running (9m) 19s ago 9m 515M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6f128d5987ef 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.y smithi044 *:8443,8765 running (7m) 41s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 853cb65869c8 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.z smithi155 *:8443,8765 running (7m) 41s ago 7m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ae6fd8769900 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a smithi019 running (10m) 19s ago 10m 50.9M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 c42d35d9ae5c 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b smithi044 running (8m) 41s ago 8m 39.5M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 4edd044bd4d6 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c smithi155 running (8m) 41s ago 8m 39.1M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 88528f6d2fdd 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (7m) 19s ago 7m 57.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 273beec7d06d 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi044 running (7m) 41s ago 7m 56.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 5661ffd87dbe 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi155 running (6m) 41s ago 6m 57.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3e41951cc3fd 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi019 running (6m) 19s ago 6m 57.5M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 2d0e7df7afe7 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi044 running (5m) 41s ago 5m 42.7M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 a14f9bf61dfc 2024-04-23T18:25:38.847 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi155 running (5m) 41s ago 5m 58.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 61b743ed3fe2 2024-04-23T18:25:38.848 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 smithi019 running (4m) 19s ago 4m 53.6M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 20f692cc5c7b 2024-04-23T18:25:38.848 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 smithi044 running (4m) 41s ago 4m 56.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 f0686006a4cc 2024-04-23T18:25:38.848 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 smithi155 running (4m) 41s ago 4m 53.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b170b60b50e9 2024-04-23T18:25:38.848 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 smithi019 running (3m) 19s ago 3m 51.9M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 955d7752da79 2024-04-23T18:25:38.848 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 smithi044 running (3m) 41s ago 3m 53.6M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 032af56efe04 2024-04-23T18:25:38.848 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 smithi155 running (2m) 41s ago 2m 55.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3962e34ce10d 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 -- 172.21.15.19:0/2414730251 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f58dc0a3c40 msgr2=0x7f58dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f58dc0a3c40 0x7f58dc0a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f58f006c250 tx=0x7f58ec009190 comp rx=0 tx=0).stop 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 -- 172.21.15.19:0/2414730251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f01019c0 msgr2=0x7f58f006bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f01019c0 0x7f58f006bcf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f58e000d980 tx=0x7f58e000dc90 comp rx=0 tx=0).stop 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 -- 172.21.15.19:0/2414730251 shutdown_connections 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f58dc0a3c40 0x7f58dc0a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f0103560 0x7f58f019ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f01019c0 0x7f58f006bcf0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 --2- 172.21.15.19:0/2414730251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f0102bc0 0x7f58f019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 -- 172.21.15.19:0/2414730251 >> 172.21.15.19:0/2414730251 conn(0x7f58f00fcf60 msgr2=0x7f58f0105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 -- 172.21.15.19:0/2414730251 shutdown_connections 2024-04-23T18:25:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:38.855+0000 7f58f81c9700 1 -- 172.21.15.19:0/2414730251 wait complete. 2024-04-23T18:25:39.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:38 smithi044 ceph-mon[32458]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:39.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:38 smithi155 ceph-mon[33468]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:39.730 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-23T18:25:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:40.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:39 smithi019 ceph-mon[29215]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:40.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:39 smithi044 ceph-mon[32458]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:40.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:39 smithi155 ceph-mon[33468]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:40.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:40 smithi019 ceph-mon[29215]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:41.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:40 smithi044 ceph-mon[32458]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:41.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:40 smithi155 ceph-mon[33468]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.883+0000 7f258c531700 1 -- 172.21.15.19:0/3088451863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2584103cc0 msgr2=0x7f2584104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.883+0000 7f258c531700 1 --2- 172.21.15.19:0/3088451863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2584103cc0 0x7f2584104120 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f25740099c0 tx=0x7f2574009cd0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.883+0000 7f258c531700 1 -- 172.21.15.19:0/3088451863 shutdown_connections 2024-04-23T18:25:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.883+0000 7f258c531700 1 --2- 172.21.15.19:0/3088451863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2584104660 0x7f258410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.883+0000 7f258c531700 1 --2- 172.21.15.19:0/3088451863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2584102ac0 0x7f2584102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.883+0000 7f258c531700 1 --2- 172.21.15.19:0/3088451863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2584103cc0 0x7f2584104120 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.883+0000 7f258c531700 1 -- 172.21.15.19:0/3088451863 >> 172.21.15.19:0/3088451863 conn(0x7f25840fe060 msgr2=0x7f25841004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.883+0000 7f258c531700 1 -- 172.21.15.19:0/3088451863 shutdown_connections 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.883+0000 7f258c531700 1 -- 172.21.15.19:0/3088451863 wait complete. 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258c531700 1 Processor -- start 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258c531700 1 -- start start 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258c531700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2584102ac0 0x7f25840767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258c531700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2584103cc0 0x7f2584074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258c531700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2584104660 0x7f2584075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258c531700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f258410e530 con 0x7f2584102ac0 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258c531700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f258410e3b0 con 0x7f2584104660 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258c531700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f258410e6b0 con 0x7f2584103cc0 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258a2cd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2584102ac0 0x7f25840767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258a2cd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2584102ac0 0x7f25840767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59192/0 (socket says 172.21.15.19:59192) 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258a2cd700 1 -- 172.21.15.19:0/1630261471 learned_addr learned my addr 172.21.15.19:0/1630261471 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f2589acc700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2584103cc0 0x7f2584074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258aace700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2584104660 0x7f2584075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258a2cd700 1 -- 172.21.15.19:0/1630261471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2584103cc0 msgr2=0x7f2584074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258a2cd700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2584103cc0 0x7f2584074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258a2cd700 1 -- 172.21.15.19:0/1630261471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2584104660 msgr2=0x7f2584075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258a2cd700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2584104660 0x7f2584075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258a2cd700 1 -- 172.21.15.19:0/1630261471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2584075c20 con 0x7f2584102ac0 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f2589acc700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2584103cc0 0x7f2584074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258aace700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2584104660 0x7f2584075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.884+0000 7f258a2cd700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2584102ac0 0x7f25840767e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f258000c970 tx=0x7f258000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.885+0000 7f257b7fe700 1 -- 172.21.15.19:0/1630261471 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580004220 con 0x7f2584102ac0 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.885+0000 7f258c531700 1 -- 172.21.15.19:0/1630261471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25841a2d30 con 0x7f2584102ac0 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.885+0000 7f257b7fe700 1 -- 172.21.15.19:0/1630261471 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2580007d50 con 0x7f2584102ac0 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.885+0000 7f257b7fe700 1 -- 172.21.15.19:0/1630261471 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580003d10 con 0x7f2584102ac0 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.885+0000 7f258c531700 1 -- 172.21.15.19:0/1630261471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25841a3190 con 0x7f2584102ac0 2024-04-23T18:25:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.886+0000 7f258c531700 1 -- 172.21.15.19:0/1630261471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258402df80 con 0x7f2584102ac0 2024-04-23T18:25:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.887+0000 7f257b7fe700 1 -- 172.21.15.19:0/1630261471 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f25800043c0 con 0x7f2584102ac0 2024-04-23T18:25:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.888+0000 7f257b7fe700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f25700a3c40 0x7f25700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.888+0000 7f2589acc700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f25700a3c40 0x7f25700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.888+0000 7f257b7fe700 1 -- 172.21.15.19:0/1630261471 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f25800d0020 con 0x7f2584102ac0 2024-04-23T18:25:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.888+0000 7f2589acc700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f25700a3c40 0x7f25700a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2574000c00 tx=0x7f2574005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:41.893+0000 7f257b7fe700 1 -- 172.21.15.19:0/1630261471 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2580014030 con 0x7f2584102ac0 2024-04-23T18:25:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.110+0000 7f258c531700 1 -- 172.21.15.19:0/1630261471 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2584062e80 con 0x7f25700a3c40 2024-04-23T18:25:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.118+0000 7f257b7fe700 1 -- 172.21.15.19:0/1630261471 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2584062e80 con 0x7f25700a3c40 2024-04-23T18:25:42.111 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-23T18:25:42.111 INFO:teuthology.orchestra.run.smithi019.stdout:mds.all 12/12 45s ago 2m smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-04-23T18:25:42.111 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 3/3 45s ago 7m smithi019=x;smithi044=y;smithi155=z;count:3 2024-04-23T18:25:42.111 INFO:teuthology.orchestra.run.smithi019.stdout:mon 3/3 45s ago 8m smithi019:172.21.15.19=a;smithi044:172.21.15.44=b;smithi155:172.21.15.155=c;count:3 2024-04-23T18:25:42.111 INFO:teuthology.orchestra.run.smithi019.stdout:osd 12 45s ago - 2024-04-23T18:25:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 -- 172.21.15.19:0/1630261471 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f25700a3c40 msgr2=0x7f25700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f25700a3c40 0x7f25700a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2574000c00 tx=0x7f2574005fd0 comp rx=0 tx=0).stop 2024-04-23T18:25:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 -- 172.21.15.19:0/1630261471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2584102ac0 msgr2=0x7f25840767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2584102ac0 0x7f25840767e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f258000c970 tx=0x7f258000cd30 comp rx=0 tx=0).stop 2024-04-23T18:25:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 -- 172.21.15.19:0/1630261471 shutdown_connections 2024-04-23T18:25:42.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f25700a3c40 0x7f25700a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:42.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2584104660 0x7f2584075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:42.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2584102ac0 0x7f25840767e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:42.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 --2- 172.21.15.19:0/1630261471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2584103cc0 0x7f2584074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:42.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 -- 172.21.15.19:0/1630261471 >> 172.21.15.19:0/1630261471 conn(0x7f25840fe060 msgr2=0x7f2584106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:42.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 -- 172.21.15.19:0/1630261471 shutdown_connections 2024-04-23T18:25:42.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:42.120+0000 7f258c531700 1 -- 172.21.15.19:0/1630261471 wait complete. 2024-04-23T18:25:42.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:42 smithi019 ceph-mon[29215]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-23T18:25:42.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:42 smithi019 ceph-mon[29215]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:42.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-23T18:25:43.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:42 smithi044 ceph-mon[32458]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-23T18:25:43.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:42 smithi044 ceph-mon[32458]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:43.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:42 smithi155 ceph-mon[33468]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-23T18:25:43.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:42 smithi155 ceph-mon[33468]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:43.250 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:44.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:44 smithi019 ceph-mon[29215]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.068+0000 7f2c799bb700 1 -- 172.21.15.19:0/2514564761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c741019a0 msgr2=0x7f2c74101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:45.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.068+0000 7f2c799bb700 1 --2- 172.21.15.19:0/2514564761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c741019a0 0x7f2c74101dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2c640099c0 tx=0x7f2c64009cd0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.068+0000 7f2c799bb700 1 -- 172.21.15.19:0/2514564761 shutdown_connections 2024-04-23T18:25:45.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.068+0000 7f2c799bb700 1 --2- 172.21.15.19:0/2514564761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c74103540 0x7f2c74109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.068+0000 7f2c799bb700 1 --2- 172.21.15.19:0/2514564761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c741019a0 0x7f2c74101dc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.068+0000 7f2c799bb700 1 --2- 172.21.15.19:0/2514564761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c74102ba0 0x7f2c74103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.068+0000 7f2c799bb700 1 -- 172.21.15.19:0/2514564761 >> 172.21.15.19:0/2514564761 conn(0x7f2c740fcf40 msgr2=0x7f2c740ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:45.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.068+0000 7f2c799bb700 1 -- 172.21.15.19:0/2514564761 shutdown_connections 2024-04-23T18:25:45.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.068+0000 7f2c799bb700 1 -- 172.21.15.19:0/2514564761 wait complete. 2024-04-23T18:25:45.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c799bb700 1 Processor -- start 2024-04-23T18:25:45.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c799bb700 1 -- start start 2024-04-23T18:25:45.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:44 smithi044 ceph-mon[32458]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c799bb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c741019a0 0x7f2c741058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c799bb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c74102ba0 0x7f2c74103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c799bb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c74103540 0x7f2c741044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c799bb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7410d410 con 0x7f2c74102ba0 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c799bb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7410d290 con 0x7f2c74103540 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c799bb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7410d590 con 0x7f2c741019a0 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c727fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c74102ba0 0x7f2c74103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c727fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c74102ba0 0x7f2c74103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59216/0 (socket says 172.21.15.19:59216) 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c727fc700 1 -- 172.21.15.19:0/4041671148 learned_addr learned my addr 172.21.15.19:0/4041671148 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c72ffd700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c741019a0 0x7f2c741058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.069+0000 7f2c737fe700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c74103540 0x7f2c741044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c727fc700 1 -- 172.21.15.19:0/4041671148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c741019a0 msgr2=0x7f2c741058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c727fc700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c741019a0 0x7f2c741058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c727fc700 1 -- 172.21.15.19:0/4041671148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c74103540 msgr2=0x7f2c741044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c727fc700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c74103540 0x7f2c741044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c727fc700 1 -- 172.21.15.19:0/4041671148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c74104d90 con 0x7f2c74102ba0 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c737fe700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c74103540 0x7f2c741044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c727fc700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c74102ba0 0x7f2c74103f80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800e9a0 tx=0x7f2c6800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c789b9700 1 -- 172.21.15.19:0/4041671148 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6800ca10 con 0x7f2c74102ba0 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c789b9700 1 -- 172.21.15.19:0/4041671148 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c6800cbb0 con 0x7f2c74102ba0 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c799bb700 1 -- 172.21.15.19:0/4041671148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c741a2e10 con 0x7f2c74102ba0 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c789b9700 1 -- 172.21.15.19:0/4041671148 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68018680 con 0x7f2c74102ba0 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.070+0000 7f2c799bb700 1 -- 172.21.15.19:0/4041671148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c741a3260 con 0x7f2c74102ba0 2024-04-23T18:25:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.071+0000 7f2c799bb700 1 -- 172.21.15.19:0/4041671148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c741a34f0 con 0x7f2c74102ba0 2024-04-23T18:25:45.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.072+0000 7f2c789b9700 1 -- 172.21.15.19:0/4041671148 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2c68010430 con 0x7f2c74102ba0 2024-04-23T18:25:45.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.073+0000 7f2c789b9700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c5c0a3cc0 0x7f2c5c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:45.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.073+0000 7f2c72ffd700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c5c0a3cc0 0x7f2c5c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:45.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.073+0000 7f2c789b9700 1 -- 172.21.15.19:0/4041671148 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2c680d07f0 con 0x7f2c74102ba0 2024-04-23T18:25:45.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.074+0000 7f2c72ffd700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c5c0a3cc0 0x7f2c5c0a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c64000c00 tx=0x7f2c64005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:45.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.077+0000 7f2c789b9700 1 -- 172.21.15.19:0/4041671148 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c6809d1e0 con 0x7f2c74102ba0 2024-04-23T18:25:45.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:44 smithi155 ceph-mon[33468]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.290+0000 7f2c799bb700 1 -- 172.21.15.19:0/4041671148 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2c7406ae80 con 0x7f2c5c0a3cc0 2024-04-23T18:25:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.291+0000 7f2c789b9700 1 -- 172.21.15.19:0/4041671148 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f2c7406ae80 con 0x7f2c5c0a3cc0 2024-04-23T18:25:45.285 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-04-23T18:25:45.285 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-04-23T18:25:45.285 INFO:teuthology.orchestra.run.smithi019.stdout:smithi044 172.21.15.44 2024-04-23T18:25:45.285 INFO:teuthology.orchestra.run.smithi019.stdout:smithi155 172.21.15.155 2024-04-23T18:25:45.285 INFO:teuthology.orchestra.run.smithi019.stdout:3 hosts in cluster 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.293+0000 7f2c799bb700 1 -- 172.21.15.19:0/4041671148 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c5c0a3cc0 msgr2=0x7f2c5c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.293+0000 7f2c799bb700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c5c0a3cc0 0x7f2c5c0a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c64000c00 tx=0x7f2c64005fb0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.293+0000 7f2c799bb700 1 -- 172.21.15.19:0/4041671148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c74102ba0 msgr2=0x7f2c74103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.293+0000 7f2c799bb700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c74102ba0 0x7f2c74103f80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800e9a0 tx=0x7f2c6800ecb0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.293+0000 7f2c799bb700 1 -- 172.21.15.19:0/4041671148 shutdown_connections 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.293+0000 7f2c799bb700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f2c5c0a3cc0 0x7f2c5c0a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.294+0000 7f2c799bb700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c74103540 0x7f2c741044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.294+0000 7f2c799bb700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c741019a0 0x7f2c741058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.294+0000 7f2c799bb700 1 --2- 172.21.15.19:0/4041671148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c74102ba0 0x7f2c74103f80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.294+0000 7f2c799bb700 1 -- 172.21.15.19:0/4041671148 >> 172.21.15.19:0/4041671148 conn(0x7f2c740fcf40 msgr2=0x7f2c7406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.294+0000 7f2c799bb700 1 -- 172.21.15.19:0/4041671148 shutdown_connections 2024-04-23T18:25:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:45.294+0000 7f2c799bb700 1 -- 172.21.15.19:0/4041671148 wait complete. 2024-04-23T18:25:46.011 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-23T18:25:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:47.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:46 smithi019 ceph-mon[29215]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:47.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:46 smithi019 ceph-mon[29215]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:47.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:46 smithi044 ceph-mon[32458]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:47.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:46 smithi044 ceph-mon[32458]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:47.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:46 smithi155 ceph-mon[33468]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:47.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:46 smithi155 ceph-mon[33468]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.198+0000 7fce1d070700 1 -- 172.21.15.19:0/2558306830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce18105f00 msgr2=0x7fce181082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.198+0000 7fce1d070700 1 --2- 172.21.15.19:0/2558306830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce18105f00 0x7fce181082f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009990 tx=0x7fce0c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.198+0000 7fce1d070700 1 -- 172.21.15.19:0/2558306830 shutdown_connections 2024-04-23T18:25:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.198+0000 7fce1d070700 1 --2- 172.21.15.19:0/2558306830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce18108830 0x7fce1810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.198+0000 7fce1d070700 1 --2- 172.21.15.19:0/2558306830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce181035d0 0x7fce181059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.198+0000 7fce1d070700 1 --2- 172.21.15.19:0/2558306830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce18105f00 0x7fce181082f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.198+0000 7fce1d070700 1 -- 172.21.15.19:0/2558306830 >> 172.21.15.19:0/2558306830 conn(0x7fce180fcf60 msgr2=0x7fce180ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.198+0000 7fce1d070700 1 -- 172.21.15.19:0/2558306830 shutdown_connections 2024-04-23T18:25:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.198+0000 7fce1d070700 1 -- 172.21.15.19:0/2558306830 wait complete. 2024-04-23T18:25:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.199+0000 7fce1d070700 1 Processor -- start 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.199+0000 7fce1d070700 1 -- start start 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.199+0000 7fce1d070700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce181035d0 0x7fce18198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.199+0000 7fce1d070700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18105f00 0x7fce18199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.199+0000 7fce1d070700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce18108830 0x7fce181a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.199+0000 7fce1d070700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce1810e570 con 0x7fce181035d0 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.199+0000 7fce1d070700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce1810e3f0 con 0x7fce18108830 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.199+0000 7fce1d070700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce1810e6f0 con 0x7fce18105f00 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.199+0000 7fce16d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce181035d0 0x7fce18198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce1759e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce18108830 0x7fce181a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce16d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce181035d0 0x7fce18198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59248/0 (socket says 172.21.15.19:59248) 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce16d9d700 1 -- 172.21.15.19:0/2195312762 learned_addr learned my addr 172.21.15.19:0/2195312762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce1759e700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce18108830 0x7fce181a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:57622/0 (socket says 172.21.15.19:57622) 2024-04-23T18:25:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce1659c700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18105f00 0x7fce18199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce16d9d700 1 -- 172.21.15.19:0/2195312762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18105f00 msgr2=0x7fce18199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce16d9d700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18105f00 0x7fce18199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce16d9d700 1 -- 172.21.15.19:0/2195312762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce18108830 msgr2=0x7fce181a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce16d9d700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce18108830 0x7fce181a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce16d9d700 1 -- 172.21.15.19:0/2195312762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce181a0890 con 0x7fce181035d0 2024-04-23T18:25:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce16d9d700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce181035d0 0x7fce18198c40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fce0800ada0 tx=0x7fce0800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fcdfffff700 1 -- 172.21.15.19:0/2195312762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08017020 con 0x7fce181035d0 2024-04-23T18:25:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce1d070700 1 -- 172.21.15.19:0/2195312762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce181a0b80 con 0x7fce181035d0 2024-04-23T18:25:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fcdfffff700 1 -- 172.21.15.19:0/2195312762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0800cd80 con 0x7fce181035d0 2024-04-23T18:25:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fcdfffff700 1 -- 172.21.15.19:0/2195312762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08005370 con 0x7fce181035d0 2024-04-23T18:25:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.200+0000 7fce1d070700 1 -- 172.21.15.19:0/2195312762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce181a10c0 con 0x7fce181035d0 2024-04-23T18:25:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.201+0000 7fce1d070700 1 -- 172.21.15.19:0/2195312762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce18069730 con 0x7fce181035d0 2024-04-23T18:25:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.202+0000 7fcdfffff700 1 -- 172.21.15.19:0/2195312762 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fce08004ad0 con 0x7fce181035d0 2024-04-23T18:25:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.203+0000 7fcdfffff700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fce000a3c40 0x7fce000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:48.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.203+0000 7fce1659c700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fce000a3c40 0x7fce000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:48.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.203+0000 7fcdfffff700 1 -- 172.21.15.19:0/2195312762 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fce080cff90 con 0x7fce181035d0 2024-04-23T18:25:48.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.204+0000 7fce1659c700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fce000a3c40 0x7fce000a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b5c0 tx=0x7fce0c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:48.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.207+0000 7fcdfffff700 1 -- 172.21.15.19:0/2195312762 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce08021020 con 0x7fce181035d0 2024-04-23T18:25:48.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.425+0000 7fce1d070700 1 -- 172.21.15.19:0/2195312762 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fce18062e80 con 0x7fce000a3c40 2024-04-23T18:25:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.427+0000 7fcdfffff700 1 -- 172.21.15.19:0/2195312762 <== mgr.14150 v2:172.21.15.19:6800/1993809272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fce18062e80 con 0x7fce000a3c40 2024-04-23T18:25:48.420 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-23T18:25:48.420 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-23T18:25:48.420 INFO:teuthology.orchestra.run.smithi019.stdout:smithi044 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001K400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-23T18:25:48.420 INFO:teuthology.orchestra.run.smithi019.stdout:smithi155 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GW400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-23T18:25:48.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 -- 172.21.15.19:0/2195312762 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fce000a3c40 msgr2=0x7fce000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:48.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fce000a3c40 0x7fce000a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b5c0 tx=0x7fce0c005fb0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 -- 172.21.15.19:0/2195312762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce181035d0 msgr2=0x7fce18198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:48.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce181035d0 0x7fce18198c40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fce0800ada0 tx=0x7fce0800c330 comp rx=0 tx=0).stop 2024-04-23T18:25:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 -- 172.21.15.19:0/2195312762 shutdown_connections 2024-04-23T18:25:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fce000a3c40 0x7fce000a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce18108830 0x7fce181a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce181035d0 0x7fce18198c40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 --2- 172.21.15.19:0/2195312762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18105f00 0x7fce18199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 -- 172.21.15.19:0/2195312762 >> 172.21.15.19:0/2195312762 conn(0x7fce180fcf60 msgr2=0x7fce181053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 -- 172.21.15.19:0/2195312762 shutdown_connections 2024-04-23T18:25:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:48.429+0000 7fce1d070700 1 -- 172.21.15.19:0/2195312762 wait complete. 2024-04-23T18:25:49.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:48 smithi019 ceph-mon[29215]: pgmap v354: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:49.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:48 smithi044 ceph-mon[32458]: pgmap v354: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:49.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:48 smithi155 ceph-mon[33468]: pgmap v354: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:49.148 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T18:25:49.158 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:25:49.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-23T18:25:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:49.877 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:49 smithi019 ceph-mon[29215]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:50.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:49 smithi044 ceph-mon[32458]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:50.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:49 smithi155 ceph-mon[33468]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:25:51.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:50 smithi019 ceph-mon[29215]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:51.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:50 smithi044 ceph-mon[32458]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:51.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:50 smithi155 ceph-mon[33468]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.322+0000 7f4805be2700 1 -- 172.21.15.19:0/1500410910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4800103ca0 msgr2=0x7f4800104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.322+0000 7f4805be2700 1 --2- 172.21.15.19:0/1500410910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4800103ca0 0x7f4800104100 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f47f40099c0 tx=0x7f47f4009cd0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.322+0000 7f4805be2700 1 -- 172.21.15.19:0/1500410910 shutdown_connections 2024-04-23T18:25:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.322+0000 7f4805be2700 1 --2- 172.21.15.19:0/1500410910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4800104640 0x7f480010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.322+0000 7f4805be2700 1 --2- 172.21.15.19:0/1500410910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4800102aa0 0x7f4800102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.322+0000 7f4805be2700 1 --2- 172.21.15.19:0/1500410910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4800103ca0 0x7f4800104100 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.322+0000 7f4805be2700 1 -- 172.21.15.19:0/1500410910 >> 172.21.15.19:0/1500410910 conn(0x7f48000fe040 msgr2=0x7f4800100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.322+0000 7f4805be2700 1 -- 172.21.15.19:0/1500410910 shutdown_connections 2024-04-23T18:25:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.322+0000 7f4805be2700 1 -- 172.21.15.19:0/1500410910 wait complete. 2024-04-23T18:25:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f4805be2700 1 Processor -- start 2024-04-23T18:25:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f4805be2700 1 -- start start 2024-04-23T18:25:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f4805be2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4800102aa0 0x7f4800198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f4805be2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4800103ca0 0x7f4800199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f4805be2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4800104640 0x7f48001a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f4805be2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f480010e510 con 0x7f4800104640 2024-04-23T18:25:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f47fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4800104640 0x7f48001a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f47fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4800104640 0x7f48001a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42854/0 (socket says 172.21.15.19:42854) 2024-04-23T18:25:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f47fffff700 1 -- 172.21.15.19:0/1640322013 learned_addr learned my addr 172.21.15.19:0/1640322013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f480010e390 con 0x7f4800103ca0 2024-04-23T18:25:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f480010e690 con 0x7f4800102aa0 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.323+0000 7f47ff7fe700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4800102aa0 0x7f4800198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f47feffd700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4800103ca0 0x7f4800199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f47fffff700 1 -- 172.21.15.19:0/1640322013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4800102aa0 msgr2=0x7f4800198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f47fffff700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4800102aa0 0x7f4800198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f47fffff700 1 -- 172.21.15.19:0/1640322013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4800103ca0 msgr2=0x7f4800199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f47fffff700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4800103ca0 0x7f4800199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f47fffff700 1 -- 172.21.15.19:0/1640322013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48001a0840 con 0x7f4800104640 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f47feffd700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4800103ca0 0x7f4800199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f47fffff700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4800104640 0x7f48001a01a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f47f8007f10 tx=0x7f47f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f47fcff9700 1 -- 172.21.15.19:0/1640322013 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f800ddd0 con 0x7f4800104640 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f47fcff9700 1 -- 172.21.15.19:0/1640322013 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f8004020 con 0x7f4800104640 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.324+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4800074cd0 con 0x7f4800104640 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.325+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48000751c0 con 0x7f4800104640 2024-04-23T18:25:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.325+0000 7f47fcff9700 1 -- 172.21.15.19:0/1640322013 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8015660 con 0x7f4800104640 2024-04-23T18:25:51.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.326+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4800069730 con 0x7f4800104640 2024-04-23T18:25:51.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.327+0000 7f47fcff9700 1 -- 172.21.15.19:0/1640322013 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f47f80044f0 con 0x7f4800104640 2024-04-23T18:25:51.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.328+0000 7f47fcff9700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f47e80a3c40 0x7f47e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.328+0000 7f47ff7fe700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f47e80a3c40 0x7f47e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.328+0000 7f47fcff9700 1 -- 172.21.15.19:0/1640322013 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f47f80d05c0 con 0x7f4800104640 2024-04-23T18:25:51.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.328+0000 7f47ff7fe700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f47e80a3c40 0x7f47e80a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4800103b00 tx=0x7f47f000f7d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.332+0000 7f47fcff9700 1 -- 172.21.15.19:0/1640322013 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47f8010070 con 0x7f4800104640 2024-04-23T18:25:51.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.593+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f480002df80 con 0x7f4800104640 2024-04-23T18:25:51.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.594+0000 7f47fcff9700 1 -- 172.21.15.19:0/1640322013 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3078 (secure 0 0 0) 0x7f47f809cfb0 con 0x7f4800104640 2024-04-23T18:25:51.587 INFO:teuthology.orchestra.run.smithi019.stdout:e26 2024-04-23T18:25:51.587 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:25:51.587 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:25:51.587 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-04-23T18:25:51.587 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:51.587 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-04-23T18:23:55.611471+0000 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-04-23T18:24:02.624724+0000 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 300 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:25:51.588 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 5 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1,2,3,4 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=24521,1=24508,2=24491,3=24487,4=24469} 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:qdb_cluster leader: 24521 members: 24487,24508,24469,24521,24491 2024-04-23T18:25:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b{0:24521} state up:active seq 3 addr [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.e{0:24467} state up:standby-replay seq 1 addr [v2:172.21.15.44:6834/659103876,v1:172.21.15.44:6835/659103876] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.i{1:24508} state up:active seq 6 addr [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.k{1:24431} state up:standby-replay seq 1 addr [v2:172.21.15.44:6832/3545031610,v1:172.21.15.44:6833/3545031610] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.h{2:24491} state up:active seq 7 addr [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.l{2:24427} state up:standby-replay seq 1 addr [v2:172.21.15.155:6832/3128313210,v1:172.21.15.155:6833/3128313210] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c{3:24487} state up:active seq 8 addr [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a{3:14604} state up:standby-replay seq 1 addr [v2:172.21.15.19:6840/2593314342,v1:172.21.15.19:6841/2593314342] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.f{4:24469} state up:active seq 9 addr [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.g{4:14586} state up:standby-replay seq 1 addr [v2:172.21.15.19:6838/716138605,v1:172.21.15.19:6839/716138605] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:51.590 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.j{-1:14514} state up:standby seq 1 addr [v2:172.21.15.19:6834/2967585130,v1:172.21.15.19:6835/2967585130] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.591 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d{-1:14526} state up:standby seq 1 addr [v2:172.21.15.19:6836/781445125,v1:172.21.15.19:6837/781445125] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:25:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.596+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f47e80a3c40 msgr2=0x7f47e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.596+0000 7f4805be2700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f47e80a3c40 0x7f47e80a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4800103b00 tx=0x7f47f000f7d0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.596+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4800104640 msgr2=0x7f48001a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.596+0000 7f4805be2700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4800104640 0x7f48001a01a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f47f8007f10 tx=0x7f47f800d3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.597+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 shutdown_connections 2024-04-23T18:25:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.597+0000 7f4805be2700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f47e80a3c40 0x7f47e80a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.597+0000 7f4805be2700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4800104640 0x7f48001a01a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.597+0000 7f4805be2700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4800102aa0 0x7f4800198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.597+0000 7f4805be2700 1 --2- 172.21.15.19:0/1640322013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4800103ca0 0x7f4800199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.597+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 >> 172.21.15.19:0/1640322013 conn(0x7f48000fe040 msgr2=0x7f48001070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.597+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 shutdown_connections 2024-04-23T18:25:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:51.597+0000 7f4805be2700 1 -- 172.21.15.19:0/1640322013 wait complete. 2024-04-23T18:25:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-23T18:25:51.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:51 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1640322013' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:25:52.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:51 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1640322013' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:25:52.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:51 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1640322013' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:25:52.455 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46706838-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-23T18:25:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:52.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:52 smithi019 ceph-mon[29215]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-23T18:25:53.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:52 smithi044 ceph-mon[32458]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-23T18:25:53.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:52 smithi155 ceph-mon[33468]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-23T18:25:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.547+0000 7fe393fa5700 1 -- 172.21.15.19:0/4161566532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe38c103cc0 msgr2=0x7fe38c104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.547+0000 7fe393fa5700 1 --2- 172.21.15.19:0/4161566532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe38c103cc0 0x7fe38c104120 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe3800099c0 tx=0x7fe380009cd0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.547+0000 7fe393fa5700 1 -- 172.21.15.19:0/4161566532 shutdown_connections 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.547+0000 7fe393fa5700 1 --2- 172.21.15.19:0/4161566532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c104660 0x7fe38c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.547+0000 7fe393fa5700 1 --2- 172.21.15.19:0/4161566532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38c102ac0 0x7fe38c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.547+0000 7fe393fa5700 1 --2- 172.21.15.19:0/4161566532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe38c103cc0 0x7fe38c104120 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.547+0000 7fe393fa5700 1 -- 172.21.15.19:0/4161566532 >> 172.21.15.19:0/4161566532 conn(0x7fe38c0fe060 msgr2=0x7fe38c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.547+0000 7fe393fa5700 1 -- 172.21.15.19:0/4161566532 shutdown_connections 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.548+0000 7fe393fa5700 1 -- 172.21.15.19:0/4161566532 wait complete. 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.548+0000 7fe393fa5700 1 Processor -- start 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.548+0000 7fe393fa5700 1 -- start start 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.548+0000 7fe393fa5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38c102ac0 0x7fe38c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.548+0000 7fe393fa5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c103cc0 0x7fe38c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.548+0000 7fe393fa5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe38c104660 0x7fe38c1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.548+0000 7fe393fa5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c10e5d0 con 0x7fe38c104660 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.548+0000 7fe393fa5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c10e450 con 0x7fe38c102ac0 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.548+0000 7fe393fa5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c10e750 con 0x7fe38c103cc0 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe392542700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe38c104660 0x7fe38c1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe392542700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe38c104660 0x7fe38c1a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42874/0 (socket says 172.21.15.19:42874) 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe392542700 1 -- 172.21.15.19:0/484269951 learned_addr learned my addr 172.21.15.19:0/484269951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe391d41700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38c102ac0 0x7fe38c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe391540700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c103cc0 0x7fe38c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe392542700 1 -- 172.21.15.19:0/484269951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c103cc0 msgr2=0x7fe38c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe392542700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c103cc0 0x7fe38c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe392542700 1 -- 172.21.15.19:0/484269951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38c102ac0 msgr2=0x7fe38c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe392542700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38c102ac0 0x7fe38c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe392542700 1 -- 172.21.15.19:0/484269951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c1a2a60 con 0x7fe38c104660 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe392542700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe38c104660 0x7fe38c1a23c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe384007da0 tx=0x7fe38400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe37effd700 1 -- 172.21.15.19:0/484269951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384016020 con 0x7fe38c104660 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe393fa5700 1 -- 172.21.15.19:0/484269951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38c1a2d50 con 0x7fe38c104660 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe37effd700 1 -- 172.21.15.19:0/484269951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe38400ad00 con 0x7fe38c104660 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe37effd700 1 -- 172.21.15.19:0/484269951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3840115e0 con 0x7fe38c104660 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.549+0000 7fe393fa5700 1 -- 172.21.15.19:0/484269951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38c1a31e0 con 0x7fe38c104660 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.550+0000 7fe393fa5700 1 -- 172.21.15.19:0/484269951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe38c069760 con 0x7fe38c104660 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.552+0000 7fe37effd700 1 -- 172.21.15.19:0/484269951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe384019020 con 0x7fe38c104660 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.553+0000 7fe37effd700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe3780a3c40 0x7fe3780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.553+0000 7fe391d41700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe3780a3c40 0x7fe3780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.553+0000 7fe37effd700 1 -- 172.21.15.19:0/484269951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe3840d0830 con 0x7fe38c104660 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.553+0000 7fe391d41700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe3780a3c40 0x7fe3780a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe38c103b20 tx=0x7fe388009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.556+0000 7fe37effd700 1 -- 172.21.15.19:0/484269951 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe38c069760 con 0x7fe38c104660 2024-04-23T18:25:54.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.766+0000 7fe393fa5700 1 -- 172.21.15.19:0/484269951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe38c1a34c0 con 0x7fe38c104660 2024-04-23T18:25:54.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.766+0000 7fe37effd700 1 -- 172.21.15.19:0/484269951 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5281 (secure 0 0 0) 0x7fe384020020 con 0x7fe38c104660 2024-04-23T18:25:54.759 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 88 2024-04-23T18:25:54.759 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-04-23T18:15:32.046592+0000 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-04-23T18:24:36.689178+0000 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 26 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client luminous 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-23T18:25:54.760 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 12 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6802/4279899748,v1:172.21.15.19:6803/4279899748] [v2:172.21.15.19:6804/4279899748,v1:172.21.15.19:6805/4279899748] exists,up 78ea8328-c293-4ef2-af0d-81bfe0dccc4c 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6800/2943619157,v1:172.21.15.44:6801/2943619157] [v2:172.21.15.44:6802/2943619157,v1:172.21.15.44:6803/2943619157] exists,up 3e9e3631-24c8-42d1-84c6-48cf0cda9283 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/3680785490,v1:172.21.15.155:6801/3680785490] [v2:172.21.15.155:6802/3680785490,v1:172.21.15.155:6803/3680785490] exists,up 680b8f94-cb10-444d-a1a4-22c12f9cbe1f 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6810/3983142737,v1:172.21.15.19:6811/3983142737] [v2:172.21.15.19:6812/3983142737,v1:172.21.15.19:6813/3983142737] exists,up 31e55972-9f96-4714-b681-fa4af600b35b 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6808/3405107831,v1:172.21.15.44:6809/3405107831] [v2:172.21.15.44:6810/3405107831,v1:172.21.15.44:6811/3405107831] exists,up a48cc342-4cea-4410-8b19-0f049a183667 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6808/2897505924,v1:172.21.15.155:6809/2897505924] [v2:172.21.15.155:6810/2897505924,v1:172.21.15.155:6811/2897505924] exists,up 777efb4b-82b9-4ce6-8187-80697bffc735 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6818/2995627219,v1:172.21.15.19:6819/2995627219] [v2:172.21.15.19:6820/2995627219,v1:172.21.15.19:6821/2995627219] exists,up a0be6429-6e18-42e7-b385-34826780f118 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6816/2128528085,v1:172.21.15.44:6817/2128528085] [v2:172.21.15.44:6818/2128528085,v1:172.21.15.44:6819/2128528085] exists,up 71f568c1-298c-4d3a-a191-02ffd057f620 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/3480056050,v1:172.21.15.155:6817/3480056050] [v2:172.21.15.155:6818/3480056050,v1:172.21.15.155:6819/3480056050] exists,up 49dae33b-4d26-4145-a4eb-3ffd2dc50e46 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6826/2566199237,v1:172.21.15.19:6827/2566199237] [v2:172.21.15.19:6828/2566199237,v1:172.21.15.19:6829/2566199237] exists,up 7715f388-45b3-47c4-82a2-d67a2de63b07 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6824/2033313090,v1:172.21.15.44:6825/2033313090] [v2:172.21.15.44:6826/2033313090,v1:172.21.15.44:6827/2033313090] exists,up 393911c2-6938-4f7a-93d6-6c187f236b60 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/3982643864,v1:172.21.15.155:6825/3982643864] [v2:172.21.15.155:6826/3982643864,v1:172.21.15.155:6827/3982643864] exists,up 35d577a0-4fd5-4e4d-a402-71434ca33c95 2024-04-23T18:25:54.761 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.0 [8,1] 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.b [8,11] 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.10 [2,4] 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.13 [2,11] 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1 [4,9] 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1b [4,8] 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1d [5,9] 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.24 [4,9] 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1473447325 expires 2024-04-24T18:16:35.264133+0000 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/2431053142 expires 2024-04-24T18:16:35.264133+0000 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3826407819 expires 2024-04-24T18:16:06.817907+0000 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/4193386941 expires 2024-04-24T18:16:06.817907+0000 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1551393169 expires 2024-04-24T18:16:35.264133+0000 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/2431053142 expires 2024-04-24T18:16:35.264133+0000 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/4087353417 expires 2024-04-24T18:16:06.817907+0000 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/1085063871 expires 2024-04-24T18:16:06.817907+0000 2024-04-23T18:25:54.762 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/1085063871 expires 2024-04-24T18:16:06.817907+0000 2024-04-23T18:25:54.763 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/2920477219 expires 2024-04-24T18:16:35.264133+0000 2024-04-23T18:25:54.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.768+0000 7fe393fa5700 1 -- 172.21.15.19:0/484269951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe3780a3c40 msgr2=0x7fe3780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.768+0000 7fe393fa5700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe3780a3c40 0x7fe3780a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe38c103b20 tx=0x7fe388009190 comp rx=0 tx=0).stop 2024-04-23T18:25:54.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.768+0000 7fe393fa5700 1 -- 172.21.15.19:0/484269951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe38c104660 msgr2=0x7fe38c1a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.768+0000 7fe393fa5700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe38c104660 0x7fe38c1a23c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe384007da0 tx=0x7fe38400a340 comp rx=0 tx=0).stop 2024-04-23T18:25:54.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.768+0000 7fe393fa5700 1 -- 172.21.15.19:0/484269951 shutdown_connections 2024-04-23T18:25:54.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.768+0000 7fe393fa5700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fe3780a3c40 0x7fe3780a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.768+0000 7fe393fa5700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe38c104660 0x7fe38c1a23c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.768+0000 7fe393fa5700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe38c102ac0 0x7fe38c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.768+0000 7fe393fa5700 1 --2- 172.21.15.19:0/484269951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c103cc0 0x7fe38c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.769+0000 7fe393fa5700 1 -- 172.21.15.19:0/484269951 >> 172.21.15.19:0/484269951 conn(0x7fe38c0fe060 msgr2=0x7fe38c074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.769+0000 7fe393fa5700 1 -- 172.21.15.19:0/484269951 shutdown_connections 2024-04-23T18:25:54.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:54.769+0000 7fe393fa5700 1 -- 172.21.15.19:0/484269951 wait complete. 2024-04-23T18:25:55.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:54 smithi019 ceph-mon[29215]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:55.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:54 smithi044 ceph-mon[32458]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:55.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:54 smithi155 ceph-mon[33468]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:25:55.524 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-23T18:25:55.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:25:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.652+0000 7ffa38b68640 1 -- 172.21.15.19:0/1918072874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa34100ea0 msgr2=0x7ffa341012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.652+0000 7ffa38b68640 1 --2- 172.21.15.19:0/1918072874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa34100ea0 0x7ffa341012c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009900 tx=0x7ffa2402a410 comp rx=0 tx=0).stop 2024-04-23T18:25:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.652+0000 7ffa38b68640 1 -- 172.21.15.19:0/1918072874 shutdown_connections 2024-04-23T18:25:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.652+0000 7ffa38b68640 1 --2- 172.21.15.19:0/1918072874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa34101ff0 0x7ffa34102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.652+0000 7ffa38b68640 1 --2- 172.21.15.19:0/1918072874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa341029b0 0x7ffa341092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.652+0000 7ffa38b68640 1 --2- 172.21.15.19:0/1918072874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa34100ea0 0x7ffa341012c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.652+0000 7ffa38b68640 1 -- 172.21.15.19:0/1918072874 >> 172.21.15.19:0/1918072874 conn(0x7ffa340fc430 msgr2=0x7ffa340fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.652+0000 7ffa38b68640 1 -- 172.21.15.19:0/1918072874 shutdown_connections 2024-04-23T18:25:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.652+0000 7ffa38b68640 1 -- 172.21.15.19:0/1918072874 wait complete. 2024-04-23T18:25:55.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.653+0000 7ffa38b68640 1 Processor -- start 2024-04-23T18:25:55.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.653+0000 7ffa38b68640 1 -- start start 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.653+0000 7ffa38b68640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa34100ea0 0x7ffa34198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.653+0000 7ffa38b68640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa34101ff0 0x7ffa341987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.653+0000 7ffa38b68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa341029b0 0x7ffa3419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.653+0000 7ffa38b68640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3410c9f0 con 0x7ffa341029b0 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.653+0000 7ffa38b68640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3410c870 con 0x7ffa34101ff0 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.653+0000 7ffa38b68640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3410cb70 con 0x7ffa34100ea0 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa32d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa341029b0 0x7ffa3419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa32d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa341029b0 0x7ffa3419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42894/0 (socket says 172.21.15.19:42894) 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa32d9d640 1 -- 172.21.15.19:0/1472115946 learned_addr learned my addr 172.21.15.19:0/1472115946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa3259c640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa34100ea0 0x7ffa34198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa31d9b640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa34101ff0 0x7ffa341987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa32d9d640 1 -- 172.21.15.19:0/1472115946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa34100ea0 msgr2=0x7ffa34198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa32d9d640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa34100ea0 0x7ffa34198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa32d9d640 1 -- 172.21.15.19:0/1472115946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa34101ff0 msgr2=0x7ffa341987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa32d9d640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa34101ff0 0x7ffa341987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa32d9d640 1 -- 172.21.15.19:0/1472115946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3419fe50 con 0x7ffa341029b0 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa31d9b640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa34101ff0 0x7ffa341987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa32d9d640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa341029b0 0x7ffa3419f7e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00eb10 tx=0x7ffa2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa237fe640 1 -- 172.21.15.19:0/1472115946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c004b20 con 0x7ffa341029b0 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa237fe640 1 -- 172.21.15.19:0/1472115946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2c004cc0 con 0x7ffa341029b0 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa38b68640 1 -- 172.21.15.19:0/1472115946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa341a0140 con 0x7ffa341029b0 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa237fe640 1 -- 172.21.15.19:0/1472115946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c010670 con 0x7ffa341029b0 2024-04-23T18:25:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.654+0000 7ffa38b68640 1 -- 172.21.15.19:0/1472115946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa341a0690 con 0x7ffa341029b0 2024-04-23T18:25:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.655+0000 7ffa38b68640 1 -- 172.21.15.19:0/1472115946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa341a0950 con 0x7ffa341029b0 2024-04-23T18:25:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.656+0000 7ffa237fe640 1 -- 172.21.15.19:0/1472115946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffa2c005020 con 0x7ffa341029b0 2024-04-23T18:25:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.657+0000 7ffa237fe640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ffa1c0a3bb0 0x7ffa1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.657+0000 7ffa3259c640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ffa1c0a3bb0 0x7ffa1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.658+0000 7ffa237fe640 1 -- 172.21.15.19:0/1472115946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ffa2c0d0a90 con 0x7ffa341029b0 2024-04-23T18:25:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.658+0000 7ffa3259c640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ffa1c0a3bb0 0x7ffa1c0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffa24000c00 tx=0x7ffa240314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.662+0000 7ffa237fe640 1 -- 172.21.15.19:0/1472115946 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa2c09d480 con 0x7ffa341029b0 2024-04-23T18:25:55.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:55 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/484269951' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:25:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.849+0000 7ffa38b68640 1 -- 172.21.15.19:0/1472115946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa341a0b20 con 0x7ffa341029b0 2024-04-23T18:25:55.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.851+0000 7ffa237fe640 1 -- 172.21.15.19:0/1472115946 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22642 (secure 0 0 0) 0x7ffa2c09cd80 con 0x7ffa341029b0 2024-04-23T18:25:55.844 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:55.844 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:24:36.689178+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 -- 172.21.15.19:0/1472115946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ffa1c0a3bb0 msgr2=0x7ffa1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ffa1c0a3bb0 0x7ffa1c0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffa24000c00 tx=0x7ffa240314a0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 -- 172.21.15.19:0/1472115946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa341029b0 msgr2=0x7ffa3419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa341029b0 0x7ffa3419f7e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00eb10 tx=0x7ffa2c00c2d0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 -- 172.21.15.19:0/1472115946 shutdown_connections 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7ffa1c0a3bb0 0x7ffa1c0a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa34101ff0 0x7ffa341987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa341029b0 0x7ffa3419f7e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 --2- 172.21.15.19:0/1472115946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa34100ea0 0x7ffa34198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 -- 172.21.15.19:0/1472115946 >> 172.21.15.19:0/1472115946 conn(0x7ffa340fc430 msgr2=0x7ffa340fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 -- 172.21.15.19:0/1472115946 shutdown_connections 2024-04-23T18:25:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.853+0000 7ffa38b68640 1 -- 172.21.15.19:0/1472115946 wait complete. 2024-04-23T18:25:55.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:23:53.296504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:23:54.573484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:25:55.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.969+0000 7efd92423640 1 -- 172.21.15.19:0/861401111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c1029a0 msgr2=0x7efd8c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.969+0000 7efd92423640 1 --2- 172.21.15.19:0/861401111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c1029a0 0x7efd8c1092c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efd84009ca0 tx=0x7efd8402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.969+0000 7efd92423640 1 -- 172.21.15.19:0/861401111 shutdown_connections 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.969+0000 7efd92423640 1 --2- 172.21.15.19:0/861401111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd8c101fe0 0x7efd8c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.969+0000 7efd92423640 1 --2- 172.21.15.19:0/861401111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c1029a0 0x7efd8c1092c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.969+0000 7efd92423640 1 --2- 172.21.15.19:0/861401111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c100e90 0x7efd8c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.969+0000 7efd92423640 1 -- 172.21.15.19:0/861401111 >> 172.21.15.19:0/861401111 conn(0x7efd8c0fc400 msgr2=0x7efd8c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.970+0000 7efd92423640 1 -- 172.21.15.19:0/861401111 shutdown_connections 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.970+0000 7efd92423640 1 -- 172.21.15.19:0/861401111 wait complete. 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.970+0000 7efd92423640 1 Processor -- start 2024-04-23T18:25:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.970+0000 7efd92423640 1 -- start start 2024-04-23T18:25:55.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.970+0000 7efd92423640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c100e90 0x7efd8c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.970+0000 7efd92423640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c101fe0 0x7efd8c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.970+0000 7efd92423640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd8c1029a0 0x7efd8c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.970+0000 7efd92423640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c10c9f0 con 0x7efd8c100e90 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.970+0000 7efd92423640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c10c870 con 0x7efd8c1029a0 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.970+0000 7efd92423640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c10cb70 con 0x7efd8c101fe0 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c100e90 0x7efd8c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c100e90 0x7efd8c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42908/0 (socket says 172.21.15.19:42908) 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8bfff640 1 -- 172.21.15.19:0/511719527 learned_addr learned my addr 172.21.15.19:0/511719527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd909c0640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd8c1029a0 0x7efd8c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8b7fe640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c101fe0 0x7efd8c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8bfff640 1 -- 172.21.15.19:0/511719527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c101fe0 msgr2=0x7efd8c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8bfff640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c101fe0 0x7efd8c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8bfff640 1 -- 172.21.15.19:0/511719527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd8c1029a0 msgr2=0x7efd8c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8bfff640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd8c1029a0 0x7efd8c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8bfff640 1 -- 172.21.15.19:0/511719527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c19fdb0 con 0x7efd8c100e90 2024-04-23T18:25:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8b7fe640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c101fe0 0x7efd8c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd8bfff640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c100e90 0x7efd8c198190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00d770 tx=0x7efd7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd897fa640 1 -- 172.21.15.19:0/511719527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c017070 con 0x7efd8c100e90 2024-04-23T18:25:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd897fa640 1 -- 172.21.15.19:0/511719527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd7c0054d0 con 0x7efd8c100e90 2024-04-23T18:25:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd92423640 1 -- 172.21.15.19:0/511719527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c1a00a0 con 0x7efd8c100e90 2024-04-23T18:25:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd897fa640 1 -- 172.21.15.19:0/511719527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c004c30 con 0x7efd8c100e90 2024-04-23T18:25:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.971+0000 7efd92423640 1 -- 172.21.15.19:0/511719527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c1a05a0 con 0x7efd8c100e90 2024-04-23T18:25:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.972+0000 7efd92423640 1 -- 172.21.15.19:0/511719527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8c1a0800 con 0x7efd8c100e90 2024-04-23T18:25:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.974+0000 7efd897fa640 1 -- 172.21.15.19:0/511719527 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efd7c004dd0 con 0x7efd8c100e90 2024-04-23T18:25:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.974+0000 7efd897fa640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efd740a3bb0 0x7efd740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.975+0000 7efd8b7fe640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efd740a3bb0 0x7efd740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.975+0000 7efd897fa640 1 -- 172.21.15.19:0/511719527 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7efd7c0d02f0 con 0x7efd8c100e90 2024-04-23T18:25:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.975+0000 7efd8b7fe640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efd740a3bb0 0x7efd740a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efd8c1996a0 tx=0x7efd80005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:55.978+0000 7efd897fa640 1 -- 172.21.15.19:0/511719527 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd7c09cce0 con 0x7efd8c100e90 2024-04-23T18:25:56.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:55 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/484269951' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:25:56.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:55 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/484269951' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:25:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.168+0000 7efd92423640 1 -- 172.21.15.19:0/511719527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd8c1a09d0 con 0x7efd8c100e90 2024-04-23T18:25:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.168+0000 7efd897fa640 1 -- 172.21.15.19:0/511719527 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7efd7c09c5e0 con 0x7efd8c100e90 2024-04-23T18:25:56.161 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:25:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.170+0000 7efd92423640 1 -- 172.21.15.19:0/511719527 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efd740a3bb0 msgr2=0x7efd740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.170+0000 7efd92423640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efd740a3bb0 0x7efd740a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efd8c1996a0 tx=0x7efd80005db0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.170+0000 7efd92423640 1 -- 172.21.15.19:0/511719527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c100e90 msgr2=0x7efd8c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.170+0000 7efd92423640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c100e90 0x7efd8c198190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00d770 tx=0x7efd7c00dc30 comp rx=0 tx=0).stop 2024-04-23T18:25:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.171+0000 7efd92423640 1 -- 172.21.15.19:0/511719527 shutdown_connections 2024-04-23T18:25:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.171+0000 7efd92423640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7efd740a3bb0 0x7efd740a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.171+0000 7efd92423640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd8c1029a0 0x7efd8c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.171+0000 7efd92423640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c101fe0 0x7efd8c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.171+0000 7efd92423640 1 --2- 172.21.15.19:0/511719527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c100e90 0x7efd8c198190 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.171+0000 7efd92423640 1 -- 172.21.15.19:0/511719527 >> 172.21.15.19:0/511719527 conn(0x7efd8c0fc400 msgr2=0x7efd8c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.171+0000 7efd92423640 1 -- 172.21.15.19:0/511719527 shutdown_connections 2024-04-23T18:25:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.171+0000 7efd92423640 1 -- 172.21.15.19:0/511719527 wait complete. 2024-04-23T18:25:56.175 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:25:56.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.287+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2833236313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70101fb0 msgr2=0x7f3c701023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.287+0000 7f3c77aa0640 1 --2- 172.21.15.19:0/2833236313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70101fb0 0x7f3c701023d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009900 tx=0x7f3c6802a410 comp rx=0 tx=0).stop 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.287+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2833236313 shutdown_connections 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.287+0000 7f3c77aa0640 1 --2- 172.21.15.19:0/2833236313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c701031a0 0x7f3c70103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.287+0000 7f3c77aa0640 1 --2- 172.21.15.19:0/2833236313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70103b60 0x7f3c7010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.287+0000 7f3c77aa0640 1 --2- 172.21.15.19:0/2833236313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70101fb0 0x7f3c701023d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.287+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2833236313 >> 172.21.15.19:0/2833236313 conn(0x7f3c700fd540 msgr2=0x7f3c700ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.287+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2833236313 shutdown_connections 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.287+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2833236313 wait complete. 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c77aa0640 1 Processor -- start 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c77aa0640 1 -- start start 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c77aa0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70101fb0 0x7f3c701981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c77aa0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c701031a0 0x7f3c70198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c77aa0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70103b60 0x7f3c7019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c77aa0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7010daf0 con 0x7f3c70103b60 2024-04-23T18:25:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c77aa0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7010d970 con 0x7f3c701031a0 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c77aa0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7010dc70 con 0x7f3c70101fb0 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c7603d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70103b60 0x7f3c7019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c7603d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70103b60 0x7f3c7019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42924/0 (socket says 172.21.15.19:42924) 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c7603d640 1 -- 172.21.15.19:0/2301606714 learned_addr learned my addr 172.21.15.19:0/2301606714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c7583c640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70101fb0 0x7f3c701981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c7503b640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c701031a0 0x7f3c70198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c7603d640 1 -- 172.21.15.19:0/2301606714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70101fb0 msgr2=0x7f3c701981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c7603d640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70101fb0 0x7f3c701981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c7603d640 1 -- 172.21.15.19:0/2301606714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c701031a0 msgr2=0x7f3c70198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c7603d640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c701031a0 0x7f3c70198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.288+0000 7f3c7603d640 1 -- 172.21.15.19:0/2301606714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7019fde0 con 0x7f3c70103b60 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.289+0000 7f3c7603d640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70103b60 0x7f3c7019f770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000eaa0 tx=0x7f3c6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.289+0000 7f3c6effd640 1 -- 172.21.15.19:0/2301606714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000cdb0 con 0x7f3c70103b60 2024-04-23T18:25:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.289+0000 7f3c6effd640 1 -- 172.21.15.19:0/2301606714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c60005020 con 0x7f3c70103b60 2024-04-23T18:25:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.289+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2301606714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c701a00d0 con 0x7f3c70103b60 2024-04-23T18:25:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.289+0000 7f3c6effd640 1 -- 172.21.15.19:0/2301606714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c600116d0 con 0x7f3c70103b60 2024-04-23T18:25:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.289+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2301606714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701a0620 con 0x7f3c70103b60 2024-04-23T18:25:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.290+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2301606714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70109350 con 0x7f3c70103b60 2024-04-23T18:25:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.291+0000 7f3c6effd640 1 -- 172.21.15.19:0/2301606714 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3c600051d0 con 0x7f3c70103b60 2024-04-23T18:25:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.292+0000 7f3c6effd640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c580a3c30 0x7f3c580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.292+0000 7f3c7583c640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c580a3c30 0x7f3c580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.292+0000 7f3c6effd640 1 -- 172.21.15.19:0/2301606714 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3c600d0a20 con 0x7f3c70103b60 2024-04-23T18:25:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.292+0000 7f3c7583c640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c580a3c30 0x7f3c580a60f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3c68000c00 tx=0x7f3c680314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.295+0000 7f3c6effd640 1 -- 172.21.15.19:0/2301606714 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c60016030 con 0x7f3c70103b60 2024-04-23T18:25:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.480+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2301606714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3c7002d5c0 con 0x7f3c70103b60 2024-04-23T18:25:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.480+0000 7f3c6effd640 1 -- 172.21.15.19:0/2301606714 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f3c6009d430 con 0x7f3c70103b60 2024-04-23T18:25:56.473 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-23T18:25:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2301606714 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c580a3c30 msgr2=0x7f3c580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c580a3c30 0x7f3c580a60f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3c68000c00 tx=0x7f3c680314a0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2301606714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70103b60 msgr2=0x7f3c7019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70103b60 0x7f3c7019f770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000eaa0 tx=0x7f3c6000ef60 comp rx=0 tx=0).stop 2024-04-23T18:25:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2301606714 shutdown_connections 2024-04-23T18:25:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c580a3c30 0x7f3c580a60f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c701031a0 0x7f3c70198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70103b60 0x7f3c7019f770 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 --2- 172.21.15.19:0/2301606714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c70101fb0 0x7f3c701981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2301606714 >> 172.21.15.19:0/2301606714 conn(0x7f3c700fd540 msgr2=0x7f3c70106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2301606714 shutdown_connections 2024-04-23T18:25:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.482+0000 7f3c77aa0640 1 -- 172.21.15.19:0/2301606714 wait complete. 2024-04-23T18:25:56.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:25:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.598+0000 7f8f92ed9640 1 -- 172.21.15.19:0/3114739180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f8c107c90 msgr2=0x7f8f8c10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.598+0000 7f8f92ed9640 1 --2- 172.21.15.19:0/3114739180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f8c107c90 0x7f8f8c10a0e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009ca0 tx=0x7f8f8402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.598+0000 7f8f92ed9640 1 -- 172.21.15.19:0/3114739180 shutdown_connections 2024-04-23T18:25:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.598+0000 7f8f92ed9640 1 --2- 172.21.15.19:0/3114739180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f8c105360 0x7f8f8c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.598+0000 7f8f92ed9640 1 --2- 172.21.15.19:0/3114739180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f8c107c90 0x7f8f8c10a0e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.598+0000 7f8f92ed9640 1 --2- 172.21.15.19:0/3114739180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f8c102a30 0x7f8f8c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.598+0000 7f8f92ed9640 1 -- 172.21.15.19:0/3114739180 >> 172.21.15.19:0/3114739180 conn(0x7f8f8c0fc440 msgr2=0x7f8f8c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.598+0000 7f8f92ed9640 1 -- 172.21.15.19:0/3114739180 shutdown_connections 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.598+0000 7f8f92ed9640 1 -- 172.21.15.19:0/3114739180 wait complete. 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f92ed9640 1 Processor -- start 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f92ed9640 1 -- start start 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f92ed9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f8c102a30 0x7f8f8c1023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f92ed9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f8c105360 0x7f8f8c1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f92ed9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f8c107c90 0x7f8f8c100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f92ed9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c10daf0 con 0x7f8f8c105360 2024-04-23T18:25:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f92ed9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c10d970 con 0x7f8f8c107c90 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f92ed9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c10dc70 con 0x7f8f8c102a30 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f8bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f8c105360 0x7f8f8c1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f8bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f8c105360 0x7f8f8c1009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42936/0 (socket says 172.21.15.19:42936) 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f8bfff640 1 -- 172.21.15.19:0/2851011467 learned_addr learned my addr 172.21.15.19:0/2851011467 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f90c75640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f8c102a30 0x7f8f8c1023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f8bfff640 1 -- 172.21.15.19:0/2851011467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f8c102a30 msgr2=0x7f8f8c1023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f8bfff640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f8c102a30 0x7f8f8c1023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f8bfff640 1 -- 172.21.15.19:0/2851011467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f8c107c90 msgr2=0x7f8f8c100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f8bfff640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f8c107c90 0x7f8f8c100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f8bfff640 1 -- 172.21.15.19:0/2851011467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c1017b0 con 0x7f8f8c105360 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f90c75640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f8c102a30 0x7f8f8c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.599+0000 7f8f8bfff640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f8c105360 0x7f8f8c1009e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800c970 tx=0x7f8f7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.600+0000 7f8f89ffb640 1 -- 172.21.15.19:0/2851011467 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78014070 con 0x7f8f8c105360 2024-04-23T18:25:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.600+0000 7f8f89ffb640 1 -- 172.21.15.19:0/2851011467 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f78004440 con 0x7f8f8c105360 2024-04-23T18:25:56.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.600+0000 7f8f92ed9640 1 -- 172.21.15.19:0/2851011467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c1a2230 con 0x7f8f8c105360 2024-04-23T18:25:56.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.600+0000 7f8f89ffb640 1 -- 172.21.15.19:0/2851011467 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78003e50 con 0x7f8f8c105360 2024-04-23T18:25:56.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.600+0000 7f8f92ed9640 1 -- 172.21.15.19:0/2851011467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c1a27d0 con 0x7f8f8c105360 2024-04-23T18:25:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.601+0000 7f8f92ed9640 1 -- 172.21.15.19:0/2851011467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8c0fdf30 con 0x7f8f8c105360 2024-04-23T18:25:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.602+0000 7f8f89ffb640 1 -- 172.21.15.19:0/2851011467 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8f78017070 con 0x7f8f8c105360 2024-04-23T18:25:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.602+0000 7f8f89ffb640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f8f7c0a3dd0 0x7f8f7c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.603+0000 7f8f90c75640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f8f7c0a3dd0 0x7f8f7c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.603+0000 7f8f89ffb640 1 -- 172.21.15.19:0/2851011467 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8f780cfff0 con 0x7f8f8c105360 2024-04-23T18:25:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.603+0000 7f8f90c75640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f8f7c0a3dd0 0x7f8f7c0a6290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f800058c0 tx=0x7f8f80005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.607+0000 7f8f89ffb640 1 -- 172.21.15.19:0/2851011467 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f7801e030 con 0x7f8f8c105360 2024-04-23T18:25:56.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:56 smithi019 ceph-mon[29215]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:56.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:56 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1472115946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:56.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:56 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/511719527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:56.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:56 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2301606714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.796+0000 7f8f92ed9640 1 -- 172.21.15.19:0/2851011467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8f8c062b50 con 0x7f8f8c105360 2024-04-23T18:25:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.796+0000 7f8f89ffb640 1 -- 172.21.15.19:0/2851011467 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f8f7800de20 con 0x7f8f8c105360 2024-04-23T18:25:56.789 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-23T18:25:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.798+0000 7f8f92ed9640 1 -- 172.21.15.19:0/2851011467 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f8f7c0a3dd0 msgr2=0x7f8f7c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.798+0000 7f8f92ed9640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f8f7c0a3dd0 0x7f8f7c0a6290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f800058c0 tx=0x7f8f80005850 comp rx=0 tx=0).stop 2024-04-23T18:25:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.798+0000 7f8f92ed9640 1 -- 172.21.15.19:0/2851011467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f8c105360 msgr2=0x7f8f8c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.799+0000 7f8f92ed9640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f8c105360 0x7f8f8c1009e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800c970 tx=0x7f8f7800ce30 comp rx=0 tx=0).stop 2024-04-23T18:25:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.799+0000 7f8f92ed9640 1 -- 172.21.15.19:0/2851011467 shutdown_connections 2024-04-23T18:25:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.799+0000 7f8f92ed9640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f8f7c0a3dd0 0x7f8f7c0a6290 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.799+0000 7f8f92ed9640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f8c107c90 0x7f8f8c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.799+0000 7f8f92ed9640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f8c105360 0x7f8f8c1009e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.799+0000 7f8f92ed9640 1 --2- 172.21.15.19:0/2851011467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f8c102a30 0x7f8f8c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.799+0000 7f8f92ed9640 1 -- 172.21.15.19:0/2851011467 >> 172.21.15.19:0/2851011467 conn(0x7f8f8c0fc440 msgr2=0x7f8f8c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.799+0000 7f8f92ed9640 1 -- 172.21.15.19:0/2851011467 shutdown_connections 2024-04-23T18:25:56.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.799+0000 7f8f92ed9640 1 -- 172.21.15.19:0/2851011467 wait complete. 2024-04-23T18:25:56.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:25:56.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.912+0000 7f36b0587640 1 -- 172.21.15.19:0/2672858931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36a8101fc0 msgr2=0x7f36a81023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.912+0000 7f36b0587640 1 --2- 172.21.15.19:0/2672858931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36a8101fc0 0x7f36a81023e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009900 tx=0x7f36a402a410 comp rx=0 tx=0).stop 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.912+0000 7f36b0587640 1 -- 172.21.15.19:0/2672858931 shutdown_connections 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.912+0000 7f36b0587640 1 --2- 172.21.15.19:0/2672858931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36a81031b0 0x7f36a8103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.912+0000 7f36b0587640 1 --2- 172.21.15.19:0/2672858931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a8103b70 0x7f36a810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.912+0000 7f36b0587640 1 --2- 172.21.15.19:0/2672858931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36a8101fc0 0x7f36a81023e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.912+0000 7f36b0587640 1 -- 172.21.15.19:0/2672858931 >> 172.21.15.19:0/2672858931 conn(0x7f36a80fd550 msgr2=0x7f36a80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.912+0000 7f36b0587640 1 -- 172.21.15.19:0/2672858931 shutdown_connections 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.912+0000 7f36b0587640 1 -- 172.21.15.19:0/2672858931 wait complete. 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36b0587640 1 Processor -- start 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36b0587640 1 -- start start 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36b0587640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36a8101fc0 0x7f36a81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36b0587640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a81031b0 0x7f36a8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36b0587640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36a8103b70 0x7f36a819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36b0587640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36a810db00 con 0x7f36a8103b70 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36b0587640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f36a810d980 con 0x7f36a8101fc0 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36b0587640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36a810dc80 con 0x7f36a81031b0 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36aeb24640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36a8103b70 0x7f36a819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36aeb24640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36a8103b70 0x7f36a819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42962/0 (socket says 172.21.15.19:42962) 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36aeb24640 1 -- 172.21.15.19:0/871979840 learned_addr learned my addr 172.21.15.19:0/871979840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36adb22640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a81031b0 0x7f36a8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36ae323640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36a8101fc0 0x7f36a81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36aeb24640 1 -- 172.21.15.19:0/871979840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a81031b0 msgr2=0x7f36a8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36aeb24640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a81031b0 0x7f36a8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36aeb24640 1 -- 172.21.15.19:0/871979840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36a8101fc0 msgr2=0x7f36a81981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36aeb24640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36a8101fc0 0x7f36a81981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36aeb24640 1 -- 172.21.15.19:0/871979840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a819fe00 con 0x7f36a8103b70 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36ae323640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36a8101fc0 0x7f36a81981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.913+0000 7f36aeb24640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36a8103b70 0x7f36a819f790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f36a000eb10 tx=0x7f36a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.914+0000 7f369f7fe640 1 -- 172.21.15.19:0/871979840 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0004b20 con 0x7f36a8103b70 2024-04-23T18:25:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.914+0000 7f36b0587640 1 -- 172.21.15.19:0/871979840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a81a00f0 con 0x7f36a8103b70 2024-04-23T18:25:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.914+0000 7f369f7fe640 1 -- 172.21.15.19:0/871979840 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a0004cc0 con 0x7f36a8103b70 2024-04-23T18:25:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.914+0000 7f369f7fe640 1 -- 172.21.15.19:0/871979840 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0010670 con 0x7f36a8103b70 2024-04-23T18:25:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.914+0000 7f36b0587640 1 -- 172.21.15.19:0/871979840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a81a0640 con 0x7f36a8103b70 2024-04-23T18:25:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.915+0000 7f36b0587640 1 -- 172.21.15.19:0/871979840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a81093b0 con 0x7f36a8103b70 2024-04-23T18:25:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.916+0000 7f369f7fe640 1 -- 172.21.15.19:0/871979840 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f36a0005020 con 0x7f36a8103b70 2024-04-23T18:25:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.916+0000 7f369f7fe640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f36940a3bb0 0x7f36940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.917+0000 7f369f7fe640 1 -- 172.21.15.19:0/871979840 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f36a00d0a20 con 0x7f36a8103b70 2024-04-23T18:25:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.917+0000 7f36ae323640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f36940a3bb0 0x7f36940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.917+0000 7f36ae323640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f36940a3bb0 0x7f36940a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f36a4000c00 tx=0x7f36a40314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:56.921+0000 7f369f7fe640 1 -- 172.21.15.19:0/871979840 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36a009d3b0 con 0x7f36a8103b70 2024-04-23T18:25:57.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:56 smithi044 ceph-mon[32458]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:57.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:56 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1472115946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:57.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:56 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/511719527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:57.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:56 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2301606714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:57.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:56 smithi155 ceph-mon[33468]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:25:57.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:56 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1472115946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:57.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:56 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/511719527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:57.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:56 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2301606714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:57.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.148+0000 7f36b0587640 1 -- 172.21.15.19:0/871979840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36a8062b50 con 0x7f36a8103b70 2024-04-23T18:25:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.148+0000 7f369f7fe640 1 -- 172.21.15.19:0/871979840 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7f36a009ccb0 con 0x7f36a8103b70 2024-04-23T18:25:57.141 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:57.142 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:24:02.624724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24508,24469,24521,24491]},"id":1}]} 2024-04-23T18:25:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-23T18:25:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.150+0000 7f36b0587640 1 -- 172.21.15.19:0/871979840 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f36940a3bb0 msgr2=0x7f36940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.150+0000 7f36b0587640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f36940a3bb0 0x7f36940a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f36a4000c00 tx=0x7f36a40314a0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.150+0000 7f36b0587640 1 -- 172.21.15.19:0/871979840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36a8103b70 msgr2=0x7f36a819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.150+0000 7f36b0587640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36a8103b70 0x7f36a819f790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f36a000eb10 tx=0x7f36a000c2d0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.151+0000 7f36b0587640 1 -- 172.21.15.19:0/871979840 shutdown_connections 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.151+0000 7f36b0587640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f36940a3bb0 0x7f36940a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.151+0000 7f36b0587640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36a8103b70 0x7f36a819f790 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.151+0000 7f36b0587640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a81031b0 0x7f36a8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.151+0000 7f36b0587640 1 --2- 172.21.15.19:0/871979840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36a8101fc0 0x7f36a81981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.151+0000 7f36b0587640 1 -- 172.21.15.19:0/871979840 >> 172.21.15.19:0/871979840 conn(0x7f36a80fd550 msgr2=0x7f36a8106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.151+0000 7f36b0587640 1 -- 172.21.15.19:0/871979840 shutdown_connections 2024-04-23T18:25:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.151+0000 7f36b0587640 1 -- 172.21.15.19:0/871979840 wait complete. 2024-04-23T18:25:57.155 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:25:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 -- 172.21.15.19:0/993298798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c181031b0 msgr2=0x7f3c18103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 --2- 172.21.15.19:0/993298798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c181031b0 0x7f3c18103630 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009900 tx=0x7f3c0c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 -- 172.21.15.19:0/993298798 shutdown_connections 2024-04-23T18:25:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 --2- 172.21.15.19:0/993298798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c18103b70 0x7f3c1810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 --2- 172.21.15.19:0/993298798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c181031b0 0x7f3c18103630 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 --2- 172.21.15.19:0/993298798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c18101fc0 0x7f3c181023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 -- 172.21.15.19:0/993298798 >> 172.21.15.19:0/993298798 conn(0x7f3c180fd550 msgr2=0x7f3c180ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 -- 172.21.15.19:0/993298798 shutdown_connections 2024-04-23T18:25:57.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 -- 172.21.15.19:0/993298798 wait complete. 2024-04-23T18:25:57.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 Processor -- start 2024-04-23T18:25:57.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.265+0000 7f3c1d95d640 1 -- start start 2024-04-23T18:25:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1d95d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18101fc0 0x7f3c18198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1d95d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c181031b0 0x7f3c181986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1d95d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c18103b70 0x7f3c1819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1d95d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1810db00 con 0x7f3c18101fc0 2024-04-23T18:25:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1d95d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1810d980 con 0x7f3c18103b70 2024-04-23T18:25:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1d95d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1810dc80 con 0x7f3c181031b0 2024-04-23T18:25:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18101fc0 0x7f3c18198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18101fc0 0x7f3c18198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42978/0 (socket says 172.21.15.19:42978) 2024-04-23T18:25:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c16ffd640 1 -- 172.21.15.19:0/3838666930 learned_addr learned my addr 172.21.15.19:0/3838666930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c167fc640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c181031b0 0x7f3c181986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c16ffd640 1 -- 172.21.15.19:0/3838666930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c181031b0 msgr2=0x7f3c181986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c16ffd640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c181031b0 0x7f3c181986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c177fe640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c18103b70 0x7f3c1819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c16ffd640 1 -- 172.21.15.19:0/3838666930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c18103b70 msgr2=0x7f3c1819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c16ffd640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c18103b70 0x7f3c1819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c16ffd640 1 -- 172.21.15.19:0/3838666930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1819fd80 con 0x7f3c18101fc0 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c177fe640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c18103b70 0x7f3c1819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c16ffd640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18101fc0 0x7f3c18198160 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800c970 tx=0x7f3c0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1c95b640 1 -- 172.21.15.19:0/3838666930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08007c10 con 0x7f3c18101fc0 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1d95d640 1 -- 172.21.15.19:0/3838666930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c181a0070 con 0x7f3c18101fc0 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1c95b640 1 -- 172.21.15.19:0/3838666930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c08007db0 con 0x7f3c18101fc0 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1c95b640 1 -- 172.21.15.19:0/3838666930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800f870 con 0x7f3c18101fc0 2024-04-23T18:25:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.266+0000 7f3c1d95d640 1 -- 172.21.15.19:0/3838666930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c181a05c0 con 0x7f3c18101fc0 2024-04-23T18:25:57.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.267+0000 7f3c1d95d640 1 -- 172.21.15.19:0/3838666930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c181092e0 con 0x7f3c18101fc0 2024-04-23T18:25:57.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.269+0000 7f3c1c95b640 1 -- 172.21.15.19:0/3838666930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3c0800fa10 con 0x7f3c18101fc0 2024-04-23T18:25:57.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.269+0000 7f3c1c95b640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c000a3bb0 0x7f3c000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.269+0000 7f3c167fc640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c000a3bb0 0x7f3c000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.269+0000 7f3c1c95b640 1 -- 172.21.15.19:0/3838666930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3c080d0830 con 0x7f3c18101fc0 2024-04-23T18:25:57.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.270+0000 7f3c167fc640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c000a3bb0 0x7f3c000a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c18199670 tx=0x7f3c0c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.274+0000 7f3c1c95b640 1 -- 172.21.15.19:0/3838666930 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c0809d2b0 con 0x7f3c18101fc0 2024-04-23T18:25:57.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.463+0000 7f3c1d95d640 1 -- 172.21.15.19:0/3838666930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c18062b80 con 0x7f3c18101fc0 2024-04-23T18:25:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.464+0000 7f3c1c95b640 1 -- 172.21.15.19:0/3838666930 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22642 (secure 0 0 0) 0x7f3c0809cbb0 con 0x7f3c18101fc0 2024-04-23T18:25:57.458 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:57.458 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:24:36.689178+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:25:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.466+0000 7f3c1d95d640 1 -- 172.21.15.19:0/3838666930 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c000a3bb0 msgr2=0x7f3c000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.466+0000 7f3c1d95d640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c000a3bb0 0x7f3c000a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c18199670 tx=0x7f3c0c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.466+0000 7f3c1d95d640 1 -- 172.21.15.19:0/3838666930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18101fc0 msgr2=0x7f3c18198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.466+0000 7f3c1d95d640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18101fc0 0x7f3c18198160 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800c970 tx=0x7f3c0800ce30 comp rx=0 tx=0).stop 2024-04-23T18:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.466+0000 7f3c1d95d640 1 -- 172.21.15.19:0/3838666930 shutdown_connections 2024-04-23T18:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.466+0000 7f3c1d95d640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3c000a3bb0 0x7f3c000a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.466+0000 7f3c1d95d640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c18103b70 0x7f3c1819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.466+0000 7f3c1d95d640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c181031b0 0x7f3c181986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.466+0000 7f3c1d95d640 1 --2- 172.21.15.19:0/3838666930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18101fc0 0x7f3c18198160 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.466+0000 7f3c1d95d640 1 -- 172.21.15.19:0/3838666930 >> 172.21.15.19:0/3838666930 conn(0x7f3c180fd550 msgr2=0x7f3c181064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.467+0000 7f3c1d95d640 1 -- 172.21.15.19:0/3838666930 shutdown_connections 2024-04-23T18:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.467+0000 7f3c1d95d640 1 -- 172.21.15.19:0/3838666930 wait complete. 2024-04-23T18:25:57.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:23:53.296504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:23:54.573484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:25:57.471 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 -- 172.21.15.19:0/4135159368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c8100e60 msgr2=0x7fb6c8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 --2- 172.21.15.19:0/4135159368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c8100e60 0x7fb6c8101280 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009900 tx=0x7fb6b802a410 comp rx=0 tx=0).stop 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 -- 172.21.15.19:0/4135159368 shutdown_connections 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 --2- 172.21.15.19:0/4135159368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8102040 0x7fb6c81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 --2- 172.21.15.19:0/4135159368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8102a00 0x7fb6c8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 --2- 172.21.15.19:0/4135159368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c8100e60 0x7fb6c8101280 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 -- 172.21.15.19:0/4135159368 >> 172.21.15.19:0/4135159368 conn(0x7fb6c80fc440 msgr2=0x7fb6c80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 -- 172.21.15.19:0/4135159368 shutdown_connections 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 -- 172.21.15.19:0/4135159368 wait complete. 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 Processor -- start 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.584+0000 7fb6cda8a640 1 -- start start 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6cda8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8100e60 0x7fb6c81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6cda8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c8102040 0x7fb6c81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6cda8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8102a00 0x7fb6c819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6cda8a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810daf0 con 0x7fb6c8102040 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6cda8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810d970 con 0x7fb6c8100e60 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6cda8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810dc70 con 0x7fb6c8102a00 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c8102040 0x7fb6c81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c8102040 0x7fb6c81986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42996/0 (socket says 172.21.15.19:42996) 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c6ffd640 1 -- 172.21.15.19:0/3356578546 learned_addr learned my addr 172.21.15.19:0/3356578546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c77fe640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8100e60 0x7fb6c81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c6ffd640 1 -- 172.21.15.19:0/3356578546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8102a00 msgr2=0x7fb6c819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c6ffd640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8102a00 0x7fb6c819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c6ffd640 1 -- 172.21.15.19:0/3356578546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8100e60 msgr2=0x7fb6c81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c6ffd640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8100e60 0x7fb6c81981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c6ffd640 1 -- 172.21.15.19:0/3356578546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c819fdd0 con 0x7fb6c8102040 2024-04-23T18:25:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c6ffd640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c8102040 0x7fb6c81986f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00c920 tx=0x7fb6bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c4ff9640 1 -- 172.21.15.19:0/3356578546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc007c70 con 0x7fb6c8102040 2024-04-23T18:25:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6cda8a640 1 -- 172.21.15.19:0/3356578546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c81a00c0 con 0x7fb6c8102040 2024-04-23T18:25:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c4ff9640 1 -- 172.21.15.19:0/3356578546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6bc007e10 con 0x7fb6c8102040 2024-04-23T18:25:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6c4ff9640 1 -- 172.21.15.19:0/3356578546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc018820 con 0x7fb6c8102040 2024-04-23T18:25:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.585+0000 7fb6cda8a640 1 -- 172.21.15.19:0/3356578546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c81a0610 con 0x7fb6c8102040 2024-04-23T18:25:57.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.586+0000 7fb6cda8a640 1 -- 172.21.15.19:0/3356578546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c81082d0 con 0x7fb6c8102040 2024-04-23T18:25:57.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.588+0000 7fb6c4ff9640 1 -- 172.21.15.19:0/3356578546 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6bc00f450 con 0x7fb6c8102040 2024-04-23T18:25:57.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.588+0000 7fb6c4ff9640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb6b00a3bb0 0x7fb6b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.588+0000 7fb6c77fe640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb6b00a3bb0 0x7fb6b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.588+0000 7fb6c4ff9640 1 -- 172.21.15.19:0/3356578546 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb6bc0d0500 con 0x7fb6c8102040 2024-04-23T18:25:57.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.589+0000 7fb6c77fe640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb6b00a3bb0 0x7fb6b00a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009230 tx=0x7fb6b80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.593+0000 7fb6c4ff9640 1 -- 172.21.15.19:0/3356578546 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6bc014030 con 0x7fb6c8102040 2024-04-23T18:25:57.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:57 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2851011467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:57.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:57 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/871979840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:57.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:57 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3838666930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.780+0000 7fb6cda8a640 1 -- 172.21.15.19:0/3356578546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb6c8062bf0 con 0x7fb6c8102040 2024-04-23T18:25:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.781+0000 7fb6c4ff9640 1 -- 172.21.15.19:0/3356578546 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fb6bc09cef0 con 0x7fb6c8102040 2024-04-23T18:25:57.774 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:25:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 -- 172.21.15.19:0/3356578546 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb6b00a3bb0 msgr2=0x7fb6b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb6b00a3bb0 0x7fb6b00a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009230 tx=0x7fb6b80314a0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 -- 172.21.15.19:0/3356578546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c8102040 msgr2=0x7fb6c81986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c8102040 0x7fb6c81986f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00c920 tx=0x7fb6bc00cde0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 -- 172.21.15.19:0/3356578546 shutdown_connections 2024-04-23T18:25:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fb6b00a3bb0 0x7fb6b00a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8102a00 0x7fb6c819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c8102040 0x7fb6c81986f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 --2- 172.21.15.19:0/3356578546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6c8100e60 0x7fb6c81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 -- 172.21.15.19:0/3356578546 >> 172.21.15.19:0/3356578546 conn(0x7fb6c80fc440 msgr2=0x7fb6c81054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 -- 172.21.15.19:0/3356578546 shutdown_connections 2024-04-23T18:25:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.783+0000 7fb6cda8a640 1 -- 172.21.15.19:0/3356578546 wait complete. 2024-04-23T18:25:57.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 -- 172.21.15.19:0/2868873377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee78100e70 msgr2=0x7fee78101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 --2- 172.21.15.19:0/2868873377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee78100e70 0x7fee78101290 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fee70009ca0 tx=0x7fee7002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 -- 172.21.15.19:0/2868873377 shutdown_connections 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 --2- 172.21.15.19:0/2868873377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78101fc0 0x7fee78102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 --2- 172.21.15.19:0/2868873377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78102980 0x7fee781092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 --2- 172.21.15.19:0/2868873377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee78100e70 0x7fee78101290 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 -- 172.21.15.19:0/2868873377 >> 172.21.15.19:0/2868873377 conn(0x7fee780fc3e0 msgr2=0x7fee780fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 -- 172.21.15.19:0/2868873377 shutdown_connections 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 -- 172.21.15.19:0/2868873377 wait complete. 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 Processor -- start 2024-04-23T18:25:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 -- start start 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78100e70 0x7fee78104df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee78101fc0 0x7fee78103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78102980 0x7fee78103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee7810c9c0 con 0x7fee78101fc0 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.899+0000 7fee7ed30640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fee7810c840 con 0x7fee78102980 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee7ed30640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee7810cb40 con 0x7fee78100e70 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee78101fc0 0x7fee78103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee78101fc0 0x7fee78103430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43012/0 (socket says 172.21.15.19:43012) 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee77fff640 1 -- 172.21.15.19:0/1785001509 learned_addr learned my addr 172.21.15.19:0/1785001509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee7cacc640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78100e70 0x7fee78104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee7d2cd640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78102980 0x7fee78103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee77fff640 1 -- 172.21.15.19:0/1785001509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78100e70 msgr2=0x7fee78104df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee77fff640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78100e70 0x7fee78104df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee77fff640 1 -- 172.21.15.19:0/1785001509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78102980 msgr2=0x7fee78103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee77fff640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78102980 0x7fee78103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee77fff640 1 -- 172.21.15.19:0/1785001509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee78104200 con 0x7fee78101fc0 2024-04-23T18:25:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee77fff640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee78101fc0 0x7fee78103430 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fee6800d950 tx=0x7fee6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee75ffb640 1 -- 172.21.15.19:0/1785001509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68017020 con 0x7fee78101fc0 2024-04-23T18:25:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee75ffb640 1 -- 172.21.15.19:0/1785001509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee68009c30 con 0x7fee78101fc0 2024-04-23T18:25:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee75ffb640 1 -- 172.21.15.19:0/1785001509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6800f6b0 con 0x7fee78101fc0 2024-04-23T18:25:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.900+0000 7fee7ed30640 1 -- 172.21.15.19:0/1785001509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee781a2300 con 0x7fee78101fc0 2024-04-23T18:25:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.901+0000 7fee7ed30640 1 -- 172.21.15.19:0/1785001509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee781a2800 con 0x7fee78101fc0 2024-04-23T18:25:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.901+0000 7fee7ed30640 1 -- 172.21.15.19:0/1785001509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee781a2b20 con 0x7fee78101fc0 2024-04-23T18:25:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.903+0000 7fee75ffb640 1 -- 172.21.15.19:0/1785001509 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fee6801a020 con 0x7fee78101fc0 2024-04-23T18:25:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.903+0000 7fee75ffb640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee640a3bb0 0x7fee640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.903+0000 7fee7cacc640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee640a3bb0 0x7fee640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.903+0000 7fee75ffb640 1 -- 172.21.15.19:0/1785001509 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fee680d0160 con 0x7fee78101fc0 2024-04-23T18:25:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.904+0000 7fee7cacc640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee640a3bb0 0x7fee640a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee70005e30 tx=0x7fee70005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:57.908+0000 7fee75ffb640 1 -- 172.21.15.19:0/1785001509 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee68009de0 con 0x7fee78101fc0 2024-04-23T18:25:58.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:57 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2851011467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:58.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:57 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/871979840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:58.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:57 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3838666930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:58.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:57 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2851011467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:58.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:57 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/871979840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:58.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:57 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3838666930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:58.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.094+0000 7fee7ed30640 1 -- 172.21.15.19:0/1785001509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fee781a2cf0 con 0x7fee78101fc0 2024-04-23T18:25:58.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.095+0000 7fee75ffb640 1 -- 172.21.15.19:0/1785001509 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fee6809caf0 con 0x7fee78101fc0 2024-04-23T18:25:58.087 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-23T18:25:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 -- 172.21.15.19:0/1785001509 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee640a3bb0 msgr2=0x7fee640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee640a3bb0 0x7fee640a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee70005e30 tx=0x7fee70005dc0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 -- 172.21.15.19:0/1785001509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee78101fc0 msgr2=0x7fee78103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee78101fc0 0x7fee78103430 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fee6800d950 tx=0x7fee6800de10 comp rx=0 tx=0).stop 2024-04-23T18:25:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 -- 172.21.15.19:0/1785001509 shutdown_connections 2024-04-23T18:25:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee640a3bb0 0x7fee640a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee78102980 0x7fee78103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee78101fc0 0x7fee78103430 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 --2- 172.21.15.19:0/1785001509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee78100e70 0x7fee78104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 -- 172.21.15.19:0/1785001509 >> 172.21.15.19:0/1785001509 conn(0x7fee780fc3e0 msgr2=0x7fee780fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:58.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 -- 172.21.15.19:0/1785001509 shutdown_connections 2024-04-23T18:25:58.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.097+0000 7fee7ed30640 1 -- 172.21.15.19:0/1785001509 wait complete. 2024-04-23T18:25:58.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:25:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.212+0000 7f1f35fbf640 1 -- 172.21.15.19:0/1159204804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f30102a70 msgr2=0x7f1f30104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.212+0000 7f1f35fbf640 1 --2- 172.21.15.19:0/1159204804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f30102a70 0x7f1f30104e60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009900 tx=0x7f1f2002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.212+0000 7f1f35fbf640 1 -- 172.21.15.19:0/1159204804 shutdown_connections 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.212+0000 7f1f35fbf640 1 --2- 172.21.15.19:0/1159204804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f301053a0 0x7f1f30107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.212+0000 7f1f35fbf640 1 --2- 172.21.15.19:0/1159204804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f30107cd0 0x7f1f3010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.212+0000 7f1f35fbf640 1 --2- 172.21.15.19:0/1159204804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f30102a70 0x7f1f30104e60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.212+0000 7f1f35fbf640 1 -- 172.21.15.19:0/1159204804 >> 172.21.15.19:0/1159204804 conn(0x7f1f300fc400 msgr2=0x7f1f300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.213+0000 7f1f35fbf640 1 -- 172.21.15.19:0/1159204804 shutdown_connections 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.213+0000 7f1f35fbf640 1 -- 172.21.15.19:0/1159204804 wait complete. 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.213+0000 7f1f35fbf640 1 Processor -- start 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.213+0000 7f1f35fbf640 1 -- start start 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.213+0000 7f1f35fbf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f30102a70 0x7f1f301023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.213+0000 7f1f35fbf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f301053a0 0x7f1f30100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.213+0000 7f1f35fbf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f30107cd0 0x7f1f30100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.213+0000 7f1f35fbf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010dae0 con 0x7f1f301053a0 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.213+0000 7f1f35fbf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010d960 con 0x7f1f30102a70 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.213+0000 7f1f35fbf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010dc60 con 0x7f1f30107cd0 2024-04-23T18:25:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f301053a0 0x7f1f30100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f301053a0 0x7f1f30100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43038/0 (socket says 172.21.15.19:43038) 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2effd640 1 -- 172.21.15.19:0/2212952982 learned_addr learned my addr 172.21.15.19:0/2212952982 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2f7fe640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f30102a70 0x7f1f301023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2ffff640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f30107cd0 0x7f1f30100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2effd640 1 -- 172.21.15.19:0/2212952982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f30107cd0 msgr2=0x7f1f30100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2effd640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f30107cd0 0x7f1f30100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2effd640 1 -- 172.21.15.19:0/2212952982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f30102a70 msgr2=0x7f1f301023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2effd640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f30102a70 0x7f1f301023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2effd640 1 -- 172.21.15.19:0/2212952982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f301017f0 con 0x7f1f301053a0 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2ffff640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f30107cd0 0x7f1f30100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2effd640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f301053a0 0x7f1f30100a20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400d770 tx=0x7f1f2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2cff9640 1 -- 172.21.15.19:0/2212952982 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24017070 con 0x7f1f301053a0 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2cff9640 1 -- 172.21.15.19:0/2212952982 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f24009c30 con 0x7f1f301053a0 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f35fbf640 1 -- 172.21.15.19:0/2212952982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f301a2360 con 0x7f1f301053a0 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f2cff9640 1 -- 172.21.15.19:0/2212952982 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400f620 con 0x7f1f301053a0 2024-04-23T18:25:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.214+0000 7f1f35fbf640 1 -- 172.21.15.19:0/2212952982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f301a28b0 con 0x7f1f301053a0 2024-04-23T18:25:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.215+0000 7f1f35fbf640 1 -- 172.21.15.19:0/2212952982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f300fdef0 con 0x7f1f301053a0 2024-04-23T18:25:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.217+0000 7f1f2cff9640 1 -- 172.21.15.19:0/2212952982 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1f2401a070 con 0x7f1f301053a0 2024-04-23T18:25:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.218+0000 7f1f2cff9640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1f180a3bb0 0x7f1f180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.218+0000 7f1f2f7fe640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1f180a3bb0 0x7f1f180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.218+0000 7f1f2cff9640 1 -- 172.21.15.19:0/2212952982 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f1f240cff80 con 0x7f1f301053a0 2024-04-23T18:25:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.218+0000 7f1f2f7fe640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1f180a3bb0 0x7f1f180a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f20000c00 tx=0x7f1f200314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:58.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.222+0000 7f1f2cff9640 1 -- 172.21.15.19:0/2212952982 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f24009de0 con 0x7f1f301053a0 2024-04-23T18:25:58.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.407+0000 7f1f35fbf640 1 -- 172.21.15.19:0/2212952982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f3002d810 con 0x7f1f301053a0 2024-04-23T18:25:58.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.408+0000 7f1f2cff9640 1 -- 172.21.15.19:0/2212952982 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f1f2409c910 con 0x7f1f301053a0 2024-04-23T18:25:58.401 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-23T18:25:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 -- 172.21.15.19:0/2212952982 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1f180a3bb0 msgr2=0x7f1f180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1f180a3bb0 0x7f1f180a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f20000c00 tx=0x7f1f200314a0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 -- 172.21.15.19:0/2212952982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f301053a0 msgr2=0x7f1f30100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f301053a0 0x7f1f30100a20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400d770 tx=0x7f1f2400dc30 comp rx=0 tx=0).stop 2024-04-23T18:25:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 -- 172.21.15.19:0/2212952982 shutdown_connections 2024-04-23T18:25:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f1f180a3bb0 0x7f1f180a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f30107cd0 0x7f1f30100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f301053a0 0x7f1f30100a20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 --2- 172.21.15.19:0/2212952982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f30102a70 0x7f1f301023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 -- 172.21.15.19:0/2212952982 >> 172.21.15.19:0/2212952982 conn(0x7f1f300fc400 msgr2=0x7f1f30104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:58.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 -- 172.21.15.19:0/2212952982 shutdown_connections 2024-04-23T18:25:58.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.410+0000 7f1f35fbf640 1 -- 172.21.15.19:0/2212952982 wait complete. 2024-04-23T18:25:58.414 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:25:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.522+0000 7f7b10093640 1 -- 172.21.15.19:0/1996720460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08101fa0 msgr2=0x7f7b081023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.522+0000 7f7b10093640 1 --2- 172.21.15.19:0/1996720460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08101fa0 0x7f7b081023c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009900 tx=0x7f7b0402a410 comp rx=0 tx=0).stop 2024-04-23T18:25:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.522+0000 7f7b10093640 1 -- 172.21.15.19:0/1996720460 shutdown_connections 2024-04-23T18:25:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.522+0000 7f7b10093640 1 --2- 172.21.15.19:0/1996720460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b08103b50 0x7f7b0810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.522+0000 7f7b10093640 1 --2- 172.21.15.19:0/1996720460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b08103190 0x7f7b08103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.522+0000 7f7b10093640 1 --2- 172.21.15.19:0/1996720460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08101fa0 0x7f7b081023c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.522+0000 7f7b10093640 1 -- 172.21.15.19:0/1996720460 >> 172.21.15.19:0/1996720460 conn(0x7f7b080fd530 msgr2=0x7f7b080ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.522+0000 7f7b10093640 1 -- 172.21.15.19:0/1996720460 shutdown_connections 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.522+0000 7f7b10093640 1 -- 172.21.15.19:0/1996720460 wait complete. 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b10093640 1 Processor -- start 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b10093640 1 -- start start 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b10093640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b08101fa0 0x7f7b08198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b10093640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b08103190 0x7f7b08198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b10093640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08103b50 0x7f7b0819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b10093640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0810dae0 con 0x7f7b08103b50 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b10093640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0810d960 con 0x7f7b08101fa0 2024-04-23T18:25:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b10093640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0810dc60 con 0x7f7b08103190 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b0e630640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08103b50 0x7f7b0819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b0e630640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08103b50 0x7f7b0819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43062/0 (socket says 172.21.15.19:43062) 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b0e630640 1 -- 172.21.15.19:0/2052267954 learned_addr learned my addr 172.21.15.19:0/2052267954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b0de2f640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b08101fa0 0x7f7b08198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b0d62e640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b08103190 0x7f7b08198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b0e630640 1 -- 172.21.15.19:0/2052267954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b08103190 msgr2=0x7f7b08198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b0e630640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b08103190 0x7f7b08198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b0e630640 1 -- 172.21.15.19:0/2052267954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b08101fa0 msgr2=0x7f7b08198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b0e630640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b08101fa0 0x7f7b08198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.523+0000 7f7b0e630640 1 -- 172.21.15.19:0/2052267954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0819fd60 con 0x7f7b08103b50 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.524+0000 7f7b0e630640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08103b50 0x7f7b0819f6f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000eb10 tx=0x7f7b0000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.524+0000 7f7afeffd640 1 -- 172.21.15.19:0/2052267954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00004b20 con 0x7f7b08103b50 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.524+0000 7f7afeffd640 1 -- 172.21.15.19:0/2052267954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b00004cc0 con 0x7f7b08103b50 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.524+0000 7f7b10093640 1 -- 172.21.15.19:0/2052267954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0819fff0 con 0x7f7b08103b50 2024-04-23T18:25:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.524+0000 7f7afeffd640 1 -- 172.21.15.19:0/2052267954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00010670 con 0x7f7b08103b50 2024-04-23T18:25:58.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.524+0000 7f7b10093640 1 -- 172.21.15.19:0/2052267954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b081a04c0 con 0x7f7b08103b50 2024-04-23T18:25:58.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.525+0000 7f7b10093640 1 -- 172.21.15.19:0/2052267954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b081092c0 con 0x7f7b08103b50 2024-04-23T18:25:58.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.526+0000 7f7afeffd640 1 -- 172.21.15.19:0/2052267954 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7b00005020 con 0x7f7b08103b50 2024-04-23T18:25:58.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.526+0000 7f7afeffd640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7af40a3bb0 0x7f7af40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.527+0000 7f7b0de2f640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7af40a3bb0 0x7f7af40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.527+0000 7f7afeffd640 1 -- 172.21.15.19:0/2052267954 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f7b000d0b20 con 0x7f7b08103b50 2024-04-23T18:25:58.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.527+0000 7f7b0de2f640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7af40a3bb0 0x7f7af40a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b04000c00 tx=0x7f7b040314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.531+0000 7f7afeffd640 1 -- 172.21.15.19:0/2052267954 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b0009d4b0 con 0x7f7b08103b50 2024-04-23T18:25:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.757+0000 7f7b10093640 1 -- 172.21.15.19:0/2052267954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b08062b50 con 0x7f7b08103b50 2024-04-23T18:25:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.758+0000 7f7afeffd640 1 -- 172.21.15.19:0/2052267954 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7f7b0009cdb0 con 0x7f7b08103b50 2024-04-23T18:25:58.751 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:58.751 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:24:02.624724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24508,24469,24521,24491]},"id":1}]} 2024-04-23T18:25:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-23T18:25:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 -- 172.21.15.19:0/2052267954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7af40a3bb0 msgr2=0x7f7af40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7af40a3bb0 0x7f7af40a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b04000c00 tx=0x7f7b040314a0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 -- 172.21.15.19:0/2052267954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08103b50 msgr2=0x7f7b0819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08103b50 0x7f7b0819f6f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000eb10 tx=0x7f7b0000c2d0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 -- 172.21.15.19:0/2052267954 shutdown_connections 2024-04-23T18:25:58.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7af40a3bb0 0x7f7af40a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08103b50 0x7f7b0819f6f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b08103190 0x7f7b08198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 --2- 172.21.15.19:0/2052267954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b08101fa0 0x7f7b08198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 -- 172.21.15.19:0/2052267954 >> 172.21.15.19:0/2052267954 conn(0x7f7b080fd530 msgr2=0x7f7b08106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:58.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 -- 172.21.15.19:0/2052267954 shutdown_connections 2024-04-23T18:25:58.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.760+0000 7f7b10093640 1 -- 172.21.15.19:0/2052267954 wait complete. 2024-04-23T18:25:58.764 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:25:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.874+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/971168700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b58102a30 msgr2=0x7f3b58104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.874+0000 7f3b5d2ef640 1 --2- 172.21.15.19:0/971168700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b58102a30 0x7f3b58104e20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009ca0 tx=0x7f3b4802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.874+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/971168700 shutdown_connections 2024-04-23T18:25:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.874+0000 7f3b5d2ef640 1 --2- 172.21.15.19:0/971168700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b58107c90 0x7f3b5810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.874+0000 7f3b5d2ef640 1 --2- 172.21.15.19:0/971168700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b58105360 0x7f3b58107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.874+0000 7f3b5d2ef640 1 --2- 172.21.15.19:0/971168700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b58102a30 0x7f3b58104e20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.874+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/971168700 >> 172.21.15.19:0/971168700 conn(0x7f3b580fc440 msgr2=0x7f3b580fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.874+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/971168700 shutdown_connections 2024-04-23T18:25:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.874+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/971168700 wait complete. 2024-04-23T18:25:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.874+0000 7f3b5d2ef640 1 Processor -- start 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b5d2ef640 1 -- start start 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b5d2ef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b58102a30 0x7f3b58198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b5d2ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b58105360 0x7f3b581987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b5d2ef640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b58107c90 0x7f3b5819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b5d2ef640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5810daf0 con 0x7f3b58102a30 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b5d2ef640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5810d970 con 0x7f3b58105360 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b5d2ef640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5810dc70 con 0x7f3b58107c90 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b56ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b58102a30 0x7f3b58198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b56ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b58102a30 0x7f3b58198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43092/0 (socket says 172.21.15.19:43092) 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b56ffd640 1 -- 172.21.15.19:0/2057334264 learned_addr learned my addr 172.21.15.19:0/2057334264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b577fe640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b58107c90 0x7f3b5819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b56ffd640 1 -- 172.21.15.19:0/2057334264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b58107c90 msgr2=0x7f3b5819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b56ffd640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b58107c90 0x7f3b5819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b56ffd640 1 -- 172.21.15.19:0/2057334264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b58105360 msgr2=0x7f3b581987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b56ffd640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b58105360 0x7f3b581987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b56ffd640 1 -- 172.21.15.19:0/2057334264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5819fe60 con 0x7f3b58102a30 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b577fe640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b58107c90 0x7f3b5819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b56ffd640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b58102a30 0x7f3b58198290 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3b48006010 tx=0x7f3b48003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.875+0000 7f3b3ffff640 1 -- 172.21.15.19:0/2057334264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b480317c0 con 0x7f3b58102a30 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.876+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/2057334264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b581a00f0 con 0x7f3b58102a30 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.876+0000 7f3b3ffff640 1 -- 172.21.15.19:0/2057334264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b480044f0 con 0x7f3b58102a30 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.876+0000 7f3b3ffff640 1 -- 172.21.15.19:0/2057334264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4802fd10 con 0x7f3b58102a30 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.876+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/2057334264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b581a0690 con 0x7f3b58102a30 2024-04-23T18:25:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.876+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/2057334264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b580fdf30 con 0x7f3b58102a30 2024-04-23T18:25:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.878+0000 7f3b3ffff640 1 -- 172.21.15.19:0/2057334264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3b48004040 con 0x7f3b58102a30 2024-04-23T18:25:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.878+0000 7f3b3ffff640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3b400a3c30 0x7f3b400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.878+0000 7f3b567fc640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3b400a3c30 0x7f3b400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.878+0000 7f3b3ffff640 1 -- 172.21.15.19:0/2057334264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3b480f19f0 con 0x7f3b58102a30 2024-04-23T18:25:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.879+0000 7f3b567fc640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3b400a3c30 0x7f3b400a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c005fd0 tx=0x7f3b4c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:58.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:58.883+0000 7f3b3ffff640 1 -- 172.21.15.19:0/2057334264 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b480be3e0 con 0x7f3b58102a30 2024-04-23T18:25:59.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:58 smithi019 ceph-mon[29215]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:59.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:58 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3356578546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:59.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:58 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1785001509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:59.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:58 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2212952982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:59.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:58 smithi044 ceph-mon[32458]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:59.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:58 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3356578546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:59.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:58 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1785001509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:59.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:58 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2212952982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.072+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/2057334264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b58062b80 con 0x7f3b58102a30 2024-04-23T18:25:59.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.073+0000 7f3b3ffff640 1 -- 172.21.15.19:0/2057334264 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22642 (secure 0 0 0) 0x7f3b480bdce0 con 0x7f3b58102a30 2024-04-23T18:25:59.066 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:25:59.066 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:24:36.689178+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:25:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/2057334264 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3b400a3c30 msgr2=0x7f3b400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3b400a3c30 0x7f3b400a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c005fd0 tx=0x7f3b4c005850 comp rx=0 tx=0).stop 2024-04-23T18:25:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/2057334264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b58102a30 msgr2=0x7f3b58198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b58102a30 0x7f3b58198290 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3b48006010 tx=0x7f3b48003ba0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/2057334264 shutdown_connections 2024-04-23T18:25:59.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f3b400a3c30 0x7f3b400a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b58105360 0x7f3b581987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b58107c90 0x7f3b5819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 --2- 172.21.15.19:0/2057334264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b58102a30 0x7f3b58198290 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/2057334264 >> 172.21.15.19:0/2057334264 conn(0x7f3b580fc440 msgr2=0x7f3b581034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/2057334264 shutdown_connections 2024-04-23T18:25:59.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.075+0000 7f3b5d2ef640 1 -- 172.21.15.19:0/2057334264 wait complete. 2024-04-23T18:25:59.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:58 smithi155 ceph-mon[33468]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:25:59.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:58 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3356578546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:59.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:58 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1785001509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:59.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:58 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2212952982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:59.080 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:23:55.611471+0000', 'modified': '2024-04-23T18:24:02.624724+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24521, 'mds_1': 24508, 'mds_2': 24491, 'mds_3': 24487, 'mds_4': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/716138605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 716138605}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 716138605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14604': {'gid': 14604, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/2593314342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 2593314342}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 2593314342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6833/3128313210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 3128313210}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 3128313210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6833/3545031610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 3545031610}, {'type': 'v1', 'addr': '172.21.15.44:6833', 'nonce': 3545031610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6835/659103876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 659103876}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 659103876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.155:6835/698922157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 698922157}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 698922157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.155:6837/980782961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 980782961}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 980782961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.44:6837/728759484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 728759484}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 728759484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.155:6839/3934638415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 3934638415}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 3934638415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6839/2221834529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 2221834529}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 2221834529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24487, 24508, 24469, 24521, 24491]} 2024-04-23T18:25:59.080 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T18:25:59.080 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.190+0000 7f20c8460640 1 -- 172.21.15.19:0/2908034767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c0103180 msgr2=0x7f20c0103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.190+0000 7f20c8460640 1 --2- 172.21.15.19:0/2908034767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c0103180 0x7f20c0103600 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009900 tx=0x7f20b002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.190+0000 7f20c8460640 1 -- 172.21.15.19:0/2908034767 shutdown_connections 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.190+0000 7f20c8460640 1 --2- 172.21.15.19:0/2908034767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20c0103b40 0x7f20c010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.190+0000 7f20c8460640 1 --2- 172.21.15.19:0/2908034767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c0103180 0x7f20c0103600 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.190+0000 7f20c8460640 1 --2- 172.21.15.19:0/2908034767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c0101f90 0x7f20c01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.190+0000 7f20c8460640 1 -- 172.21.15.19:0/2908034767 >> 172.21.15.19:0/2908034767 conn(0x7f20c00fd500 msgr2=0x7f20c00ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.190+0000 7f20c8460640 1 -- 172.21.15.19:0/2908034767 shutdown_connections 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.190+0000 7f20c8460640 1 -- 172.21.15.19:0/2908034767 wait complete. 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c8460640 1 Processor -- start 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c8460640 1 -- start start 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c8460640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20c0101f90 0x7f20c0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c8460640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c0103180 0x7f20c01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c8460640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c0103b40 0x7f20c019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c8460640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20c010dad0 con 0x7f20c0103180 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c8460640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f20c010d950 con 0x7f20c0103b40 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c8460640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20c010dc50 con 0x7f20c0101f90 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c59fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c0103180 0x7f20c01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c59fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c0103180 0x7f20c01986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43106/0 (socket says 172.21.15.19:43106) 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c59fb640 1 -- 172.21.15.19:0/1635813115 learned_addr learned my addr 172.21.15.19:0/1635813115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c61fc640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20c0101f90 0x7f20c0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c69fd640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c0103b40 0x7f20c019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c59fb640 1 -- 172.21.15.19:0/1635813115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20c0101f90 msgr2=0x7f20c0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c59fb640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20c0101f90 0x7f20c0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c59fb640 1 -- 172.21.15.19:0/1635813115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c0103b40 msgr2=0x7f20c019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c59fb640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c0103b40 0x7f20c019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c59fb640 1 -- 172.21.15.19:0/1635813115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c019fda0 con 0x7f20c0103180 2024-04-23T18:25:59.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c61fc640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20c0101f90 0x7f20c0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:59.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c69fd640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c0103b40 0x7f20c019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:59.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.191+0000 7f20c59fb640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c0103180 0x7f20c01986c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f20b0005940 tx=0x7f20b0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.192+0000 7f20b77fe640 1 -- 172.21.15.19:0/1635813115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b003f030 con 0x7f20c0103180 2024-04-23T18:25:59.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.192+0000 7f20b77fe640 1 -- 172.21.15.19:0/1635813115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b0003aa0 con 0x7f20c0103180 2024-04-23T18:25:59.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.192+0000 7f20c8460640 1 -- 172.21.15.19:0/1635813115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c01a0030 con 0x7f20c0103180 2024-04-23T18:25:59.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.192+0000 7f20b77fe640 1 -- 172.21.15.19:0/1635813115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0035070 con 0x7f20c0103180 2024-04-23T18:25:59.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.192+0000 7f20c8460640 1 -- 172.21.15.19:0/1635813115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c01a0520 con 0x7f20c0103180 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.194+0000 7f20b77fe640 1 -- 172.21.15.19:0/1635813115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f20b0042030 con 0x7f20c0103180 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.194+0000 7f20cb175740 1 Processor -- start 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.194+0000 7f20cb175740 1 -- start start 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.194+0000 7f20cb175740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9ab80 0x5638d2f9afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20cb175740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9bd50 0x5638d2f9c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20cb175740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fa2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20cb175740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638d2effc50 con 0x5638d2f9c710 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20cb175740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638d2e9c830 con 0x5638d2f9bd50 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20cb175740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638d2efbc10 con 0x5638d2f9ab80 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c59fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fa2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20b77fe640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20ac0a3bb0 0x7f20ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c59fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fa2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43122/0 (socket says 172.21.15.19:43122) 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c59fb640 1 -- 172.21.15.19:0/3585647292 learned_addr learned my addr 172.21.15.19:0/3585647292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c61fc640 1 --2- 172.21.15.19:0/3585647292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9bd50 0x5638d2f9c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c69fd640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20ac0a3bb0 0x7f20ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c69fd640 1 --2- 172.21.15.19:0/3585647292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9ab80 0x5638d2f9afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20b77fe640 1 -- 172.21.15.19:0/1635813115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f20b00f0ae0 con 0x7f20c0103180 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c59fb640 1 -- 172.21.15.19:0/3585647292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9ab80 msgr2=0x5638d2f9afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c59fb640 1 --2- 172.21.15.19:0/3585647292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9ab80 0x5638d2f9afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c59fb640 1 -- 172.21.15.19:0/3585647292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9bd50 msgr2=0x5638d2f9c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c59fb640 1 --2- 172.21.15.19:0/3585647292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9bd50 0x5638d2f9c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c59fb640 1 -- 172.21.15.19:0/3585647292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638d2c122d0 con 0x5638d2f9c710 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c69fd640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20ac0a3bb0 0x7f20ac0a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20b8005d00 tx=0x7f20b8009a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20c59fb640 1 --2- 172.21.15.19:0/3585647292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fa2fa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f20b0046000 tx=0x7f20b002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c8bb70a3f16a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.195+0000 7f20b4bfe640 1 -- 172.21.15.19:0/3585647292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0049020 con 0x5638d2f9c710 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20b4bfe640 1 -- 172.21.15.19:0/3585647292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b00c5a40 con 0x5638d2f9c710 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20b4bfe640 1 -- 172.21.15.19:0/3585647292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0003d70 con 0x5638d2f9c710 2024-04-23T18:25:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20cb175740 1 -- 172.21.15.19:0/3585647292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 msgr2=0x5638d2fa2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20cb175740 1 --2- 172.21.15.19:0/3585647292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fa2fa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f20b0046000 tx=0x7f20b002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20cb175740 1 -- 172.21.15.19:0/3585647292 shutdown_connections 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20cb175740 1 --2- 172.21.15.19:0/3585647292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9bd50 0x5638d2f9c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20cb175740 1 --2- 172.21.15.19:0/3585647292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fa2fa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20cb175740 1 --2- 172.21.15.19:0/3585647292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9ab80 0x5638d2f9afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20cb175740 1 -- 172.21.15.19:0/3585647292 >> 172.21.15.19:0/3585647292 conn(0x5638d2f96070 msgr2=0x5638d2f96470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20cb175740 1 -- 172.21.15.19:0/3585647292 shutdown_connections 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.196+0000 7f20cb175740 1 -- 172.21.15.19:0/3585647292 wait complete. 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20cb175740 1 -- start start 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20cb175740 1 Processor -- start 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20cb175740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2fa4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20cb175740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 0x5638d2fa52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20cb175740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20cb175740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638d2fa34e0 con 0x5638d2f9c710 2024-04-23T18:25:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20cb175740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638d2e9a470 con 0x5638d2f9ab80 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20cb175740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638d2fa3660 con 0x5638d2f9bd50 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20cb175740 10 client.0 _ll_register_callbacks cb 0x5638d2ebff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20c69fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20c69fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fac300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43134/0 (socket says 172.21.15.19:43134) 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20c69fd640 1 -- 172.21.15.19:0/177365874 learned_addr learned my addr 172.21.15.19:0/177365874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20c59fb640 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 0x5638d2fa52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20c61fc640 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2fa4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20c69fd640 1 -- 172.21.15.19:0/177365874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 msgr2=0x5638d2fa52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20c69fd640 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 0x5638d2fa52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20c69fd640 1 -- 172.21.15.19:0/177365874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 msgr2=0x5638d2fa4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20c69fd640 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2fa4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.197+0000 7f20c69fd640 1 -- 172.21.15.19:0/177365874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638d2cdc6b0 con 0x5638d2f9c710 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20c69fd640 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fac300 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f20b8014440 tx=0x7f20b8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321e72a81179d345 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20a9ffb640 1 -- 172.21.15.19:0/177365874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8003a20 con 0x5638d2f9c710 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20a9ffb640 1 -- 172.21.15.19:0/177365874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f20b8003bc0 con 0x5638d2f9c710 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20cb175740 1 -- 172.21.15.19:0/177365874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b00c5a40 con 0x5638d2f9c710 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20a9ffb640 1 -- 172.21.15.19:0/177365874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20b8011e60 con 0x5638d2f9c710 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20cb175740 10 client.15213 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20cb175740 1 -- 172.21.15.19:0/177365874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638d2faf6e0 con 0x5638d2f9c710 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20a9ffb640 1 -- 172.21.15.19:0/177365874 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b00c5a40 con 0x5638d2f9c710 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20a9ffb640 1 -- 172.21.15.19:0/177365874 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f20b801d040 con 0x5638d2f9c710 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20cb175740 10 client.15213 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20cb175740 10 client.15213 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20cb175740 20 client.15213 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20cb175740 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x5638d2fb1db0 0x5638d2fb3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20cb175740 4 client.15213 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.198+0000 7f20cb175740 1 -- 172.21.15.19:0/177365874 --> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638d2f0d670 con 0x5638d2fb1db0 2024-04-23T18:25:59.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.199+0000 7f20c61fc640 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x5638d2fb1db0 0x5638d2fb3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.199+0000 7f20c61fc640 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x5638d2fb1db0 0x5638d2fb3220 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16474101e6dd2915 server_cookie=e358c2d9a3615ab5 in_seq=0 out_seq=0 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.199+0000 7f20a9ffb640 10 client.15213 ms_handle_connect on v2:172.21.15.155:6834/698922157 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20a9ffb640 1 -- 172.21.15.19:0/177365874 <== mds.4 v2:172.21.15.155:6834/698922157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638d2f0d670 con 0x5638d2fb1db0 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20a9ffb640 10 client.15213 handle_command_reply: tid=0 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 client.15213 shutdown 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 -- 172.21.15.19:0/177365874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 msgr2=0x5638d2fac300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fac300 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f20b8014440 tx=0x7f20b8005f90 comp rx=0 tx=0).stop 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 -- 172.21.15.19:0/177365874 shutdown_connections 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x5638d2fb1db0 0x5638d2fb3220 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2fac300 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20a9ffb640 0 client.15213 ms_handle_reset on v2:172.21.15.155:6834/698922157 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 0x5638d2fa52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 --2- 172.21.15.19:0/177365874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2fa4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 -- 172.21.15.19:0/177365874 >> 172.21.15.19:0/177365874 conn(0x5638d2f96070 msgr2=0x5638d2f951d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 -- 172.21.15.19:0/177365874 shutdown_connections 2024-04-23T18:25:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 1 -- 172.21.15.19:0/177365874 wait complete. 2024-04-23T18:25:59.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.200+0000 7f20cb175740 20 client.15213 trim_cache size 0 max 16384 2024-04-23T18:25:59.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.218+0000 7f20cb175740 1 Processor -- start 2024-04-23T18:25:59.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.218+0000 7f20cb175740 1 -- start start 2024-04-23T18:25:59.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.218+0000 7f20cb175740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2f96450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20cb175740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9bd50 0x5638d2fac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20cb175740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9c710 0x5638d2f9f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20cb175740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638d2fb1560 con 0x5638d2f9bd50 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20cb175740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638d2fb1c30 con 0x5638d2f9ab80 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20cb175740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638d2fb2300 con 0x5638d2f9c710 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20c59fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9bd50 0x5638d2fac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20c59fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9bd50 0x5638d2fac220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43144/0 (socket says 172.21.15.19:43144) 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20c59fb640 1 -- 172.21.15.19:0/3992092702 learned_addr learned my addr 172.21.15.19:0/3992092702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20c61fc640 1 --2- 172.21.15.19:0/3992092702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2f96450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20c59fb640 1 -- 172.21.15.19:0/3992092702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9c710 msgr2=0x5638d2f9f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20c59fb640 1 --2- 172.21.15.19:0/3992092702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9c710 0x5638d2f9f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20c59fb640 1 -- 172.21.15.19:0/3992092702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 msgr2=0x5638d2f96450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20c59fb640 1 --2- 172.21.15.19:0/3992092702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2f96450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20c59fb640 1 -- 172.21.15.19:0/3992092702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638d2f9b7d0 con 0x5638d2f9bd50 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20c59fb640 1 --2- 172.21.15.19:0/3992092702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9bd50 0x5638d2fac220 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f20b00ee5f0 tx=0x7f20b003b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55a0c2560422f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20abfff640 1 -- 172.21.15.19:0/3992092702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b00c0040 con 0x5638d2f9bd50 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20abfff640 1 -- 172.21.15.19:0/3992092702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b00bae70 con 0x5638d2f9bd50 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.219+0000 7f20abfff640 1 -- 172.21.15.19:0/3992092702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0049040 con 0x5638d2f9bd50 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 -- 172.21.15.19:0/3992092702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9bd50 msgr2=0x5638d2fac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 --2- 172.21.15.19:0/3992092702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9bd50 0x5638d2fac220 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f20b00ee5f0 tx=0x7f20b003b6b0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 -- 172.21.15.19:0/3992092702 shutdown_connections 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 --2- 172.21.15.19:0/3992092702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9c710 0x5638d2f9f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 --2- 172.21.15.19:0/3992092702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9bd50 0x5638d2fac220 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 --2- 172.21.15.19:0/3992092702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2f96450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 -- 172.21.15.19:0/3992092702 >> 172.21.15.19:0/3992092702 conn(0x5638d2f96070 msgr2=0x5638d2fa4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 -- 172.21.15.19:0/3992092702 shutdown_connections 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 -- 172.21.15.19:0/3992092702 wait complete. 2024-04-23T18:25:59.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 -- start start 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 Processor -- start 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2fa7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 0x5638d2fa7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2faaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638d2fb30a0 con 0x5638d2f9c710 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638d2fb29d0 con 0x5638d2f9ab80 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638d2fb3770 con 0x5638d2f9bd50 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20cb175740 10 client.0 _ll_register_callbacks cb 0x7f20b8009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20c69fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2faaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20c69fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2faaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43154/0 (socket says 172.21.15.19:43154) 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.220+0000 7f20c69fd640 1 -- 172.21.15.19:0/33269305 learned_addr learned my addr 172.21.15.19:0/33269305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20c61fc640 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2fa7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20c59fb640 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 0x5638d2fa7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20c69fd640 1 -- 172.21.15.19:0/33269305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 msgr2=0x5638d2fa7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20c69fd640 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 0x5638d2fa7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20c69fd640 1 -- 172.21.15.19:0/33269305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 msgr2=0x5638d2fa7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20c69fd640 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2fa7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20c69fd640 1 -- 172.21.15.19:0/33269305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638d2fab4d0 con 0x5638d2f9c710 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20c61fc640 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2fa7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20c59fb640 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 0x5638d2fa7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20c69fd640 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2faaee0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009d50 tx=0x7f20b8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5cd62e1472d569 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20a9ffb640 1 -- 172.21.15.19:0/33269305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8012c10 con 0x5638d2f9c710 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20a9ffb640 1 -- 172.21.15.19:0/33269305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f20b8012db0 con 0x5638d2f9c710 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20cb175740 1 -- 172.21.15.19:0/33269305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638d2f074e0 con 0x5638d2f9c710 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20a9ffb640 1 -- 172.21.15.19:0/33269305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20b8003820 con 0x5638d2f9c710 2024-04-23T18:25:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20cb175740 10 client.15219 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20cb175740 1 -- 172.21.15.19:0/33269305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638d3017d60 con 0x5638d2f9c710 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.221+0000 7f20a9ffb640 1 -- 172.21.15.19:0/33269305 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5638d2f074e0 con 0x5638d2f9c710 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20a9ffb640 1 -- 172.21.15.19:0/33269305 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f20b8020860 con 0x5638d2f9c710 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20cb175740 10 client.15219 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20cb175740 10 client.15219 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20cb175740 20 client.15219 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20cb175740 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x5638d301c850 0x5638d301b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20cb175740 4 client.15219 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20cb175740 1 -- 172.21.15.19:0/33269305 --> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] -- command(tid 0: {"prefix": "status"}) -- 0x5638d2e839a0 con 0x5638d301c850 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20c61fc640 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x5638d301c850 0x5638d301b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20a9ffb640 10 client.15219 ms_handle_connect on v2:172.21.15.155:6834/698922157 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20c61fc640 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x5638d301c850 0x5638d301b560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bbb2b1cf32a76234 server_cookie=725cd1138dd327b0 in_seq=0 out_seq=0 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20a9ffb640 1 -- 172.21.15.19:0/33269305 <== mds.4 v2:172.21.15.155:6834/698922157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5638d2e839a0 con 0x5638d301c850 2024-04-23T18:25:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20a9ffb640 10 client.15219 handle_command_reply: tid=0 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.222+0000 7f20cb175740 1 client.15219 shutdown 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 1 -- 172.21.15.19:0/33269305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 msgr2=0x5638d2faaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2faaee0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009d50 tx=0x7f20b8005740 comp rx=0 tx=0).stop 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 1 -- 172.21.15.19:0/33269305 shutdown_connections 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x5638d301c850 0x5638d301b560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638d2f9c710 0x5638d2faaee0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d2f9bd50 0x5638d2fa7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 1 --2- 172.21.15.19:0/33269305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d2f9ab80 0x5638d2fa7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20a9ffb640 0 client.15219 ms_handle_reset on v2:172.21.15.155:6834/698922157 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 1 -- 172.21.15.19:0/33269305 >> 172.21.15.19:0/33269305 conn(0x5638d2f96070 msgr2=0x5638d2fa4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 1 -- 172.21.15.19:0/33269305 shutdown_connections 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 1 -- 172.21.15.19:0/33269305 wait complete. 2024-04-23T18:25:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20cb175740 20 client.15219 trim_cache size 0 max 16384 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24469, 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 116.834350078, 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 88, 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 146.687280961 2024-04-23T18:25:59.223 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:25:59.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 -- 172.21.15.19:0/1635813115 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20ac0a3bb0 msgr2=0x7f20ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20ac0a3bb0 0x7f20ac0a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20b8005d00 tx=0x7f20b8009a90 comp rx=0 tx=0).stop 2024-04-23T18:25:59.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 -- 172.21.15.19:0/1635813115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c0103180 msgr2=0x7f20c01986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c0103180 0x7f20c01986c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f20b0005940 tx=0x7f20b0031a50 comp rx=0 tx=0).stop 2024-04-23T18:25:59.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 -- 172.21.15.19:0/1635813115 shutdown_connections 2024-04-23T18:25:59.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20ac0a3bb0 0x7f20ac0a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c0103b40 0x7f20c019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c0103180 0x7f20c01986c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 --2- 172.21.15.19:0/1635813115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20c0101f90 0x7f20c0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 -- 172.21.15.19:0/1635813115 >> 172.21.15.19:0/1635813115 conn(0x7f20c00fd500 msgr2=0x7f20c0106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 -- 172.21.15.19:0/1635813115 shutdown_connections 2024-04-23T18:25:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.223+0000 7f20c8460640 1 -- 172.21.15.19:0/1635813115 wait complete. 2024-04-23T18:25:59.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T18:25:59.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.336+0000 7feeab007640 1 -- 172.21.15.19:0/3684918002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4100f30 msgr2=0x7feea4101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.336+0000 7feeab007640 1 --2- 172.21.15.19:0/3684918002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4100f30 0x7feea4101350 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fee94009900 tx=0x7fee9402a410 comp rx=0 tx=0).stop 2024-04-23T18:25:59.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.336+0000 7feeab007640 1 -- 172.21.15.19:0/3684918002 shutdown_connections 2024-04-23T18:25:59.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.336+0000 7feeab007640 1 --2- 172.21.15.19:0/3684918002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea4102a40 0x7feea4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.336+0000 7feeab007640 1 --2- 172.21.15.19:0/3684918002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feea4102080 0x7feea4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.336+0000 7feeab007640 1 --2- 172.21.15.19:0/3684918002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4100f30 0x7feea4101350 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.336+0000 7feeab007640 1 -- 172.21.15.19:0/3684918002 >> 172.21.15.19:0/3684918002 conn(0x7feea40fc4a0 msgr2=0x7feea40fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.336+0000 7feeab007640 1 -- 172.21.15.19:0/3684918002 shutdown_connections 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.336+0000 7feeab007640 1 -- 172.21.15.19:0/3684918002 wait complete. 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feeab007640 1 Processor -- start 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feeab007640 1 -- start start 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feeab007640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feea4100f30 0x7feea4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feeab007640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea4102080 0x7feea4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feeab007640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4102a40 0x7feea419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feeab007640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feea410cb00 con 0x7feea4102a40 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feeab007640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feea410c980 con 0x7feea4102080 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feeab007640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feea410cc80 con 0x7feea4100f30 2024-04-23T18:25:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea95a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4102a40 0x7feea419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea95a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4102a40 0x7feea419f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43182/0 (socket says 172.21.15.19:43182) 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea95a4640 1 -- 172.21.15.19:0/1055776957 learned_addr learned my addr 172.21.15.19:0/1055776957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea8da3640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feea4100f30 0x7feea4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea95a4640 1 -- 172.21.15.19:0/1055776957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feea4100f30 msgr2=0x7feea4198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea95a4640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feea4100f30 0x7feea4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea95a4640 1 -- 172.21.15.19:0/1055776957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea4102080 msgr2=0x7feea4198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea3fff640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea4102080 0x7feea4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea95a4640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea4102080 0x7feea4198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea95a4640 1 -- 172.21.15.19:0/1055776957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea419fed0 con 0x7feea4102a40 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.337+0000 7feea95a4640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4102a40 0x7feea419f860 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00cce0 tx=0x7fee9c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.338+0000 7feea1ffb640 1 -- 172.21.15.19:0/1055776957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00f450 con 0x7feea4102a40 2024-04-23T18:25:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.338+0000 7feea1ffb640 1 -- 172.21.15.19:0/1055776957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c00faa0 con 0x7feea4102a40 2024-04-23T18:25:59.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.338+0000 7feeab007640 1 -- 172.21.15.19:0/1055776957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea41a01c0 con 0x7feea4102a40 2024-04-23T18:25:59.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.338+0000 7feea1ffb640 1 -- 172.21.15.19:0/1055776957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c018b70 con 0x7feea4102a40 2024-04-23T18:25:59.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.338+0000 7feeab007640 1 -- 172.21.15.19:0/1055776957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea41a06c0 con 0x7feea4102a40 2024-04-23T18:25:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.340+0000 7feea1ffb640 1 -- 172.21.15.19:0/1055776957 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fee9c018d10 con 0x7feea4102a40 2024-04-23T18:25:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.340+0000 7feeaddf0740 1 Processor -- start 2024-04-23T18:25:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.340+0000 7feeaddf0740 1 -- start start 2024-04-23T18:25:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.340+0000 7feea1ffb640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee900a3bb0 0x7fee900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.340+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcab80 0x56169cdcafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.340+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcbd50 0x56169cdcc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.340+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.340+0000 7feeaddf0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56169cd2fc50 con 0x56169cdcc710 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.340+0000 7feeaddf0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56169cccc830 con 0x56169cdcbd50 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.340+0000 7feeaddf0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56169cd2bc10 con 0x56169cdcab80 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea3fff640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee900a3bb0 0x7fee900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea1ffb640 1 -- 172.21.15.19:0/1055776957 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fee9c0d08e0 con 0x7feea4102a40 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea95a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea95a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43196/0 (socket says 172.21.15.19:43196) 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea8da3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcab80 0x56169cdcafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea95a4640 1 -- 172.21.15.19:0/2027350496 learned_addr learned my addr 172.21.15.19:0/2027350496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea3fff640 1 --2- 172.21.15.19:0/2027350496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcbd50 0x56169cdcc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea95a4640 1 -- 172.21.15.19:0/2027350496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcab80 msgr2=0x56169cdcafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea95a4640 1 --2- 172.21.15.19:0/2027350496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcab80 0x56169cdcafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea95a4640 1 -- 172.21.15.19:0/2027350496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcbd50 msgr2=0x56169cdcc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea95a4640 1 --2- 172.21.15.19:0/2027350496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcbd50 0x56169cdcc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea95a4640 1 -- 172.21.15.19:0/2027350496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56169ca422d0 con 0x56169cdcc710 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea3fff640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee900a3bb0 0x7fee900a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fee8c006fd0 tx=0x7fee8c013650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7feea95a4640 1 --2- 172.21.15.19:0/2027350496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd2fa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee9c09cb60 tx=0x7fee9c04adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd799ce60073dc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7fee9affd640 1 -- 172.21.15.19:0/2027350496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0530b0 con 0x56169cdcc710 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7fee9affd640 1 -- 172.21.15.19:0/2027350496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c09cbd0 con 0x56169cdcc710 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.341+0000 7fee9affd640 1 -- 172.21.15.19:0/2027350496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c099240 con 0x56169cdcc710 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.342+0000 7feeaddf0740 1 -- 172.21.15.19:0/2027350496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 msgr2=0x56169cdd2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.342+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2027350496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd2fa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee9c09cb60 tx=0x7fee9c04adb0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.342+0000 7feeaddf0740 1 -- 172.21.15.19:0/2027350496 shutdown_connections 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.342+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2027350496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcbd50 0x56169cdcc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.342+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2027350496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd2fa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.342+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2027350496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcab80 0x56169cdcafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.342+0000 7feeaddf0740 1 -- 172.21.15.19:0/2027350496 >> 172.21.15.19:0/2027350496 conn(0x56169cdc6070 msgr2=0x56169cdc6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.342+0000 7feeaddf0740 1 -- 172.21.15.19:0/2027350496 shutdown_connections 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.342+0000 7feeaddf0740 1 -- 172.21.15.19:0/2027350496 wait complete. 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.342+0000 7feeaddf0740 1 -- start start 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feeaddf0740 1 Processor -- start 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cdd9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cdd99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cde0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feeaddf0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56169cdd3720 con 0x56169cdcc710 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feeaddf0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56169ccca470 con 0x56169cdcab80 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feeaddf0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56169cdd38a0 con 0x56169cdcbd50 2024-04-23T18:25:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feeaddf0740 10 client.0 _ll_register_callbacks cb 0x56169cceff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feea95a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cde0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feea95a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cde0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43210/0 (socket says 172.21.15.19:43210) 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feea95a4640 1 -- 172.21.15.19:0/474872584 learned_addr learned my addr 172.21.15.19:0/474872584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feea3fff640 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cdd99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feea8da3640 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cdd9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feea95a4640 1 -- 172.21.15.19:0/474872584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 msgr2=0x56169cdd99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feea95a4640 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cdd99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feea95a4640 1 -- 172.21.15.19:0/474872584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 msgr2=0x56169cdd9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feea95a4640 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cdd9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.343+0000 7feea95a4640 1 -- 172.21.15.19:0/474872584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56169cb0c6b0 con 0x56169cdcc710 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7feea95a4640 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cde0a10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fee9c098fe0 tx=0x7fee9c018400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff20a9eff66ac735 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7fee83fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7fee83fff640 1 -- 172.21.15.19:0/474872584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0b6060 con 0x56169cdcc710 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7fee83fff640 1 -- 172.21.15.19:0/474872584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fee9c09aed0 con 0x56169cdcc710 2024-04-23T18:25:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7feeaddf0740 1 -- 172.21.15.19:0/474872584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9c09cbd0 con 0x56169cdcc710 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7fee83fff640 1 -- 172.21.15.19:0/474872584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee9c0530b0 con 0x56169cdcc710 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7feeaddf0740 10 client.15231 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7feeaddf0740 1 -- 172.21.15.19:0/474872584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56169cde3d80 con 0x56169cdcc710 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7fee83fff640 1 -- 172.21.15.19:0/474872584 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c09cbd0 con 0x56169cdcc710 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7fee83fff640 1 -- 172.21.15.19:0/474872584 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fee9c016b90 con 0x56169cdcc710 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7feeaddf0740 10 client.15231 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7feeaddf0740 10 client.15231 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7feeaddf0740 20 client.15231 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7feeaddf0740 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x56169cde50d0 0x56169cde7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7feeaddf0740 4 client.15231 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.344+0000 7feeaddf0740 1 -- 172.21.15.19:0/474872584 --> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56169cd3d670 con 0x56169cde50d0 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.345+0000 7feea8da3640 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x56169cde50d0 0x56169cde7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.345+0000 7feea8da3640 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x56169cde50d0 0x56169cde7220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d061adf8384597ec server_cookie=df54abc46262a9df in_seq=0 out_seq=0 2024-04-23T18:25:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.345+0000 7fee83fff640 10 client.15231 ms_handle_connect on v2:172.21.15.155:6836/980782961 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7fee83fff640 1 -- 172.21.15.19:0/474872584 <== mds.3 v2:172.21.15.155:6836/980782961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56169cd3d670 con 0x56169cde50d0 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7fee83fff640 10 client.15231 handle_command_reply: tid=0 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 client.15231 shutdown 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 -- 172.21.15.19:0/474872584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 msgr2=0x56169cde0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cde0a10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fee9c098fe0 tx=0x7fee9c018400 comp rx=0 tx=0).stop 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 -- 172.21.15.19:0/474872584 shutdown_connections 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cde0a10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cdd99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x56169cde50d0 0x56169cde7220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 --2- 172.21.15.19:0/474872584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cdd9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 -- 172.21.15.19:0/474872584 >> 172.21.15.19:0/474872584 conn(0x56169cdc6070 msgr2=0x56169cdc51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7fee83fff640 0 client.15231 ms_handle_reset on v2:172.21.15.155:6836/980782961 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 -- 172.21.15.19:0/474872584 shutdown_connections 2024-04-23T18:25:59.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 1 -- 172.21.15.19:0/474872584 wait complete. 2024-04-23T18:25:59.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.346+0000 7feeaddf0740 20 client.15231 trim_cache size 0 max 16384 2024-04-23T18:25:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.364+0000 7feeaddf0740 1 Processor -- start 2024-04-23T18:25:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.364+0000 7feeaddf0740 1 -- start start 2024-04-23T18:25:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.364+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cdd44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.364+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cde09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.364+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.364+0000 7feeaddf0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56169cde5560 con 0x56169cdcc710 2024-04-23T18:25:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.364+0000 7feeaddf0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56169cde5c30 con 0x56169cdcab80 2024-04-23T18:25:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.364+0000 7feeaddf0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56169cde6300 con 0x56169cdcbd50 2024-04-23T18:25:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea95a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea95a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43212/0 (socket says 172.21.15.19:43212) 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea95a4640 1 -- 172.21.15.19:0/2366863201 learned_addr learned my addr 172.21.15.19:0/2366863201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea3fff640 1 --2- 172.21.15.19:0/2366863201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cde09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea95a4640 1 -- 172.21.15.19:0/2366863201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 msgr2=0x56169cde09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea8da3640 1 --2- 172.21.15.19:0/2366863201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cdd44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea95a4640 1 --2- 172.21.15.19:0/2366863201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cde09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea95a4640 1 -- 172.21.15.19:0/2366863201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 msgr2=0x56169cdd44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea95a4640 1 --2- 172.21.15.19:0/2366863201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cdd44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea95a4640 1 -- 172.21.15.19:0/2366863201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56169cdc1700 con 0x56169cdcc710 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea3fff640 1 --2- 172.21.15.19:0/2366863201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cde09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7feea95a4640 1 --2- 172.21.15.19:0/2366863201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd49e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fee9c09cb60 tx=0x7fee9c018610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdda58cc1ccc9f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7fee9a7fc640 1 -- 172.21.15.19:0/2366863201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0b6040 con 0x56169cdcc710 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7fee9a7fc640 1 -- 172.21.15.19:0/2366863201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c09cbd0 con 0x56169cdcc710 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.365+0000 7fee9a7fc640 1 -- 172.21.15.19:0/2366863201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c09afb0 con 0x56169cdcc710 2024-04-23T18:25:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 -- 172.21.15.19:0/2366863201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 msgr2=0x56169cdd49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2366863201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd49e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fee9c09cb60 tx=0x7fee9c018610 comp rx=0 tx=0).stop 2024-04-23T18:25:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 -- 172.21.15.19:0/2366863201 shutdown_connections 2024-04-23T18:25:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2366863201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cdd49e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2366863201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cde09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2366863201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cdd44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 -- 172.21.15.19:0/2366863201 >> 172.21.15.19:0/2366863201 conn(0x56169cdc6070 msgr2=0x56169cdd9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 -- 172.21.15.19:0/2366863201 shutdown_connections 2024-04-23T18:25:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 -- 172.21.15.19:0/2366863201 wait complete. 2024-04-23T18:25:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 -- start start 2024-04-23T18:25:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.366+0000 7feeaddf0740 1 Processor -- start 2024-04-23T18:25:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cddbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cddc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feeaddf0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cddf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feeaddf0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56169cde70a0 con 0x56169cdcc710 2024-04-23T18:25:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feeaddf0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56169cde69d0 con 0x56169cdcab80 2024-04-23T18:25:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feeaddf0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56169cde7770 con 0x56169cdcbd50 2024-04-23T18:25:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feeaddf0740 10 client.0 _ll_register_callbacks cb 0x7fee9c04af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea95a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cddf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea95a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cddf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43216/0 (socket says 172.21.15.19:43216) 2024-04-23T18:25:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea95a4640 1 -- 172.21.15.19:0/2013878474 learned_addr learned my addr 172.21.15.19:0/2013878474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea8da3640 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cddbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea3fff640 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cddc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea95a4640 1 -- 172.21.15.19:0/2013878474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 msgr2=0x56169cddc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea95a4640 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cddc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea95a4640 1 -- 172.21.15.19:0/2013878474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 msgr2=0x56169cddbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea95a4640 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cddbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea95a4640 1 -- 172.21.15.19:0/2013878474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56169cddf8a0 con 0x56169cdcc710 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea3fff640 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cddc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feea95a4640 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cddf2b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0995a0 tx=0x7fee9c04aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9f0e5cfe54816a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7fee83fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7fee83fff640 1 -- 172.21.15.19:0/2013878474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c09afb0 con 0x56169cdcc710 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7fee83fff640 1 -- 172.21.15.19:0/2013878474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fee9c016b90 con 0x56169cdcc710 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.367+0000 7feeaddf0740 1 -- 172.21.15.19:0/2013878474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9c09cbd0 con 0x56169cdcc710 2024-04-23T18:25:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7fee83fff640 1 -- 172.21.15.19:0/2013878474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee9c0a0f80 con 0x56169cdcc710 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7feeaddf0740 10 client.15237 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7feeaddf0740 1 -- 172.21.15.19:0/2013878474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56169cddf8a0 con 0x56169cdcc710 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7fee83fff640 1 -- 172.21.15.19:0/2013878474 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c09cbd0 con 0x56169cdcc710 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7fee83fff640 1 -- 172.21.15.19:0/2013878474 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fee9c098970 con 0x56169cdcc710 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7feeaddf0740 10 client.15237 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7feeaddf0740 10 client.15237 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7feeaddf0740 20 client.15237 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x56169ce4fe80 0x56169ce4c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7feeaddf0740 4 client.15237 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7feeaddf0740 1 -- 172.21.15.19:0/2013878474 --> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] -- command(tid 0: {"prefix": "status"}) -- 0x56169cd374e0 con 0x56169ce4fe80 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.368+0000 7feea8da3640 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x56169ce4fe80 0x56169ce4c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feea8da3640 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x56169ce4fe80 0x56169ce4c990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6807e6749d37ee21 server_cookie=8e5980a9f2b88cb3 in_seq=0 out_seq=0 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7fee83fff640 10 client.15237 ms_handle_connect on v2:172.21.15.155:6836/980782961 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7fee83fff640 1 -- 172.21.15.19:0/2013878474 <== mds.3 v2:172.21.15.155:6836/980782961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56169cd374e0 con 0x56169ce4fe80 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7fee83fff640 10 client.15237 handle_command_reply: tid=0 2024-04-23T18:25:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 client.15237 shutdown 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 -- 172.21.15.19:0/2013878474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 msgr2=0x56169cddf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cddf2b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0995a0 tx=0x7fee9c04aee0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 -- 172.21.15.19:0/2013878474 shutdown_connections 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56169cdcc710 0x56169cddf2b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56169cdcbd50 0x56169cddc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x56169ce4fe80 0x56169ce4c990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 --2- 172.21.15.19:0/2013878474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56169cdcab80 0x56169cddbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7fee83fff640 0 client.15237 ms_handle_reset on v2:172.21.15.155:6836/980782961 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 -- 172.21.15.19:0/2013878474 >> 172.21.15.19:0/2013878474 conn(0x56169cdc6070 msgr2=0x56169cdd9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 -- 172.21.15.19:0/2013878474 shutdown_connections 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 1 -- 172.21.15.19:0/2013878474 wait complete. 2024-04-23T18:25:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.369+0000 7feeaddf0740 20 client.15237 trim_cache size 0 max 16384 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24487, 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 117.98610595300001, 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 88, 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 143.887108052 2024-04-23T18:25:59.369 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:25:59.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 -- 172.21.15.19:0/1055776957 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee900a3bb0 msgr2=0x7fee900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee900a3bb0 0x7fee900a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fee8c006fd0 tx=0x7fee8c013650 comp rx=0 tx=0).stop 2024-04-23T18:25:59.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 -- 172.21.15.19:0/1055776957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4102a40 msgr2=0x7feea419f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4102a40 0x7feea419f860 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00cce0 tx=0x7fee9c00a620 comp rx=0 tx=0).stop 2024-04-23T18:25:59.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 -- 172.21.15.19:0/1055776957 shutdown_connections 2024-04-23T18:25:59.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fee900a3bb0 0x7fee900a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea4102080 0x7feea4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4102a40 0x7feea419f860 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 --2- 172.21.15.19:0/1055776957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feea4100f30 0x7feea4198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 -- 172.21.15.19:0/1055776957 >> 172.21.15.19:0/1055776957 conn(0x7feea40fc4a0 msgr2=0x7feea40fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 -- 172.21.15.19:0/1055776957 shutdown_connections 2024-04-23T18:25:59.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.370+0000 7feeab007640 1 -- 172.21.15.19:0/1055776957 wait complete. 2024-04-23T18:25:59.374 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:25:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.483+0000 7f4ea59cd640 1 -- 172.21.15.19:0/2218247582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea0102a90 msgr2=0x7f4ea0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.483+0000 7f4ea59cd640 1 --2- 172.21.15.19:0/2218247582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea0102a90 0x7f4ea0104e80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009900 tx=0x7f4e9002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.483+0000 7f4ea59cd640 1 -- 172.21.15.19:0/2218247582 shutdown_connections 2024-04-23T18:25:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.483+0000 7f4ea59cd640 1 --2- 172.21.15.19:0/2218247582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ea01053c0 0x7f4ea01077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.483+0000 7f4ea59cd640 1 --2- 172.21.15.19:0/2218247582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ea0107cf0 0x7f4ea010a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.483+0000 7f4ea59cd640 1 --2- 172.21.15.19:0/2218247582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea0102a90 0x7f4ea0104e80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.483+0000 7f4ea59cd640 1 -- 172.21.15.19:0/2218247582 >> 172.21.15.19:0/2218247582 conn(0x7f4ea00fc4a0 msgr2=0x7f4ea00fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.483+0000 7f4ea59cd640 1 -- 172.21.15.19:0/2218247582 shutdown_connections 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.483+0000 7f4ea59cd640 1 -- 172.21.15.19:0/2218247582 wait complete. 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4ea59cd640 1 Processor -- start 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4ea59cd640 1 -- start start 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4ea59cd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ea0102a90 0x7f4ea0198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4ea59cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea01053c0 0x7f4ea0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4ea59cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ea0107cf0 0x7f4ea019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4ea59cd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea010db50 con 0x7f4ea01053c0 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4ea59cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea010d9d0 con 0x7f4ea0107cf0 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4ea59cd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea010dcd0 con 0x7f4ea0102a90 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4e97fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea01053c0 0x7f4ea0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4e97fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea01053c0 0x7f4ea0198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43240/0 (socket says 172.21.15.19:43240) 2024-04-23T18:25:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4e97fff640 1 -- 172.21.15.19:0/3374520492 learned_addr learned my addr 172.21.15.19:0/3374520492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.484+0000 7f4e9f7fe640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ea0107cf0 0x7f4ea019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4e9effd640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ea0102a90 0x7f4ea0198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4e97fff640 1 -- 172.21.15.19:0/3374520492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ea0102a90 msgr2=0x7f4ea0198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4e97fff640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ea0102a90 0x7f4ea0198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4e97fff640 1 -- 172.21.15.19:0/3374520492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ea0107cf0 msgr2=0x7f4ea019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4e97fff640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ea0107cf0 0x7f4ea019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4e97fff640 1 -- 172.21.15.19:0/3374520492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea019fe40 con 0x7f4ea01053c0 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4e97fff640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea01053c0 0x7f4ea0198760 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800e9c0 tx=0x7f4e8800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4ea49cb640 1 -- 172.21.15.19:0/3374520492 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8800cd80 con 0x7f4ea01053c0 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4ea59cd640 1 -- 172.21.15.19:0/3374520492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea01a0130 con 0x7f4ea01053c0 2024-04-23T18:25:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4ea49cb640 1 -- 172.21.15.19:0/3374520492 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e880054d0 con 0x7f4ea01053c0 2024-04-23T18:25:59.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4ea49cb640 1 -- 172.21.15.19:0/3374520492 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88010430 con 0x7f4ea01053c0 2024-04-23T18:25:59.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.485+0000 7f4ea59cd640 1 -- 172.21.15.19:0/3374520492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea01a0680 con 0x7f4ea01053c0 2024-04-23T18:25:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.487+0000 7f4ea49cb640 1 -- 172.21.15.19:0/3374520492 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4e88004fd0 con 0x7f4ea01053c0 2024-04-23T18:25:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.487+0000 7f4ea87c3740 1 Processor -- start 2024-04-23T18:25:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4ea87c3740 1 -- start start 2024-04-23T18:25:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bdf90 0x555d2a3be3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bf160 0x555d2a3bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d2a31ec50 con 0x555d2a3bfb20 2024-04-23T18:25:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555d2a2bb830 con 0x555d2a3bdf90 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555d2a31ac10 con 0x555d2a3bf160 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e97fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4ea49cb640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f4e8c0a3bb0 0x7f4e8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e97fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43254/0 (socket says 172.21.15.19:43254) 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e97fff640 1 -- 172.21.15.19:0/4212776959 learned_addr learned my addr 172.21.15.19:0/4212776959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e9f7fe640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f4e8c0a3bb0 0x7f4e8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e9f7fe640 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bdf90 0x555d2a3be3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e97fff640 1 -- 172.21.15.19:0/4212776959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bf160 msgr2=0x555d2a3bf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e9effd640 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bf160 0x555d2a3bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e97fff640 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bf160 0x555d2a3bf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e97fff640 1 -- 172.21.15.19:0/4212776959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bdf90 msgr2=0x555d2a3be3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e97fff640 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bdf90 0x555d2a3be3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4ea49cb640 1 -- 172.21.15.19:0/3374520492 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4e880d0890 con 0x7f4ea01053c0 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e97fff640 1 -- 172.21.15.19:0/4212776959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d2a0312d0 con 0x555d2a3bfb20 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e9f7fe640 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bdf90 0x555d2a3be3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.488+0000 7f4e9effd640 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bf160 0x555d2a3bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4e97fff640 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3c63b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4e88099550 tx=0x7f4e88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425f62ecff332a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4e95ffb640 1 -- 172.21.15.19:0/4212776959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e880046c0 con 0x555d2a3bfb20 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4e9f7fe640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f4e8c0a3bb0 0x7f4e8c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x555d2a3bf060 tx=0x7f4e980072a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4e95ffb640 1 -- 172.21.15.19:0/4212776959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e8809cb80 con 0x555d2a3bfb20 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4e95ffb640 1 -- 172.21.15.19:0/4212776959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e880bd0e0 con 0x555d2a3bfb20 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4ea87c3740 1 -- 172.21.15.19:0/4212776959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 msgr2=0x555d2a3c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3c63b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4e88099550 tx=0x7f4e88005880 comp rx=0 tx=0).stop 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4ea87c3740 1 -- 172.21.15.19:0/4212776959 shutdown_connections 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3c63b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bf160 0x555d2a3bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/4212776959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bdf90 0x555d2a3be3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4ea87c3740 1 -- 172.21.15.19:0/4212776959 >> 172.21.15.19:0/4212776959 conn(0x555d2a3b5070 msgr2=0x555d2a3b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4ea87c3740 1 -- 172.21.15.19:0/4212776959 shutdown_connections 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.489+0000 7f4ea87c3740 1 -- 172.21.15.19:0/4212776959 wait complete. 2024-04-23T18:25:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4ea87c3740 1 -- start start 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4ea87c3740 1 Processor -- start 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bdf90 0x555d2a3c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bf160 0x555d2a3c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bfb20 0x555d2a3cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d2a3c6ad0 con 0x555d2a3bdf90 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555d2a2b9470 con 0x555d2a3bf160 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555d2a3c6c50 con 0x555d2a3bfb20 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4ea87c3740 10 client.0 _ll_register_callbacks cb 0x555d2a2def30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4e9effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bdf90 0x555d2a3c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4e9effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bdf90 0x555d2a3c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43268/0 (socket says 172.21.15.19:43268) 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4e9effd640 1 -- 172.21.15.19:0/2473407303 learned_addr learned my addr 172.21.15.19:0/2473407303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4e9f7fe640 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bfb20 0x555d2a3cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4e97fff640 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bf160 0x555d2a3c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4e9effd640 1 -- 172.21.15.19:0/2473407303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bfb20 msgr2=0x555d2a3cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.490+0000 7f4e9effd640 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bfb20 0x555d2a3cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e9effd640 1 -- 172.21.15.19:0/2473407303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bf160 msgr2=0x555d2a3c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e9effd640 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bf160 0x555d2a3c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e9effd640 1 -- 172.21.15.19:0/2473407303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2a0fb6b0 con 0x555d2a3bdf90 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e9f7fe640 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bfb20 0x555d2a3cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e9effd640 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bdf90 0x555d2a3c83a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4e90005ab0 tx=0x7f4e90031e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b32258485b20359 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e7effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e7effd640 1 -- 172.21.15.19:0/2473407303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90035030 con 0x555d2a3bdf90 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e7effd640 1 -- 172.21.15.19:0/2473407303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4e90004040 con 0x555d2a3bdf90 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4ea87c3740 1 -- 172.21.15.19:0/2473407303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d2a32c670 con 0x555d2a3bdf90 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e7effd640 1 -- 172.21.15.19:0/2473407303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e90004240 con 0x555d2a3bdf90 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4ea87c3740 10 client.15249 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4ea87c3740 1 -- 172.21.15.19:0/2473407303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555d2a3d2c00 con 0x555d2a3bdf90 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e7effd640 1 -- 172.21.15.19:0/2473407303 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555d2a32c670 con 0x555d2a3bdf90 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4e7effd640 1 -- 172.21.15.19:0/2473407303 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4e90004a30 con 0x555d2a3bdf90 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4ea87c3740 10 client.15249 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4ea87c3740 10 client.15249 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4ea87c3740 20 client.15249 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x555d2a3d52c0 0x555d2a3d6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4ea87c3740 4 client.15249 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.491+0000 7f4ea87c3740 1 -- 172.21.15.19:0/2473407303 --> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d2a3264e0 con 0x555d2a3d52c0 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.492+0000 7f4e97fff640 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x555d2a3d52c0 0x555d2a3d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.492+0000 7f4e97fff640 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x555d2a3d52c0 0x555d2a3d6730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=245338c9961e2f4e server_cookie=453a8f577a488367 in_seq=0 out_seq=0 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.492+0000 7f4e7effd640 10 client.15249 ms_handle_connect on v2:172.21.15.44:6836/728759484 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4e7effd640 1 -- 172.21.15.19:0/2473407303 <== mds.2 v2:172.21.15.44:6836/728759484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d2a3264e0 con 0x555d2a3d52c0 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4e7effd640 10 client.15249 handle_command_reply: tid=0 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 client.15249 shutdown 2024-04-23T18:25:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 -- 172.21.15.19:0/2473407303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bdf90 msgr2=0x555d2a3c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bdf90 0x555d2a3c83a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4e90005ab0 tx=0x7f4e90031e10 comp rx=0 tx=0).stop 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 -- 172.21.15.19:0/2473407303 shutdown_connections 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x555d2a3d52c0 0x555d2a3d6730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bf160 0x555d2a3c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bfb20 0x555d2a3cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4e7effd640 0 client.15249 ms_handle_reset on v2:172.21.15.44:6836/728759484 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/2473407303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bdf90 0x555d2a3c83a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 -- 172.21.15.19:0/2473407303 >> 172.21.15.19:0/2473407303 conn(0x555d2a3b5070 msgr2=0x555d2a3b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 -- 172.21.15.19:0/2473407303 shutdown_connections 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 1 -- 172.21.15.19:0/2473407303 wait complete. 2024-04-23T18:25:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.493+0000 7f4ea87c3740 20 client.15249 trim_cache size 0 max 16384 2024-04-23T18:25:59.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.511+0000 7f4ea87c3740 1 Processor -- start 2024-04-23T18:25:59.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.511+0000 7f4ea87c3740 1 -- start start 2024-04-23T18:25:59.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.511+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bdf90 0x555d2a3b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.511+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bf160 0x555d2a3cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.511+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bfb20 0x555d2a3b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.511+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d2a3d4a70 con 0x555d2a3bf160 2024-04-23T18:25:59.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.511+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555d2a3d5140 con 0x555d2a3bfb20 2024-04-23T18:25:59.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.511+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555d2a3d5810 con 0x555d2a3bdf90 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e97fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bf160 0x555d2a3cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e97fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bf160 0x555d2a3cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43274/0 (socket says 172.21.15.19:43274) 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e97fff640 1 -- 172.21.15.19:0/3418047784 learned_addr learned my addr 172.21.15.19:0/3418047784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e97fff640 1 -- 172.21.15.19:0/3418047784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bdf90 msgr2=0x555d2a3b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e97fff640 1 --2- 172.21.15.19:0/3418047784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bdf90 0x555d2a3b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e97fff640 1 -- 172.21.15.19:0/3418047784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bfb20 msgr2=0x555d2a3b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e97fff640 1 --2- 172.21.15.19:0/3418047784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bfb20 0x555d2a3b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e97fff640 1 -- 172.21.15.19:0/3418047784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d2a3c5f20 con 0x555d2a3bf160 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e97fff640 1 --2- 172.21.15.19:0/3418047784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bf160 0x555d2a3cf870 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4e880057a0 tx=0x7f4e880987a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f19298b5d89ccf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e957fa640 1 -- 172.21.15.19:0/3418047784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e880b5020 con 0x555d2a3bf160 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e957fa640 1 -- 172.21.15.19:0/3418047784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e880046c0 con 0x555d2a3bf160 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.512+0000 7f4e957fa640 1 -- 172.21.15.19:0/3418047784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e880991f0 con 0x555d2a3bf160 2024-04-23T18:25:59.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 -- 172.21.15.19:0/3418047784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bf160 msgr2=0x555d2a3cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/3418047784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bf160 0x555d2a3cf870 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4e880057a0 tx=0x7f4e880987a0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 -- 172.21.15.19:0/3418047784 shutdown_connections 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/3418047784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bfb20 0x555d2a3b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/3418047784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bf160 0x555d2a3cf870 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/3418047784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bdf90 0x555d2a3b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 -- 172.21.15.19:0/3418047784 >> 172.21.15.19:0/3418047784 conn(0x555d2a3b5070 msgr2=0x555d2a3b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 -- 172.21.15.19:0/3418047784 shutdown_connections 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 -- 172.21.15.19:0/3418047784 wait complete. 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 -- start start 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 Processor -- start 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bdf90 0x555d2a3c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bf160 0x555d2a3c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d2a3d65b0 con 0x555d2a3bfb20 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555d2a3d5ee0 con 0x555d2a3bf160 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555d2a3d6c80 con 0x555d2a3bdf90 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.513+0000 7f4ea87c3740 10 client.0 _ll_register_callbacks cb 0x7f4e88005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e9f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e9f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43280/0 (socket says 172.21.15.19:43280) 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e9f7fe640 1 -- 172.21.15.19:0/182360127 learned_addr learned my addr 172.21.15.19:0/182360127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e97fff640 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bf160 0x555d2a3c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e9effd640 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bdf90 0x555d2a3c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e9f7fe640 1 -- 172.21.15.19:0/182360127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bdf90 msgr2=0x555d2a3c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e9f7fe640 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bdf90 0x555d2a3c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e9f7fe640 1 -- 172.21.15.19:0/182360127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bf160 msgr2=0x555d2a3c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e9f7fe640 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bf160 0x555d2a3c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e9f7fe640 1 -- 172.21.15.19:0/182360127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2a3ce3f0 con 0x555d2a3bfb20 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e9f7fe640 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3cddb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4e98013f20 tx=0x7f4e98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c517cad5a5af3cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e7effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e7effd640 1 -- 172.21.15.19:0/182360127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98027630 con 0x555d2a3bfb20 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4ea87c3740 1 -- 172.21.15.19:0/182360127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d2a2a29a0 con 0x555d2a3bfb20 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e7effd640 1 -- 172.21.15.19:0/182360127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4e9801e420 con 0x555d2a3bfb20 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e7effd640 1 -- 172.21.15.19:0/182360127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e9801de80 con 0x555d2a3bfb20 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4ea87c3740 10 client.15255 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4ea87c3740 1 -- 172.21.15.19:0/182360127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555d2a3cf780 con 0x555d2a3bfb20 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.514+0000 7f4e7effd640 1 -- 172.21.15.19:0/182360127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555d2a2a29a0 con 0x555d2a3bfb20 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.515+0000 7f4e7effd640 1 -- 172.21.15.19:0/182360127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4e9801eb40 con 0x555d2a3bfb20 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.515+0000 7f4ea87c3740 10 client.15255 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.515+0000 7f4ea87c3740 10 client.15255 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.515+0000 7f4ea87c3740 20 client.15255 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.515+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x555d2a43f7e0 0x555d2a43e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.515+0000 7f4ea87c3740 4 client.15255 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.515+0000 7f4ea87c3740 1 -- 172.21.15.19:0/182360127 --> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] -- command(tid 0: {"prefix": "status"}) -- 0x555d2a2ef660 con 0x555d2a43f7e0 2024-04-23T18:25:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.515+0000 7f4e9effd640 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x555d2a43f7e0 0x555d2a43e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.515+0000 7f4e7effd640 10 client.15255 ms_handle_connect on v2:172.21.15.44:6836/728759484 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.515+0000 7f4e9effd640 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x555d2a43f7e0 0x555d2a43e4f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c556cc4575a1f780 server_cookie=c43d678fa5c1b7c3 in_seq=0 out_seq=0 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4e7effd640 1 -- 172.21.15.19:0/182360127 <== mds.2 v2:172.21.15.44:6836/728759484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555d2a2ef660 con 0x555d2a43f7e0 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4e7effd640 10 client.15255 handle_command_reply: tid=0 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 client.15255 shutdown 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 -- 172.21.15.19:0/182360127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 msgr2=0x555d2a3cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3cddb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4e98013f20 tx=0x7f4e98004a40 comp rx=0 tx=0).stop 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 -- 172.21.15.19:0/182360127 shutdown_connections 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x555d2a43f7e0 0x555d2a43e4f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d2a3bf160 0x555d2a3c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d2a3bfb20 0x555d2a3cddb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 --2- 172.21.15.19:0/182360127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d2a3bdf90 0x555d2a3c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 -- 172.21.15.19:0/182360127 >> 172.21.15.19:0/182360127 conn(0x555d2a3b5070 msgr2=0x555d2a3b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4e7effd640 0 client.15255 ms_handle_reset on v2:172.21.15.44:6836/728759484 2024-04-23T18:25:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 -- 172.21.15.19:0/182360127 shutdown_connections 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 1 -- 172.21.15.19:0/182360127 wait complete. 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea87c3740 20 client.15255 trim_cache size 0 max 16384 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24491, 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 119.142489473, 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 88, 2024-04-23T18:25:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:25:59.516 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 141.08375783400001 2024-04-23T18:25:59.516 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:25:59.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 -- 172.21.15.19:0/3374520492 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f4e8c0a3bb0 msgr2=0x7f4e8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f4e8c0a3bb0 0x7f4e8c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x555d2a3bf060 tx=0x7f4e980072a0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 -- 172.21.15.19:0/3374520492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea01053c0 msgr2=0x7f4ea0198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea01053c0 0x7f4ea0198760 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800e9c0 tx=0x7f4e8800ee80 comp rx=0 tx=0).stop 2024-04-23T18:25:59.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 -- 172.21.15.19:0/3374520492 shutdown_connections 2024-04-23T18:25:59.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f4e8c0a3bb0 0x7f4e8c0a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ea0107cf0 0x7f4ea019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea01053c0 0x7f4ea0198760 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 --2- 172.21.15.19:0/3374520492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ea0102a90 0x7f4ea0198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 -- 172.21.15.19:0/3374520492 >> 172.21.15.19:0/3374520492 conn(0x7f4ea00fc4a0 msgr2=0x7f4ea01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.516+0000 7f4ea59cd640 1 -- 172.21.15.19:0/3374520492 shutdown_connections 2024-04-23T18:25:59.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.517+0000 7f4ea59cd640 1 -- 172.21.15.19:0/3374520492 wait complete. 2024-04-23T18:25:59.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:25:59.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/2598097944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a94106b10 msgr2=0x7f5a94108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 --2- 172.21.15.19:0/2598097944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a94106b10 0x7f5a94108fd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009ca0 tx=0x7f5a8c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/2598097944 shutdown_connections 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 --2- 172.21.15.19:0/2598097944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a94106b10 0x7f5a94108fd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 --2- 172.21.15.19:0/2598097944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a94104080 0x7f5a94106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 --2- 172.21.15.19:0/2598097944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a941016c0 0x7f5a94103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/2598097944 >> 172.21.15.19:0/2598097944 conn(0x7f5a940fb740 msgr2=0x7f5a940fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/2598097944 shutdown_connections 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/2598097944 wait complete. 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 Processor -- start 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 -- start start 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a941016c0 0x7f5a94103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a94104080 0x7f5a94101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94106b10 0x7f5a941021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9410c9f0 con 0x7f5a94104080 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9410c870 con 0x7f5a94106b10 2024-04-23T18:25:59.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.630+0000 7f5a9bb4f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9410cb70 con 0x7f5a941016c0 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a990ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a94104080 0x7f5a94101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a990ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a94104080 0x7f5a94101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43284/0 (socket says 172.21.15.19:43284) 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a990ea640 1 -- 172.21.15.19:0/4253495608 learned_addr learned my addr 172.21.15.19:0/4253495608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a998eb640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a941016c0 0x7f5a94103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94106b10 0x7f5a941021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a990ea640 1 -- 172.21.15.19:0/4253495608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a941016c0 msgr2=0x7f5a94103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a990ea640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a941016c0 0x7f5a94103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a990ea640 1 -- 172.21.15.19:0/4253495608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94106b10 msgr2=0x7f5a941021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a990ea640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94106b10 0x7f5a941021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a990ea640 1 -- 172.21.15.19:0/4253495608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a94102a30 con 0x7f5a94104080 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94106b10 0x7f5a941021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a990ea640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a94104080 0x7f5a94101c60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400d770 tx=0x7f5a8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a8affd640 1 -- 172.21.15.19:0/4253495608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84017070 con 0x7f5a94104080 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a8affd640 1 -- 172.21.15.19:0/4253495608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a840054d0 con 0x7f5a94104080 2024-04-23T18:25:59.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/4253495608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a941a22a0 con 0x7f5a94104080 2024-04-23T18:25:59.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a8affd640 1 -- 172.21.15.19:0/4253495608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84004c30 con 0x7f5a94104080 2024-04-23T18:25:59.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.631+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/4253495608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a941a27f0 con 0x7f5a94104080 2024-04-23T18:25:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.633+0000 7f5a8affd640 1 -- 172.21.15.19:0/4253495608 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5a84004e50 con 0x7f5a94104080 2024-04-23T18:25:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.634+0000 7f5a9e86e740 1 Processor -- start 2024-04-23T18:25:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.634+0000 7f5a9e86e740 1 -- start start 2024-04-23T18:25:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.634+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873077b80 0x55d873077fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.634+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d8730791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.634+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873079710 0x55d87307ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.634+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d872fdcc50 con 0x55d873078d50 2024-04-23T18:25:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.634+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d872f79830 con 0x55d873079710 2024-04-23T18:25:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.634+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d872fd8c10 con 0x55d873077b80 2024-04-23T18:25:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.634+0000 7f5a8affd640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5a800a3c30 0x7f5a800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.634+0000 7f5a998eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d8730791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a998eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d8730791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43294/0 (socket says 172.21.15.19:43294) 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a998eb640 1 -- 172.21.15.19:0/167704756 learned_addr learned my addr 172.21.15.19:0/167704756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/167704756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873077b80 0x55d873077fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5a800a3c30 0x7f5a800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a8affd640 1 -- 172.21.15.19:0/4253495608 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5a840cfe50 con 0x7f5a94104080 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a990ea640 1 --2- 172.21.15.19:0/167704756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873079710 0x55d87307ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a998eb640 1 -- 172.21.15.19:0/167704756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873077b80 msgr2=0x55d873077fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a998eb640 1 --2- 172.21.15.19:0/167704756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873077b80 0x55d873077fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a998eb640 1 -- 172.21.15.19:0/167704756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873079710 msgr2=0x55d87307ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a998eb640 1 --2- 172.21.15.19:0/167704756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873079710 0x55d87307ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a998eb640 1 -- 172.21.15.19:0/167704756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d872cef2d0 con 0x55d873078d50 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a998eb640 1 --2- 172.21.15.19:0/167704756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d8730791d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000ac40 tx=0x7f5a9000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8546bd99b9ad759 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5a800a3c30 0x7f5a800a60f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55d873078c50 tx=0x7f5a8c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a7ffff640 1 -- 172.21.15.19:0/167704756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90005220 con 0x55d873078d50 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a7ffff640 1 -- 172.21.15.19:0/167704756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a900047a0 con 0x55d873078d50 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.635+0000 7f5a7ffff640 1 -- 172.21.15.19:0/167704756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a900187e0 con 0x55d873078d50 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.636+0000 7f5a9e86e740 1 -- 172.21.15.19:0/167704756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 msgr2=0x55d8730791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.636+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/167704756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d8730791d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000ac40 tx=0x7f5a9000f790 comp rx=0 tx=0).stop 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.636+0000 7f5a9e86e740 1 -- 172.21.15.19:0/167704756 shutdown_connections 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.636+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/167704756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873079710 0x55d87307ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.636+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/167704756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d8730791d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.636+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/167704756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873077b80 0x55d873077fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.636+0000 7f5a9e86e740 1 -- 172.21.15.19:0/167704756 >> 172.21.15.19:0/167704756 conn(0x55d873073070 msgr2=0x55d873073470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.636+0000 7f5a9e86e740 1 -- 172.21.15.19:0/167704756 shutdown_connections 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.636+0000 7f5a9e86e740 1 -- 172.21.15.19:0/167704756 wait complete. 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.636+0000 7f5a9e86e740 1 -- start start 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a9e86e740 1 Processor -- start 2024-04-23T18:25:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d873081fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d873082520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873079710 0x55d873089560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d873080720 con 0x55d873078d50 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d872f77470 con 0x55d873077b80 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8730808a0 con 0x55d873079710 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a9e86e740 10 client.0 _ll_register_callbacks cb 0x55d872f9cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a990ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d873082520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a990ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d873082520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43304/0 (socket says 172.21.15.19:43304) 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a990ea640 1 -- 172.21.15.19:0/1315430212 learned_addr learned my addr 172.21.15.19:0/1315430212 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a998eb640 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d873081fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873079710 0x55d873089560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a990ea640 1 -- 172.21.15.19:0/1315430212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873079710 msgr2=0x55d873089560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a990ea640 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873079710 0x55d873089560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a990ea640 1 -- 172.21.15.19:0/1315430212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 msgr2=0x55d873081fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a990ea640 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d873081fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a990ea640 1 -- 172.21.15.19:0/1315430212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d872db96b0 con 0x55d873078d50 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a998eb640 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d873081fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a990ea640 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d873082520 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5a84098b10 tx=0x7f5a8409c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18846eb15aed2e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.637+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1315430212 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a840b5060 con 0x55d873078d50 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1315430212 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5a840a0170 con 0x55d873078d50 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1315430212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a900047a0 con 0x55d873078d50 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1315430212 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a84020040 con 0x55d873078d50 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a9e86e740 10 client.15267 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1315430212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d87308c910 con 0x55d873078d50 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1315430212 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a900047a0 con 0x55d873078d50 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1315430212 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5a84005140 con 0x55d873078d50 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a9e86e740 10 client.15267 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a9e86e740 10 client.15267 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a9e86e740 20 client.15267 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x55d87308ef20 0x55d873090390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a9e86e740 4 client.15267 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1315430212 --> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d872fea670 con 0x55d87308ef20 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.638+0000 7f5a998eb640 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x55d87308ef20 0x55d873090390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.639+0000 7f5a998eb640 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x55d87308ef20 0x55d873090390 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab7535b48a5fa0bd server_cookie=39e49788b5a2bb2 in_seq=0 out_seq=0 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.639+0000 7f5a7d7fa640 10 client.15267 ms_handle_connect on v2:172.21.15.155:6838/3934638415 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.639+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1315430212 <== mds.1 v2:172.21.15.155:6838/3934638415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d872fea670 con 0x55d87308ef20 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.639+0000 7f5a7d7fa640 10 client.15267 handle_command_reply: tid=0 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.639+0000 7f5a9e86e740 1 client.15267 shutdown 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1315430212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 msgr2=0x55d873082520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d873082520 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5a84098b10 tx=0x7f5a8409c980 comp rx=0 tx=0).stop 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1315430212 shutdown_connections 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x55d87308ef20 0x55d873090390 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873079710 0x55d873089560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873078d50 0x55d873082520 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1315430212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d873081fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a7d7fa640 0 client.15267 ms_handle_reset on v2:172.21.15.155:6838/3934638415 2024-04-23T18:25:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1315430212 >> 172.21.15.19:0/1315430212 conn(0x55d873073070 msgr2=0x55d8730721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1315430212 shutdown_connections 2024-04-23T18:25:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1315430212 wait complete. 2024-04-23T18:25:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.640+0000 7f5a9e86e740 20 client.15267 trim_cache size 0 max 16384 2024-04-23T18:25:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9e86e740 1 Processor -- start 2024-04-23T18:25:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9e86e740 1 -- start start 2024-04-23T18:25:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d873079130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873078d50 0x55d873089520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d87307c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d87308e6d0 con 0x55d873079710 2024-04-23T18:25:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d87308eda0 con 0x55d873077b80 2024-04-23T18:25:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d87308f470 con 0x55d873078d50 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9a0ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d87307c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9a0ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d87307c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43320/0 (socket says 172.21.15.19:43320) 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a9a0ec640 1 -- 172.21.15.19:0/1828973855 learned_addr learned my addr 172.21.15.19:0/1828973855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.658+0000 7f5a998eb640 1 --2- 172.21.15.19:0/1828973855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d873079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a990ea640 1 --2- 172.21.15.19:0/1828973855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873078d50 0x55d873089520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a9a0ec640 1 -- 172.21.15.19:0/1828973855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873078d50 msgr2=0x55d873089520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/1828973855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873078d50 0x55d873089520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a9a0ec640 1 -- 172.21.15.19:0/1828973855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 msgr2=0x55d873079130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/1828973855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d873079130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a9a0ec640 1 -- 172.21.15.19:0/1828973855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d873073d40 con 0x55d873079710 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a998eb640 1 --2- 172.21.15.19:0/1828973855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d873079130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/1828973855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d87307c7f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c032fe0 tx=0x7f5a8c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce6b352688a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a7f7fe640 1 -- 172.21.15.19:0/1828973855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c003a30 con 0x55d873079710 2024-04-23T18:25:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a7f7fe640 1 -- 172.21.15.19:0/1828973855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a8c036040 con 0x55d873079710 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.659+0000 7f5a7f7fe640 1 -- 172.21.15.19:0/1828973855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c02f8c0 con 0x55d873079710 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1828973855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 msgr2=0x55d87307c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1828973855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d87307c7f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c032fe0 tx=0x7f5a8c042000 comp rx=0 tx=0).stop 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1828973855 shutdown_connections 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1828973855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d87307c7f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1828973855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873078d50 0x55d873089520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1828973855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d873079130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1828973855 >> 172.21.15.19:0/1828973855 conn(0x55d873073070 msgr2=0x55d87306e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1828973855 shutdown_connections 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1828973855 wait complete. 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 -- start start 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 Processor -- start 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d8730847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873078d50 0x55d873084d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d873087d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d873090210 con 0x55d873079710 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d87308fb40 con 0x55d873077b80 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8730908e0 con 0x55d873078d50 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9e86e740 10 client.0 _ll_register_callbacks cb 0x7f5a840983e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9a0ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d873087d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9a0ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d873087d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43322/0 (socket says 172.21.15.19:43322) 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.660+0000 7f5a9a0ec640 1 -- 172.21.15.19:0/1035852352 learned_addr learned my addr 172.21.15.19:0/1035852352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a998eb640 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d8730847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a990ea640 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873078d50 0x55d873084d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a9a0ec640 1 -- 172.21.15.19:0/1035852352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873078d50 msgr2=0x55d873084d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873078d50 0x55d873084d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a9a0ec640 1 -- 172.21.15.19:0/1035852352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 msgr2=0x55d8730847c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d8730847c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a9a0ec640 1 -- 172.21.15.19:0/1035852352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d873088310 con 0x55d873079710 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a9a0ec640 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d873087d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c003cc0 tx=0x7f5a8c003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6e544e0d099d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1035852352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c004250 con 0x55d873079710 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1035852352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5a8c0043f0 con 0x55d873079710 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1035852352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a8c036040 con 0x55d873079710 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1035852352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a8c02f660 con 0x55d873079710 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a9e86e740 10 client.15273 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1035852352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d873089610 con 0x55d873079710 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.661+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1035852352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a8c036040 con 0x55d873079710 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.662+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1035852352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5a8c049040 con 0x55d873079710 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.662+0000 7f5a9e86e740 10 client.15273 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.662+0000 7f5a9e86e740 10 client.15273 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.662+0000 7f5a9e86e740 20 client.15273 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.662+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x55d8730f92e0 0x55d8730f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.662+0000 7f5a9e86e740 4 client.15273 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.662+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1035852352 --> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] -- command(tid 0: {"prefix": "status"}) -- 0x55d872fe44e0 con 0x55d8730f92e0 2024-04-23T18:25:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.662+0000 7f5a998eb640 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x55d8730f92e0 0x55d8730f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.662+0000 7f5a7d7fa640 10 client.15273 ms_handle_connect on v2:172.21.15.155:6838/3934638415 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.662+0000 7f5a998eb640 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x55d8730f92e0 0x55d8730f7ff0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=750cb65d57004fd6 server_cookie=12c47bdaf82d3335 in_seq=0 out_seq=0 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a7d7fa640 1 -- 172.21.15.19:0/1035852352 <== mds.1 v2:172.21.15.155:6838/3934638415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d872fe44e0 con 0x55d8730f92e0 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a7d7fa640 10 client.15273 handle_command_reply: tid=0 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 client.15273 shutdown 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1035852352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 msgr2=0x55d873087d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d873087d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c003cc0 tx=0x7f5a8c003cf0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1035852352 shutdown_connections 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x55d8730f92e0 0x55d8730f7ff0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d873079710 0x55d873087d20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d873078d50 0x55d873084d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a7d7fa640 0 client.15273 ms_handle_reset on v2:172.21.15.155:6838/3934638415 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 --2- 172.21.15.19:0/1035852352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d873077b80 0x55d8730847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1035852352 >> 172.21.15.19:0/1035852352 conn(0x55d873073070 msgr2=0x55d87307f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1035852352 shutdown_connections 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 1 -- 172.21.15.19:0/1035852352 wait complete. 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9e86e740 20 client.15273 trim_cache size 0 max 16384 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24508, 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:25:59.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 120.296935378, 2024-04-23T18:25:59.663 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-23T18:25:59.663 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 88, 2024-04-23T18:25:59.663 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:25:59.663 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 138.29807557500001 2024-04-23T18:25:59.663 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:25:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/4253495608 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5a800a3c30 msgr2=0x7f5a800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9bb4f640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5a800a3c30 0x7f5a800a60f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55d873078c50 tx=0x7f5a8c0095c0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/4253495608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a94104080 msgr2=0x7f5a94101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9bb4f640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a94104080 0x7f5a94101c60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400d770 tx=0x7f5a8400dc30 comp rx=0 tx=0).stop 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/4253495608 shutdown_connections 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.663+0000 7f5a9bb4f640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f5a800a3c30 0x7f5a800a60f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.664+0000 7f5a9bb4f640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94106b10 0x7f5a941021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.664+0000 7f5a9bb4f640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a94104080 0x7f5a94101c60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.664+0000 7f5a9bb4f640 1 --2- 172.21.15.19:0/4253495608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a941016c0 0x7f5a94103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.664+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/4253495608 >> 172.21.15.19:0/4253495608 conn(0x7f5a940fb740 msgr2=0x7f5a941008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.664+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/4253495608 shutdown_connections 2024-04-23T18:25:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.664+0000 7f5a9bb4f640 1 -- 172.21.15.19:0/4253495608 wait complete. 2024-04-23T18:25:59.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:25:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.778+0000 7f0236205640 1 -- 172.21.15.19:0/1327792182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0230101fc0 msgr2=0x7f02301023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.778+0000 7f0236205640 1 --2- 172.21.15.19:0/1327792182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0230101fc0 0x7f02301023e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0220009ca0 tx=0x7f022002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.778+0000 7f0236205640 1 -- 172.21.15.19:0/1327792182 shutdown_connections 2024-04-23T18:25:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.778+0000 7f0236205640 1 --2- 172.21.15.19:0/1327792182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02301031b0 0x7f0230103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.778+0000 7f0236205640 1 --2- 172.21.15.19:0/1327792182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0230103b70 0x7f023010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.778+0000 7f0236205640 1 --2- 172.21.15.19:0/1327792182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0230101fc0 0x7f02301023e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.778+0000 7f0236205640 1 -- 172.21.15.19:0/1327792182 >> 172.21.15.19:0/1327792182 conn(0x7f02300fd530 msgr2=0x7f02300ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.778+0000 7f0236205640 1 -- 172.21.15.19:0/1327792182 shutdown_connections 2024-04-23T18:25:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.778+0000 7f0236205640 1 -- 172.21.15.19:0/1327792182 wait complete. 2024-04-23T18:25:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.778+0000 7f0236205640 1 Processor -- start 2024-04-23T18:25:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f0236205640 1 -- start start 2024-04-23T18:25:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f0236205640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0230101fc0 0x7f0230198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f0236205640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02301031b0 0x7f02301986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f0236205640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0230103b70 0x7f023019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f0236205640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f023010db00 con 0x7f02301031b0 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f0236205640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f023010d980 con 0x7f0230103b70 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f0236205640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f023010dc80 con 0x7f0230101fc0 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02301031b0 0x7f02301986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02301031b0 0x7f02301986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40838/0 (socket says 172.21.15.19:40838) 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022effd640 1 -- 172.21.15.19:0/4197308908 learned_addr learned my addr 172.21.15.19:0/4197308908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022ffff640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0230103b70 0x7f023019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022effd640 1 -- 172.21.15.19:0/4197308908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0230101fc0 msgr2=0x7f0230198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022effd640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0230101fc0 0x7f0230198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022effd640 1 -- 172.21.15.19:0/4197308908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0230103b70 msgr2=0x7f023019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022effd640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0230103b70 0x7f023019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022effd640 1 -- 172.21.15.19:0/4197308908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023019fd80 con 0x7f02301031b0 2024-04-23T18:25:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022effd640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02301031b0 0x7f02301986a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f022400c920 tx=0x7f022400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022cff9640 1 -- 172.21.15.19:0/4197308908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0224007c70 con 0x7f02301031b0 2024-04-23T18:25:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f0236205640 1 -- 172.21.15.19:0/4197308908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02301a0070 con 0x7f02301031b0 2024-04-23T18:25:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022cff9640 1 -- 172.21.15.19:0/4197308908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0224007e10 con 0x7f02301031b0 2024-04-23T18:25:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.779+0000 7f022cff9640 1 -- 172.21.15.19:0/4197308908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0224018820 con 0x7f02301031b0 2024-04-23T18:25:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.780+0000 7f0236205640 1 -- 172.21.15.19:0/4197308908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02301a05c0 con 0x7f02301031b0 2024-04-23T18:25:59.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f022cff9640 1 -- 172.21.15.19:0/4197308908 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f022400f450 con 0x7f02301031b0 2024-04-23T18:25:59.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f0238ffa740 1 Processor -- start 2024-04-23T18:25:59.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f0238ffa740 1 -- start start 2024-04-23T18:25:59.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a41b80 0x5585a1a41fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 0x5585a1a431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a49fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f0238ffa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5585a19a6c50 con 0x5585a1a43710 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f0238ffa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5585a1943830 con 0x5585a1a41b80 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f0238ffa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5585a19a2c10 con 0x5585a1a42d50 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f022effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a49fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 0x5585a1a431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f022effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a49fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40840/0 (socket says 172.21.15.19:40840) 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f022effd640 1 -- 172.21.15.19:0/2866028218 learned_addr learned my addr 172.21.15.19:0/2866028218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.782+0000 7f022cff9640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f02180a3bb0 0x7f02180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f022ffff640 1 --2- 172.21.15.19:0/2866028218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a41b80 0x5585a1a41fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f022ffff640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f02180a3bb0 0x7f02180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f022cff9640 1 -- 172.21.15.19:0/4197308908 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f02240d05b0 con 0x7f02301031b0 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f022f7fe640 1 -- 172.21.15.19:0/2866028218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a41b80 msgr2=0x5585a1a41fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f022f7fe640 1 --2- 172.21.15.19:0/2866028218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a41b80 0x5585a1a41fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f022f7fe640 1 -- 172.21.15.19:0/2866028218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 msgr2=0x5585a1a49fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f022f7fe640 1 --2- 172.21.15.19:0/2866028218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a49fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f022f7fe640 1 -- 172.21.15.19:0/2866028218 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a16b92d0 con 0x5585a1a42d50 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f022f7fe640 1 --2- 172.21.15.19:0/2866028218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 0x5585a1a431d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0220009ca0 tx=0x7f0220009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb14f19babca3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f022ffff640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f02180a3bb0 0x7f02180a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f022800cfd0 tx=0x7f0228004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f0215ffb640 1 -- 172.21.15.19:0/2866028218 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220031710 con 0x5585a1a42d50 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f0215ffb640 1 -- 172.21.15.19:0/2866028218 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f022003f070 con 0x5585a1a42d50 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.783+0000 7f0215ffb640 1 -- 172.21.15.19:0/2866028218 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220004730 con 0x5585a1a42d50 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.784+0000 7f0238ffa740 1 -- 172.21.15.19:0/2866028218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 msgr2=0x5585a1a431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.784+0000 7f0238ffa740 1 --2- 172.21.15.19:0/2866028218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 0x5585a1a431d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0220009ca0 tx=0x7f0220009880 comp rx=0 tx=0).stop 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.784+0000 7f0238ffa740 1 -- 172.21.15.19:0/2866028218 shutdown_connections 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.784+0000 7f0238ffa740 1 --2- 172.21.15.19:0/2866028218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a49fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.784+0000 7f0238ffa740 1 --2- 172.21.15.19:0/2866028218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 0x5585a1a431d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.784+0000 7f0238ffa740 1 --2- 172.21.15.19:0/2866028218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a41b80 0x5585a1a41fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.784+0000 7f0238ffa740 1 -- 172.21.15.19:0/2866028218 >> 172.21.15.19:0/2866028218 conn(0x5585a1a3d070 msgr2=0x5585a1a3d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.784+0000 7f0238ffa740 1 -- 172.21.15.19:0/2866028218 shutdown_connections 2024-04-23T18:25:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.784+0000 7f0238ffa740 1 -- 172.21.15.19:0/2866028218 wait complete. 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.784+0000 7f0238ffa740 1 -- start start 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f0238ffa740 1 Processor -- start 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 0x5585a1a4bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a42d50 0x5585a1a4c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a43710 0x5585a1a53520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f0238ffa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5585a1a4a6c0 con 0x5585a1a42d50 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f0238ffa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5585a1941470 con 0x5585a1a43710 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f0238ffa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5585a1a4a840 con 0x5585a1a41b80 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f0238ffa740 10 client.0 _ll_register_callbacks cb 0x5585a1966f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a42d50 0x5585a1a4c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a42d50 0x5585a1a4c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40854/0 (socket says 172.21.15.19:40854) 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022effd640 1 -- 172.21.15.19:0/3228682438 learned_addr learned my addr 172.21.15.19:0/3228682438 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022f7fe640 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 0x5585a1a4bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022ffff640 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a43710 0x5585a1a53520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022effd640 1 -- 172.21.15.19:0/3228682438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 msgr2=0x5585a1a4bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022effd640 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 0x5585a1a4bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022effd640 1 -- 172.21.15.19:0/3228682438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a43710 msgr2=0x5585a1a53520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022effd640 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a43710 0x5585a1a53520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022effd640 1 -- 172.21.15.19:0/3228682438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a17836b0 con 0x5585a1a42d50 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022ffff640 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a43710 0x5585a1a53520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022f7fe640 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 0x5585a1a4bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f022effd640 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a42d50 0x5585a1a4c4e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f02240a4fe0 tx=0x7f022409c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23601e9c6974dea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.785+0000 7f020effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f020effd640 1 -- 172.21.15.19:0/3228682438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022409cfa0 con 0x5585a1a42d50 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f020effd640 1 -- 172.21.15.19:0/3228682438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f022400fa90 con 0x5585a1a42d50 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f0238ffa740 1 -- 172.21.15.19:0/3228682438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f022003f070 con 0x5585a1a42d50 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f020effd640 1 -- 172.21.15.19:0/3228682438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f022409c8a0 con 0x5585a1a42d50 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f0238ffa740 10 client.15285 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f0238ffa740 1 -- 172.21.15.19:0/3228682438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5585a1a56900 con 0x5585a1a42d50 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f020effd640 1 -- 172.21.15.19:0/3228682438 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f022003f070 con 0x5585a1a42d50 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f020effd640 1 -- 172.21.15.19:0/3228682438 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f022409cb00 con 0x5585a1a42d50 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f0238ffa740 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f0238ffa740 10 client.15285 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f0238ffa740 20 client.15285 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f0238ffa740 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5585a1a58f10 0x5585a1a5a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f0238ffa740 4 client.15285 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.786+0000 7f0238ffa740 1 -- 172.21.15.19:0/3228682438 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585a19b4670 con 0x5585a1a58f10 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.787+0000 7f022f7fe640 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5585a1a58f10 0x5585a1a5a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.787+0000 7f022f7fe640 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5585a1a58f10 0x5585a1a5a380 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=561ab1080c4fa33b server_cookie=c05ae7929eca3da5 in_seq=0 out_seq=0 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.787+0000 7f020effd640 10 client.15285 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f020effd640 1 -- 172.21.15.19:0/3228682438 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585a19b4670 con 0x5585a1a58f10 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f020effd640 10 client.15285 handle_command_reply: tid=0 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 client.15285 shutdown 2024-04-23T18:25:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 -- 172.21.15.19:0/3228682438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a42d50 msgr2=0x5585a1a4c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a42d50 0x5585a1a4c4e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f02240a4fe0 tx=0x7f022409c670 comp rx=0 tx=0).stop 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 -- 172.21.15.19:0/3228682438 shutdown_connections 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5585a1a58f10 0x5585a1a5a380 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a43710 0x5585a1a53520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a42d50 0x5585a1a4c4e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f020effd640 0 client.15285 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 --2- 172.21.15.19:0/3228682438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 0x5585a1a4bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 -- 172.21.15.19:0/3228682438 >> 172.21.15.19:0/3228682438 conn(0x5585a1a3d070 msgr2=0x5585a1a3c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 -- 172.21.15.19:0/3228682438 shutdown_connections 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 1 -- 172.21.15.19:0/3228682438 wait complete. 2024-04-23T18:25:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.788+0000 7f0238ffa740 20 client.15285 trim_cache size 0 max 16384 2024-04-23T18:25:59.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f0238ffa740 1 Processor -- start 2024-04-23T18:25:59.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f0238ffa740 1 -- start start 2024-04-23T18:25:59.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a41b80 0x5585a1a49e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 0x5585a1a534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a43710 0x5585a1a46830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f0238ffa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5585a1a586c0 con 0x5585a1a41b80 2024-04-23T18:25:59.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f0238ffa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5585a1a58d90 con 0x5585a1a43710 2024-04-23T18:25:59.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f0238ffa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5585a1a59460 con 0x5585a1a42d50 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a41b80 0x5585a1a49e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a41b80 0x5585a1a49e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40856/0 (socket says 172.21.15.19:40856) 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f022f7fe640 1 -- 172.21.15.19:0/3309697801 learned_addr learned my addr 172.21.15.19:0/3309697801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f022effd640 1 --2- 172.21.15.19:0/3309697801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 0x5585a1a534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f022f7fe640 1 -- 172.21.15.19:0/3309697801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 msgr2=0x5585a1a534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f022f7fe640 1 --2- 172.21.15.19:0/3309697801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 0x5585a1a534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f022f7fe640 1 -- 172.21.15.19:0/3309697801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a43710 msgr2=0x5585a1a46830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f022f7fe640 1 --2- 172.21.15.19:0/3309697801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a43710 0x5585a1a46830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f022f7fe640 1 -- 172.21.15.19:0/3309697801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a1a41790 con 0x5585a1a41b80 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f022f7fe640 1 --2- 172.21.15.19:0/3309697801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a41b80 0x5585a1a49e70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0220031a20 tx=0x7f022003d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b346f61122b7e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.807+0000 7f02157fa640 1 -- 172.21.15.19:0/3309697801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220031720 con 0x5585a1a41b80 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f02157fa640 1 -- 172.21.15.19:0/3309697801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f022003f040 con 0x5585a1a41b80 2024-04-23T18:25:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f02157fa640 1 -- 172.21.15.19:0/3309697801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220051130 con 0x5585a1a41b80 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f0238ffa740 1 -- 172.21.15.19:0/3309697801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a41b80 msgr2=0x5585a1a49e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f0238ffa740 1 --2- 172.21.15.19:0/3309697801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a41b80 0x5585a1a49e70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0220031a20 tx=0x7f022003d040 comp rx=0 tx=0).stop 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f0238ffa740 1 -- 172.21.15.19:0/3309697801 shutdown_connections 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f0238ffa740 1 --2- 172.21.15.19:0/3309697801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a43710 0x5585a1a46830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f0238ffa740 1 --2- 172.21.15.19:0/3309697801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a42d50 0x5585a1a534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f0238ffa740 1 --2- 172.21.15.19:0/3309697801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a41b80 0x5585a1a49e70 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f0238ffa740 1 -- 172.21.15.19:0/3309697801 >> 172.21.15.19:0/3309697801 conn(0x5585a1a3d070 msgr2=0x5585a1a380b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f0238ffa740 1 -- 172.21.15.19:0/3309697801 shutdown_connections 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f0238ffa740 1 -- 172.21.15.19:0/3309697801 wait complete. 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.808+0000 7f0238ffa740 1 -- start start 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f0238ffa740 1 Processor -- start 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 0x5585a1a4e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a42d50 0x5585a1a4ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f0238ffa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a51c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f0238ffa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5585a1a5a200 con 0x5585a1a43710 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f0238ffa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5585a1a59b30 con 0x5585a1a42d50 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f0238ffa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5585a1a5a8d0 con 0x5585a1a41b80 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f0238ffa740 10 client.0 _ll_register_callbacks cb 0x7f0224097240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a51c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a51c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40872/0 (socket says 172.21.15.19:40872) 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022ffff640 1 -- 172.21.15.19:0/2155130223 learned_addr learned my addr 172.21.15.19:0/2155130223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022f7fe640 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 0x5585a1a4e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022effd640 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a42d50 0x5585a1a4ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022ffff640 1 -- 172.21.15.19:0/2155130223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 msgr2=0x5585a1a4e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022ffff640 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 0x5585a1a4e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022ffff640 1 -- 172.21.15.19:0/2155130223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a42d50 msgr2=0x5585a1a4ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022ffff640 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a42d50 0x5585a1a4ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022ffff640 1 -- 172.21.15.19:0/2155130223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a1a52250 con 0x5585a1a43710 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.809+0000 7f022effd640 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a42d50 0x5585a1a4ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f022ffff640 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a51c60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f022801ed60 tx=0x7f022800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4767e1e6ccd28ae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f020effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f020effd640 1 -- 172.21.15.19:0/2155130223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228018070 con 0x5585a1a43710 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f020effd640 1 -- 172.21.15.19:0/2155130223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f022801c820 con 0x5585a1a43710 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f0238ffa740 1 -- 172.21.15.19:0/2155130223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f022003f040 con 0x5585a1a43710 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f020effd640 1 -- 172.21.15.19:0/2155130223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f022801ca20 con 0x5585a1a43710 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f0238ffa740 10 client.15291 fetch_fsmap learned FSMap version 26 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f0238ffa740 1 -- 172.21.15.19:0/2155130223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5585a1a535c0 con 0x5585a1a43710 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f020effd640 1 -- 172.21.15.19:0/2155130223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f022003f040 con 0x5585a1a43710 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f020effd640 1 -- 172.21.15.19:0/2155130223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0228033bd0 con 0x5585a1a43710 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f0238ffa740 10 client.15291 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f0238ffa740 10 client.15291 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f0238ffa740 20 client.15291 populate_metadata read hostname 'smithi019' 2024-04-23T18:25:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f0238ffa740 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5585a1ac36c0 0x5585a1ac23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f0238ffa740 4 client.15291 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.810+0000 7f0238ffa740 1 -- 172.21.15.19:0/2155130223 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "status"}) -- 0x5585a19ae4e0 con 0x5585a1ac36c0 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f022f7fe640 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5585a1ac36c0 0x5585a1ac23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f022f7fe640 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5585a1ac36c0 0x5585a1ac23d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53afbf531fe8e9e server_cookie=4f4cccd3390485ee in_seq=0 out_seq=0 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f020effd640 10 client.15291 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f020effd640 1 -- 172.21.15.19:0/2155130223 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5585a19ae4e0 con 0x5585a1ac36c0 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f020effd640 10 client.15291 handle_command_reply: tid=0 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f0238ffa740 1 client.15291 shutdown 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f0238ffa740 1 -- 172.21.15.19:0/2155130223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 msgr2=0x5585a1a51c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f0238ffa740 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a51c60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f022801ed60 tx=0x7f022800c980 comp rx=0 tx=0).stop 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f0238ffa740 1 -- 172.21.15.19:0/2155130223 shutdown_connections 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f0238ffa740 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5585a1ac36c0 0x5585a1ac23d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f0238ffa740 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585a1a42d50 0x5585a1a4ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f0238ffa740 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585a1a43710 0x5585a1a51c60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f0238ffa740 1 --2- 172.21.15.19:0/2155130223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a1a41b80 0x5585a1a4e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f020effd640 0 client.15291 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f0238ffa740 1 -- 172.21.15.19:0/2155130223 >> 172.21.15.19:0/2155130223 conn(0x5585a1a3d070 msgr2=0x5585a1a380b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.811+0000 7f0238ffa740 1 -- 172.21.15.19:0/2155130223 shutdown_connections 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0238ffa740 1 -- 172.21.15.19:0/2155130223 wait complete. 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0238ffa740 20 client.15291 trim_cache size 0 max 16384 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-23T18:25:59.811 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24521, 2024-04-23T18:25:59.812 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:25:59.812 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:25:59.812 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:25:59.812 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 123.46392536800001, 2024-04-23T18:25:59.812 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-23T18:25:59.812 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 88, 2024-04-23T18:25:59.812 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:25:59.812 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 129.53125040899999 2024-04-23T18:25:59.812 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:25:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 -- 172.21.15.19:0/4197308908 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f02180a3bb0 msgr2=0x7f02180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f02180a3bb0 0x7f02180a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f022800cfd0 tx=0x7f0228004730 comp rx=0 tx=0).stop 2024-04-23T18:25:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 -- 172.21.15.19:0/4197308908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02301031b0 msgr2=0x7f02301986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02301031b0 0x7f02301986a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f022400c920 tx=0x7f022400cde0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 -- 172.21.15.19:0/4197308908 shutdown_connections 2024-04-23T18:25:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f02180a3bb0 0x7f02180a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0230103b70 0x7f023019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02301031b0 0x7f02301986a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 --2- 172.21.15.19:0/4197308908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0230101fc0 0x7f0230198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 -- 172.21.15.19:0/4197308908 >> 172.21.15.19:0/4197308908 conn(0x7f02300fd530 msgr2=0x7f02301064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 -- 172.21.15.19:0/4197308908 shutdown_connections 2024-04-23T18:25:59.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.812+0000 7f0236205640 1 -- 172.21.15.19:0/4197308908 wait complete. 2024-04-23T18:25:59.816 INFO:teuthology.run_tasks:Running task sequential... 2024-04-23T18:25:59.826 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-23T18:25:59.827 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-23T18:25:59.827 INFO:teuthology.task.print:Enabling mgr modules 2024-04-23T18:25:59.827 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-23T18:25:59.828 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:25:59.828 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:25:59.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-23T18:25:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.936+0000 7fcc03849640 1 -- 172.21.15.19:0/1532869673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbfc103b90 msgr2=0x7fcbfc10a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.936+0000 7fcc03849640 1 --2- 172.21.15.19:0/1532869673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbfc103b90 0x7fcbfc10a420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009ca0 tx=0x7fcbec02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.936+0000 7fcc03849640 1 -- 172.21.15.19:0/1532869673 shutdown_connections 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.936+0000 7fcc03849640 1 --2- 172.21.15.19:0/1532869673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbfc103b90 0x7fcbfc10a420 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.936+0000 7fcc03849640 1 --2- 172.21.15.19:0/1532869673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc1031d0 0x7fcbfc103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.936+0000 7fcc03849640 1 --2- 172.21.15.19:0/1532869673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbfc101fe0 0x7fcbfc102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.936+0000 7fcc03849640 1 -- 172.21.15.19:0/1532869673 >> 172.21.15.19:0/1532869673 conn(0x7fcbfc0fd570 msgr2=0x7fcbfc0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc03849640 1 -- 172.21.15.19:0/1532869673 shutdown_connections 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc03849640 1 -- 172.21.15.19:0/1532869673 wait complete. 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc03849640 1 Processor -- start 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc03849640 1 -- start start 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc03849640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbfc101fe0 0x7fcbfc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc03849640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc1031d0 0x7fcbfc198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc03849640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbfc103b90 0x7fcbfc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc03849640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc10dae0 con 0x7fcbfc101fe0 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc03849640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc10d960 con 0x7fcbfc103b90 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc03849640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc10dc60 con 0x7fcbfc1031d0 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc015e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbfc101fe0 0x7fcbfc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc015e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbfc101fe0 0x7fcbfc1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40890/0 (socket says 172.21.15.19:40890) 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc015e5640 1 -- 172.21.15.19:0/1896967352 learned_addr learned my addr 172.21.15.19:0/1896967352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.937+0000 7fcc01de6640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbfc103b90 0x7fcbfc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcc00de4640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc1031d0 0x7fcbfc198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcc015e5640 1 -- 172.21.15.19:0/1896967352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc1031d0 msgr2=0x7fcbfc198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcc015e5640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc1031d0 0x7fcbfc198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcc015e5640 1 -- 172.21.15.19:0/1896967352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbfc103b90 msgr2=0x7fcbfc19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcc015e5640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbfc103b90 0x7fcbfc19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcc015e5640 1 -- 172.21.15.19:0/1896967352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc19fcf0 con 0x7fcbfc101fe0 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcc00de4640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc1031d0 0x7fcbfc198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcc015e5640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbfc101fe0 0x7fcbfc1980d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400d770 tx=0x7fcbf400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcbfa7fc640 1 -- 172.21.15.19:0/1896967352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4017070 con 0x7fcbfc101fe0 2024-04-23T18:25:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcc03849640 1 -- 172.21.15.19:0/1896967352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbfc19ffe0 con 0x7fcbfc101fe0 2024-04-23T18:25:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcbfa7fc640 1 -- 172.21.15.19:0/1896967352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf4009c30 con 0x7fcbfc101fe0 2024-04-23T18:25:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcbfa7fc640 1 -- 172.21.15.19:0/1896967352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4003cc0 con 0x7fcbfc101fe0 2024-04-23T18:25:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.938+0000 7fcc03849640 1 -- 172.21.15.19:0/1896967352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbfc1a04a0 con 0x7fcbfc101fe0 2024-04-23T18:25:59.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.939+0000 7fcc03849640 1 -- 172.21.15.19:0/1896967352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbfc109300 con 0x7fcbfc101fe0 2024-04-23T18:25:59.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.940+0000 7fcbfa7fc640 1 -- 172.21.15.19:0/1896967352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcbf4003e60 con 0x7fcbfc101fe0 2024-04-23T18:25:59.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.941+0000 7fcbfa7fc640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcbe40a3cd0 0x7fcbe40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:59.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.941+0000 7fcc00de4640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcbe40a3cd0 0x7fcbe40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:59.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.941+0000 7fcbfa7fc640 1 -- 172.21.15.19:0/1896967352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fcbf40cfd60 con 0x7fcbfc101fe0 2024-04-23T18:25:59.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.941+0000 7fcc00de4640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcbe40a3cd0 0x7fcbe40a6190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00058c0 tx=0x7fcbf0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:25:59.945+0000 7fcbfa7fc640 1 -- 172.21.15.19:0/1896967352 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcbf4009de0 con 0x7fcbfc101fe0 2024-04-23T18:26:00.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:59 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2052267954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:00.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:25:59 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2057334264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:00.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:59 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2052267954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:00.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:25:59 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2057334264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:00.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:59 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2052267954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:00.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:25:59 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2057334264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.184+0000 7fcc03849640 1 -- 172.21.15.19:0/1896967352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fcbfc062c50 con 0x7fcbfc101fe0 2024-04-23T18:26:00.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.737+0000 7fcbfa7fc640 1 -- 172.21.15.19:0/1896967352 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fcbf40bc0a0 con 0x7fcbfc101fe0 2024-04-23T18:26:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 -- 172.21.15.19:0/1896967352 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcbe40a3cd0 msgr2=0x7fcbe40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcbe40a3cd0 0x7fcbe40a6190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00058c0 tx=0x7fcbf0005850 comp rx=0 tx=0).stop 2024-04-23T18:26:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 -- 172.21.15.19:0/1896967352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbfc101fe0 msgr2=0x7fcbfc1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbfc101fe0 0x7fcbfc1980d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400d770 tx=0x7fcbf400dc30 comp rx=0 tx=0).stop 2024-04-23T18:26:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 -- 172.21.15.19:0/1896967352 shutdown_connections 2024-04-23T18:26:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fcbe40a3cd0 0x7fcbe40a6190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbfc103b90 0x7fcbfc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc1031d0 0x7fcbfc198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 --2- 172.21.15.19:0/1896967352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbfc101fe0 0x7fcbfc1980d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 -- 172.21.15.19:0/1896967352 >> 172.21.15.19:0/1896967352 conn(0x7fcbfc0fd570 msgr2=0x7fcbfc1064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:00.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 -- 172.21.15.19:0/1896967352 shutdown_connections 2024-04-23T18:26:00.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.740+0000 7fcc03849640 1 -- 172.21.15.19:0/1896967352 wait complete. 2024-04-23T18:26:00.754 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 -- 172.21.15.19:0/1123527986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4604107cf0 msgr2=0x7f460410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 --2- 172.21.15.19:0/1123527986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4604107cf0 0x7f460410a0e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009ca0 tx=0x7f45fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 -- 172.21.15.19:0/1123527986 shutdown_connections 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 --2- 172.21.15.19:0/1123527986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46041053c0 0x7f46041077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 --2- 172.21.15.19:0/1123527986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4604107cf0 0x7f460410a0e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 --2- 172.21.15.19:0/1123527986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4604102a90 0x7f4604104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 -- 172.21.15.19:0/1123527986 >> 172.21.15.19:0/1123527986 conn(0x7f46040fc420 msgr2=0x7f46040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 -- 172.21.15.19:0/1123527986 shutdown_connections 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 -- 172.21.15.19:0/1123527986 wait complete. 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 Processor -- start 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.873+0000 7f460a6ea640 1 -- start start 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f460a6ea640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4604102a90 0x7f4604198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f460a6ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46041053c0 0x7f4604198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f460a6ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4604107cf0 0x7f460419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f460a6ea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f460410daf0 con 0x7f46041053c0 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f460a6ea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f460410d970 con 0x7f4604107cf0 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f460a6ea640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f460410dc70 con 0x7f4604102a90 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46041053c0 0x7f4604198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46041053c0 0x7f4604198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40922/0 (socket says 172.21.15.19:40922) 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46037fe640 1 -- 172.21.15.19:0/1342900648 learned_addr learned my addr 172.21.15.19:0/1342900648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f4603fff640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4604102a90 0x7f4604198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f4608c87640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4604107cf0 0x7f460419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46037fe640 1 -- 172.21.15.19:0/1342900648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4604102a90 msgr2=0x7f4604198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46037fe640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4604102a90 0x7f4604198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46037fe640 1 -- 172.21.15.19:0/1342900648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4604107cf0 msgr2=0x7f460419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46037fe640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4604107cf0 0x7f460419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46037fe640 1 -- 172.21.15.19:0/1342900648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460419fd60 con 0x7f46041053c0 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46037fe640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46041053c0 0x7f4604198680 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f45f800e9c0 tx=0x7f45f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46017fa640 1 -- 172.21.15.19:0/1342900648 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f800cd80 con 0x7f46041053c0 2024-04-23T18:26:00.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46017fa640 1 -- 172.21.15.19:0/1342900648 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f80054d0 con 0x7f46041053c0 2024-04-23T18:26:00.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f460a6ea640 1 -- 172.21.15.19:0/1342900648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460419fff0 con 0x7f46041053c0 2024-04-23T18:26:00.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.874+0000 7f46017fa640 1 -- 172.21.15.19:0/1342900648 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8010430 con 0x7f46041053c0 2024-04-23T18:26:00.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.875+0000 7f460a6ea640 1 -- 172.21.15.19:0/1342900648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46041a0510 con 0x7f46041053c0 2024-04-23T18:26:00.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.876+0000 7f460a6ea640 1 -- 172.21.15.19:0/1342900648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46040fdf10 con 0x7f46041053c0 2024-04-23T18:26:00.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.877+0000 7f46017fa640 1 -- 172.21.15.19:0/1342900648 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f45f8004fd0 con 0x7f46041053c0 2024-04-23T18:26:00.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.877+0000 7f46017fa640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f45ec0a3ca0 0x7f45ec0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:00.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.877+0000 7f4603fff640 1 -- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f45ec0a3ca0 msgr2=0x7f45ec0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1993809272 2024-04-23T18:26:00.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.877+0000 7f46017fa640 1 -- 172.21.15.19:0/1342900648 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f45f80d0820 con 0x7f46041053c0 2024-04-23T18:26:00.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.877+0000 7f4603fff640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f45ec0a3ca0 0x7f45ec0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:00.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:00.882+0000 7f46017fa640 1 -- 172.21.15.19:0/1342900648 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45f809d210 con 0x7f46041053c0 2024-04-23T18:26:01.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:00 smithi019 ceph-mon[29215]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:26:01.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:00 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1896967352' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:26:01.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: ignoring --setuser ceph since I am not root 2024-04-23T18:26:01.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: ignoring --setgroup ceph since I am not root 2024-04-23T18:26:01.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:00.934+0000 7f6baa684200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:26:01.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:00 smithi044 ceph-mon[32458]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:26:01.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:00 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1896967352' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:26:01.062 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: ignoring --setuser ceph since I am not root 2024-04-23T18:26:01.062 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: ignoring --setgroup ceph since I am not root 2024-04-23T18:26:01.062 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:00 smithi044 ceph-mgr[33894]: -- 172.21.15.44:0/3040173754 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55834cd365a0 con 0x55834cd0ac00 2024-04-23T18:26:01.063 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:00.927+0000 7f6088804200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:26:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.073+0000 7f460a6ea640 1 -- 172.21.15.19:0/1342900648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f4604062bf0 con 0x7f46041053c0 2024-04-23T18:26:01.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.078+0000 7f4603fff640 1 -- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f45ec0a3ca0 msgr2=0x7f45ec0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1993809272 2024-04-23T18:26:01.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.078+0000 7f46017fa640 1 -- 172.21.15.19:0/1342900648 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v34)=0 v34) ==== 138+0+0 (secure 0 0 0) 0x7f45f809cb10 con 0x7f46041053c0 2024-04-23T18:26:01.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.078+0000 7f4603fff640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f45ec0a3ca0 0x7f45ec0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:01.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:00 smithi155 ceph-mon[33468]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:26:01.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:00 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1896967352' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:26:01.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: ignoring --setuser ceph since I am not root 2024-04-23T18:26:01.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: ignoring --setgroup ceph since I am not root 2024-04-23T18:26:01.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:00 smithi155 ceph-mgr[35294]: -- 172.21.15.155:0/2206732329 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba547cc5a0 con 0x55ba547a0800 2024-04-23T18:26:01.073 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:00.920+0000 7f241224e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:26:01.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 -- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f45ec0a3ca0 msgr2=0x7f45ec0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:01.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f45ec0a3ca0 0x7f45ec0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 -- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46041053c0 msgr2=0x7f4604198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46041053c0 0x7f4604198680 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f45f800e9c0 tx=0x7f45f800ee80 comp rx=0 tx=0).stop 2024-04-23T18:26:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 -- 172.21.15.19:0/1342900648 shutdown_connections 2024-04-23T18:26:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f45ec0a3ca0 0x7f45ec0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4604107cf0 0x7f460419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46041053c0 0x7f4604198680 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 --2- 172.21.15.19:0/1342900648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4604102a90 0x7f4604198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 -- 172.21.15.19:0/1342900648 >> 172.21.15.19:0/1342900648 conn(0x7f46040fc420 msgr2=0x7f46041033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 -- 172.21.15.19:0/1342900648 shutdown_connections 2024-04-23T18:26:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.080+0000 7f460a6ea640 1 -- 172.21.15.19:0/1342900648 wait complete. 2024-04-23T18:26:01.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-23T18:26:01.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.196+0000 7f7b794a9640 1 -- 172.21.15.19:0/1961483197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b74100e60 msgr2=0x7f7b74101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:01.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.196+0000 7f7b794a9640 1 --2- 172.21.15.19:0/1961483197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b74100e60 0x7f7b74101280 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:01.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.196+0000 7f7b794a9640 1 -- 172.21.15.19:0/1961483197 shutdown_connections 2024-04-23T18:26:01.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.196+0000 7f7b794a9640 1 --2- 172.21.15.19:0/1961483197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b74102a00 0x7f7b74109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.196+0000 7f7b794a9640 1 --2- 172.21.15.19:0/1961483197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b74102040 0x7f7b741024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.196+0000 7f7b794a9640 1 --2- 172.21.15.19:0/1961483197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b74100e60 0x7f7b74101280 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.196+0000 7f7b794a9640 1 -- 172.21.15.19:0/1961483197 >> 172.21.15.19:0/1961483197 conn(0x7f7b740fc440 msgr2=0x7f7b740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:01.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.197+0000 7f7b794a9640 1 -- 172.21.15.19:0/1961483197 shutdown_connections 2024-04-23T18:26:01.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.197+0000 7f7b794a9640 1 -- 172.21.15.19:0/1961483197 wait complete. 2024-04-23T18:26:01.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.197+0000 7f7b794a9640 1 Processor -- start 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b794a9640 1 -- start start 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b794a9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b74100e60 0x7f7b74198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b794a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b74102040 0x7f7b74198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b794a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b74102a00 0x7f7b7419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b794a9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410ca20 con 0x7f7b74102a00 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b794a9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410c8a0 con 0x7f7b74102040 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b794a9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410cba0 con 0x7f7b74100e60 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b74102a00 0x7f7b7419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b72ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b74100e60 0x7f7b74198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b74102a00 0x7f7b7419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40938/0 (socket says 172.21.15.19:40938) 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b727fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b74102040 0x7f7b74198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b737fe640 1 -- 172.21.15.19:0/1650142159 learned_addr learned my addr 172.21.15.19:0/1650142159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b737fe640 1 -- 172.21.15.19:0/1650142159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b74100e60 msgr2=0x7f7b74198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b737fe640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b74100e60 0x7f7b74198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b737fe640 1 -- 172.21.15.19:0/1650142159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b74102040 msgr2=0x7f7b74198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b737fe640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b74102040 0x7f7b74198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b737fe640 1 -- 172.21.15.19:0/1650142159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7419fd60 con 0x7f7b74102a00 2024-04-23T18:26:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.198+0000 7f7b737fe640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b74102a00 0x7f7b7419f6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009fd0 tx=0x7f7b6c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.199+0000 7f7b5bfff640 1 -- 172.21.15.19:0/1650142159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c017070 con 0x7f7b74102a00 2024-04-23T18:26:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.199+0000 7f7b5bfff640 1 -- 172.21.15.19:0/1650142159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c003e20 con 0x7f7b74102a00 2024-04-23T18:26:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.199+0000 7f7b794a9640 1 -- 172.21.15.19:0/1650142159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7419fff0 con 0x7f7b74102a00 2024-04-23T18:26:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.199+0000 7f7b5bfff640 1 -- 172.21.15.19:0/1650142159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c010460 con 0x7f7b74102a00 2024-04-23T18:26:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.199+0000 7f7b794a9640 1 -- 172.21.15.19:0/1650142159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741a04c0 con 0x7f7b74102a00 2024-04-23T18:26:01.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.200+0000 7f7b794a9640 1 -- 172.21.15.19:0/1650142159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b741a0720 con 0x7f7b74102a00 2024-04-23T18:26:01.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.201+0000 7f7b5bfff640 1 -- 172.21.15.19:0/1650142159 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f7b6c003970 con 0x7f7b74102a00 2024-04-23T18:26:01.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.202+0000 7f7b5bfff640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7b5c0a3ca0 0x7f7b5c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:01.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.202+0000 7f7b72ffd640 1 -- 172.21.15.19:0/1650142159 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7b5c0a3ca0 msgr2=0x7f7b5c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1993809272 2024-04-23T18:26:01.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.202+0000 7f7b5bfff640 1 -- 172.21.15.19:0/1650142159 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f7b6c0d01b0 con 0x7f7b74102a00 2024-04-23T18:26:01.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.202+0000 7f7b72ffd640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7b5c0a3ca0 0x7f7b5c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:01.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.206+0000 7f7b5bfff640 1 -- 172.21.15.19:0/1650142159 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b6c09c4f0 con 0x7f7b74102a00 2024-04-23T18:26:01.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.394+0000 7f7b794a9640 1 -- 172.21.15.19:0/1650142159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f7b741a0990 con 0x7f7b74102a00 2024-04-23T18:26:01.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.399+0000 7f7b5bfff640 1 -- 172.21.15.19:0/1650142159 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v35)=0 v35) ==== 133+0+0 (secure 0 0 0) 0x7f7b6c0a07f0 con 0x7f7b74102a00 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.401+0000 7f7b794a9640 1 -- 172.21.15.19:0/1650142159 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7b5c0a3ca0 msgr2=0x7f7b5c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.401+0000 7f7b794a9640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7b5c0a3ca0 0x7f7b5c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.401+0000 7f7b794a9640 1 -- 172.21.15.19:0/1650142159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b74102a00 msgr2=0x7f7b7419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.401+0000 7f7b794a9640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b74102a00 0x7f7b7419f6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009fd0 tx=0x7f7b6c00b260 comp rx=0 tx=0).stop 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.401+0000 7f7b794a9640 1 -- 172.21.15.19:0/1650142159 shutdown_connections 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.401+0000 7f7b794a9640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f7b5c0a3ca0 0x7f7b5c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.401+0000 7f7b794a9640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b74102040 0x7f7b74198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.401+0000 7f7b794a9640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b74102a00 0x7f7b7419f6f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.401+0000 7f7b794a9640 1 --2- 172.21.15.19:0/1650142159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b74100e60 0x7f7b74198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.401+0000 7f7b794a9640 1 -- 172.21.15.19:0/1650142159 >> 172.21.15.19:0/1650142159 conn(0x7f7b740fc440 msgr2=0x7f7b7406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.402+0000 7f7b794a9640 1 -- 172.21.15.19:0/1650142159 shutdown_connections 2024-04-23T18:26:01.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:01.402+0000 7f7b794a9640 1 -- 172.21.15.19:0/1650142159 wait complete. 2024-04-23T18:26:01.404 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:01.132+0000 7f6baa684200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:01.405 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:01.244+0000 7f6baa684200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:26:01.406 INFO:teuthology.run_tasks:Running task kclient... 2024-04-23T18:26:01.430 INFO:tasks.kclient:Mounting kernel clients... 2024-04-23T18:26:01.430 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-23T18:26:01.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:01.431 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-23T18:26:01.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:01.446 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-23T18:26:01.514 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:01.105+0000 7f241224e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:01.514 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:01.210+0000 7f241224e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:26:01.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:26:01.515 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-23T18:26:01.515 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-23T18:26:01.516 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T18:26:01.516 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:01.516 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-23T18:26:01.545 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:01.122+0000 7f6088804200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:01.545 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:01.229+0000 7f6088804200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:26:01.546 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:01.552+0000 7f6088804200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:26:01.657 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:01.574+0000 7f6baa684200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:26:01.658 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:01.664+0000 7f6baa684200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:26:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:26:01.812 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:01.640+0000 7f6088804200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:26:01.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:01 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1896967352' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:26:01.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:01 smithi044 ceph-mon[32458]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-23T18:26:01.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:01 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1342900648' entity='client.admin' 2024-04-23T18:26:01.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:01 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1650142159' entity='client.admin' 2024-04-23T18:26:01.822 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:01.519+0000 7f241224e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:26:01.822 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:01.604+0000 7f241224e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:26:01.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:01 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1896967352' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:26:01.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:01 smithi155 ceph-mon[33468]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-23T18:26:01.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:01 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1342900648' entity='client.admin' 2024-04-23T18:26:01.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:01 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1650142159' entity='client.admin' 2024-04-23T18:26:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:01 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1896967352' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:26:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:01 smithi019 ceph-mon[29215]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-23T18:26:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:01 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1342900648' entity='client.admin' 2024-04-23T18:26:01.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:01 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1650142159' entity='client.admin' 2024-04-23T18:26:03.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:02 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:02.787+0000 7f241224e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:26:03.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:02 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:02.959+0000 7f241224e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:03.237 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:02 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:02.890+0000 7f6088804200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:26:03.238 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:03.067+0000 7f6088804200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:03.238 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:03.243+0000 7f6088804200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:26:03.248 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:02.941+0000 7f6baa684200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:26:03.248 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:03.122+0000 7f6baa684200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:03.414 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:03.129+0000 7f241224e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:26:03.510 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:03.309+0000 7f6baa684200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:26:03.712 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:03.421+0000 7f241224e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:26:03.712 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:03.514+0000 7f241224e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:26:03.788 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:03.620+0000 7f6baa684200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:26:03.788 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:03.719+0000 7f6baa684200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:26:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.794+0000 7fa5a718c700 1 -- 172.21.15.19:0/3484259645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a0102300 msgr2=0x7fa5a010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.794+0000 7fa5a718c700 1 --2- 172.21.15.19:0/3484259645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a0102300 0x7fa5a010e7f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa59800b0a0 tx=0x7fa59800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:26:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.794+0000 7fa5a718c700 1 -- 172.21.15.19:0/3484259645 shutdown_connections 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.794+0000 7fa5a718c700 1 --2- 172.21.15.19:0/3484259645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a0102300 0x7fa5a010e7f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.794+0000 7fa5a718c700 1 --2- 172.21.15.19:0/3484259645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a0107990 0x7fa5a0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.794+0000 7fa5a718c700 1 --2- 172.21.15.19:0/3484259645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a0101940 0x7fa5a0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.794+0000 7fa5a718c700 1 -- 172.21.15.19:0/3484259645 >> 172.21.15.19:0/3484259645 conn(0x7fa5a00788f0 msgr2=0x7fa5a0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.794+0000 7fa5a718c700 1 -- 172.21.15.19:0/3484259645 shutdown_connections 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.795+0000 7fa5a718c700 1 -- 172.21.15.19:0/3484259645 wait complete. 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.795+0000 7fa5a718c700 1 Processor -- start 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.795+0000 7fa5a718c700 1 -- start start 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.795+0000 7fa5a718c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a0101940 0x7fa5a0199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.795+0000 7fa5a718c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a0102300 0x7fa5a019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a718c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a0107990 0x7fa5a01a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a718c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0111b00 con 0x7fa5a0101940 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a718c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0111980 con 0x7fa5a0107990 2024-04-23T18:26:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a718c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0111c80 con 0x7fa5a0102300 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a4f28700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a0101940 0x7fa5a0199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a5729700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a0107990 0x7fa5a01a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a5729700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a0107990 0x7fa5a01a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:38590/0 (socket says 172.21.15.19:38590) 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa59ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a0102300 0x7fa5a019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a5729700 1 -- 172.21.15.19:0/2968122701 learned_addr learned my addr 172.21.15.19:0/2968122701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a5729700 1 -- 172.21.15.19:0/2968122701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a0102300 msgr2=0x7fa5a019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a5729700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a0102300 0x7fa5a019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a5729700 1 -- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a0101940 msgr2=0x7fa5a0199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a5729700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a0101940 0x7fa5a0199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a5729700 1 -- 172.21.15.19:0/2968122701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a01a54e0 con 0x7fa5a0107990 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa59ffff700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a0102300 0x7fa5a019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a5729700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a0107990 0x7fa5a01a4fa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa59800b780 tx=0x7fa598004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa59dffb700 1 -- 172.21.15.19:0/2968122701 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598021070 con 0x7fa5a0107990 2024-04-23T18:26:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.796+0000 7fa5a718c700 1 -- 172.21.15.19:0/2968122701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a01a56b0 con 0x7fa5a0107990 2024-04-23T18:26:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.797+0000 7fa59dffb700 1 -- 172.21.15.19:0/2968122701 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa598007e30 con 0x7fa5a0107990 2024-04-23T18:26:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.797+0000 7fa59dffb700 1 -- 172.21.15.19:0/2968122701 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598004700 con 0x7fa5a0107990 2024-04-23T18:26:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.797+0000 7fa5a718c700 1 -- 172.21.15.19:0/2968122701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a01a5b60 con 0x7fa5a0107990 2024-04-23T18:26:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.798+0000 7fa5a718c700 1 -- 172.21.15.19:0/2968122701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a0109eb0 con 0x7fa5a0107990 2024-04-23T18:26:03.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.799+0000 7fa59dffb700 1 -- 172.21.15.19:0/2968122701 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fa598007980 con 0x7fa5a0107990 2024-04-23T18:26:03.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.800+0000 7fa59dffb700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa58c0a3c90 0x7fa58c0a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:03.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.800+0000 7fa5a4f28700 1 -- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa58c0a3c90 msgr2=0x7fa58c0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1993809272 2024-04-23T18:26:03.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.800+0000 7fa59dffb700 1 -- 172.21.15.19:0/2968122701 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fa59800f020 con 0x7fa5a0107990 2024-04-23T18:26:03.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.800+0000 7fa5a4f28700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa58c0a3c90 0x7fa58c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:03.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:03.808+0000 7fa59dffb700 1 -- 172.21.15.19:0/2968122701 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa59802a020 con 0x7fa5a0107990 2024-04-23T18:26:03.812 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:03.547+0000 7f6088804200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:26:03.812 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:03.643+0000 7f6088804200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:26:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.001+0000 7fa5a4f28700 1 -- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa58c0a3c90 msgr2=0x7fa58c0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1993809272 2024-04-23T18:26:03.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.001+0000 7fa5a4f28700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa58c0a3c90 0x7fa58c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:04.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.063+0000 7fa5a718c700 1 -- 172.21.15.19:0/2968122701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa5a006ad70 con 0x7fa5a0107990 2024-04-23T18:26:04.057 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:03.935+0000 7f6baa684200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:26:04.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.064+0000 7fa59dffb700 1 -- 172.21.15.19:0/2968122701 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fa59809e630 con 0x7fa5a0107990 2024-04-23T18:26:04.057 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T18:26:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 -- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa58c0a3c90 msgr2=0x7fa58c0a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa58c0a3c90 0x7fa58c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 -- 172.21.15.19:0/2968122701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a0107990 msgr2=0x7fa5a01a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a0107990 0x7fa5a01a4fa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa59800b780 tx=0x7fa598004d00 comp rx=0 tx=0).stop 2024-04-23T18:26:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 -- 172.21.15.19:0/2968122701 shutdown_connections 2024-04-23T18:26:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7fa58c0a3c90 0x7fa58c0a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a0107990 0x7fa5a01a4fa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a0101940 0x7fa5a0199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 --2- 172.21.15.19:0/2968122701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a0102300 0x7fa5a019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 -- 172.21.15.19:0/2968122701 >> 172.21.15.19:0/2968122701 conn(0x7fa5a00788f0 msgr2=0x7fa5a00ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 -- 172.21.15.19:0/2968122701 shutdown_connections 2024-04-23T18:26:04.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04.067+0000 7fa5a718c700 1 -- 172.21.15.19:0/2968122701 wait complete. 2024-04-23T18:26:04.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:03.718+0000 7f241224e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:26:04.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:03.911+0000 7f241224e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:04.109 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:03.857+0000 7f6088804200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:26:04.109 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:04 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:04.058+0000 7f6088804200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:04.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:04 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2968122701' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:26:04.444 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:04.151+0000 7f6baa684200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:04.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:04 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2968122701' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:26:04.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:04 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2968122701' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:26:04.822 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:04 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:04.518+0000 7f241224e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:26:04.833 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T18:26:04.833 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T18:26:04.833 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-23T18:26:04.833 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T18:26:04.833 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:04.833 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T18:26:04.833 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T18:26:04.833 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T18:26:04.833 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T18:26:04.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:04.834 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-23T18:26:04.850 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T18:26:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T18:26:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T18:26:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:26:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-23T18:26:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:26:04.850 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41993sec preferred_lft 41993sec 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-23T18:26:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-23T18:26:04.852 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T18:26:04.852 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:26:04.852 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-23T18:26:04.852 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-04-23T18:26:04.852 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-04-23T18:26:04.852 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-04-23T18:26:04.852 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T18:26:04.852 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-23T18:26:04.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:05.062 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:04 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:04.692+0000 7f6088804200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:26:05.063 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:04.783+0000 7f6baa684200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:26:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:05.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:05.108 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T18:26:05.132 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-23T18:26:05.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:05.134 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-23T18:26:05.203 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-23T18:26:05.203 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T18:26:05.203 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T18:26:05.203 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T18:26:05.203 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T18:26:05.204 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:26:05.204 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-23T18:26:05.204 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T18:26:05.204 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T18:26:05.204 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T18:26:05.204 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-23T18:26:05.228 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:04 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:04.940+0000 7f241224e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:26:05.229 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:05.035+0000 7f241224e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:26:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:05.346 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:05.248+0000 7f6baa684200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:26:05.346 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:05.353+0000 7f6baa684200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:26:05.437 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:05.134+0000 7f6088804200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:26:05.437 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:05.234+0000 7f6088804200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:26:05.438 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:05.444+0000 7f6088804200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:26:05.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:05.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:05.471 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-23T18:26:05.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:05.486 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-04-23T18:26:05.542 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:26:05.542 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-23T18:26:05.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T18:26:05.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-23T18:26:05.542 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-23T18:26:05.572 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:05.235+0000 7f241224e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:26:05.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:05.641 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:05.575+0000 7f6baa684200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:26:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:05.663 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-23T18:26:05.663 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:26:05.663 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-23T18:26:05.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-23T18:26:05.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T18:26:05.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-23T18:26:05.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-23T18:26:05.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-23T18:26:05.663 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-23T18:26:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:05.790 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:26:05.790 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-23T18:26:05.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-23T18:26:05.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T18:26:05.790 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-23T18:26:05.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:05.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:05.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T18:26:05.869 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:05.884 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T18:26:05.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T18:26:05.884 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:05.938 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-23T18:26:05.938 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-23T18:26:05.939 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/897ae47f-ad36-4531-8493-4f897c3ecfab 2024-04-23T18:26:05.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:05.939 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/897ae47f-ad36-4531-8493-4f897c3ecfab /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-23T18:26:06.030 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-23T18:26:06.031 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-23T18:26:06.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:05.760+0000 7f241224e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:26:06.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:05.853+0000 7f241224e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:26:06.133 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-23T18:26:06.133 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-23T18:26:06.133 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.44:3300,172.21.15.155:3300" 2024-04-23T18:26:06.133 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@46706838-019d-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/897ae47f-ad36-4531-8493-4f897c3ecfab 2024-04-23T18:26:06.134 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.19:3300/172.21.15.44:3300/172.21.15.155:3300" will pass to kernel 2024-04-23T18:26:06.134 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-23T18:26:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-23T18:26:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-23T18:26:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-23T18:26:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-23T18:26:06.137 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-23T18:26:06.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:06.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:06.163 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:26:06.253 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:06.153+0000 7f6baa684200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:26:06.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:26:06.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:06.270 INFO:teuthology.orchestra.run.smithi019.stdout:client15336 2024-04-23T18:26:06.270 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:26:06.270 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:26:06.287 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:26:06.312 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:05.991+0000 7f6088804200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:26:06.312 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:06.087+0000 7f6088804200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:26:06.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:26:06.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:06.387 INFO:teuthology.orchestra.run.smithi019.stdout:client15336 2024-04-23T18:26:06.387 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:26:06.432 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:06.095+0000 7f241224e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:26:06.432 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:06.227+0000 7f241224e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:26:06.432 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:06.323+0000 7f241224e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:26:06.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:26:06.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:06.446 INFO:teuthology.orchestra.run.smithi019.stdout:client15336 2024-04-23T18:26:06.446 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/46706838-019d-11ef-bc93-c7b262605968.client15336/status') 2024-04-23T18:26:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:06.503 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:06.260+0000 7f6baa684200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:26:06.506 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.15336 (3)192.168.0.1:0/445122245 2024-04-23T18:26:06.506 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-04-23T18:26:06.506 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-04-23T18:26:06.506 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-04-23T18:26:06.506 INFO:teuthology.orchestra.run.smithi019.stderr:69 bytes copied, 2.2823e-05 s, 3.0 MB/s 2024-04-23T18:26:06.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:06.513 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T18:26:06.523 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:26:06.523 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:26:06.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-23T18:26:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 -- 172.21.15.19:0/3786631852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8102040 msgr2=0x7f20b81024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 --2- 172.21.15.19:0/3786631852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8102040 0x7f20b81024c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009860 tx=0x7f20a802a340 comp rx=0 tx=0).stop 2024-04-23T18:26:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 -- 172.21.15.19:0/3786631852 shutdown_connections 2024-04-23T18:26:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 --2- 172.21.15.19:0/3786631852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8102040 0x7f20b81024c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 --2- 172.21.15.19:0/3786631852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b8102a00 0x7f20b8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 --2- 172.21.15.19:0/3786631852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20b8100e60 0x7f20b8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 -- 172.21.15.19:0/3786631852 >> 172.21.15.19:0/3786631852 conn(0x7f20b80fc420 msgr2=0x7f20b80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 -- 172.21.15.19:0/3786631852 shutdown_connections 2024-04-23T18:26:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 -- 172.21.15.19:0/3786631852 wait complete. 2024-04-23T18:26:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 Processor -- start 2024-04-23T18:26:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.636+0000 7f20c014f640 1 -- start start 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20c014f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20b8100e60 0x7f20b8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20c014f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8102040 0x7f20b81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20c014f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b8102a00 0x7f20b819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20c014f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20b810c9f0 con 0x7f20b8100e60 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20c014f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f20b810c870 con 0x7f20b8102040 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20c014f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20b810cb70 con 0x7f20b8102a00 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bdeeb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20b8100e60 0x7f20b8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bdeeb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20b8100e60 0x7f20b8198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40968/0 (socket says 172.21.15.19:40968) 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bdeeb640 1 -- 172.21.15.19:0/1019518334 learned_addr learned my addr 172.21.15.19:0/1019518334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bd6ea640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8102040 0x7f20b81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20be6ec640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b8102a00 0x7f20b819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bdeeb640 1 -- 172.21.15.19:0/1019518334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b8102a00 msgr2=0x7f20b819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bdeeb640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b8102a00 0x7f20b819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bdeeb640 1 -- 172.21.15.19:0/1019518334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8102040 msgr2=0x7f20b81986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bdeeb640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8102040 0x7f20b81986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bdeeb640 1 -- 172.21.15.19:0/1019518334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b819fd80 con 0x7f20b8100e60 2024-04-23T18:26:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bd6ea640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8102040 0x7f20b81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20bdeeb640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20b8100e60 0x7f20b8198160 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f20b400e9a0 tx=0x7f20b400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20aeffd640 1 -- 172.21.15.19:0/1019518334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b400ccc0 con 0x7f20b8100e60 2024-04-23T18:26:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20aeffd640 1 -- 172.21.15.19:0/1019518334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b40108e0 con 0x7f20b8100e60 2024-04-23T18:26:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20c014f640 1 -- 172.21.15.19:0/1019518334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b81a0070 con 0x7f20b8100e60 2024-04-23T18:26:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20aeffd640 1 -- 172.21.15.19:0/1019518334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4018980 con 0x7f20b8100e60 2024-04-23T18:26:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.637+0000 7f20c014f640 1 -- 172.21.15.19:0/1019518334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b81a0590 con 0x7f20b8100e60 2024-04-23T18:26:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.638+0000 7f20c014f640 1 -- 172.21.15.19:0/1019518334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b81a07f0 con 0x7f20b8100e60 2024-04-23T18:26:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.640+0000 7f20aeffd640 1 -- 172.21.15.19:0/1019518334 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f20b401f020 con 0x7f20b8100e60 2024-04-23T18:26:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.640+0000 7f20aeffd640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20a40a3ca0 0x7f20a40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.640+0000 7f20bd6ea640 1 -- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20a40a3ca0 msgr2=0x7f20a40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1993809272 2024-04-23T18:26:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.640+0000 7f20aeffd640 1 -- 172.21.15.19:0/1019518334 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f20b40d0ae0 con 0x7f20b8100e60 2024-04-23T18:26:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.640+0000 7f20bd6ea640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20a40a3ca0 0x7f20a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.644+0000 7f20aeffd640 1 -- 172.21.15.19:0/1019518334 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20b409d4d0 con 0x7f20b8100e60 2024-04-23T18:26:06.693 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:06.341+0000 7f6088804200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:26:06.693 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:06.480+0000 7f6088804200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:26:06.693 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:06.578+0000 7f6088804200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:26:06.694 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:06.699+0000 7f6088804200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:06.759 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:06.534+0000 7f6baa684200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:26:06.759 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:06.684+0000 7f6baa684200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:26:06.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.841+0000 7f20bd6ea640 1 -- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20a40a3ca0 msgr2=0x7f20a40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1993809272 2024-04-23T18:26:06.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.841+0000 7f20bd6ea640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20a40a3ca0 0x7f20a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:06.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.873+0000 7f20c014f640 1 -- 172.21.15.19:0/1019518334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f20b81a09c0 con 0x7f20b8100e60 2024-04-23T18:26:06.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.925+0000 7f20aeffd640 1 -- 172.21.15.19:0/1019518334 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f20b409cdd0 con 0x7f20b8100e60 2024-04-23T18:26:06.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 -- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20a40a3ca0 msgr2=0x7f20a40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:06.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20a40a3ca0 0x7f20a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 -- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20b8100e60 msgr2=0x7f20b8198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:06.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20b8100e60 0x7f20b8198160 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f20b400e9a0 tx=0x7f20b400eda0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 -- 172.21.15.19:0/1019518334 shutdown_connections 2024-04-23T18:26:06.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f20a40a3ca0 0x7f20a40a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8102040 0x7f20b81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b8102a00 0x7f20b819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 --2- 172.21.15.19:0/1019518334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20b8100e60 0x7f20b8198160 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 -- 172.21.15.19:0/1019518334 >> 172.21.15.19:0/1019518334 conn(0x7f20b80fc420 msgr2=0x7f20b80fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 -- 172.21.15.19:0/1019518334 shutdown_connections 2024-04-23T18:26:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:06.927+0000 7f20c014f640 1 -- 172.21.15.19:0/1019518334 wait complete. 2024-04-23T18:26:06.932 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-23T18:26:07.009 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:06 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1019518334' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:26:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.046+0000 7f0155c42640 1 -- 172.21.15.19:0/3067593398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0150107c90 msgr2=0x7f015010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.046+0000 7f0155c42640 1 --2- 172.21.15.19:0/3067593398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0150107c90 0x7f015010a0e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0148009ca0 tx=0x7f014802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.046+0000 7f0155c42640 1 -- 172.21.15.19:0/3067593398 shutdown_connections 2024-04-23T18:26:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.046+0000 7f0155c42640 1 --2- 172.21.15.19:0/3067593398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0150107c90 0x7f015010a0e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.046+0000 7f0155c42640 1 --2- 172.21.15.19:0/3067593398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0150105360 0x7f0150107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.046+0000 7f0155c42640 1 --2- 172.21.15.19:0/3067593398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0150102a30 0x7f0150104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.046+0000 7f0155c42640 1 -- 172.21.15.19:0/3067593398 >> 172.21.15.19:0/3067593398 conn(0x7f01500fc420 msgr2=0x7f01500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.046+0000 7f0155c42640 1 -- 172.21.15.19:0/3067593398 shutdown_connections 2024-04-23T18:26:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.046+0000 7f0155c42640 1 -- 172.21.15.19:0/3067593398 wait complete. 2024-04-23T18:26:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f0155c42640 1 Processor -- start 2024-04-23T18:26:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f0155c42640 1 -- start start 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f0155c42640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0150102a30 0x7f015019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f0155c42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0150105360 0x7f015019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f0155c42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0150107c90 0x7f01501a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f0155c42640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f015010daf0 con 0x7f0150107c90 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f0155c42640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f015010d970 con 0x7f0150102a30 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f0155c42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f015010dc70 con 0x7f0150105360 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f014ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0150107c90 0x7f01501a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f014ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0150107c90 0x7f01501a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40974/0 (socket says 172.21.15.19:40974) 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f014ffff640 1 -- 172.21.15.19:0/2894670854 learned_addr learned my addr 172.21.15.19:0/2894670854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f0146ffd640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0150105360 0x7f015019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.047+0000 7f014f7fe640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0150102a30 0x7f015019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f014ffff640 1 -- 172.21.15.19:0/2894670854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0150105360 msgr2=0x7f015019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f014ffff640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0150105360 0x7f015019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f014ffff640 1 -- 172.21.15.19:0/2894670854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0150102a30 msgr2=0x7f015019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f014ffff640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0150102a30 0x7f015019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f014ffff640 1 -- 172.21.15.19:0/2894670854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01501a1f20 con 0x7f0150107c90 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f0146ffd640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0150105360 0x7f015019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f014ffff640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0150107c90 0x7f01501a18b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f01480097b0 tx=0x7f0148031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148003b70 con 0x7f0150107c90 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0148003d10 con 0x7f0150107c90 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f0155c42640 1 -- 172.21.15.19:0/2894670854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01501a21b0 con 0x7f0150107c90 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014802fc10 con 0x7f0150107c90 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.048+0000 7f0155c42640 1 -- 172.21.15.19:0/2894670854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01501a2700 con 0x7f0150107c90 2024-04-23T18:26:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.049+0000 7f0155c42640 1 -- 172.21.15.19:0/2894670854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0150109780 con 0x7f0150107c90 2024-04-23T18:26:07.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.050+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0148004040 con 0x7f0150107c90 2024-04-23T18:26:07.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.051+0000 7f014d7fa640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f013c0a3ca0 0x7f013c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:07.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.051+0000 7f014f7fe640 1 -- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f013c0a3ca0 msgr2=0x7f013c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1993809272 2024-04-23T18:26:07.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.051+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f01480f1b70 con 0x7f0150107c90 2024-04-23T18:26:07.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.051+0000 7f014f7fe640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f013c0a3ca0 0x7f013c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.055+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01480be560 con 0x7f0150107c90 2024-04-23T18:26:07.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:06 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1019518334' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:26:07.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:06.438+0000 7f241224e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:07.072 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:06.745+0000 7f241224e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:26:07.229 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:06 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1019518334' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:26:07.230 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:06.789+0000 7f6baa684200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:26:07.230 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:06.914+0000 7f6baa684200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:07.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.252+0000 7f014f7fe640 1 -- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f013c0a3ca0 msgr2=0x7f013c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1993809272 2024-04-23T18:26:07.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.252+0000 7f014f7fe640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f013c0a3ca0 0x7f013c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.270+0000 7f0155c42640 1 -- 172.21.15.19:0/2894670854 --> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f0150062390 con 0x7f013c0a3ca0 2024-04-23T18:26:07.312 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:07.015+0000 7f6088804200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:26:07.510 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:07.236+0000 7f6baa684200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:26:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.653+0000 7f014f7fe640 1 -- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f013c0a3ca0 msgr2=0x7f013c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1993809272 2024-04-23T18:26:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.653+0000 7f014f7fe640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f013c0a3ca0 0x7f013c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:26:07.822 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:07.349+0000 7f241224e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:26:07.822 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z[35240]: 2024-04-23T18:26:07.565+0000 7f241224e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:26:07.924 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:07.647+0000 7f6088804200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:26:07.924 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y[33890]: 2024-04-23T18:26:07.871+0000 7f6088804200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:26:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:07.942+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f01480ba4d0 con 0x7f0150107c90 2024-04-23T18:26:08.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:08.127+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f014802fc10 con 0x7f0150107c90 2024-04-23T18:26:08.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:08.127+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f013c0a3ca0 msgr2=0x7f013c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:08.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:08.127+0000 7f014d7fa640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1993809272,v1:172.21.15.19:6801/1993809272] conn(0x7f013c0a3ca0 0x7f013c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:08.262 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:07.878+0000 7f6baa684200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:26:08.262 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.113+0000 7f6baa684200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:26:08.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1019518334' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: Standby manager daemon z restarted 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: Standby manager daemon z started 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: Standby manager daemon y restarted 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: Standby manager daemon y started 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:26:08.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:07 smithi019 ceph-mon[29215]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:08.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1019518334' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T18:26:08.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:26:08.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: Standby manager daemon z restarted 2024-04-23T18:26:08.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: Standby manager daemon z started 2024-04-23T18:26:08.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: Standby manager daemon y restarted 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: Standby manager daemon y started 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:07 smithi155 ceph-mon[33468]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1019518334' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T18:26:08.269 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: Standby manager daemon z restarted 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: Standby manager daemon z started 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.155:0/1972698103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: Standby manager daemon y restarted 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: Standby manager daemon y started 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:26:08.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:07 smithi044 ceph-mon[32458]: from='mgr.? 172.21.15.44:0/1258369270' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:08.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.381+0000 7f6b0cfbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:08.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.381+0000 7f6b0cfbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:08.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.381+0000 7f6b0cfbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:08.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.381+0000 7f6b0cfbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:08.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.381+0000 7f6b0cfbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:08.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.382+0000 7f6b09f78700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:08.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.382+0000 7f6b09f78700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:08.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.382+0000 7f6b09f78700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:08.760 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.382+0000 7f6b09f78700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:08.761 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:08.382+0000 7f6b09f78700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.132+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 <== mon.0 v2:172.21.15.19:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f01480ef6d0 con 0x7f0150107c90 2024-04-23T18:26:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.133+0000 7f014d7fa640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f013c0324b0 0x7f013c032890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.133+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 --> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f01480bcb50 con 0x7f013c0324b0 2024-04-23T18:26:09.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.135+0000 7f014f7fe640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f013c0324b0 0x7f013c032890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.141+0000 7f014f7fe640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f013c0324b0 0x7f013c032890 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f01400097f0 tx=0x7f0140005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:09.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.162+0000 7f014d7fa640 1 -- 172.21.15.19:0/2894670854 <== mgr.25004 v2:172.21.15.19:6800/1177101685 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f01480bcb50 con 0x7f013c0324b0 2024-04-23T18:26:09.154 INFO:teuthology.orchestra.run.smithi019.stdout:{} 2024-04-23T18:26:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.164+0000 7f0155c42640 1 -- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f013c0324b0 msgr2=0x7f013c032890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.164+0000 7f0155c42640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f013c0324b0 0x7f013c032890 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f01400097f0 tx=0x7f0140005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.164+0000 7f0155c42640 1 -- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0150107c90 msgr2=0x7f01501a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.164+0000 7f0155c42640 1 --2- 172.21.15.19:0/2894670854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0150107c90 0x7f01501a18b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f01480097b0 tx=0x7f0148031770 comp rx=0 tx=0).stop 2024-04-23T18:26:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.164+0000 7f014ffff640 1 -- 172.21.15.19:0/2894670854 reap_dead start 2024-04-23T18:26:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.164+0000 7f0155c42640 1 -- 172.21.15.19:0/2894670854 shutdown_connections 2024-04-23T18:26:09.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.164+0000 7f0155c42640 1 -- 172.21.15.19:0/2894670854 >> 172.21.15.19:0/2894670854 conn(0x7f01500fc420 msgr2=0x7f0150102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:09.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.164+0000 7f0155c42640 1 -- 172.21.15.19:0/2894670854 shutdown_connections 2024-04-23T18:26:09.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.164+0000 7f0155c42640 1 -- 172.21.15.19:0/2894670854 wait complete. 2024-04-23T18:26:09.169 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-23T18:26:09.180 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T18:26:09.189 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:26:09.189 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:26:09.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-23T18:26:09.210 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:09.149+0000 7f6b06f72700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:09.210 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:09.149+0000 7f6b06f72700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:09.210 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:09.149+0000 7f6b06f72700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:09.210 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:09.149+0000 7f6b06f72700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:09.211 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:26:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:26:09.149+0000 7f6b06f72700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:09.211 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-23T18:26:09.211 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: Active manager daemon x restarted 2024-04-23T18:26:09.211 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: Activating manager daemon x 2024-04-23T18:26:09.211 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:26:09.211 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: mgrmap e21: x(active, starting, since 0.0124466s), standbys: y, z 2024-04-23T18:26:09.211 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:26:09.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:26:09.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:26:09.214 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:26:09.214 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: Manager daemon x is now available 2024-04-23T18:26:09.214 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:09.214 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:26:09.214 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:08 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 -- 172.21.15.19:0/1661059336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f14102a00 msgr2=0x7f7f14109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 --2- 172.21.15.19:0/1661059336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f14102a00 0x7f7f14109320 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009ca0 tx=0x7f7f0c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 -- 172.21.15.19:0/1661059336 shutdown_connections 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 --2- 172.21.15.19:0/1661059336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f14102a00 0x7f7f14109320 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 --2- 172.21.15.19:0/1661059336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14102040 0x7f7f141024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 --2- 172.21.15.19:0/1661059336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f14100e60 0x7f7f14101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 -- 172.21.15.19:0/1661059336 >> 172.21.15.19:0/1661059336 conn(0x7f7f140fc440 msgr2=0x7f7f140fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 -- 172.21.15.19:0/1661059336 shutdown_connections 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 -- 172.21.15.19:0/1661059336 wait complete. 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 Processor -- start 2024-04-23T18:26:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.302+0000 7f7f19a2a640 1 -- start start 2024-04-23T18:26:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f19a2a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f14100e60 0x7f7f14196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f19a2a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14102040 0x7f7f141974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f19a2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f14102a00 0x7f7f1419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f19a2a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1410ca00 con 0x7f7f14102a00 2024-04-23T18:26:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f19a2a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1410c880 con 0x7f7f14100e60 2024-04-23T18:26:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f19a2a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1410cb80 con 0x7f7f14102040 2024-04-23T18:26:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f14102a00 0x7f7f1419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f14102a00 0x7f7f1419e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41148/0 (socket says 172.21.15.19:41148) 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f137fe640 1 -- 172.21.15.19:0/2688944412 learned_addr learned my addr 172.21.15.19:0/2688944412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f127fc640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14102040 0x7f7f141974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f12ffd640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f14100e60 0x7f7f14196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f137fe640 1 -- 172.21.15.19:0/2688944412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14102040 msgr2=0x7f7f141974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f137fe640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14102040 0x7f7f141974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f137fe640 1 -- 172.21.15.19:0/2688944412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f14100e60 msgr2=0x7f7f14196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f137fe640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f14100e60 0x7f7f14196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f137fe640 1 -- 172.21.15.19:0/2688944412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1419ebd0 con 0x7f7f14102a00 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f137fe640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f14102a00 0x7f7f1419e560 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0097b0 tx=0x7f7f0c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f18a28640 1 -- 172.21.15.19:0/2688944412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c003bf0 con 0x7f7f14102a00 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f19a2a640 1 -- 172.21.15.19:0/2688944412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1419ee60 con 0x7f7f14102a00 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f18a28640 1 -- 172.21.15.19:0/2688944412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c003d90 con 0x7f7f14102a00 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f18a28640 1 -- 172.21.15.19:0/2688944412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c02fc60 con 0x7f7f14102a00 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.303+0000 7f7f19a2a640 1 -- 172.21.15.19:0/2688944412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1419f370 con 0x7f7f14102a00 2024-04-23T18:26:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.304+0000 7f7f19a2a640 1 -- 172.21.15.19:0/2688944412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f14108130 con 0x7f7f14102a00 2024-04-23T18:26:09.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.306+0000 7f7f18a28640 1 -- 172.21.15.19:0/2688944412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f7f0c0046a0 con 0x7f7f14102a00 2024-04-23T18:26:09.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.307+0000 7f7f18a28640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7f000a3a60 0x7f7f000a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.307+0000 7f7f12ffd640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7f000a3a60 0x7f7f000a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.307+0000 7f7f18a28640 1 -- 172.21.15.19:0/2688944412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7f0c0f1a70 con 0x7f7f14102a00 2024-04-23T18:26:09.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.307+0000 7f7f12ffd640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7f000a3a60 0x7f7f000a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f04005fd0 tx=0x7f7f04005850 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.311+0000 7f7f18a28640 1 -- 172.21.15.19:0/2688944412 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f0c0be650 con 0x7f7f14102a00 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: Active manager daemon x restarted 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: Activating manager daemon x 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: mgrmap e21: x(active, starting, since 0.0124466s), standbys: y, z 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:26:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:26:09.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: Manager daemon x is now available 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:26:09.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:08 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:26:09.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-23T18:26:09.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: Active manager daemon x restarted 2024-04-23T18:26:09.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: Activating manager daemon x 2024-04-23T18:26:09.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:26:09.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: mgrmap e21: x(active, starting, since 0.0124466s), standbys: y, z 2024-04-23T18:26:09.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:26:09.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:26:09.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:26:09.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:26:09.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:26:09.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: Manager daemon x is now available 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:26:09.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:08 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:26:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.522+0000 7f7f19a2a640 1 -- 172.21.15.19:0/2688944412 --> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7f14062390 con 0x7f7f000a3a60 2024-04-23T18:26:09.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.590+0000 7f7f18a28640 1 -- 172.21.15.19:0/2688944412 <== mgr.25004 v2:172.21.15.19:6800/1177101685 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7f14062390 con 0x7f7f000a3a60 2024-04-23T18:26:09.583 INFO:teuthology.orchestra.run.smithi019.stdout:Schedule set for path / 2024-04-23T18:26:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 -- 172.21.15.19:0/2688944412 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7f000a3a60 msgr2=0x7f7f000a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7f000a3a60 0x7f7f000a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f04005fd0 tx=0x7f7f04005850 comp rx=0 tx=0).stop 2024-04-23T18:26:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 -- 172.21.15.19:0/2688944412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f14102a00 msgr2=0x7f7f1419e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f14102a00 0x7f7f1419e560 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0097b0 tx=0x7f7f0c031840 comp rx=0 tx=0).stop 2024-04-23T18:26:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 -- 172.21.15.19:0/2688944412 shutdown_connections 2024-04-23T18:26:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7f000a3a60 0x7f7f000a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f14102a00 0x7f7f1419e560 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14102040 0x7f7f141974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 --2- 172.21.15.19:0/2688944412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f14100e60 0x7f7f14196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 -- 172.21.15.19:0/2688944412 >> 172.21.15.19:0/2688944412 conn(0x7f7f140fc440 msgr2=0x7f7f14105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:09.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 -- 172.21.15.19:0/2688944412 shutdown_connections 2024-04-23T18:26:09.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.592+0000 7f7f19a2a640 1 -- 172.21.15.19:0/2688944412 wait complete. 2024-04-23T18:26:09.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-23T18:26:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.750+0000 7f3969a49640 1 -- 172.21.15.19:0/2539352622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3964101fb0 msgr2=0x7f39641023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.750+0000 7f3969a49640 1 --2- 172.21.15.19:0/2539352622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3964101fb0 0x7f39641023d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3954009900 tx=0x7f395402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.750+0000 7f3969a49640 1 -- 172.21.15.19:0/2539352622 shutdown_connections 2024-04-23T18:26:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.750+0000 7f3969a49640 1 --2- 172.21.15.19:0/2539352622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39641031a0 0x7f3964103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 --2- 172.21.15.19:0/2539352622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3964103b60 0x7f396410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 --2- 172.21.15.19:0/2539352622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3964101fb0 0x7f39641023d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 -- 172.21.15.19:0/2539352622 >> 172.21.15.19:0/2539352622 conn(0x7f39640fd540 msgr2=0x7f39640ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 -- 172.21.15.19:0/2539352622 shutdown_connections 2024-04-23T18:26:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 -- 172.21.15.19:0/2539352622 wait complete. 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 Processor -- start 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 -- start start 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3964101fb0 0x7f3964198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39641031a0 0x7f3964198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3964103b60 0x7f396419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f396410daf0 con 0x7f39641031a0 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f396410d970 con 0x7f3964101fb0 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.751+0000 7f3969a49640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f396410dc70 con 0x7f3964103b60 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f39627fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39641031a0 0x7f3964198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f39627fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39641031a0 0x7f3964198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53196/0 (socket says 172.21.15.19:53196) 2024-04-23T18:26:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f3962ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3964101fb0 0x7f3964198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f39627fc640 1 -- 172.21.15.19:0/194757728 learned_addr learned my addr 172.21.15.19:0/194757728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f39637fe640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3964103b60 0x7f396419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f39627fc640 1 -- 172.21.15.19:0/194757728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3964103b60 msgr2=0x7f396419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f39627fc640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3964103b60 0x7f396419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f39627fc640 1 -- 172.21.15.19:0/194757728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3964101fb0 msgr2=0x7f3964198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f39627fc640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3964101fb0 0x7f3964198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f39627fc640 1 -- 172.21.15.19:0/194757728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396419fd30 con 0x7f39641031a0 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f39627fc640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39641031a0 0x7f3964198650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f395800d950 tx=0x7f395800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f3968a47640 1 -- 172.21.15.19:0/194757728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958017030 con 0x7f39641031a0 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f3969a49640 1 -- 172.21.15.19:0/194757728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396419ffc0 con 0x7f39641031a0 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f3968a47640 1 -- 172.21.15.19:0/194757728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3958004020 con 0x7f39641031a0 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f3968a47640 1 -- 172.21.15.19:0/194757728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580047a0 con 0x7f39641031a0 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.752+0000 7f3969a49640 1 -- 172.21.15.19:0/194757728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39641a0530 con 0x7f39641031a0 2024-04-23T18:26:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.753+0000 7f3969a49640 1 -- 172.21.15.19:0/194757728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39641092d0 con 0x7f39641031a0 2024-04-23T18:26:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.754+0000 7f3968a47640 1 -- 172.21.15.19:0/194757728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f3958009a30 con 0x7f39641031a0 2024-04-23T18:26:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.755+0000 7f3968a47640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f394c0a3a60 0x7f394c0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.755+0000 7f3962ffd640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f394c0a3a60 0x7f394c0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.755+0000 7f3968a47640 1 -- 172.21.15.19:0/194757728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f39580d0190 con 0x7f39641031a0 2024-04-23T18:26:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.756+0000 7f3962ffd640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f394c0a3a60 0x7f394c0a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3954009230 tx=0x7f39540314a0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:09.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.759+0000 7f3968a47640 1 -- 172.21.15.19:0/194757728 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f395809cd30 con 0x7f39641031a0 2024-04-23T18:26:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:09.972+0000 7f3969a49640 1 -- 172.21.15.19:0/194757728 --> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f3964062430 con 0x7f394c0a3a60 2024-04-23T18:26:10.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.034+0000 7f3968a47640 1 -- 172.21.15.19:0/194757728 <== mgr.25004 v2:172.21.15.19:6800/1177101685 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f3964062430 con 0x7f394c0a3a60 2024-04-23T18:26:10.027 INFO:teuthology.orchestra.run.smithi019.stdout:Retention added to path / 2024-04-23T18:26:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 -- 172.21.15.19:0/194757728 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f394c0a3a60 msgr2=0x7f394c0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f394c0a3a60 0x7f394c0a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3954009230 tx=0x7f39540314a0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 -- 172.21.15.19:0/194757728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39641031a0 msgr2=0x7f3964198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39641031a0 0x7f3964198650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f395800d950 tx=0x7f395800de10 comp rx=0 tx=0).stop 2024-04-23T18:26:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 -- 172.21.15.19:0/194757728 shutdown_connections 2024-04-23T18:26:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f394c0a3a60 0x7f394c0a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3964103b60 0x7f396419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39641031a0 0x7f3964198650 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 --2- 172.21.15.19:0/194757728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3964101fb0 0x7f3964198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 -- 172.21.15.19:0/194757728 >> 172.21.15.19:0/194757728 conn(0x7f39640fd540 msgr2=0x7f39641064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 -- 172.21.15.19:0/194757728 shutdown_connections 2024-04-23T18:26:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.036+0000 7f3969a49640 1 -- 172.21.15.19:0/194757728 wait complete. 2024-04-23T18:26:10.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T18:26:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.166+0000 7f2a47cbb640 1 -- 172.21.15.19:0/3248738909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a40105410 msgr2=0x7f2a40107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.166+0000 7f2a47cbb640 1 --2- 172.21.15.19:0/3248738909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a40105410 0x7f2a40107800 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009900 tx=0x7f2a3002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.166+0000 7f2a47cbb640 1 -- 172.21.15.19:0/3248738909 shutdown_connections 2024-04-23T18:26:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.166+0000 7f2a47cbb640 1 --2- 172.21.15.19:0/3248738909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40107d40 0x7f2a4010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.166+0000 7f2a47cbb640 1 --2- 172.21.15.19:0/3248738909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a40105410 0x7f2a40107800 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.166+0000 7f2a47cbb640 1 --2- 172.21.15.19:0/3248738909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a40102ae0 0x7f2a40104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.166+0000 7f2a47cbb640 1 -- 172.21.15.19:0/3248738909 >> 172.21.15.19:0/3248738909 conn(0x7f2a400fc4d0 msgr2=0x7f2a400fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.166+0000 7f2a47cbb640 1 -- 172.21.15.19:0/3248738909 shutdown_connections 2024-04-23T18:26:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.166+0000 7f2a47cbb640 1 -- 172.21.15.19:0/3248738909 wait complete. 2024-04-23T18:26:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.166+0000 7f2a47cbb640 1 Processor -- start 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a47cbb640 1 -- start start 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a47cbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a40102ae0 0x7f2a40102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a47cbb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a40105410 0x7f2a40100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a47cbb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40107d40 0x7f2a40100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a47cbb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010daf0 con 0x7f2a40102ae0 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a47cbb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010d970 con 0x7f2a40107d40 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a47cbb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010dc70 con 0x7f2a40105410 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a45a57640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a40102ae0 0x7f2a40102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a45a57640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a40102ae0 0x7f2a40102450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53210/0 (socket says 172.21.15.19:53210) 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a45a57640 1 -- 172.21.15.19:0/1267955900 learned_addr learned my addr 172.21.15.19:0/1267955900 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a45256640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a40105410 0x7f2a40100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a46258640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40107d40 0x7f2a40100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a45a57640 1 -- 172.21.15.19:0/1267955900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a40105410 msgr2=0x7f2a40100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a45a57640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a40105410 0x7f2a40100a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a45a57640 1 -- 172.21.15.19:0/1267955900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40107d40 msgr2=0x7f2a40100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a45a57640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40107d40 0x7f2a40100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a45a57640 1 -- 172.21.15.19:0/1267955900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a40101860 con 0x7f2a40102ae0 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a46258640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40107d40 0x7f2a40100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a45a57640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a40102ae0 0x7f2a40102450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e9f0 tx=0x7f2a3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a36ffd640 1 -- 172.21.15.19:0/1267955900 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c00cd80 con 0x7f2a40102ae0 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.167+0000 7f2a36ffd640 1 -- 172.21.15.19:0/1267955900 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a3c0108e0 con 0x7f2a40102ae0 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.168+0000 7f2a47cbb640 1 -- 172.21.15.19:0/1267955900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a401a2260 con 0x7f2a40102ae0 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.168+0000 7f2a36ffd640 1 -- 172.21.15.19:0/1267955900 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c018980 con 0x7f2a40102ae0 2024-04-23T18:26:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.168+0000 7f2a47cbb640 1 -- 172.21.15.19:0/1267955900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a401a2780 con 0x7f2a40102ae0 2024-04-23T18:26:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.169+0000 7f2a47cbb640 1 -- 172.21.15.19:0/1267955900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a400fdfc0 con 0x7f2a40102ae0 2024-04-23T18:26:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.171+0000 7f2a36ffd640 1 -- 172.21.15.19:0/1267955900 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f2a3c010430 con 0x7f2a40102ae0 2024-04-23T18:26:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.171+0000 7f2a36ffd640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2a2c0a3e60 0x7f2a2c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.171+0000 7f2a36ffd640 1 -- 172.21.15.19:0/1267955900 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2a3c0d11c0 con 0x7f2a40102ae0 2024-04-23T18:26:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.172+0000 7f2a45256640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2a2c0a3e60 0x7f2a2c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.172+0000 7f2a45256640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2a2c0a3e60 0x7f2a2c0a6320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009230 tx=0x7f2a300314a0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.175+0000 7f2a36ffd640 1 -- 172.21.15.19:0/1267955900 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a3c09d720 con 0x7f2a40102ae0 2024-04-23T18:26:10.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.420+0000 7f2a47cbb640 1 -- 172.21.15.19:0/1267955900 --> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2a40062390 con 0x7f2a2c0a3e60 2024-04-23T18:26:10.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.428+0000 7f2a36ffd640 1 -- 172.21.15.19:0/1267955900 <== mgr.25004 v2:172.21.15.19:6800/1177101685 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2a40062390 con 0x7f2a2c0a3e60 2024-04-23T18:26:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T18:26:09", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 -- 172.21.15.19:0/1267955900 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2a2c0a3e60 msgr2=0x7f2a2c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2a2c0a3e60 0x7f2a2c0a6320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009230 tx=0x7f2a300314a0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 -- 172.21.15.19:0/1267955900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a40102ae0 msgr2=0x7f2a40102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a40102ae0 0x7f2a40102450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e9f0 tx=0x7f2a3c00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 -- 172.21.15.19:0/1267955900 shutdown_connections 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2a2c0a3e60 0x7f2a2c0a6320 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40107d40 0x7f2a40100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a40105410 0x7f2a40100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 --2- 172.21.15.19:0/1267955900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a40102ae0 0x7f2a40102450 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 -- 172.21.15.19:0/1267955900 >> 172.21.15.19:0/1267955900 conn(0x7f2a400fc4d0 msgr2=0x7f2a401034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 -- 172.21.15.19:0/1267955900 shutdown_connections 2024-04-23T18:26:10.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.430+0000 7f2a47cbb640 1 -- 172.21.15.19:0/1267955900 wait complete. 2024-04-23T18:26:10.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:10 smithi019 ceph-mon[29215]: mgrmap e22: x(active, since 1.02628s), standbys: y, z 2024-04-23T18:26:10.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:10 smithi019 ceph-mon[29215]: [23/Apr/2024:18:26:09] ENGINE Bus STARTING 2024-04-23T18:26:10.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:10 smithi019 ceph-mon[29215]: [23/Apr/2024:18:26:09] ENGINE Serving on https://172.21.15.19:7150 2024-04-23T18:26:10.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:10 smithi019 ceph-mon[29215]: [23/Apr/2024:18:26:09] ENGINE Serving on http://172.21.15.19:8765 2024-04-23T18:26:10.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:10 smithi019 ceph-mon[29215]: [23/Apr/2024:18:26:09] ENGINE Bus STARTED 2024-04-23T18:26:10.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:10 smithi019 ceph-mon[29215]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:10.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T18:26:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.548+0000 7fa0498ea640 1 -- 172.21.15.19:0/3304274235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa044102060 msgr2=0x7fa0441024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.548+0000 7fa0498ea640 1 --2- 172.21.15.19:0/3304274235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa044102060 0x7fa0441024e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa038009900 tx=0x7fa03802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.548+0000 7fa0498ea640 1 -- 172.21.15.19:0/3304274235 shutdown_connections 2024-04-23T18:26:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.548+0000 7fa0498ea640 1 --2- 172.21.15.19:0/3304274235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa044102a20 0x7fa044109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.548+0000 7fa0498ea640 1 --2- 172.21.15.19:0/3304274235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa044102060 0x7fa0441024e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.548+0000 7fa0498ea640 1 --2- 172.21.15.19:0/3304274235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa044100e80 0x7fa0441012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.548+0000 7fa0498ea640 1 -- 172.21.15.19:0/3304274235 >> 172.21.15.19:0/3304274235 conn(0x7fa0440fc460 msgr2=0x7fa0440fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.548+0000 7fa0498ea640 1 -- 172.21.15.19:0/3304274235 shutdown_connections 2024-04-23T18:26:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.548+0000 7fa0498ea640 1 -- 172.21.15.19:0/3304274235 wait complete. 2024-04-23T18:26:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0498ea640 1 Processor -- start 2024-04-23T18:26:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0498ea640 1 -- start start 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0498ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa044100e80 0x7fa044198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0498ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa044102060 0x7fa0441986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0498ea640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa044102a20 0x7fa04419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0498ea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa04410db70 con 0x7fa044102060 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0498ea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa04410d9f0 con 0x7fa044100e80 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0498ea640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa04410dcf0 con 0x7fa044102a20 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0427fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa044102060 0x7fa0441986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0427fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa044102060 0x7fa0441986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53246/0 (socket says 172.21.15.19:53246) 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0427fc640 1 -- 172.21.15.19:0/1339690720 learned_addr learned my addr 172.21.15.19:0/1339690720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa042ffd640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa044100e80 0x7fa044198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.549+0000 7fa0437fe640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa044102a20 0x7fa04419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0427fc640 1 -- 172.21.15.19:0/1339690720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa044102a20 msgr2=0x7fa04419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0427fc640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa044102a20 0x7fa04419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0427fc640 1 -- 172.21.15.19:0/1339690720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa044100e80 msgr2=0x7fa044198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0427fc640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa044100e80 0x7fa044198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0427fc640 1 -- 172.21.15.19:0/1339690720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04419fdb0 con 0x7fa044102060 2024-04-23T18:26:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0427fc640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa044102060 0x7fa0441986d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa038005ab0 tx=0x7fa038003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0488e8640 1 -- 172.21.15.19:0/1339690720 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03803f070 con 0x7fa044102060 2024-04-23T18:26:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0488e8640 1 -- 172.21.15.19:0/1339690720 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa03803b460 con 0x7fa044102060 2024-04-23T18:26:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0498ea640 1 -- 172.21.15.19:0/1339690720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0441a0040 con 0x7fa044102060 2024-04-23T18:26:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0488e8640 1 -- 172.21.15.19:0/1339690720 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038035030 con 0x7fa044102060 2024-04-23T18:26:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.550+0000 7fa0498ea640 1 -- 172.21.15.19:0/1339690720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0441a0530 con 0x7fa044102060 2024-04-23T18:26:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.551+0000 7fa0498ea640 1 -- 172.21.15.19:0/1339690720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa044108220 con 0x7fa044102060 2024-04-23T18:26:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.552+0000 7fa0488e8640 1 -- 172.21.15.19:0/1339690720 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fa038042070 con 0x7fa044102060 2024-04-23T18:26:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.553+0000 7fa0488e8640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa02c0a3e60 0x7fa02c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.553+0000 7fa042ffd640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa02c0a3e60 0x7fa02c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.553+0000 7fa0488e8640 1 -- 172.21.15.19:0/1339690720 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa0380f1230 con 0x7fa044102060 2024-04-23T18:26:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.553+0000 7fa042ffd640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa02c0a3e60 0x7fa02c0a6320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa044101bd0 tx=0x7fa03400b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.558+0000 7fa0488e8640 1 -- 172.21.15.19:0/1339690720 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0380bd0e0 con 0x7fa044102060 2024-04-23T18:26:10.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:10 smithi044 ceph-mon[32458]: mgrmap e22: x(active, since 1.02628s), standbys: y, z 2024-04-23T18:26:10.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:10 smithi044 ceph-mon[32458]: [23/Apr/2024:18:26:09] ENGINE Bus STARTING 2024-04-23T18:26:10.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:10 smithi044 ceph-mon[32458]: [23/Apr/2024:18:26:09] ENGINE Serving on https://172.21.15.19:7150 2024-04-23T18:26:10.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:10 smithi044 ceph-mon[32458]: [23/Apr/2024:18:26:09] ENGINE Serving on http://172.21.15.19:8765 2024-04-23T18:26:10.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:10 smithi044 ceph-mon[32458]: [23/Apr/2024:18:26:09] ENGINE Bus STARTED 2024-04-23T18:26:10.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:10 smithi044 ceph-mon[32458]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:10.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:10 smithi155 ceph-mon[33468]: mgrmap e22: x(active, since 1.02628s), standbys: y, z 2024-04-23T18:26:10.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:10 smithi155 ceph-mon[33468]: [23/Apr/2024:18:26:09] ENGINE Bus STARTING 2024-04-23T18:26:10.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:10 smithi155 ceph-mon[33468]: [23/Apr/2024:18:26:09] ENGINE Serving on https://172.21.15.19:7150 2024-04-23T18:26:10.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:10 smithi155 ceph-mon[33468]: [23/Apr/2024:18:26:09] ENGINE Serving on http://172.21.15.19:8765 2024-04-23T18:26:10.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:10 smithi155 ceph-mon[33468]: [23/Apr/2024:18:26:09] ENGINE Bus STARTED 2024-04-23T18:26:10.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:10 smithi155 ceph-mon[33468]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.811+0000 7fa0498ea640 1 -- 172.21.15.19:0/1339690720 --> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa044062390 con 0x7fa02c0a3e60 2024-04-23T18:26:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.835+0000 7fa0488e8640 1 -- 172.21.15.19:0/1339690720 <== mgr.25004 v2:172.21.15.19:6800/1177101685 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa044062390 con 0x7fa02c0a3e60 2024-04-23T18:26:10.828 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-04-23T18:26:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 -- 172.21.15.19:0/1339690720 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa02c0a3e60 msgr2=0x7fa02c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa02c0a3e60 0x7fa02c0a6320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa044101bd0 tx=0x7fa03400b390 comp rx=0 tx=0).stop 2024-04-23T18:26:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 -- 172.21.15.19:0/1339690720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa044102060 msgr2=0x7fa0441986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa044102060 0x7fa0441986d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa038005ab0 tx=0x7fa038003cd0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 -- 172.21.15.19:0/1339690720 shutdown_connections 2024-04-23T18:26:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa02c0a3e60 0x7fa02c0a6320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa044102a20 0x7fa04419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa044102060 0x7fa0441986d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 --2- 172.21.15.19:0/1339690720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa044100e80 0x7fa044198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 -- 172.21.15.19:0/1339690720 >> 172.21.15.19:0/1339690720 conn(0x7fa0440fc460 msgr2=0x7fa0441053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 -- 172.21.15.19:0/1339690720 shutdown_connections 2024-04-23T18:26:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:10.837+0000 7fa0498ea640 1 -- 172.21.15.19:0/1339690720 wait complete. 2024-04-23T18:26:10.841 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-23T18:26:10.867 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-23T18:26:10.879 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-23T18:26:10.894 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:26:10.894 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-23T18:26:10.909 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-23T18:26:10.921 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:11.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.039+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2327954433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b68101fe0 msgr2=0x7f4b68102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.039+0000 7f4b6d00b640 1 --2- 172.21.15.19:0/2327954433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b68101fe0 0x7f4b68102460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009860 tx=0x7f4b5802a340 comp rx=0 tx=0).stop 2024-04-23T18:26:11.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.039+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2327954433 shutdown_connections 2024-04-23T18:26:11.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.039+0000 7f4b6d00b640 1 --2- 172.21.15.19:0/2327954433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b68101fe0 0x7f4b68102460 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.039+0000 7f4b6d00b640 1 --2- 172.21.15.19:0/2327954433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b681029a0 0x7f4b681092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.039+0000 7f4b6d00b640 1 --2- 172.21.15.19:0/2327954433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b68100e90 0x7f4b681012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.039+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2327954433 >> 172.21.15.19:0/2327954433 conn(0x7f4b680fc400 msgr2=0x7f4b680fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.039+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2327954433 shutdown_connections 2024-04-23T18:26:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.039+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2327954433 wait complete. 2024-04-23T18:26:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.040+0000 7f4b6d00b640 1 Processor -- start 2024-04-23T18:26:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.040+0000 7f4b6d00b640 1 -- start start 2024-04-23T18:26:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6d00b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b68100e90 0x7f4b6810b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6d00b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b68101fe0 0x7f4b68109e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6d00b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b681029a0 0x7f4b6810a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6d00b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6810daf0 con 0x7f4b681029a0 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6d00b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6810d970 con 0x7f4b68101fe0 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6d00b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6810dc70 con 0x7f4b68100e90 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b681029a0 0x7f4b6810a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b681029a0 0x7f4b6810a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53264/0 (socket says 172.21.15.19:53264) 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6759e640 1 -- 172.21.15.19:0/2390328909 learned_addr learned my addr 172.21.15.19:0/2390328909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6659c640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b68101fe0 0x7f4b68109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b66d9d640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b68100e90 0x7f4b6810b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6759e640 1 -- 172.21.15.19:0/2390328909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b68100e90 msgr2=0x7f4b6810b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6759e640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b68100e90 0x7f4b6810b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6759e640 1 -- 172.21.15.19:0/2390328909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b68101fe0 msgr2=0x7f4b68109e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6759e640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b68101fe0 0x7f4b68109e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6759e640 1 -- 172.21.15.19:0/2390328909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6810ac40 con 0x7f4b681029a0 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b6759e640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b681029a0 0x7f4b6810a3b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000efc0 tx=0x7f4b6000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.041+0000 7f4b4ffff640 1 -- 172.21.15.19:0/2390328909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60003970 con 0x7f4b681029a0 2024-04-23T18:26:11.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.042+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2390328909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b681a2340 con 0x7f4b681029a0 2024-04-23T18:26:11.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.042+0000 7f4b4ffff640 1 -- 172.21.15.19:0/2390328909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b60004020 con 0x7f4b681029a0 2024-04-23T18:26:11.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.042+0000 7f4b4ffff640 1 -- 172.21.15.19:0/2390328909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000b920 con 0x7f4b681029a0 2024-04-23T18:26:11.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.042+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2390328909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b681a2860 con 0x7f4b681029a0 2024-04-23T18:26:11.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.043+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2390328909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b68108270 con 0x7f4b681029a0 2024-04-23T18:26:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.044+0000 7f4b4ffff640 1 -- 172.21.15.19:0/2390328909 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f4b60003b10 con 0x7f4b681029a0 2024-04-23T18:26:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.045+0000 7f4b4ffff640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b500a3db0 0x7f4b500a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.045+0000 7f4b66d9d640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b500a3db0 0x7f4b500a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.045+0000 7f4b4ffff640 1 -- 172.21.15.19:0/2390328909 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f4b600d12c0 con 0x7f4b681029a0 2024-04-23T18:26:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.045+0000 7f4b66d9d640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b500a3db0 0x7f4b500a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0095f0 tx=0x7f4b5c005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:11.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.049+0000 7f4b4ffff640 1 -- 172.21.15.19:0/2390328909 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b60024070 con 0x7f4b681029a0 2024-04-23T18:26:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.243+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2390328909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b6802d810 con 0x7f4b681029a0 2024-04-23T18:26:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.246+0000 7f4b4ffff640 1 -- 172.21.15.19:0/2390328909 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23018 (secure 0 0 0) 0x7f4b6009d820 con 0x7f4b681029a0 2024-04-23T18:26:11.239 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:26:08.116951+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/304874449":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1108384152":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6801/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6800/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/983971095":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2556512605":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2390328909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b500a3db0 msgr2=0x7f4b500a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b500a3db0 0x7f4b500a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0095f0 tx=0x7f4b5c005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2390328909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b681029a0 msgr2=0x7f4b6810a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b681029a0 0x7f4b6810a3b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000efc0 tx=0x7f4b6000c800 comp rx=0 tx=0).stop 2024-04-23T18:26:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2390328909 shutdown_connections 2024-04-23T18:26:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b500a3db0 0x7f4b500a6270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b68101fe0 0x7f4b68109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b681029a0 0x7f4b6810a3b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 --2- 172.21.15.19:0/2390328909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b68100e90 0x7f4b6810b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2390328909 >> 172.21.15.19:0/2390328909 conn(0x7f4b680fc400 msgr2=0x7f4b68105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2390328909 shutdown_connections 2024-04-23T18:26:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.248+0000 7f4b6d00b640 1 -- 172.21.15.19:0/2390328909 wait complete. 2024-04-23T18:26:11.253 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:23:53.296504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:23:54.573484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:26:11.254 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:26:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.368+0000 7fc7bb691640 1 -- 172.21.15.19:0/2432908041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b4105400 msgr2=0x7fc7b41077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.368+0000 7fc7bb691640 1 --2- 172.21.15.19:0/2432908041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b4105400 0x7fc7b41077f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009900 tx=0x7fc7b002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.368+0000 7fc7bb691640 1 -- 172.21.15.19:0/2432908041 shutdown_connections 2024-04-23T18:26:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.368+0000 7fc7bb691640 1 --2- 172.21.15.19:0/2432908041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b4107d30 0x7fc7b410a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.368+0000 7fc7bb691640 1 --2- 172.21.15.19:0/2432908041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b4105400 0x7fc7b41077f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.368+0000 7fc7bb691640 1 --2- 172.21.15.19:0/2432908041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b4102ad0 0x7fc7b4104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.368+0000 7fc7bb691640 1 -- 172.21.15.19:0/2432908041 >> 172.21.15.19:0/2432908041 conn(0x7fc7b40fc400 msgr2=0x7fc7b40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.368+0000 7fc7bb691640 1 -- 172.21.15.19:0/2432908041 shutdown_connections 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.368+0000 7fc7bb691640 1 -- 172.21.15.19:0/2432908041 wait complete. 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7bb691640 1 Processor -- start 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7bb691640 1 -- start start 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7bb691640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b4102ad0 0x7fc7b4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7bb691640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b4105400 0x7fc7b4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7bb691640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b4107d30 0x7fc7b419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7bb691640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b410daa0 con 0x7fc7b4105400 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7bb691640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b410d920 con 0x7fc7b4102ad0 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7bb691640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b410dc20 con 0x7fc7b4107d30 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7b8c2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b4105400 0x7fc7b4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7b8c2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b4105400 0x7fc7b4198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53270/0 (socket says 172.21.15.19:53270) 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7b8c2c640 1 -- 172.21.15.19:0/1984008909 learned_addr learned my addr 172.21.15.19:0/1984008909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7b942d640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b4102ad0 0x7fc7b4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7b9c2e640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b4107d30 0x7fc7b419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.369+0000 7fc7b8c2c640 1 -- 172.21.15.19:0/1984008909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b4107d30 msgr2=0x7fc7b419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.370+0000 7fc7b8c2c640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b4107d30 0x7fc7b419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.370+0000 7fc7b8c2c640 1 -- 172.21.15.19:0/1984008909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b4102ad0 msgr2=0x7fc7b4198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.370+0000 7fc7b8c2c640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b4102ad0 0x7fc7b4198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.370+0000 7fc7b8c2c640 1 -- 172.21.15.19:0/1984008909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b419fe50 con 0x7fc7b4105400 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.370+0000 7fc7b8c2c640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b4105400 0x7fc7b4198770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0006010 tx=0x7fc7b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.370+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1984008909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b003f030 con 0x7fc7b4105400 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.370+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1984008909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b00318f0 con 0x7fc7b4105400 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.370+0000 7fc7bb691640 1 -- 172.21.15.19:0/1984008909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b41a00e0 con 0x7fc7b4105400 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.370+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1984008909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0035070 con 0x7fc7b4105400 2024-04-23T18:26:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.370+0000 7fc7bb691640 1 -- 172.21.15.19:0/1984008909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b41a0620 con 0x7fc7b4105400 2024-04-23T18:26:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.371+0000 7fc7bb691640 1 -- 172.21.15.19:0/1984008909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b40fdef0 con 0x7fc7b4105400 2024-04-23T18:26:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.372+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1984008909 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fc7b0042030 con 0x7fc7b4105400 2024-04-23T18:26:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.373+0000 7fc7aa7fc640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7a00a3e30 0x7fc7a00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.373+0000 7fc7b942d640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7a00a3e30 0x7fc7a00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.373+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1984008909 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc7b00f0ea0 con 0x7fc7b4105400 2024-04-23T18:26:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.373+0000 7fc7b942d640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7a00a3e30 0x7fc7a00a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4005d00 tx=0x7fc7a400b410 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.377+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1984008909 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7b00bd400 con 0x7fc7b4105400 2024-04-23T18:26:11.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:11 smithi019 ceph-mon[29215]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:11.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:11 smithi019 ceph-mon[29215]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:11.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:11 smithi019 ceph-mon[29215]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-23T18:26:11.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:11 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:11 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:11 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:11 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:11 smithi019 ceph-mon[29215]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:11.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:11 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:11 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.568+0000 7fc7bb691640 1 -- 172.21.15.19:0/1984008909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7b402d770 con 0x7fc7b4105400 2024-04-23T18:26:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.568+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1984008909 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fc7b00bcd00 con 0x7fc7b4105400 2024-04-23T18:26:11.561 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:26:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:11 smithi044 ceph-mon[32458]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:11 smithi044 ceph-mon[32458]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:11 smithi044 ceph-mon[32458]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-23T18:26:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:11 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:11 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:11 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:11 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:11 smithi044 ceph-mon[32458]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:11.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:11 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:11 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 -- 172.21.15.19:0/1984008909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7a00a3e30 msgr2=0x7fc7a00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7a00a3e30 0x7fc7a00a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4005d00 tx=0x7fc7a400b410 comp rx=0 tx=0).stop 2024-04-23T18:26:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 -- 172.21.15.19:0/1984008909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b4105400 msgr2=0x7fc7b4198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b4105400 0x7fc7b4198770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0006010 tx=0x7fc7b00042f0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 -- 172.21.15.19:0/1984008909 shutdown_connections 2024-04-23T18:26:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7a00a3e30 0x7fc7a00a62f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b4107d30 0x7fc7b419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b4105400 0x7fc7b4198770 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 --2- 172.21.15.19:0/1984008909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b4102ad0 0x7fc7b4198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 -- 172.21.15.19:0/1984008909 >> 172.21.15.19:0/1984008909 conn(0x7fc7b40fc400 msgr2=0x7fc7b41035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 -- 172.21.15.19:0/1984008909 shutdown_connections 2024-04-23T18:26:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.570+0000 7fc7bb691640 1 -- 172.21.15.19:0/1984008909 wait complete. 2024-04-23T18:26:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:11 smithi155 ceph-mon[33468]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:11 smithi155 ceph-mon[33468]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:11 smithi155 ceph-mon[33468]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-23T18:26:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:11 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:11 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:11 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:11 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:11 smithi155 ceph-mon[33468]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:11.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:11 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:11 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:11.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:26:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.689+0000 7fefc6f43640 1 -- 172.21.15.19:0/514089081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc0105320 msgr2=0x7fefc0107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.689+0000 7fefc6f43640 1 --2- 172.21.15.19:0/514089081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc0105320 0x7fefc0107710 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fefac009900 tx=0x7fefac02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.689+0000 7fefc6f43640 1 -- 172.21.15.19:0/514089081 shutdown_connections 2024-04-23T18:26:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.689+0000 7fefc6f43640 1 --2- 172.21.15.19:0/514089081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc0107c50 0x7fefc010a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.689+0000 7fefc6f43640 1 --2- 172.21.15.19:0/514089081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc0105320 0x7fefc0107710 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.689+0000 7fefc6f43640 1 --2- 172.21.15.19:0/514089081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefc01029f0 0x7fefc0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.689+0000 7fefc6f43640 1 -- 172.21.15.19:0/514089081 >> 172.21.15.19:0/514089081 conn(0x7fefc00fc3e0 msgr2=0x7fefc00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.689+0000 7fefc6f43640 1 -- 172.21.15.19:0/514089081 shutdown_connections 2024-04-23T18:26:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.689+0000 7fefc6f43640 1 -- 172.21.15.19:0/514089081 wait complete. 2024-04-23T18:26:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.690+0000 7fefc6f43640 1 Processor -- start 2024-04-23T18:26:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.690+0000 7fefc6f43640 1 -- start start 2024-04-23T18:26:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.690+0000 7fefc6f43640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc01029f0 0x7fefc0198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.690+0000 7fefc6f43640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefc0105320 0x7fefc01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.690+0000 7fefc6f43640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc0107c50 0x7fefc019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.690+0000 7fefc6f43640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010da80 con 0x7fefc0107c50 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.690+0000 7fefc6f43640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010d900 con 0x7fefc0105320 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.690+0000 7fefc6f43640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010dc00 con 0x7fefc01029f0 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefc54e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc0107c50 0x7fefc019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefc0105320 0x7fefc01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefc0105320 0x7fefc01986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:54048/0 (socket says 172.21.15.19:54048) 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefc54e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc0107c50 0x7fefc019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53298/0 (socket says 172.21.15.19:53298) 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbffff640 1 -- 172.21.15.19:0/4000943173 learned_addr learned my addr 172.21.15.19:0/4000943173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefc4cdf640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc01029f0 0x7fefc0198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbffff640 1 -- 172.21.15.19:0/4000943173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc01029f0 msgr2=0x7fefc0198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbffff640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc01029f0 0x7fefc0198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbffff640 1 -- 172.21.15.19:0/4000943173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc0107c50 msgr2=0x7fefc019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbffff640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc0107c50 0x7fefc019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbffff640 1 -- 172.21.15.19:0/4000943173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc019fd80 con 0x7fefc0105320 2024-04-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefc54e0640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc0107c50 0x7fefc019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbffff640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefc0105320 0x7fefc01986a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fefac006010 tx=0x7fefac0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbdffb640 1 -- 172.21.15.19:0/4000943173 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac03f030 con 0x7fefc0105320 2024-04-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbdffb640 1 -- 172.21.15.19:0/4000943173 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefac003cb0 con 0x7fefc0105320 2024-04-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefbdffb640 1 -- 172.21.15.19:0/4000943173 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac035070 con 0x7fefc0105320 2024-04-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.691+0000 7fefc6f43640 1 -- 172.21.15.19:0/4000943173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc01a0010 con 0x7fefc0105320 2024-04-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.692+0000 7fefc6f43640 1 -- 172.21.15.19:0/4000943173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc01a0500 con 0x7fefc0105320 2024-04-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.693+0000 7fefc6f43640 1 -- 172.21.15.19:0/4000943173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc00fded0 con 0x7fefc0105320 2024-04-23T18:26:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.694+0000 7fefbdffb640 1 -- 172.21.15.19:0/4000943173 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fefac042030 con 0x7fefc0105320 2024-04-23T18:26:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.694+0000 7fefbdffb640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fefb00a3db0 0x7fefb00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.694+0000 7fefbdffb640 1 -- 172.21.15.19:0/4000943173 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fefac0f1120 con 0x7fefc0105320 2024-04-23T18:26:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.694+0000 7fefc4cdf640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fefb00a3db0 0x7fefb00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.695+0000 7fefc4cdf640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fefb00a3db0 0x7fefb00a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fefb400abe0 tx=0x7fefb400a2d0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.698+0000 7fefbdffb640 1 -- 172.21.15.19:0/4000943173 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefac0bd680 con 0x7fefc0105320 2024-04-23T18:26:11.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.925+0000 7fefc6f43640 1 -- 172.21.15.19:0/4000943173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fefc002d810 con 0x7fefc0105320 2024-04-23T18:26:11.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.925+0000 7fefbdffb640 1 -- 172.21.15.19:0/4000943173 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fefac0bcf80 con 0x7fefc0105320 2024-04-23T18:26:11.918 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-23T18:26:11.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.927+0000 7fefc6f43640 1 -- 172.21.15.19:0/4000943173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fefb00a3db0 msgr2=0x7fefb00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.927+0000 7fefc6f43640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fefb00a3db0 0x7fefb00a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fefb400abe0 tx=0x7fefb400a2d0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.927+0000 7fefc6f43640 1 -- 172.21.15.19:0/4000943173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefc0105320 msgr2=0x7fefc01986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.927+0000 7fefc6f43640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefc0105320 0x7fefc01986a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fefac006010 tx=0x7fefac0042f0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.928+0000 7fefc6f43640 1 -- 172.21.15.19:0/4000943173 shutdown_connections 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.928+0000 7fefc6f43640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fefb00a3db0 0x7fefb00a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.928+0000 7fefc6f43640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefc0105320 0x7fefc01986a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.928+0000 7fefc6f43640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc0107c50 0x7fefc019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.928+0000 7fefc6f43640 1 --2- 172.21.15.19:0/4000943173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefc01029f0 0x7fefc0198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.928+0000 7fefc6f43640 1 -- 172.21.15.19:0/4000943173 >> 172.21.15.19:0/4000943173 conn(0x7fefc00fc3e0 msgr2=0x7fefc0103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.928+0000 7fefc6f43640 1 -- 172.21.15.19:0/4000943173 shutdown_connections 2024-04-23T18:26:11.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:11.928+0000 7fefc6f43640 1 -- 172.21.15.19:0/4000943173 wait complete. 2024-04-23T18:26:11.932 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:26:12.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.047+0000 7fce31872640 1 -- 172.21.15.19:0/1996932061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce2c105360 msgr2=0x7fce2c107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.047+0000 7fce31872640 1 --2- 172.21.15.19:0/1996932061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce2c105360 0x7fce2c107750 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009900 tx=0x7fce1c02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:12.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.047+0000 7fce31872640 1 -- 172.21.15.19:0/1996932061 shutdown_connections 2024-04-23T18:26:12.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.047+0000 7fce31872640 1 --2- 172.21.15.19:0/1996932061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce2c107c90 0x7fce2c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.047+0000 7fce31872640 1 --2- 172.21.15.19:0/1996932061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce2c105360 0x7fce2c107750 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.048+0000 7fce31872640 1 --2- 172.21.15.19:0/1996932061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce2c102a30 0x7fce2c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.048+0000 7fce31872640 1 -- 172.21.15.19:0/1996932061 >> 172.21.15.19:0/1996932061 conn(0x7fce2c0fc440 msgr2=0x7fce2c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:12.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.048+0000 7fce31872640 1 -- 172.21.15.19:0/1996932061 shutdown_connections 2024-04-23T18:26:12.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.048+0000 7fce31872640 1 -- 172.21.15.19:0/1996932061 wait complete. 2024-04-23T18:26:12.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.048+0000 7fce31872640 1 Processor -- start 2024-04-23T18:26:12.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce31872640 1 -- start start 2024-04-23T18:26:12.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce31872640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce2c102a30 0x7fce2c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce31872640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce2c105360 0x7fce2c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce31872640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce2c107c90 0x7fce2c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce31872640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c10daf0 con 0x7fce2c102a30 2024-04-23T18:26:12.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce31872640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c10d970 con 0x7fce2c107c90 2024-04-23T18:26:12.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce31872640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c10dc70 con 0x7fce2c105360 2024-04-23T18:26:12.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce2affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce2c102a30 0x7fce2c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce2affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce2c102a30 0x7fce2c198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53314/0 (socket says 172.21.15.19:53314) 2024-04-23T18:26:12.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce2affd640 1 -- 172.21.15.19:0/41697506 learned_addr learned my addr 172.21.15.19:0/41697506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:12.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce2b7fe640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce2c107c90 0x7fce2c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce2a7fc640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce2c105360 0x7fce2c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce2affd640 1 -- 172.21.15.19:0/41697506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce2c105360 msgr2=0x7fce2c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce2affd640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce2c105360 0x7fce2c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce2affd640 1 -- 172.21.15.19:0/41697506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce2c107c90 msgr2=0x7fce2c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce2affd640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce2c107c90 0x7fce2c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.049+0000 7fce2affd640 1 -- 172.21.15.19:0/41697506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c19fe70 con 0x7fce2c102a30 2024-04-23T18:26:12.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.050+0000 7fce2affd640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce2c102a30 0x7fce2c198250 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fce2000e9f0 tx=0x7fce2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:12.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.050+0000 7fce30870640 1 -- 172.21.15.19:0/41697506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2000cd80 con 0x7fce2c102a30 2024-04-23T18:26:12.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.050+0000 7fce31872640 1 -- 172.21.15.19:0/41697506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2c1a0160 con 0x7fce2c102a30 2024-04-23T18:26:12.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.050+0000 7fce30870640 1 -- 172.21.15.19:0/41697506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce200054d0 con 0x7fce2c102a30 2024-04-23T18:26:12.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.050+0000 7fce30870640 1 -- 172.21.15.19:0/41697506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20010690 con 0x7fce2c102a30 2024-04-23T18:26:12.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.050+0000 7fce31872640 1 -- 172.21.15.19:0/41697506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2c1a0680 con 0x7fce2c102a30 2024-04-23T18:26:12.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.051+0000 7fce31872640 1 -- 172.21.15.19:0/41697506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2c0fdf30 con 0x7fce2c102a30 2024-04-23T18:26:12.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.052+0000 7fce30870640 1 -- 172.21.15.19:0/41697506 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fce20004e90 con 0x7fce2c102a30 2024-04-23T18:26:12.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.053+0000 7fce30870640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fce140a3db0 0x7fce140a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.053+0000 7fce2a7fc640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fce140a3db0 0x7fce140a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.053+0000 7fce30870640 1 -- 172.21.15.19:0/41697506 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fce200d0dd0 con 0x7fce2c102a30 2024-04-23T18:26:12.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.054+0000 7fce2a7fc640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fce140a3db0 0x7fce140a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009230 tx=0x7fce1c0314a0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:12.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.057+0000 7fce30870640 1 -- 172.21.15.19:0/41697506 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce2009d2d0 con 0x7fce2c102a30 2024-04-23T18:26:12.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.257+0000 7fce31872640 1 -- 172.21.15.19:0/41697506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce2c02d5c0 con 0x7fce2c102a30 2024-04-23T18:26:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.258+0000 7fce30870640 1 -- 172.21.15.19:0/41697506 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fce2c02d5c0 con 0x7fce2c102a30 2024-04-23T18:26:12.251 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-23T18:26:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 -- 172.21.15.19:0/41697506 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fce140a3db0 msgr2=0x7fce140a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fce140a3db0 0x7fce140a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009230 tx=0x7fce1c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 -- 172.21.15.19:0/41697506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce2c102a30 msgr2=0x7fce2c198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce2c102a30 0x7fce2c198250 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fce2000e9f0 tx=0x7fce2000edf0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 -- 172.21.15.19:0/41697506 shutdown_connections 2024-04-23T18:26:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fce140a3db0 0x7fce140a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce2c107c90 0x7fce2c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce2c105360 0x7fce2c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 --2- 172.21.15.19:0/41697506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce2c102a30 0x7fce2c198250 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 -- 172.21.15.19:0/41697506 >> 172.21.15.19:0/41697506 conn(0x7fce2c0fc440 msgr2=0x7fce2c1048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:12.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 -- 172.21.15.19:0/41697506 shutdown_connections 2024-04-23T18:26:12.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.260+0000 7fce31872640 1 -- 172.21.15.19:0/41697506 wait complete. 2024-04-23T18:26:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2390328909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-23T18:26:12.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1984008909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:12.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:12.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-23T18:26:12.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-23T18:26:12.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: Updating smithi155:/etc/ceph/ceph.conf 2024-04-23T18:26:12.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:12 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4000943173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:12.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:12.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.390+0000 7f43f74d0640 1 -- 172.21.15.19:0/1865289998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f0102a10 msgr2=0x7f43f0104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.390+0000 7f43f74d0640 1 --2- 172.21.15.19:0/1865289998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f0102a10 0x7f43f0104e00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009900 tx=0x7f43e002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:12.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.390+0000 7f43f74d0640 1 -- 172.21.15.19:0/1865289998 shutdown_connections 2024-04-23T18:26:12.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.390+0000 7f43f74d0640 1 --2- 172.21.15.19:0/1865289998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f0105340 0x7f43f0107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.390+0000 7f43f74d0640 1 --2- 172.21.15.19:0/1865289998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f0107c70 0x7f43f010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.390+0000 7f43f74d0640 1 --2- 172.21.15.19:0/1865289998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f0102a10 0x7f43f0104e00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.390+0000 7f43f74d0640 1 -- 172.21.15.19:0/1865289998 >> 172.21.15.19:0/1865289998 conn(0x7f43f00fc400 msgr2=0x7f43f00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.390+0000 7f43f74d0640 1 -- 172.21.15.19:0/1865289998 shutdown_connections 2024-04-23T18:26:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.390+0000 7f43f74d0640 1 -- 172.21.15.19:0/1865289998 wait complete. 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f74d0640 1 Processor -- start 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f74d0640 1 -- start start 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f74d0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f0102a10 0x7f43f01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f74d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f0105340 0x7f43f0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f74d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f0107c70 0x7f43f019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f74d0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010dac0 con 0x7f43f0107c70 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f74d0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010d940 con 0x7f43f0105340 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f74d0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010dc40 con 0x7f43f0102a10 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f5a6d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f0107c70 0x7f43f019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f5a6d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f0107c70 0x7f43f019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53330/0 (socket says 172.21.15.19:53330) 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f5a6d640 1 -- 172.21.15.19:0/2893709260 learned_addr learned my addr 172.21.15.19:0/2893709260 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f4a6b640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f0105340 0x7f43f0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f526c640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f0102a10 0x7f43f01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f5a6d640 1 -- 172.21.15.19:0/2893709260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f0102a10 msgr2=0x7f43f01981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f5a6d640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f0102a10 0x7f43f01981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.391+0000 7f43f5a6d640 1 -- 172.21.15.19:0/2893709260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f0105340 msgr2=0x7f43f0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.392+0000 7f43f5a6d640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f0105340 0x7f43f0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.392+0000 7f43f5a6d640 1 -- 172.21.15.19:0/2893709260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f019fe10 con 0x7f43f0107c70 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.392+0000 7f43f5a6d640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f0107c70 0x7f43f019f7a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f43e800eb10 tx=0x7f43e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.392+0000 7f43e67fc640 1 -- 172.21.15.19:0/2893709260 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8004b20 con 0x7f43f0107c70 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.392+0000 7f43e67fc640 1 -- 172.21.15.19:0/2893709260 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43e8004cc0 con 0x7f43f0107c70 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.392+0000 7f43f74d0640 1 -- 172.21.15.19:0/2893709260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f01a0100 con 0x7f43f0107c70 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.392+0000 7f43e67fc640 1 -- 172.21.15.19:0/2893709260 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8010670 con 0x7f43f0107c70 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.392+0000 7f43f74d0640 1 -- 172.21.15.19:0/2893709260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f01a0650 con 0x7f43f0107c70 2024-04-23T18:26:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.393+0000 7f43f74d0640 1 -- 172.21.15.19:0/2893709260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f00fdef0 con 0x7f43f0107c70 2024-04-23T18:26:12.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.394+0000 7f43e67fc640 1 -- 172.21.15.19:0/2893709260 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f43e8005000 con 0x7f43f0107c70 2024-04-23T18:26:12.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.395+0000 7f43e67fc640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f43dc0a3e30 0x7f43dc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.396+0000 7f43f526c640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f43dc0a3e30 0x7f43dc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.396+0000 7f43e67fc640 1 -- 172.21.15.19:0/2893709260 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f43e80d1120 con 0x7f43f0107c70 2024-04-23T18:26:12.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.396+0000 7f43f526c640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f43dc0a3e30 0x7f43dc0a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43e0000c00 tx=0x7f43e00314a0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:12.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.400+0000 7f43e67fc640 1 -- 172.21.15.19:0/2893709260 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43e8053ff0 con 0x7f43f0107c70 2024-04-23T18:26:12.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2390328909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1984008909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:12.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:12.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-23T18:26:12.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-23T18:26:12.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: Updating smithi155:/etc/ceph/ceph.conf 2024-04-23T18:26:12.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:12 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4000943173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:12.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:12.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2390328909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1984008909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:12.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-23T18:26:12.575 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-23T18:26:12.575 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: Updating smithi155:/etc/ceph/ceph.conf 2024-04-23T18:26:12.575 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:12 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4000943173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:12.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.590+0000 7f43f74d0640 1 -- 172.21.15.19:0/2893709260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43f0062b50 con 0x7f43f0107c70 2024-04-23T18:26:12.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.591+0000 7f43e67fc640 1 -- 172.21.15.19:0/2893709260 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23018 (secure 0 0 0) 0x7f43e809d620 con 0x7f43f0107c70 2024-04-23T18:26:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:12.585 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:26:08.116951+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/304874449":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1108384152":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6801/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6800/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/983971095":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2556512605":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:12.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.593+0000 7f43f74d0640 1 -- 172.21.15.19:0/2893709260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f43dc0a3e30 msgr2=0x7f43dc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.593+0000 7f43f74d0640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f43dc0a3e30 0x7f43dc0a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43e0000c00 tx=0x7f43e00314a0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.593+0000 7f43f74d0640 1 -- 172.21.15.19:0/2893709260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f0107c70 msgr2=0x7f43f019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.593+0000 7f43f74d0640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f0107c70 0x7f43f019f7a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f43e800eb10 tx=0x7f43e800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.593+0000 7f43f74d0640 1 -- 172.21.15.19:0/2893709260 shutdown_connections 2024-04-23T18:26:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.593+0000 7f43f74d0640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f43dc0a3e30 0x7f43dc0a62f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.593+0000 7f43f74d0640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f0105340 0x7f43f0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.593+0000 7f43f74d0640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f0107c70 0x7f43f019f7a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.593+0000 7f43f74d0640 1 --2- 172.21.15.19:0/2893709260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f0102a10 0x7f43f01981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.593+0000 7f43f74d0640 1 -- 172.21.15.19:0/2893709260 >> 172.21.15.19:0/2893709260 conn(0x7f43f00fc400 msgr2=0x7f43f0103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.594+0000 7f43f74d0640 1 -- 172.21.15.19:0/2893709260 shutdown_connections 2024-04-23T18:26:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.594+0000 7f43f74d0640 1 -- 172.21.15.19:0/2893709260 wait complete. 2024-04-23T18:26:12.598 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:23:53.296504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:23:54.573484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:26:12.599 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:26:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.715+0000 7f28cb76f640 1 -- 172.21.15.19:0/4245347699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c4102b50 msgr2=0x7f28c4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.715+0000 7f28cb76f640 1 --2- 172.21.15.19:0/4245347699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c4102b50 0x7f28c4106b90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009900 tx=0x7f28b402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.715+0000 7f28cb76f640 1 -- 172.21.15.19:0/4245347699 shutdown_connections 2024-04-23T18:26:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.715+0000 7f28cb76f640 1 --2- 172.21.15.19:0/4245347699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c4107270 0x7f28c410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.715+0000 7f28cb76f640 1 --2- 172.21.15.19:0/4245347699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c4102b50 0x7f28c4106b90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.715+0000 7f28cb76f640 1 --2- 172.21.15.19:0/4245347699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c41021a0 0x7f28c4102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.715+0000 7f28cb76f640 1 -- 172.21.15.19:0/4245347699 >> 172.21.15.19:0/4245347699 conn(0x7f28c4079350 msgr2=0x7f28c4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.715+0000 7f28cb76f640 1 -- 172.21.15.19:0/4245347699 shutdown_connections 2024-04-23T18:26:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.715+0000 7f28cb76f640 1 -- 172.21.15.19:0/4245347699 wait complete. 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28cb76f640 1 Processor -- start 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28cb76f640 1 -- start start 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28cb76f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c41021a0 0x7f28c419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28cb76f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c4107270 0x7f28c419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28cb76f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c4198590 0x7f28c4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28cb76f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4112190 con 0x7f28c4198590 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28cb76f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4112010 con 0x7f28c41021a0 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28cb76f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4112310 con 0x7f28c4107270 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c9d0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c4198590 0x7f28c4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c9d0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c4198590 0x7f28c4198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53336/0 (socket says 172.21.15.19:53336) 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c9d0c640 1 -- 172.21.15.19:0/122691039 learned_addr learned my addr 172.21.15.19:0/122691039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c8d0a640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c4107270 0x7f28c419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c950b640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c41021a0 0x7f28c419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c9d0c640 1 -- 172.21.15.19:0/122691039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c4107270 msgr2=0x7f28c419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c9d0c640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c4107270 0x7f28c419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c9d0c640 1 -- 172.21.15.19:0/122691039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c41021a0 msgr2=0x7f28c419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c9d0c640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c41021a0 0x7f28c419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c9d0c640 1 -- 172.21.15.19:0/122691039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c41990b0 con 0x7f28c4198590 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.716+0000 7f28c950b640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c41021a0 0x7f28c419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.717+0000 7f28c9d0c640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c4198590 0x7f28c4198a40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00eb10 tx=0x7f28bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.717+0000 7f28ba7fc640 1 -- 172.21.15.19:0/122691039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc004b20 con 0x7f28c4198590 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.717+0000 7f28ba7fc640 1 -- 172.21.15.19:0/122691039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28bc004cc0 con 0x7f28c4198590 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.717+0000 7f28cb76f640 1 -- 172.21.15.19:0/122691039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c41993a0 con 0x7f28c4198590 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.717+0000 7f28ba7fc640 1 -- 172.21.15.19:0/122691039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc010670 con 0x7f28c4198590 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.717+0000 7f28cb76f640 1 -- 172.21.15.19:0/122691039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c41a55a0 con 0x7f28c4198590 2024-04-23T18:26:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.718+0000 7f28cb76f640 1 -- 172.21.15.19:0/122691039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c4068d70 con 0x7f28c4198590 2024-04-23T18:26:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.719+0000 7f28ba7fc640 1 -- 172.21.15.19:0/122691039 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f28bc005020 con 0x7f28c4198590 2024-04-23T18:26:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.720+0000 7f28ba7fc640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28b00a3db0 0x7f28b00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.720+0000 7f28c950b640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28b00a3db0 0x7f28b00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.720+0000 7f28ba7fc640 1 -- 172.21.15.19:0/122691039 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f28bc0d0f20 con 0x7f28c4198590 2024-04-23T18:26:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.720+0000 7f28c950b640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28b00a3db0 0x7f28b00a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009c20 tx=0x7f28c0005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.725+0000 7f28ba7fc640 1 -- 172.21.15.19:0/122691039 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28bc09d480 con 0x7f28c4198590 2024-04-23T18:26:12.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.753+0000 7f28ba7fc640 1 -- 172.21.15.19:0/122691039 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f28bc0993f0 con 0x7f28c4198590 2024-04-23T18:26:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.973+0000 7f28cb76f640 1 -- 172.21.15.19:0/122691039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28c4102b50 con 0x7f28c4198590 2024-04-23T18:26:12.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.973+0000 7f28ba7fc640 1 -- 172.21.15.19:0/122691039 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f28bc051de0 con 0x7f28c4198590 2024-04-23T18:26:12.966 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 -- 172.21.15.19:0/122691039 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28b00a3db0 msgr2=0x7f28b00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28b00a3db0 0x7f28b00a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009c20 tx=0x7f28c0005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 -- 172.21.15.19:0/122691039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c4198590 msgr2=0x7f28c4198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c4198590 0x7f28c4198a40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00eb10 tx=0x7f28bc00c2d0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 -- 172.21.15.19:0/122691039 shutdown_connections 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28b00a3db0 0x7f28b00a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c4198590 0x7f28c4198a40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c4107270 0x7f28c419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 --2- 172.21.15.19:0/122691039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c41021a0 0x7f28c419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 -- 172.21.15.19:0/122691039 >> 172.21.15.19:0/122691039 conn(0x7f28c4079350 msgr2=0x7f28c4100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:12.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 -- 172.21.15.19:0/122691039 shutdown_connections 2024-04-23T18:26:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:12.975+0000 7f28cb76f640 1 -- 172.21.15.19:0/122691039 wait complete. 2024-04-23T18:26:12.979 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:26:13.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.096+0000 7f537339b640 1 -- 172.21.15.19:0/3044892284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f536c101a30 msgr2=0x7f536c105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.096+0000 7f537339b640 1 --2- 172.21.15.19:0/3044892284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f536c101a30 0x7f536c105a70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f535c009900 tx=0x7f535c02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.096+0000 7f537339b640 1 -- 172.21.15.19:0/3044892284 shutdown_connections 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.096+0000 7f537339b640 1 --2- 172.21.15.19:0/3044892284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c106150 0x7f536c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.096+0000 7f537339b640 1 --2- 172.21.15.19:0/3044892284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f536c101a30 0x7f536c105a70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.096+0000 7f537339b640 1 --2- 172.21.15.19:0/3044892284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f536c101080 0x7f536c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.096+0000 7f537339b640 1 -- 172.21.15.19:0/3044892284 >> 172.21.15.19:0/3044892284 conn(0x7f536c078000 msgr2=0x7f536c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.097+0000 7f537339b640 1 -- 172.21.15.19:0/3044892284 shutdown_connections 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.097+0000 7f537339b640 1 -- 172.21.15.19:0/3044892284 wait complete. 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.097+0000 7f537339b640 1 Processor -- start 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.097+0000 7f537339b640 1 -- start start 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.097+0000 7f537339b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c101080 0x7f536c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.097+0000 7f537339b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f536c106150 0x7f536c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.097+0000 7f537339b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f536c19cff0 0x7f536c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.097+0000 7f537339b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f536c111090 con 0x7f536c106150 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.097+0000 7f537339b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f536c110f10 con 0x7f536c101080 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.097+0000 7f537339b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f536c111210 con 0x7f536c19cff0 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5370936640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f536c106150 0x7f536c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5371137640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c101080 0x7f536c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5370936640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f536c106150 0x7f536c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53354/0 (socket says 172.21.15.19:53354) 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5370936640 1 -- 172.21.15.19:0/1647890227 learned_addr learned my addr 172.21.15.19:0/1647890227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5371938640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f536c19cff0 0x7f536c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5370936640 1 -- 172.21.15.19:0/1647890227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f536c19cff0 msgr2=0x7f536c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5370936640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f536c19cff0 0x7f536c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5370936640 1 -- 172.21.15.19:0/1647890227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c101080 msgr2=0x7f536c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5370936640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c101080 0x7f536c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5370936640 1 -- 172.21.15.19:0/1647890227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c19dae0 con 0x7f536c106150 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5371137640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c101080 0x7f536c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f5370936640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f536c106150 0x7f536c198c70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f535c006010 tx=0x7f535c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f53627fc640 1 -- 172.21.15.19:0/1647890227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c03f030 con 0x7f536c106150 2024-04-23T18:26:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f53627fc640 1 -- 172.21.15.19:0/1647890227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c003aa0 con 0x7f536c106150 2024-04-23T18:26:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f537339b640 1 -- 172.21.15.19:0/1647890227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536c19dd70 con 0x7f536c106150 2024-04-23T18:26:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f53627fc640 1 -- 172.21.15.19:0/1647890227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c035070 con 0x7f536c106150 2024-04-23T18:26:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.098+0000 7f537339b640 1 -- 172.21.15.19:0/1647890227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536c102cd0 con 0x7f536c106150 2024-04-23T18:26:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.099+0000 7f537339b640 1 -- 172.21.15.19:0/1647890227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536c068d70 con 0x7f536c106150 2024-04-23T18:26:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.101+0000 7f53627fc640 1 -- 172.21.15.19:0/1647890227 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f535c042030 con 0x7f536c106150 2024-04-23T18:26:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.101+0000 7f53627fc640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53580a3e40 0x7f53580a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.102+0000 7f5371137640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53580a3e40 0x7f53580a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.102+0000 7f53627fc640 1 -- 172.21.15.19:0/1647890227 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f535c0f0a90 con 0x7f536c106150 2024-04-23T18:26:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.102+0000 7f5371137640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53580a3e40 0x7f53580a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f53680096a0 tx=0x7f5368012040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.106+0000 7f53627fc640 1 -- 172.21.15.19:0/1647890227 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f535c0bcfe0 con 0x7f536c106150 2024-04-23T18:26:13.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:13.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:13.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/41697506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:13.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2893709260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/122691039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:13.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:13.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:13.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:13 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.302+0000 7f537339b640 1 -- 172.21.15.19:0/1647890227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f536c101a30 con 0x7f536c106150 2024-04-23T18:26:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.302+0000 7f53627fc640 1 -- 172.21.15.19:0/1647890227 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f535c0a5e20 con 0x7f536c106150 2024-04-23T18:26:13.295 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-23T18:26:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.304+0000 7f537339b640 1 -- 172.21.15.19:0/1647890227 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53580a3e40 msgr2=0x7f53580a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.304+0000 7f537339b640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53580a3e40 0x7f53580a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f53680096a0 tx=0x7f5368012040 comp rx=0 tx=0).stop 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.305+0000 7f537339b640 1 -- 172.21.15.19:0/1647890227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f536c106150 msgr2=0x7f536c198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.305+0000 7f537339b640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f536c106150 0x7f536c198c70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f535c006010 tx=0x7f535c004060 comp rx=0 tx=0).stop 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.305+0000 7f537339b640 1 -- 172.21.15.19:0/1647890227 shutdown_connections 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.305+0000 7f537339b640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53580a3e40 0x7f53580a6300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.305+0000 7f537339b640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f536c19cff0 0x7f536c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.305+0000 7f537339b640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f536c106150 0x7f536c198c70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.305+0000 7f537339b640 1 --2- 172.21.15.19:0/1647890227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c101080 0x7f536c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.305+0000 7f537339b640 1 -- 172.21.15.19:0/1647890227 >> 172.21.15.19:0/1647890227 conn(0x7f536c078000 msgr2=0x7f536c069540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.305+0000 7f537339b640 1 -- 172.21.15.19:0/1647890227 shutdown_connections 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.305+0000 7f537339b640 1 -- 172.21.15.19:0/1647890227 wait complete. 2024-04-23T18:26:13.308 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:26:13.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.421+0000 7f4946956640 1 -- 172.21.15.19:0/3675334167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4940102b40 msgr2=0x7f4940106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.421+0000 7f4946956640 1 --2- 172.21.15.19:0/3675334167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4940102b40 0x7f4940106b80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4930009900 tx=0x7f493002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:13.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.421+0000 7f4946956640 1 -- 172.21.15.19:0/3675334167 shutdown_connections 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.421+0000 7f4946956640 1 --2- 172.21.15.19:0/3675334167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4940107260 0x7f494010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.421+0000 7f4946956640 1 --2- 172.21.15.19:0/3675334167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4940102b40 0x7f4940106b80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.421+0000 7f4946956640 1 --2- 172.21.15.19:0/3675334167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4940102190 0x7f4940102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.421+0000 7f4946956640 1 -- 172.21.15.19:0/3675334167 >> 172.21.15.19:0/3675334167 conn(0x7f49400793a0 msgr2=0x7f49400797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.421+0000 7f4946956640 1 -- 172.21.15.19:0/3675334167 shutdown_connections 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.421+0000 7f4946956640 1 -- 172.21.15.19:0/3675334167 wait complete. 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f4946956640 1 Processor -- start 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f4946956640 1 -- start start 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f4946956640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4940102190 0x7f494019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f4946956640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4940107260 0x7f494019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f4946956640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4940198600 0x7f4940198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f4946956640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4940112180 con 0x7f4940102190 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f4946956640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4940112000 con 0x7f4940198600 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f4946956640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4940112300 con 0x7f4940107260 2024-04-23T18:26:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f493ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4940102190 0x7f494019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f493ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4940102190 0x7f494019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53376/0 (socket says 172.21.15.19:53376) 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f493ffff640 1 -- 172.21.15.19:0/828083611 learned_addr learned my addr 172.21.15.19:0/828083611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f4944ef3640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4940198600 0x7f4940198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.422+0000 7f493f7fe640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4940107260 0x7f494019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f493ffff640 1 -- 172.21.15.19:0/828083611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4940107260 msgr2=0x7f494019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f493ffff640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4940107260 0x7f494019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f493ffff640 1 -- 172.21.15.19:0/828083611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4940198600 msgr2=0x7f4940198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f493ffff640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4940198600 0x7f4940198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f493ffff640 1 -- 172.21.15.19:0/828083611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4940199120 con 0x7f4940102190 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f4944ef3640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4940198600 0x7f4940198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f493ffff640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4940102190 0x7f494019e400 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f492800d950 tx=0x7f492800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f493d7fa640 1 -- 172.21.15.19:0/828083611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928017030 con 0x7f4940102190 2024-04-23T18:26:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f4946956640 1 -- 172.21.15.19:0/828083611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4940199410 con 0x7f4940102190 2024-04-23T18:26:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f493d7fa640 1 -- 172.21.15.19:0/828083611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4928004020 con 0x7f4940102190 2024-04-23T18:26:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f493d7fa640 1 -- 172.21.15.19:0/828083611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49280047a0 con 0x7f4940102190 2024-04-23T18:26:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.423+0000 7f4946956640 1 -- 172.21.15.19:0/828083611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49401a5610 con 0x7f4940102190 2024-04-23T18:26:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.424+0000 7f4946956640 1 -- 172.21.15.19:0/828083611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49400691c0 con 0x7f4940102190 2024-04-23T18:26:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.425+0000 7f493d7fa640 1 -- 172.21.15.19:0/828083611 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4928009a30 con 0x7f4940102190 2024-04-23T18:26:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.426+0000 7f493d7fa640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f492c0a3e40 0x7f492c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.426+0000 7f493f7fe640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f492c0a3e40 0x7f492c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.426+0000 7f493d7fa640 1 -- 172.21.15.19:0/828083611 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f49280d0a90 con 0x7f4940102190 2024-04-23T18:26:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.426+0000 7f493f7fe640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f492c0a3e40 0x7f492c0a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4930000c00 tx=0x7f4930031400 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.429+0000 7f493d7fa640 1 -- 172.21.15.19:0/828083611 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f492809cf10 con 0x7f4940102190 2024-04-23T18:26:13.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:13.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/41697506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2893709260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/122691039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:13.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:13.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:13.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:13.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:13 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:13.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:13.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/41697506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: Updating smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: Updating smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: Updating smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2893709260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/122691039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:13.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:13.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:13.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:13.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:13 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:26:13.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.616+0000 7f4946956640 1 -- 172.21.15.19:0/828083611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4940102b40 con 0x7f4940102190 2024-04-23T18:26:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.616+0000 7f493d7fa640 1 -- 172.21.15.19:0/828083611 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f492809c810 con 0x7f4940102190 2024-04-23T18:26:13.609 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-23T18:26:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 -- 172.21.15.19:0/828083611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f492c0a3e40 msgr2=0x7f492c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f492c0a3e40 0x7f492c0a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4930000c00 tx=0x7f4930031400 comp rx=0 tx=0).stop 2024-04-23T18:26:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 -- 172.21.15.19:0/828083611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4940102190 msgr2=0x7f494019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4940102190 0x7f494019e400 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f492800d950 tx=0x7f492800de10 comp rx=0 tx=0).stop 2024-04-23T18:26:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 -- 172.21.15.19:0/828083611 shutdown_connections 2024-04-23T18:26:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f492c0a3e40 0x7f492c0a6300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4940198600 0x7f4940198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4940107260 0x7f494019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 --2- 172.21.15.19:0/828083611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4940102190 0x7f494019e400 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 -- 172.21.15.19:0/828083611 >> 172.21.15.19:0/828083611 conn(0x7f49400793a0 msgr2=0x7f4940100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 -- 172.21.15.19:0/828083611 shutdown_connections 2024-04-23T18:26:13.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.618+0000 7f4946956640 1 -- 172.21.15.19:0/828083611 wait complete. 2024-04-23T18:26:13.622 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-23T18:26:13.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:26:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.731+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/1748948732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c106190 msgr2=0x7f0f9c10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.731+0000 7f0fa0b4a640 1 --2- 172.21.15.19:0/1748948732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c106190 0x7f0f9c10dd60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009ca0 tx=0x7f0f9402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.732+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/1748948732 shutdown_connections 2024-04-23T18:26:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.732+0000 7f0fa0b4a640 1 --2- 172.21.15.19:0/1748948732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f9c101ac0 0x7f0f9c105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.732+0000 7f0fa0b4a640 1 --2- 172.21.15.19:0/1748948732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c106190 0x7f0f9c10dd60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.732+0000 7f0fa0b4a640 1 --2- 172.21.15.19:0/1748948732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c101110 0x7f0f9c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.732+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/1748948732 >> 172.21.15.19:0/1748948732 conn(0x7f0f9c077fb0 msgr2=0x7f0f9c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.732+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/1748948732 shutdown_connections 2024-04-23T18:26:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.732+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/1748948732 wait complete. 2024-04-23T18:26:13.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.732+0000 7f0fa0b4a640 1 Processor -- start 2024-04-23T18:26:13.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.732+0000 7f0fa0b4a640 1 -- start start 2024-04-23T18:26:13.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0fa0b4a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f9c101110 0x7f0f9c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0fa0b4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c106190 0x7f0f9c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0fa0b4a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c19d000 0x7f0f9c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0fa0b4a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c111100 con 0x7f0f9c106190 2024-04-23T18:26:13.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0fa0b4a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c110f80 con 0x7f0f9c101110 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0fa0b4a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c111280 con 0x7f0f9c19d000 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f93fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c106190 0x7f0f9c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f93fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c106190 0x7f0f9c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53390/0 (socket says 172.21.15.19:53390) 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f93fff640 1 -- 172.21.15.19:0/3365064603 learned_addr learned my addr 172.21.15.19:0/3365064603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f9a59c640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f9c101110 0x7f0f9c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f93fff640 1 -- 172.21.15.19:0/3365064603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c19d000 msgr2=0x7f0f9c19d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f93fff640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c19d000 0x7f0f9c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f93fff640 1 -- 172.21.15.19:0/3365064603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f9c101110 msgr2=0x7f0f9c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f93fff640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f9c101110 0x7f0f9c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f93fff640 1 -- 172.21.15.19:0/3365064603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c19daf0 con 0x7f0f9c106190 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f93fff640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c106190 0x7f0f9c198c80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0f84005fd0 tx=0x7f0f8400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f937fe640 1 -- 172.21.15.19:0/3365064603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84017070 con 0x7f0f9c106190 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f937fe640 1 -- 172.21.15.19:0/3365064603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f84009c30 con 0x7f0f9c106190 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/3365064603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c19dde0 con 0x7f0f9c106190 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0f937fe640 1 -- 172.21.15.19:0/3365064603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84003d30 con 0x7f0f9c106190 2024-04-23T18:26:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.733+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/3365064603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c1a55d0 con 0x7f0f9c106190 2024-04-23T18:26:13.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.734+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/3365064603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f9c0684e0 con 0x7f0f9c106190 2024-04-23T18:26:13.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.735+0000 7f0f937fe640 1 -- 172.21.15.19:0/3365064603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0f84003ed0 con 0x7f0f9c106190 2024-04-23T18:26:13.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.736+0000 7f0f937fe640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0f880a3e40 0x7f0f880a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.736+0000 7f0f9a59c640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0f880a3e40 0x7f0f880a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.736+0000 7f0f937fe640 1 -- 172.21.15.19:0/3365064603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0f840d0660 con 0x7f0f9c106190 2024-04-23T18:26:13.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.737+0000 7f0f9a59c640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0f880a3e40 0x7f0f880a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009bd0 tx=0x7f0f8c005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.741+0000 7f0f937fe640 1 -- 172.21.15.19:0/3365064603 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f84020030 con 0x7f0f9c106190 2024-04-23T18:26:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.923+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/3365064603 --> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f9c101ac0 con 0x7f0f880a3e40 2024-04-23T18:26:13.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.931+0000 7f0f937fe640 1 -- 172.21.15.19:0/3365064603 <== mgr.25004 v2:172.21.15.19:6800/1177101685 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258073 (secure 0 0 0) 0x7f0f9c101ac0 con 0x7f0f880a3e40 2024-04-23T18:26:13.928 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-23T18:26:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/3365064603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0f880a3e40 msgr2=0x7f0f880a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0f880a3e40 0x7f0f880a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009bd0 tx=0x7f0f8c005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/3365064603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c106190 msgr2=0x7f0f9c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c106190 0x7f0f9c198c80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0f84005fd0 tx=0x7f0f8400dde0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/3365064603 shutdown_connections 2024-04-23T18:26:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0f880a3e40 0x7f0f880a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c19d000 0x7f0f9c19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c106190 0x7f0f9c198c80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 --2- 172.21.15.19:0/3365064603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f9c101110 0x7f0f9c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/3365064603 >> 172.21.15.19:0/3365064603 conn(0x7f0f9c077fb0 msgr2=0x7f0f9c06a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/3365064603 shutdown_connections 2024-04-23T18:26:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:13.934+0000 7f0fa0b4a640 1 -- 172.21.15.19:0/3365064603 wait complete. 2024-04-23T18:26:13.938 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-23T18:26:12.135471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192925,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1542,"num_read_kb":940,"num_write":556,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":941,"ondisk_log_size":941,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382112,"kb_used_data":12132,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559712,"statfs":{"total":1151940427776,"available":1151549145088,"internally_reserved":0,"allocated":12423168,"data_stored":9197594,"data_compressed":220211,"data_compressed_allocated":290816,"data_compressed_original":3432448,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.985641"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292994+0000","last_change":"2024-04-23T18:23:56.098292+0000","last_active":"2024-04-23T18:26:08.292994+0000","last_peered":"2024-04-23T18:26:08.292994+0000","last_clean":"2024-04-23T18:26:08.292994+0000","last_became_active":"2024-04-23T18:23:56.096623+0000","last_became_peered":"2024-04-23T18:23:56.096623+0000","last_unstale":"2024-04-23T18:26:08.292994+0000","last_undegraded":"2024-04-23T18:26:08.292994+0000","last_fullsized":"2024-04-23T18:26:08.292994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:06.186581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293701+0000","last_change":"2024-04-23T18:23:56.096807+0000","last_active":"2024-04-23T18:26:08.293701+0000","last_peered":"2024-04-23T18:26:08.293701+0000","last_clean":"2024-04-23T18:26:08.293701+0000","last_became_active":"2024-04-23T18:23:56.096327+0000","last_became_peered":"2024-04-23T18:23:56.096327+0000","last_unstale":"2024-04-23T18:26:08.293701+0000","last_undegraded":"2024-04-23T18:26:08.293701+0000","last_fullsized":"2024-04-23T18:26:08.293701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:56:16.391821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293324+0000","last_change":"2024-04-23T18:23:56.101615+0000","last_active":"2024-04-23T18:26:08.293324+0000","last_peered":"2024-04-23T18:26:08.293324+0000","last_clean":"2024-04-23T18:26:08.293324+0000","last_became_active":"2024-04-23T18:23:56.100929+0000","last_became_peered":"2024-04-23T18:23:56.100929+0000","last_unstale":"2024-04-23T18:26:08.293324+0000","last_undegraded":"2024-04-23T18:26:08.293324+0000","last_fullsized":"2024-04-23T18:26:08.293324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:25.547029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293249+0000","last_change":"2024-04-23T18:23:56.103627+0000","last_active":"2024-04-23T18:26:08.293249+0000","last_peered":"2024-04-23T18:26:08.293249+0000","last_clean":"2024-04-23T18:26:08.293249+0000","last_became_active":"2024-04-23T18:23:56.102752+0000","last_became_peered":"2024-04-23T18:23:56.102752+0000","last_unstale":"2024-04-23T18:26:08.293249+0000","last_undegraded":"2024-04-23T18:26:08.293249+0000","last_fullsized":"2024-04-23T18:26:08.293249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:28.460342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218930+0000","last_change":"2024-04-23T18:23:56.094889+0000","last_active":"2024-04-23T18:26:08.218930+0000","last_peered":"2024-04-23T18:26:08.218930+0000","last_clean":"2024-04-23T18:26:08.218930+0000","last_became_active":"2024-04-23T18:23:56.094262+0000","last_became_peered":"2024-04-23T18:23:56.094262+0000","last_unstale":"2024-04-23T18:26:08.218930+0000","last_undegraded":"2024-04-23T18:26:08.218930+0000","last_fullsized":"2024-04-23T18:26:08.218930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:22.332114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229648+0000","last_change":"2024-04-23T18:23:56.108893+0000","last_active":"2024-04-23T18:26:08.229648+0000","last_peered":"2024-04-23T18:26:08.229648+0000","last_clean":"2024-04-23T18:26:08.229648+0000","last_became_active":"2024-04-23T18:23:56.108229+0000","last_became_peered":"2024-04-23T18:23:56.108229+0000","last_unstale":"2024-04-23T18:26:08.229648+0000","last_undegraded":"2024-04-23T18:26:08.229648+0000","last_fullsized":"2024-04-23T18:26:08.229648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:50.930321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123153+0000","last_change":"2024-04-23T18:23:56.098674+0000","last_active":"2024-04-23T18:26:08.123153+0000","last_peered":"2024-04-23T18:26:08.123153+0000","last_clean":"2024-04-23T18:26:08.123153+0000","last_became_active":"2024-04-23T18:23:56.098299+0000","last_became_peered":"2024-04-23T18:23:56.098299+0000","last_unstale":"2024-04-23T18:26:08.123153+0000","last_undegraded":"2024-04-23T18:26:08.123153+0000","last_fullsized":"2024-04-23T18:26:08.123153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:05.711410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218497+0000","last_change":"2024-04-23T18:23:56.097437+0000","last_active":"2024-04-23T18:26:08.218497+0000","last_peered":"2024-04-23T18:26:08.218497+0000","last_clean":"2024-04-23T18:26:08.218497+0000","last_became_active":"2024-04-23T18:23:56.095702+0000","last_became_peered":"2024-04-23T18:23:56.095702+0000","last_unstale":"2024-04-23T18:26:08.218497+0000","last_undegraded":"2024-04-23T18:26:08.218497+0000","last_fullsized":"2024-04-23T18:26:08.218497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:35.754832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293226+0000","last_change":"2024-04-23T18:23:56.096060+0000","last_active":"2024-04-23T18:26:08.293226+0000","last_peered":"2024-04-23T18:26:08.293226+0000","last_clean":"2024-04-23T18:26:08.293226+0000","last_became_active":"2024-04-23T18:23:56.095597+0000","last_became_peered":"2024-04-23T18:23:56.095597+0000","last_unstale":"2024-04-23T18:26:08.293226+0000","last_undegraded":"2024-04-23T18:26:08.293226+0000","last_fullsized":"2024-04-23T18:26:08.293226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:59.735347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229983+0000","last_change":"2024-04-23T18:23:56.109033+0000","last_active":"2024-04-23T18:26:08.229983+0000","last_peered":"2024-04-23T18:26:08.229983+0000","last_clean":"2024-04-23T18:26:08.229983+0000","last_became_active":"2024-04-23T18:23:56.108472+0000","last_became_peered":"2024-04-23T18:23:56.108472+0000","last_unstale":"2024-04-23T18:26:08.229983+0000","last_undegraded":"2024-04-23T18:26:08.229983+0000","last_fullsized":"2024-04-23T18:26:08.229983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:32.491181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123484+0000","last_change":"2024-04-23T18:23:56.097771+0000","last_active":"2024-04-23T18:26:08.123484+0000","last_peered":"2024-04-23T18:26:08.123484+0000","last_clean":"2024-04-23T18:26:08.123484+0000","last_became_active":"2024-04-23T18:23:56.097403+0000","last_became_peered":"2024-04-23T18:23:56.097403+0000","last_unstale":"2024-04-23T18:26:08.123484+0000","last_undegraded":"2024-04-23T18:26:08.123484+0000","last_fullsized":"2024-04-23T18:26:08.123484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:48:36.261543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293091+0000","last_change":"2024-04-23T18:23:56.108550+0000","last_active":"2024-04-23T18:26:08.293091+0000","last_peered":"2024-04-23T18:26:08.293091+0000","last_clean":"2024-04-23T18:26:08.293091+0000","last_became_active":"2024-04-23T18:23:56.108191+0000","last_became_peered":"2024-04-23T18:23:56.108191+0000","last_unstale":"2024-04-23T18:26:08.293091+0000","last_undegraded":"2024-04-23T18:26:08.293091+0000","last_fullsized":"2024-04-23T18:26:08.293091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:25.142466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293119+0000","last_change":"2024-04-23T18:23:56.092573+0000","last_active":"2024-04-23T18:26:08.293119+0000","last_peered":"2024-04-23T18:26:08.293119+0000","last_clean":"2024-04-23T18:26:08.293119+0000","last_became_active":"2024-04-23T18:23:56.092095+0000","last_became_peered":"2024-04-23T18:23:56.092095+0000","last_unstale":"2024-04-23T18:26:08.293119+0000","last_undegraded":"2024-04-23T18:26:08.293119+0000","last_fullsized":"2024-04-23T18:26:08.293119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:01.601206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292816+0000","last_change":"2024-04-23T18:23:56.102073+0000","last_active":"2024-04-23T18:26:08.292816+0000","last_peered":"2024-04-23T18:26:08.292816+0000","last_clean":"2024-04-23T18:26:08.292816+0000","last_became_active":"2024-04-23T18:23:56.101059+0000","last_became_peered":"2024-04-23T18:23:56.101059+0000","last_unstale":"2024-04-23T18:26:08.292816+0000","last_undegraded":"2024-04-23T18:26:08.292816+0000","last_fullsized":"2024-04-23T18:26:08.292816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:37.904099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292355+0000","last_change":"2024-04-23T18:23:56.108482+0000","last_active":"2024-04-23T18:26:08.292355+0000","last_peered":"2024-04-23T18:26:08.292355+0000","last_clean":"2024-04-23T18:26:08.292355+0000","last_became_active":"2024-04-23T18:23:56.108128+0000","last_became_peered":"2024-04-23T18:23:56.108128+0000","last_unstale":"2024-04-23T18:26:08.292355+0000","last_undegraded":"2024-04-23T18:26:08.292355+0000","last_fullsized":"2024-04-23T18:26:08.292355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:51.925881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292324+0000","last_change":"2024-04-23T18:23:56.099691+0000","last_active":"2024-04-23T18:26:08.292324+0000","last_peered":"2024-04-23T18:26:08.292324+0000","last_clean":"2024-04-23T18:26:08.292324+0000","last_became_active":"2024-04-23T18:23:56.099272+0000","last_became_peered":"2024-04-23T18:23:56.099272+0000","last_unstale":"2024-04-23T18:26:08.292324+0000","last_undegraded":"2024-04-23T18:26:08.292324+0000","last_fullsized":"2024-04-23T18:26:08.292324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:41.834104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292954+0000","last_change":"2024-04-23T18:23:56.097965+0000","last_active":"2024-04-23T18:26:08.292954+0000","last_peered":"2024-04-23T18:26:08.292954+0000","last_clean":"2024-04-23T18:26:08.292954+0000","last_became_active":"2024-04-23T18:23:56.096766+0000","last_became_peered":"2024-04-23T18:23:56.096766+0000","last_unstale":"2024-04-23T18:26:08.292954+0000","last_undegraded":"2024-04-23T18:26:08.292954+0000","last_fullsized":"2024-04-23T18:26:08.292954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:40:00.076939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122351+0000","last_change":"2024-04-23T18:23:56.095158+0000","last_active":"2024-04-23T18:26:08.122351+0000","last_peered":"2024-04-23T18:26:08.122351+0000","last_clean":"2024-04-23T18:26:08.122351+0000","last_became_active":"2024-04-23T18:23:56.094812+0000","last_became_peered":"2024-04-23T18:23:56.094812+0000","last_unstale":"2024-04-23T18:26:08.122351+0000","last_undegraded":"2024-04-23T18:26:08.122351+0000","last_fullsized":"2024-04-23T18:26:08.122351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:40.874698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.126943+0000","last_change":"2024-04-23T18:23:56.099358+0000","last_active":"2024-04-23T18:26:08.126943+0000","last_peered":"2024-04-23T18:26:08.126943+0000","last_clean":"2024-04-23T18:26:08.126943+0000","last_became_active":"2024-04-23T18:23:56.099031+0000","last_became_peered":"2024-04-23T18:23:56.099031+0000","last_unstale":"2024-04-23T18:26:08.126943+0000","last_undegraded":"2024-04-23T18:26:08.126943+0000","last_fullsized":"2024-04-23T18:26:08.126943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:29.634932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.805802+0000","last_change":"2024-04-23T18:23:55.262939+0000","last_active":"2024-04-23T18:26:08.805802+0000","last_peered":"2024-04-23T18:26:08.805802+0000","last_clean":"2024-04-23T18:26:08.805802+0000","last_became_active":"2024-04-23T18:23:55.262609+0000","last_became_peered":"2024-04-23T18:23:55.262609+0000","last_unstale":"2024-04-23T18:26:08.805802+0000","last_undegraded":"2024-04-23T18:26:08.805802+0000","last_fullsized":"2024-04-23T18:26:08.805802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:12.418349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218972+0000","last_change":"2024-04-23T18:23:56.092182+0000","last_active":"2024-04-23T18:26:08.218972+0000","last_peered":"2024-04-23T18:26:08.218972+0000","last_clean":"2024-04-23T18:26:08.218972+0000","last_became_active":"2024-04-23T18:23:56.091444+0000","last_became_peered":"2024-04-23T18:23:56.091444+0000","last_unstale":"2024-04-23T18:26:08.218972+0000","last_undegraded":"2024-04-23T18:26:08.218972+0000","last_fullsized":"2024-04-23T18:26:08.218972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:00.775498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219056+0000","last_change":"2024-04-23T18:23:55.259880+0000","last_active":"2024-04-23T18:26:08.219056+0000","last_peered":"2024-04-23T18:26:08.219056+0000","last_clean":"2024-04-23T18:26:08.219056+0000","last_became_active":"2024-04-23T18:23:55.259576+0000","last_became_peered":"2024-04-23T18:23:55.259576+0000","last_unstale":"2024-04-23T18:26:08.219056+0000","last_undegraded":"2024-04-23T18:26:08.219056+0000","last_fullsized":"2024-04-23T18:26:08.219056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:13.783776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293588+0000","last_change":"2024-04-23T18:23:56.094307+0000","last_active":"2024-04-23T18:26:08.293588+0000","last_peered":"2024-04-23T18:26:08.293588+0000","last_clean":"2024-04-23T18:26:08.293588+0000","last_became_active":"2024-04-23T18:23:56.093893+0000","last_became_peered":"2024-04-23T18:23:56.093893+0000","last_unstale":"2024-04-23T18:26:08.293588+0000","last_undegraded":"2024-04-23T18:26:08.293588+0000","last_fullsized":"2024-04-23T18:26:08.293588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:12:35.385497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219326+0000","last_change":"2024-04-23T18:23:55.263137+0000","last_active":"2024-04-23T18:26:08.219326+0000","last_peered":"2024-04-23T18:26:08.219326+0000","last_clean":"2024-04-23T18:26:08.219326+0000","last_became_active":"2024-04-23T18:23:55.262553+0000","last_became_peered":"2024-04-23T18:23:55.262553+0000","last_unstale":"2024-04-23T18:26:08.219326+0000","last_undegraded":"2024-04-23T18:26:08.219326+0000","last_fullsized":"2024-04-23T18:26:08.219326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:33:42.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292576+0000","last_change":"2024-04-23T18:23:56.091098+0000","last_active":"2024-04-23T18:26:08.292576+0000","last_peered":"2024-04-23T18:26:08.292576+0000","last_clean":"2024-04-23T18:26:08.292576+0000","last_became_active":"2024-04-23T18:23:56.090635+0000","last_became_peered":"2024-04-23T18:23:56.090635+0000","last_unstale":"2024-04-23T18:26:08.292576+0000","last_undegraded":"2024-04-23T18:26:08.292576+0000","last_fullsized":"2024-04-23T18:26:08.292576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:17.787937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219108+0000","last_change":"2024-04-23T18:23:55.262146+0000","last_active":"2024-04-23T18:26:08.219108+0000","last_peered":"2024-04-23T18:26:08.219108+0000","last_clean":"2024-04-23T18:26:08.219108+0000","last_became_active":"2024-04-23T18:23:55.261838+0000","last_became_peered":"2024-04-23T18:23:55.261838+0000","last_unstale":"2024-04-23T18:26:08.219108+0000","last_undegraded":"2024-04-23T18:26:08.219108+0000","last_fullsized":"2024-04-23T18:26:08.219108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:42:39.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122228+0000","last_change":"2024-04-23T18:24:36.915394+0000","last_active":"2024-04-23T18:26:08.122228+0000","last_peered":"2024-04-23T18:26:08.122228+0000","last_clean":"2024-04-23T18:26:08.122228+0000","last_became_active":"2024-04-23T18:24:36.915072+0000","last_became_peered":"2024-04-23T18:24:36.915072+0000","last_unstale":"2024-04-23T18:26:08.122228+0000","last_undegraded":"2024-04-23T18:26:08.122228+0000","last_fullsized":"2024-04-23T18:26:08.122228+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.132410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226797+0000","last_change":"2024-04-23T18:23:55.261399+0000","last_active":"2024-04-23T18:26:08.226797+0000","last_peered":"2024-04-23T18:26:08.226797+0000","last_clean":"2024-04-23T18:26:08.226797+0000","last_became_active":"2024-04-23T18:23:55.261012+0000","last_became_peered":"2024-04-23T18:23:55.261012+0000","last_unstale":"2024-04-23T18:26:08.226797+0000","last_undegraded":"2024-04-23T18:26:08.226797+0000","last_fullsized":"2024-04-23T18:26:08.226797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:31.377776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218869+0000","last_change":"2024-04-23T18:23:56.096785+0000","last_active":"2024-04-23T18:26:08.218869+0000","last_peered":"2024-04-23T18:26:08.218869+0000","last_clean":"2024-04-23T18:26:08.218869+0000","last_became_active":"2024-04-23T18:23:56.096420+0000","last_became_peered":"2024-04-23T18:23:56.096420+0000","last_unstale":"2024-04-23T18:26:08.218869+0000","last_undegraded":"2024-04-23T18:26:08.218869+0000","last_fullsized":"2024-04-23T18:26:08.218869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:20:49.215071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229615+0000","last_change":"2024-04-23T18:23:55.260645+0000","last_active":"2024-04-23T18:26:08.229615+0000","last_peered":"2024-04-23T18:26:08.229615+0000","last_clean":"2024-04-23T18:26:08.229615+0000","last_became_active":"2024-04-23T18:23:55.260118+0000","last_became_peered":"2024-04-23T18:23:55.260118+0000","last_unstale":"2024-04-23T18:26:08.229615+0000","last_undegraded":"2024-04-23T18:26:08.229615+0000","last_fullsized":"2024-04-23T18:26:08.229615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:14:12.066297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226773+0000","last_change":"2024-04-23T18:24:36.702766+0000","last_active":"2024-04-23T18:26:08.226773+0000","last_peered":"2024-04-23T18:26:08.226773+0000","last_clean":"2024-04-23T18:26:08.226773+0000","last_became_active":"2024-04-23T18:24:36.702309+0000","last_became_peered":"2024-04-23T18:24:36.702309+0000","last_unstale":"2024-04-23T18:26:08.226773+0000","last_undegraded":"2024-04-23T18:26:08.226773+0000","last_fullsized":"2024-04-23T18:26:08.226773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:10.803924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292712+0000","last_change":"2024-04-23T18:23:55.258636+0000","last_active":"2024-04-23T18:26:08.292712+0000","last_peered":"2024-04-23T18:26:08.292712+0000","last_clean":"2024-04-23T18:26:08.292712+0000","last_became_active":"2024-04-23T18:23:55.257744+0000","last_became_peered":"2024-04-23T18:23:55.257744+0000","last_unstale":"2024-04-23T18:26:08.292712+0000","last_undegraded":"2024-04-23T18:26:08.292712+0000","last_fullsized":"2024-04-23T18:26:08.292712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:46:07.454725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218571+0000","last_change":"2024-04-23T18:23:56.094822+0000","last_active":"2024-04-23T18:26:08.218571+0000","last_peered":"2024-04-23T18:26:08.218571+0000","last_clean":"2024-04-23T18:26:08.218571+0000","last_became_active":"2024-04-23T18:23:56.094489+0000","last_became_peered":"2024-04-23T18:23:56.094489+0000","last_unstale":"2024-04-23T18:26:08.218571+0000","last_undegraded":"2024-04-23T18:26:08.218571+0000","last_fullsized":"2024-04-23T18:26:08.218571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:21.234363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226519+0000","last_change":"2024-04-23T18:23:55.261322+0000","last_active":"2024-04-23T18:26:08.226519+0000","last_peered":"2024-04-23T18:26:08.226519+0000","last_clean":"2024-04-23T18:26:08.226519+0000","last_became_active":"2024-04-23T18:23:55.260990+0000","last_became_peered":"2024-04-23T18:23:55.260990+0000","last_unstale":"2024-04-23T18:26:08.226519+0000","last_undegraded":"2024-04-23T18:26:08.226519+0000","last_fullsized":"2024-04-23T18:26:08.226519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:31.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122279+0000","last_change":"2024-04-23T18:23:56.093105+0000","last_active":"2024-04-23T18:26:08.122279+0000","last_peered":"2024-04-23T18:26:08.122279+0000","last_clean":"2024-04-23T18:26:08.122279+0000","last_became_active":"2024-04-23T18:23:56.092717+0000","last_became_peered":"2024-04-23T18:23:56.092717+0000","last_unstale":"2024-04-23T18:26:08.122279+0000","last_undegraded":"2024-04-23T18:26:08.122279+0000","last_fullsized":"2024-04-23T18:26:08.122279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:13:59.574057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":281,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.807789+0000","last_change":"2024-04-23T18:23:55.256307+0000","last_active":"2024-04-23T18:26:08.807789+0000","last_peered":"2024-04-23T18:26:08.807789+0000","last_clean":"2024-04-23T18:26:08.807789+0000","last_became_active":"2024-04-23T18:23:55.255959+0000","last_became_peered":"2024-04-23T18:23:55.255959+0000","last_unstale":"2024-04-23T18:26:08.807789+0000","last_undegraded":"2024-04-23T18:26:08.807789+0000","last_fullsized":"2024-04-23T18:26:08.807789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:04.945919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293141+0000","last_change":"2024-04-23T18:23:56.095177+0000","last_active":"2024-04-23T18:26:08.293141+0000","last_peered":"2024-04-23T18:26:08.293141+0000","last_clean":"2024-04-23T18:26:08.293141+0000","last_became_active":"2024-04-23T18:23:56.094825+0000","last_became_peered":"2024-04-23T18:23:56.094825+0000","last_unstale":"2024-04-23T18:26:08.293141+0000","last_undegraded":"2024-04-23T18:26:08.293141+0000","last_fullsized":"2024-04-23T18:26:08.293141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:07:33.499817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123551+0000","last_change":"2024-04-23T18:23:55.264258+0000","last_active":"2024-04-23T18:26:08.123551+0000","last_peered":"2024-04-23T18:26:08.123551+0000","last_clean":"2024-04-23T18:26:08.123551+0000","last_became_active":"2024-04-23T18:23:55.263698+0000","last_became_peered":"2024-04-23T18:23:55.263698+0000","last_unstale":"2024-04-23T18:26:08.123551+0000","last_undegraded":"2024-04-23T18:26:08.123551+0000","last_fullsized":"2024-04-23T18:26:08.123551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:00.566948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293683+0000","last_change":"2024-04-23T18:23:56.109085+0000","last_active":"2024-04-23T18:26:08.293683+0000","last_peered":"2024-04-23T18:26:08.293683+0000","last_clean":"2024-04-23T18:26:08.293683+0000","last_became_active":"2024-04-23T18:23:56.108033+0000","last_became_peered":"2024-04-23T18:23:56.108033+0000","last_unstale":"2024-04-23T18:26:08.293683+0000","last_undegraded":"2024-04-23T18:26:08.293683+0000","last_fullsized":"2024-04-23T18:26:08.293683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:59.980512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226931+0000","last_change":"2024-04-23T18:23:55.256728+0000","last_active":"2024-04-23T18:26:08.226931+0000","last_peered":"2024-04-23T18:26:08.226931+0000","last_clean":"2024-04-23T18:26:08.226931+0000","last_became_active":"2024-04-23T18:23:55.256367+0000","last_became_peered":"2024-04-23T18:23:55.256367+0000","last_unstale":"2024-04-23T18:26:08.226931+0000","last_undegraded":"2024-04-23T18:26:08.226931+0000","last_fullsized":"2024-04-23T18:26:08.226931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:08.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292666+0000","last_change":"2024-04-23T18:23:56.102898+0000","last_active":"2024-04-23T18:26:08.292666+0000","last_peered":"2024-04-23T18:26:08.292666+0000","last_clean":"2024-04-23T18:26:08.292666+0000","last_became_active":"2024-04-23T18:23:56.102373+0000","last_became_peered":"2024-04-23T18:23:56.102373+0000","last_unstale":"2024-04-23T18:26:08.292666+0000","last_undegraded":"2024-04-23T18:26:08.292666+0000","last_fullsized":"2024-04-23T18:26:08.292666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:49.316219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'37","reported_seq":322,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.808335+0000","last_change":"2024-04-23T18:23:55.255094+0000","last_active":"2024-04-23T18:26:08.808335+0000","last_peered":"2024-04-23T18:26:08.808335+0000","last_clean":"2024-04-23T18:26:08.808335+0000","last_became_active":"2024-04-23T18:23:55.254726+0000","last_became_peered":"2024-04-23T18:23:55.254726+0000","last_unstale":"2024-04-23T18:26:08.808335+0000","last_undegraded":"2024-04-23T18:26:08.808335+0000","last_fullsized":"2024-04-23T18:26:08.808335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:26.988184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":264,"num_read_kb":200,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219090+0000","last_change":"2024-04-23T18:23:56.099866+0000","last_active":"2024-04-23T18:26:08.219090+0000","last_peered":"2024-04-23T18:26:08.219090+0000","last_clean":"2024-04-23T18:26:08.219090+0000","last_became_active":"2024-04-23T18:23:56.098606+0000","last_became_peered":"2024-04-23T18:23:56.098606+0000","last_unstale":"2024-04-23T18:26:08.219090+0000","last_undegraded":"2024-04-23T18:26:08.219090+0000","last_fullsized":"2024-04-23T18:26:08.219090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:26.661857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292679+0000","last_change":"2024-04-23T18:24:36.710791+0000","last_active":"2024-04-23T18:26:08.292679+0000","last_peered":"2024-04-23T18:26:08.292679+0000","last_clean":"2024-04-23T18:26:08.292679+0000","last_became_active":"2024-04-23T18:24:36.697731+0000","last_became_peered":"2024-04-23T18:24:36.697731+0000","last_unstale":"2024-04-23T18:26:08.292679+0000","last_undegraded":"2024-04-23T18:26:08.292679+0000","last_fullsized":"2024-04-23T18:26:08.292679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:16:46.722132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292845+0000","last_change":"2024-04-23T18:23:56.099109+0000","last_active":"2024-04-23T18:26:08.292845+0000","last_peered":"2024-04-23T18:26:08.292845+0000","last_clean":"2024-04-23T18:26:08.292845+0000","last_became_active":"2024-04-23T18:23:56.098608+0000","last_became_peered":"2024-04-23T18:23:56.098608+0000","last_unstale":"2024-04-23T18:26:08.292845+0000","last_undegraded":"2024-04-23T18:26:08.292845+0000","last_fullsized":"2024-04-23T18:26:08.292845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:43.720160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226641+0000","last_change":"2024-04-23T18:23:55.262523+0000","last_active":"2024-04-23T18:26:08.226641+0000","last_peered":"2024-04-23T18:26:08.226641+0000","last_clean":"2024-04-23T18:26:08.226641+0000","last_became_active":"2024-04-23T18:23:55.262216+0000","last_became_peered":"2024-04-23T18:23:55.262216+0000","last_unstale":"2024-04-23T18:26:08.226641+0000","last_undegraded":"2024-04-23T18:26:08.226641+0000","last_fullsized":"2024-04-23T18:26:08.226641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:35.268435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123437+0000","last_change":"2024-04-23T18:23:56.106573+0000","last_active":"2024-04-23T18:26:08.123437+0000","last_peered":"2024-04-23T18:26:08.123437+0000","last_clean":"2024-04-23T18:26:08.123437+0000","last_became_active":"2024-04-23T18:23:56.106181+0000","last_became_peered":"2024-04-23T18:23:56.106181+0000","last_unstale":"2024-04-23T18:26:08.123437+0000","last_undegraded":"2024-04-23T18:26:08.123437+0000","last_fullsized":"2024-04-23T18:26:08.123437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:15.422647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'9","reported_seq":162,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.811881+0000","last_change":"2024-04-23T18:23:55.262813+0000","last_active":"2024-04-23T18:26:08.811881+0000","last_peered":"2024-04-23T18:26:08.811881+0000","last_clean":"2024-04-23T18:26:08.811881+0000","last_became_active":"2024-04-23T18:23:55.262365+0000","last_became_peered":"2024-04-23T18:23:55.262365+0000","last_unstale":"2024-04-23T18:26:08.811881+0000","last_undegraded":"2024-04-23T18:26:08.811881+0000","last_fullsized":"2024-04-23T18:26:08.811881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:15.234355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1819,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123074+0000","last_change":"2024-04-23T18:23:56.101491+0000","last_active":"2024-04-23T18:26:08.123074+0000","last_peered":"2024-04-23T18:26:08.123074+0000","last_clean":"2024-04-23T18:26:08.123074+0000","last_became_active":"2024-04-23T18:23:56.101069+0000","last_became_peered":"2024-04-23T18:23:56.101069+0000","last_unstale":"2024-04-23T18:26:08.123074+0000","last_undegraded":"2024-04-23T18:26:08.123074+0000","last_fullsized":"2024-04-23T18:26:08.123074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:42.484653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218843+0000","last_change":"2024-04-23T18:24:36.750105+0000","last_active":"2024-04-23T18:26:08.218843+0000","last_peered":"2024-04-23T18:26:08.218843+0000","last_clean":"2024-04-23T18:26:08.218843+0000","last_became_active":"2024-04-23T18:24:36.738326+0000","last_became_peered":"2024-04-23T18:24:36.738326+0000","last_unstale":"2024-04-23T18:26:08.218843+0000","last_undegraded":"2024-04-23T18:26:08.218843+0000","last_fullsized":"2024-04-23T18:26:08.218843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:07.414513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226554+0000","last_change":"2024-04-23T18:23:56.105691+0000","last_active":"2024-04-23T18:26:08.226554+0000","last_peered":"2024-04-23T18:26:08.226554+0000","last_clean":"2024-04-23T18:26:08.226554+0000","last_became_active":"2024-04-23T18:23:56.105205+0000","last_became_peered":"2024-04-23T18:23:56.105205+0000","last_unstale":"2024-04-23T18:26:08.226554+0000","last_undegraded":"2024-04-23T18:26:08.226554+0000","last_fullsized":"2024-04-23T18:26:08.226554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:30.779821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219253+0000","last_change":"2024-04-23T18:23:55.261198+0000","last_active":"2024-04-23T18:26:08.219253+0000","last_peered":"2024-04-23T18:26:08.219253+0000","last_clean":"2024-04-23T18:26:08.219253+0000","last_became_active":"2024-04-23T18:23:55.260884+0000","last_became_peered":"2024-04-23T18:23:55.260884+0000","last_unstale":"2024-04-23T18:26:08.219253+0000","last_undegraded":"2024-04-23T18:26:08.219253+0000","last_fullsized":"2024-04-23T18:26:08.219253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:00.841671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292792+0000","last_change":"2024-04-23T18:23:56.099450+0000","last_active":"2024-04-23T18:26:08.292792+0000","last_peered":"2024-04-23T18:26:08.292792+0000","last_clean":"2024-04-23T18:26:08.292792+0000","last_became_active":"2024-04-23T18:23:56.098723+0000","last_became_peered":"2024-04-23T18:23:56.098723+0000","last_unstale":"2024-04-23T18:26:08.292792+0000","last_undegraded":"2024-04-23T18:26:08.292792+0000","last_fullsized":"2024-04-23T18:26:08.292792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:53.027495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122351+0000","last_change":"2024-04-23T18:23:55.262030+0000","last_active":"2024-04-23T18:26:08.122351+0000","last_peered":"2024-04-23T18:26:08.122351+0000","last_clean":"2024-04-23T18:26:08.122351+0000","last_became_active":"2024-04-23T18:23:55.261659+0000","last_became_peered":"2024-04-23T18:23:55.261659+0000","last_unstale":"2024-04-23T18:26:08.122351+0000","last_undegraded":"2024-04-23T18:26:08.122351+0000","last_fullsized":"2024-04-23T18:26:08.122351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:17.036709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.230040+0000","last_change":"2024-04-23T18:23:56.096511+0000","last_active":"2024-04-23T18:26:08.230040+0000","last_peered":"2024-04-23T18:26:08.230040+0000","last_clean":"2024-04-23T18:26:08.230040+0000","last_became_active":"2024-04-23T18:23:56.096154+0000","last_became_peered":"2024-04-23T18:23:56.096154+0000","last_unstale":"2024-04-23T18:26:08.230040+0000","last_undegraded":"2024-04-23T18:26:08.230040+0000","last_fullsized":"2024-04-23T18:26:08.230040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:03.926178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.806819+0000","last_change":"2024-04-23T18:23:55.263618+0000","last_active":"2024-04-23T18:26:08.806819+0000","last_peered":"2024-04-23T18:26:08.806819+0000","last_clean":"2024-04-23T18:26:08.806819+0000","last_became_active":"2024-04-23T18:23:55.261758+0000","last_became_peered":"2024-04-23T18:23:55.261758+0000","last_unstale":"2024-04-23T18:26:08.806819+0000","last_undegraded":"2024-04-23T18:26:08.806819+0000","last_fullsized":"2024-04-23T18:26:08.806819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:23:27.745668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229933+0000","last_change":"2024-04-23T18:23:56.109292+0000","last_active":"2024-04-23T18:26:08.229933+0000","last_peered":"2024-04-23T18:26:08.229933+0000","last_clean":"2024-04-23T18:26:08.229933+0000","last_became_active":"2024-04-23T18:23:56.108414+0000","last_became_peered":"2024-04-23T18:23:56.108414+0000","last_unstale":"2024-04-23T18:26:08.229933+0000","last_undegraded":"2024-04-23T18:26:08.229933+0000","last_fullsized":"2024-04-23T18:26:08.229933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:13.428782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226673+0000","last_change":"2024-04-23T18:23:56.097122+0000","last_active":"2024-04-23T18:26:08.226673+0000","last_peered":"2024-04-23T18:26:08.226673+0000","last_clean":"2024-04-23T18:26:08.226673+0000","last_became_active":"2024-04-23T18:23:56.096751+0000","last_became_peered":"2024-04-23T18:23:56.096751+0000","last_unstale":"2024-04-23T18:26:08.226673+0000","last_undegraded":"2024-04-23T18:26:08.226673+0000","last_fullsized":"2024-04-23T18:26:08.226673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:00:55.090370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":157,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.811347+0000","last_change":"2024-04-23T18:23:55.256458+0000","last_active":"2024-04-23T18:26:08.811347+0000","last_peered":"2024-04-23T18:26:08.811347+0000","last_clean":"2024-04-23T18:26:08.811347+0000","last_became_active":"2024-04-23T18:23:55.256127+0000","last_became_peered":"2024-04-23T18:23:55.256127+0000","last_unstale":"2024-04-23T18:26:08.811347+0000","last_undegraded":"2024-04-23T18:26:08.811347+0000","last_fullsized":"2024-04-23T18:26:08.811347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:03.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218803+0000","last_change":"2024-04-23T18:23:56.103743+0000","last_active":"2024-04-23T18:26:08.218803+0000","last_peered":"2024-04-23T18:26:08.218803+0000","last_clean":"2024-04-23T18:26:08.218803+0000","last_became_active":"2024-04-23T18:23:56.103041+0000","last_became_peered":"2024-04-23T18:23:56.103041+0000","last_unstale":"2024-04-23T18:26:08.218803+0000","last_undegraded":"2024-04-23T18:26:08.218803+0000","last_fullsized":"2024-04-23T18:26:08.218803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:22.419555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292990+0000","last_change":"2024-04-23T18:23:55.257008+0000","last_active":"2024-04-23T18:26:08.292990+0000","last_peered":"2024-04-23T18:26:08.292990+0000","last_clean":"2024-04-23T18:26:08.292990+0000","last_became_active":"2024-04-23T18:23:55.256648+0000","last_became_peered":"2024-04-23T18:23:55.256648+0000","last_unstale":"2024-04-23T18:26:08.292990+0000","last_undegraded":"2024-04-23T18:26:08.292990+0000","last_fullsized":"2024-04-23T18:26:08.292990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:00.867083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123176+0000","last_change":"2024-04-23T18:22:38.610948+0000","last_active":"2024-04-23T18:26:08.123176+0000","last_peered":"2024-04-23T18:26:08.123176+0000","last_clean":"2024-04-23T18:26:08.123176+0000","last_became_active":"2024-04-23T18:22:38.610475+0000","last_became_peered":"2024-04-23T18:22:38.610475+0000","last_unstale":"2024-04-23T18:26:08.123176+0000","last_undegraded":"2024-04-23T18:26:08.123176+0000","last_fullsized":"2024-04-23T18:26:08.123176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_clean_scrub_stamp":"2024-04-23T18:19:07.980519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:08.209087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292611+0000","last_change":"2024-04-23T18:24:36.699127+0000","last_active":"2024-04-23T18:26:08.292611+0000","last_peered":"2024-04-23T18:26:08.292611+0000","last_clean":"2024-04-23T18:26:08.292611+0000","last_became_active":"2024-04-23T18:24:36.698734+0000","last_became_peered":"2024-04-23T18:24:36.698734+0000","last_unstale":"2024-04-23T18:26:08.292611+0000","last_undegraded":"2024-04-23T18:26:08.292611+0000","last_fullsized":"2024-04-23T18:26:08.292611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:28.239466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123284+0000","last_change":"2024-04-23T18:24:36.918722+0000","last_active":"2024-04-23T18:26:08.123284+0000","last_peered":"2024-04-23T18:26:08.123284+0000","last_clean":"2024-04-23T18:26:08.123284+0000","last_became_active":"2024-04-23T18:24:36.915658+0000","last_became_peered":"2024-04-23T18:24:36.915658+0000","last_unstale":"2024-04-23T18:26:08.123284+0000","last_undegraded":"2024-04-23T18:26:08.123284+0000","last_fullsized":"2024-04-23T18:26:08.123284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:34.706193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219189+0000","last_change":"2024-04-23T18:23:55.265853+0000","last_active":"2024-04-23T18:26:08.219189+0000","last_peered":"2024-04-23T18:26:08.219189+0000","last_clean":"2024-04-23T18:26:08.219189+0000","last_became_active":"2024-04-23T18:23:55.265435+0000","last_became_peered":"2024-04-23T18:23:55.265435+0000","last_unstale":"2024-04-23T18:26:08.219189+0000","last_undegraded":"2024-04-23T18:26:08.219189+0000","last_fullsized":"2024-04-23T18:26:08.219189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.969720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293278+0000","last_change":"2024-04-23T18:23:56.085609+0000","last_active":"2024-04-23T18:26:08.293278+0000","last_peered":"2024-04-23T18:26:08.293278+0000","last_clean":"2024-04-23T18:26:08.293278+0000","last_became_active":"2024-04-23T18:23:56.085244+0000","last_became_peered":"2024-04-23T18:23:56.085244+0000","last_unstale":"2024-04-23T18:26:08.293278+0000","last_undegraded":"2024-04-23T18:26:08.293278+0000","last_fullsized":"2024-04-23T18:26:08.293278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.211717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'48","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.341583+0000","last_change":"2024-04-23T18:23:55.260260+0000","last_active":"2024-04-23T18:26:08.341583+0000","last_peered":"2024-04-23T18:26:08.341583+0000","last_clean":"2024-04-23T18:26:08.341583+0000","last_became_active":"2024-04-23T18:23:55.259862+0000","last_became_peered":"2024-04-23T18:23:55.259862+0000","last_unstale":"2024-04-23T18:26:08.341583+0000","last_undegraded":"2024-04-23T18:26:08.341583+0000","last_fullsized":"2024-04-23T18:26:08.341583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:57.833293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219052+0000","last_change":"2024-04-23T18:23:56.098826+0000","last_active":"2024-04-23T18:26:08.219052+0000","last_peered":"2024-04-23T18:26:08.219052+0000","last_clean":"2024-04-23T18:26:08.219052+0000","last_became_active":"2024-04-23T18:23:56.098494+0000","last_became_peered":"2024-04-23T18:23:56.098494+0000","last_unstale":"2024-04-23T18:26:08.219052+0000","last_undegraded":"2024-04-23T18:26:08.219052+0000","last_fullsized":"2024-04-23T18:26:08.219052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:12:24.953330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293238+0000","last_change":"2024-04-23T18:23:55.258174+0000","last_active":"2024-04-23T18:26:08.293238+0000","last_peered":"2024-04-23T18:26:08.293238+0000","last_clean":"2024-04-23T18:26:08.293238+0000","last_became_active":"2024-04-23T18:23:55.257846+0000","last_became_peered":"2024-04-23T18:23:55.257846+0000","last_unstale":"2024-04-23T18:26:08.293238+0000","last_undegraded":"2024-04-23T18:26:08.293238+0000","last_fullsized":"2024-04-23T18:26:08.293238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:35.131566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293829+0000","last_change":"2024-04-23T18:23:56.110758+0000","last_active":"2024-04-23T18:26:08.293829+0000","last_peered":"2024-04-23T18:26:08.293829+0000","last_clean":"2024-04-23T18:26:08.293829+0000","last_became_active":"2024-04-23T18:23:56.110372+0000","last_became_peered":"2024-04-23T18:23:56.110372+0000","last_unstale":"2024-04-23T18:26:08.293829+0000","last_undegraded":"2024-04-23T18:26:08.293829+0000","last_fullsized":"2024-04-23T18:26:08.293829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:40.193088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292460+0000","last_change":"2024-04-23T18:23:55.265714+0000","last_active":"2024-04-23T18:26:08.292460+0000","last_peered":"2024-04-23T18:26:08.292460+0000","last_clean":"2024-04-23T18:26:08.292460+0000","last_became_active":"2024-04-23T18:23:55.265198+0000","last_became_peered":"2024-04-23T18:23:55.265198+0000","last_unstale":"2024-04-23T18:26:08.292460+0000","last_undegraded":"2024-04-23T18:26:08.292460+0000","last_fullsized":"2024-04-23T18:26:08.292460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:10.770748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218909+0000","last_change":"2024-04-23T18:23:56.102347+0000","last_active":"2024-04-23T18:26:08.218909+0000","last_peered":"2024-04-23T18:26:08.218909+0000","last_clean":"2024-04-23T18:26:08.218909+0000","last_became_active":"2024-04-23T18:23:56.100687+0000","last_became_peered":"2024-04-23T18:23:56.100687+0000","last_unstale":"2024-04-23T18:26:08.218909+0000","last_undegraded":"2024-04-23T18:26:08.218909+0000","last_fullsized":"2024-04-23T18:26:08.218909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:20.378730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218670+0000","last_change":"2024-04-23T18:23:55.255943+0000","last_active":"2024-04-23T18:26:08.218670+0000","last_peered":"2024-04-23T18:26:08.218670+0000","last_clean":"2024-04-23T18:26:08.218670+0000","last_became_active":"2024-04-23T18:23:55.255618+0000","last_became_peered":"2024-04-23T18:23:55.255618+0000","last_unstale":"2024-04-23T18:26:08.218670+0000","last_undegraded":"2024-04-23T18:26:08.218670+0000","last_fullsized":"2024-04-23T18:26:08.218670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:01.408533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292542+0000","last_change":"2024-04-23T18:23:56.091664+0000","last_active":"2024-04-23T18:26:08.292542+0000","last_peered":"2024-04-23T18:26:08.292542+0000","last_clean":"2024-04-23T18:26:08.292542+0000","last_became_active":"2024-04-23T18:23:56.091308+0000","last_became_peered":"2024-04-23T18:23:56.091308+0000","last_unstale":"2024-04-23T18:26:08.292542+0000","last_undegraded":"2024-04-23T18:26:08.292542+0000","last_fullsized":"2024-04-23T18:26:08.292542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:31.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.405535+0000","last_change":"2024-04-23T18:23:55.260992+0000","last_active":"2024-04-23T18:26:08.405535+0000","last_peered":"2024-04-23T18:26:08.405535+0000","last_clean":"2024-04-23T18:26:08.405535+0000","last_became_active":"2024-04-23T18:23:55.260679+0000","last_became_peered":"2024-04-23T18:23:55.260679+0000","last_unstale":"2024-04-23T18:26:08.405535+0000","last_undegraded":"2024-04-23T18:26:08.405535+0000","last_fullsized":"2024-04-23T18:26:08.405535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:02:50.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219168+0000","last_change":"2024-04-23T18:23:56.097956+0000","last_active":"2024-04-23T18:26:08.219168+0000","last_peered":"2024-04-23T18:26:08.219168+0000","last_clean":"2024-04-23T18:26:08.219168+0000","last_became_active":"2024-04-23T18:23:56.097231+0000","last_became_peered":"2024-04-23T18:23:56.097231+0000","last_unstale":"2024-04-23T18:26:08.219168+0000","last_undegraded":"2024-04-23T18:26:08.219168+0000","last_fullsized":"2024-04-23T18:26:08.219168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:10.945857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292582+0000","last_change":"2024-04-23T18:24:36.714223+0000","last_active":"2024-04-23T18:26:08.292582+0000","last_peered":"2024-04-23T18:26:08.292582+0000","last_clean":"2024-04-23T18:26:08.292582+0000","last_became_active":"2024-04-23T18:24:36.700975+0000","last_became_peered":"2024-04-23T18:24:36.700975+0000","last_unstale":"2024-04-23T18:26:08.292582+0000","last_undegraded":"2024-04-23T18:26:08.292582+0000","last_fullsized":"2024-04-23T18:26:08.292582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:58:50.227699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123311+0000","last_change":"2024-04-23T18:23:56.096889+0000","last_active":"2024-04-23T18:26:08.123311+0000","last_peered":"2024-04-23T18:26:08.123311+0000","last_clean":"2024-04-23T18:26:08.123311+0000","last_became_active":"2024-04-23T18:23:56.096520+0000","last_became_peered":"2024-04-23T18:23:56.096520+0000","last_unstale":"2024-04-23T18:26:08.123311+0000","last_undegraded":"2024-04-23T18:26:08.123311+0000","last_fullsized":"2024-04-23T18:26:08.123311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:34:53.785481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":159,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.808804+0000","last_change":"2024-04-23T18:23:55.258710+0000","last_active":"2024-04-23T18:26:08.808804+0000","last_peered":"2024-04-23T18:26:08.808804+0000","last_clean":"2024-04-23T18:26:08.808804+0000","last_became_active":"2024-04-23T18:23:55.256247+0000","last_became_peered":"2024-04-23T18:23:55.256247+0000","last_unstale":"2024-04-23T18:26:08.808804+0000","last_undegraded":"2024-04-23T18:26:08.808804+0000","last_fullsized":"2024-04-23T18:26:08.808804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:20.551614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123532+0000","last_change":"2024-04-23T18:23:56.101862+0000","last_active":"2024-04-23T18:26:08.123532+0000","last_peered":"2024-04-23T18:26:08.123532+0000","last_clean":"2024-04-23T18:26:08.123532+0000","last_became_active":"2024-04-23T18:23:56.101458+0000","last_became_peered":"2024-04-23T18:23:56.101458+0000","last_unstale":"2024-04-23T18:26:08.123532+0000","last_undegraded":"2024-04-23T18:26:08.123532+0000","last_fullsized":"2024-04-23T18:26:08.123532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:59.292077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'41","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.333872+0000","last_change":"2024-04-23T18:23:55.258292+0000","last_active":"2024-04-23T18:26:08.333872+0000","last_peered":"2024-04-23T18:26:08.333872+0000","last_clean":"2024-04-23T18:26:08.333872+0000","last_became_active":"2024-04-23T18:23:55.257954+0000","last_became_peered":"2024-04-23T18:23:55.257954+0000","last_unstale":"2024-04-23T18:26:08.333872+0000","last_undegraded":"2024-04-23T18:26:08.333872+0000","last_fullsized":"2024-04-23T18:26:08.333872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:39.105967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123377+0000","last_change":"2024-04-23T18:23:56.105977+0000","last_active":"2024-04-23T18:26:08.123377+0000","last_peered":"2024-04-23T18:26:08.123377+0000","last_clean":"2024-04-23T18:26:08.123377+0000","last_became_active":"2024-04-23T18:23:56.105582+0000","last_became_peered":"2024-04-23T18:23:56.105582+0000","last_unstale":"2024-04-23T18:26:08.123377+0000","last_undegraded":"2024-04-23T18:26:08.123377+0000","last_fullsized":"2024-04-23T18:26:08.123377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:28:10.700684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293502+0000","last_change":"2024-04-23T18:23:55.252095+0000","last_active":"2024-04-23T18:26:08.293502+0000","last_peered":"2024-04-23T18:26:08.293502+0000","last_clean":"2024-04-23T18:26:08.293502+0000","last_became_active":"2024-04-23T18:23:55.251702+0000","last_became_peered":"2024-04-23T18:23:55.251702+0000","last_unstale":"2024-04-23T18:26:08.293502+0000","last_undegraded":"2024-04-23T18:26:08.293502+0000","last_fullsized":"2024-04-23T18:26:08.293502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:21.625905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123737+0000","last_change":"2024-04-23T18:23:56.101997+0000","last_active":"2024-04-23T18:26:08.123737+0000","last_peered":"2024-04-23T18:26:08.123737+0000","last_clean":"2024-04-23T18:26:08.123737+0000","last_became_active":"2024-04-23T18:23:56.101629+0000","last_became_peered":"2024-04-23T18:23:56.101629+0000","last_unstale":"2024-04-23T18:26:08.123737+0000","last_undegraded":"2024-04-23T18:26:08.123737+0000","last_fullsized":"2024-04-23T18:26:08.123737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:42.484413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292621+0000","last_change":"2024-04-23T18:23:56.090543+0000","last_active":"2024-04-23T18:26:08.292621+0000","last_peered":"2024-04-23T18:26:08.292621+0000","last_clean":"2024-04-23T18:26:08.292621+0000","last_became_active":"2024-04-23T18:23:56.090191+0000","last_became_peered":"2024-04-23T18:23:56.090191+0000","last_unstale":"2024-04-23T18:26:08.292621+0000","last_undegraded":"2024-04-23T18:26:08.292621+0000","last_fullsized":"2024-04-23T18:26:08.292621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:01.401822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.127145+0000","last_change":"2024-04-23T18:23:56.105537+0000","last_active":"2024-04-23T18:26:08.127145+0000","last_peered":"2024-04-23T18:26:08.127145+0000","last_clean":"2024-04-23T18:26:08.127145+0000","last_became_active":"2024-04-23T18:23:56.105130+0000","last_became_peered":"2024-04-23T18:23:56.105130+0000","last_unstale":"2024-04-23T18:26:08.127145+0000","last_undegraded":"2024-04-23T18:26:08.127145+0000","last_fullsized":"2024-04-23T18:26:08.127145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:59.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.126910+0000","last_change":"2024-04-23T18:23:56.097765+0000","last_active":"2024-04-23T18:26:08.126910+0000","last_peered":"2024-04-23T18:26:08.126910+0000","last_clean":"2024-04-23T18:26:08.126910+0000","last_became_active":"2024-04-23T18:23:56.097439+0000","last_became_peered":"2024-04-23T18:23:56.097439+0000","last_unstale":"2024-04-23T18:26:08.126910+0000","last_undegraded":"2024-04-23T18:26:08.126910+0000","last_fullsized":"2024-04-23T18:26:08.126910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:31.529751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293019+0000","last_change":"2024-04-23T18:23:56.108548+0000","last_active":"2024-04-23T18:26:08.293019+0000","last_peered":"2024-04-23T18:26:08.293019+0000","last_clean":"2024-04-23T18:26:08.293019+0000","last_became_active":"2024-04-23T18:23:56.108214+0000","last_became_peered":"2024-04-23T18:23:56.108214+0000","last_unstale":"2024-04-23T18:26:08.293019+0000","last_undegraded":"2024-04-23T18:26:08.293019+0000","last_fullsized":"2024-04-23T18:26:08.293019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:41.738754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293157+0000","last_change":"2024-04-23T18:24:36.701809+0000","last_active":"2024-04-23T18:26:08.293157+0000","last_peered":"2024-04-23T18:26:08.293157+0000","last_clean":"2024-04-23T18:26:08.293157+0000","last_became_active":"2024-04-23T18:24:36.701405+0000","last_became_peered":"2024-04-23T18:24:36.701405+0000","last_unstale":"2024-04-23T18:26:08.293157+0000","last_undegraded":"2024-04-23T18:26:08.293157+0000","last_fullsized":"2024-04-23T18:26:08.293157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:03.215622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292776+0000","last_change":"2024-04-23T18:23:56.093969+0000","last_active":"2024-04-23T18:26:08.292776+0000","last_peered":"2024-04-23T18:26:08.292776+0000","last_clean":"2024-04-23T18:26:08.292776+0000","last_became_active":"2024-04-23T18:23:56.093568+0000","last_became_peered":"2024-04-23T18:23:56.093568+0000","last_unstale":"2024-04-23T18:26:08.292776+0000","last_undegraded":"2024-04-23T18:26:08.292776+0000","last_fullsized":"2024-04-23T18:26:08.292776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:08.361305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.127150+0000","last_change":"2024-04-23T18:23:56.096106+0000","last_active":"2024-04-23T18:26:08.127150+0000","last_peered":"2024-04-23T18:26:08.127150+0000","last_clean":"2024-04-23T18:26:08.127150+0000","last_became_active":"2024-04-23T18:23:56.095745+0000","last_became_peered":"2024-04-23T18:23:56.095745+0000","last_unstale":"2024-04-23T18:26:08.127150+0000","last_undegraded":"2024-04-23T18:26:08.127150+0000","last_fullsized":"2024-04-23T18:26:08.127150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:01.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218970+0000","last_change":"2024-04-23T18:23:56.095474+0000","last_active":"2024-04-23T18:26:08.218970+0000","last_peered":"2024-04-23T18:26:08.218970+0000","last_clean":"2024-04-23T18:26:08.218970+0000","last_became_active":"2024-04-23T18:23:56.095116+0000","last_became_peered":"2024-04-23T18:23:56.095116+0000","last_unstale":"2024-04-23T18:26:08.218970+0000","last_undegraded":"2024-04-23T18:26:08.218970+0000","last_fullsized":"2024-04-23T18:26:08.218970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:50:00.339558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226608+0000","last_change":"2024-04-23T18:23:56.096022+0000","last_active":"2024-04-23T18:26:08.226608+0000","last_peered":"2024-04-23T18:26:08.226608+0000","last_clean":"2024-04-23T18:26:08.226608+0000","last_became_active":"2024-04-23T18:23:56.095493+0000","last_became_peered":"2024-04-23T18:23:56.095493+0000","last_unstale":"2024-04-23T18:26:08.226608+0000","last_undegraded":"2024-04-23T18:26:08.226608+0000","last_fullsized":"2024-04-23T18:26:08.226608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:42.209072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218607+0000","last_change":"2024-04-23T18:23:56.101697+0000","last_active":"2024-04-23T18:26:08.218607+0000","last_peered":"2024-04-23T18:26:08.218607+0000","last_clean":"2024-04-23T18:26:08.218607+0000","last_became_active":"2024-04-23T18:23:56.098708+0000","last_became_peered":"2024-04-23T18:23:56.098708+0000","last_unstale":"2024-04-23T18:26:08.218607+0000","last_undegraded":"2024-04-23T18:26:08.218607+0000","last_fullsized":"2024-04-23T18:26:08.218607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:14.170700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292721+0000","last_change":"2024-04-23T18:23:56.096982+0000","last_active":"2024-04-23T18:26:08.292721+0000","last_peered":"2024-04-23T18:26:08.292721+0000","last_clean":"2024-04-23T18:26:08.292721+0000","last_became_active":"2024-04-23T18:23:56.096608+0000","last_became_peered":"2024-04-23T18:23:56.096608+0000","last_unstale":"2024-04-23T18:26:08.292721+0000","last_undegraded":"2024-04-23T18:26:08.292721+0000","last_fullsized":"2024-04-23T18:26:08.292721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:50.329337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293339+0000","last_change":"2024-04-23T18:23:56.100345+0000","last_active":"2024-04-23T18:26:08.293339+0000","last_peered":"2024-04-23T18:26:08.293339+0000","last_clean":"2024-04-23T18:26:08.293339+0000","last_became_active":"2024-04-23T18:23:56.100043+0000","last_became_peered":"2024-04-23T18:23:56.100043+0000","last_unstale":"2024-04-23T18:26:08.293339+0000","last_undegraded":"2024-04-23T18:26:08.293339+0000","last_fullsized":"2024-04-23T18:26:08.293339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:36.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292603+0000","last_change":"2024-04-23T18:23:56.101947+0000","last_active":"2024-04-23T18:26:08.292603+0000","last_peered":"2024-04-23T18:26:08.292603+0000","last_clean":"2024-04-23T18:26:08.292603+0000","last_became_active":"2024-04-23T18:23:56.101627+0000","last_became_peered":"2024-04-23T18:23:56.101627+0000","last_unstale":"2024-04-23T18:26:08.292603+0000","last_undegraded":"2024-04-23T18:26:08.292603+0000","last_fullsized":"2024-04-23T18:26:08.292603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:21:43.703653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106755,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602323,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1406,"num_read_kb":821,"num_write":373,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1806969,"data_compressed":81501,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":461377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":1261854,"data_compressed":46511,"data_compressed_allocated":61440,"data_compressed_original":729088,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":863382,"data_compressed":22051,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":863760,"data_compressed":22051,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":1065216,"data_compressed":38490,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":470665,"data_compressed":2905,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":539336,"data_compressed":4770,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":472004,"data_compressed":2905,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":539908,"data_compressed":4770,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":994524,"data_compressed":28207,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":471087,"data_compressed":2905,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1194481,"data_compressed":44646,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70292,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70864,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134952,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":202325,"data_compressed":8021,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:26:13.943 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:26:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.088+0000 7f5321be1640 1 -- 172.21.15.19:0/2976635204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c103b00 msgr2=0x7f531c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.088+0000 7f5321be1640 1 --2- 172.21.15.19:0/2976635204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c103b00 0x7f531c10bfe0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5310009900 tx=0x7f531002a510 comp rx=0 tx=0).stop 2024-04-23T18:26:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.088+0000 7f5321be1640 1 -- 172.21.15.19:0/2976635204 shutdown_connections 2024-04-23T18:26:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.088+0000 7f5321be1640 1 --2- 172.21.15.19:0/2976635204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531c10c520 0x7f531c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.088+0000 7f5321be1640 1 --2- 172.21.15.19:0/2976635204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c103b00 0x7f531c10bfe0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.088+0000 7f5321be1640 1 --2- 172.21.15.19:0/2976635204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c1031e0 0x7f531c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.088+0000 7f5321be1640 1 -- 172.21.15.19:0/2976635204 >> 172.21.15.19:0/2976635204 conn(0x7f531c077f50 msgr2=0x7f531c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.088+0000 7f5321be1640 1 -- 172.21.15.19:0/2976635204 shutdown_connections 2024-04-23T18:26:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.088+0000 7f5321be1640 1 -- 172.21.15.19:0/2976635204 wait complete. 2024-04-23T18:26:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.088+0000 7f5321be1640 1 Processor -- start 2024-04-23T18:26:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f5321be1640 1 -- start start 2024-04-23T18:26:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f5321be1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c1031e0 0x7f531c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f5321be1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531c103b00 0x7f531c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f5321be1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c10c520 0x7f531c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f5321be1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f531c112190 con 0x7f531c1031e0 2024-04-23T18:26:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f5321be1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f531c112010 con 0x7f531c10c520 2024-04-23T18:26:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f5321be1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f531c112310 con 0x7f531c103b00 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c1031e0 0x7f531c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c1031e0 0x7f531c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53410/0 (socket says 172.21.15.19:53410) 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531b7fe640 1 -- 172.21.15.19:0/2380877530 learned_addr learned my addr 172.21.15.19:0/2380877530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531affd640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531c103b00 0x7f531c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531bfff640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c10c520 0x7f531c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531b7fe640 1 -- 172.21.15.19:0/2380877530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531c103b00 msgr2=0x7f531c198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531b7fe640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531c103b00 0x7f531c198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531b7fe640 1 -- 172.21.15.19:0/2380877530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c10c520 msgr2=0x7f531c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531b7fe640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c10c520 0x7f531c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531b7fe640 1 -- 172.21.15.19:0/2380877530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c19d610 con 0x7f531c1031e0 2024-04-23T18:26:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.089+0000 7f531b7fe640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c1031e0 0x7f531c198670 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f530c00c920 tx=0x7f530c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.090+0000 7f5318ff9640 1 -- 172.21.15.19:0/2380877530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c007c70 con 0x7f531c1031e0 2024-04-23T18:26:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.090+0000 7f5318ff9640 1 -- 172.21.15.19:0/2380877530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f530c007e10 con 0x7f531c1031e0 2024-04-23T18:26:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.090+0000 7f5321be1640 1 -- 172.21.15.19:0/2380877530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531c19d900 con 0x7f531c1031e0 2024-04-23T18:26:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.090+0000 7f5318ff9640 1 -- 172.21.15.19:0/2380877530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c018820 con 0x7f531c1031e0 2024-04-23T18:26:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.090+0000 7f5321be1640 1 -- 172.21.15.19:0/2380877530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531c1a5190 con 0x7f531c1031e0 2024-04-23T18:26:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.091+0000 7f5321be1640 1 -- 172.21.15.19:0/2380877530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531c068d70 con 0x7f531c1031e0 2024-04-23T18:26:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.092+0000 7f5318ff9640 1 -- 172.21.15.19:0/2380877530 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f530c00f450 con 0x7f531c1031e0 2024-04-23T18:26:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.092+0000 7f5318ff9640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53040a3e40 0x7f53040a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.093+0000 7f531affd640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53040a3e40 0x7f53040a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.093+0000 7f5318ff9640 1 -- 172.21.15.19:0/2380877530 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f530c0d0c50 con 0x7f531c1031e0 2024-04-23T18:26:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.093+0000 7f531affd640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53040a3e40 0x7f53040a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5310031440 tx=0x7f53100095f0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.097+0000 7f5318ff9640 1 -- 172.21.15.19:0/2380877530 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f530c0d6020 con 0x7f531c1031e0 2024-04-23T18:26:14.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:14 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1647890227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:14.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:14 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/828083611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.281+0000 7f5321be1640 1 -- 172.21.15.19:0/2380877530 --> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f531c062390 con 0x7f53040a3e40 2024-04-23T18:26:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.289+0000 7f5318ff9640 1 -- 172.21.15.19:0/2380877530 <== mgr.25004 v2:172.21.15.19:6800/1177101685 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258074 (secure 0 0 0) 0x7f531c062390 con 0x7f53040a3e40 2024-04-23T18:26:14.285 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:14.286 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 -- 172.21.15.19:0/2380877530 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53040a3e40 msgr2=0x7f53040a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53040a3e40 0x7f53040a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5310031440 tx=0x7f53100095f0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 -- 172.21.15.19:0/2380877530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c1031e0 msgr2=0x7f531c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c1031e0 0x7f531c198670 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f530c00c920 tx=0x7f530c00cde0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 -- 172.21.15.19:0/2380877530 shutdown_connections 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f53040a3e40 0x7f53040a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c10c520 0x7f531c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531c103b00 0x7f531c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 --2- 172.21.15.19:0/2380877530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c1031e0 0x7f531c198670 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 -- 172.21.15.19:0/2380877530 >> 172.21.15.19:0/2380877530 conn(0x7f531c077f50 msgr2=0x7f531c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 -- 172.21.15.19:0/2380877530 shutdown_connections 2024-04-23T18:26:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.292+0000 7f5321be1640 1 -- 172.21.15.19:0/2380877530 wait complete. 2024-04-23T18:26:14.296 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-23T18:26:14.135885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192925,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1547,"num_read_kb":945,"num_write":556,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":941,"ondisk_log_size":941,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382144,"kb_used_data":12156,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559680,"statfs":{"total":1151940427776,"available":1151549112320,"internally_reserved":0,"allocated":12447744,"data_stored":9394617,"data_compressed":229400,"data_compressed_allocated":303104,"data_compressed_original":3616768,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.986055"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292994+0000","last_change":"2024-04-23T18:23:56.098292+0000","last_active":"2024-04-23T18:26:08.292994+0000","last_peered":"2024-04-23T18:26:08.292994+0000","last_clean":"2024-04-23T18:26:08.292994+0000","last_became_active":"2024-04-23T18:23:56.096623+0000","last_became_peered":"2024-04-23T18:23:56.096623+0000","last_unstale":"2024-04-23T18:26:08.292994+0000","last_undegraded":"2024-04-23T18:26:08.292994+0000","last_fullsized":"2024-04-23T18:26:08.292994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:06.186581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293701+0000","last_change":"2024-04-23T18:23:56.096807+0000","last_active":"2024-04-23T18:26:08.293701+0000","last_peered":"2024-04-23T18:26:08.293701+0000","last_clean":"2024-04-23T18:26:08.293701+0000","last_became_active":"2024-04-23T18:23:56.096327+0000","last_became_peered":"2024-04-23T18:23:56.096327+0000","last_unstale":"2024-04-23T18:26:08.293701+0000","last_undegraded":"2024-04-23T18:26:08.293701+0000","last_fullsized":"2024-04-23T18:26:08.293701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:56:16.391821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293324+0000","last_change":"2024-04-23T18:23:56.101615+0000","last_active":"2024-04-23T18:26:08.293324+0000","last_peered":"2024-04-23T18:26:08.293324+0000","last_clean":"2024-04-23T18:26:08.293324+0000","last_became_active":"2024-04-23T18:23:56.100929+0000","last_became_peered":"2024-04-23T18:23:56.100929+0000","last_unstale":"2024-04-23T18:26:08.293324+0000","last_undegraded":"2024-04-23T18:26:08.293324+0000","last_fullsized":"2024-04-23T18:26:08.293324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:25.547029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293249+0000","last_change":"2024-04-23T18:23:56.103627+0000","last_active":"2024-04-23T18:26:08.293249+0000","last_peered":"2024-04-23T18:26:08.293249+0000","last_clean":"2024-04-23T18:26:08.293249+0000","last_became_active":"2024-04-23T18:23:56.102752+0000","last_became_peered":"2024-04-23T18:23:56.102752+0000","last_unstale":"2024-04-23T18:26:08.293249+0000","last_undegraded":"2024-04-23T18:26:08.293249+0000","last_fullsized":"2024-04-23T18:26:08.293249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:28.460342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218930+0000","last_change":"2024-04-23T18:23:56.094889+0000","last_active":"2024-04-23T18:26:08.218930+0000","last_peered":"2024-04-23T18:26:08.218930+0000","last_clean":"2024-04-23T18:26:08.218930+0000","last_became_active":"2024-04-23T18:23:56.094262+0000","last_became_peered":"2024-04-23T18:23:56.094262+0000","last_unstale":"2024-04-23T18:26:08.218930+0000","last_undegraded":"2024-04-23T18:26:08.218930+0000","last_fullsized":"2024-04-23T18:26:08.218930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:22.332114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229648+0000","last_change":"2024-04-23T18:23:56.108893+0000","last_active":"2024-04-23T18:26:08.229648+0000","last_peered":"2024-04-23T18:26:08.229648+0000","last_clean":"2024-04-23T18:26:08.229648+0000","last_became_active":"2024-04-23T18:23:56.108229+0000","last_became_peered":"2024-04-23T18:23:56.108229+0000","last_unstale":"2024-04-23T18:26:08.229648+0000","last_undegraded":"2024-04-23T18:26:08.229648+0000","last_fullsized":"2024-04-23T18:26:08.229648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:50.930321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123153+0000","last_change":"2024-04-23T18:23:56.098674+0000","last_active":"2024-04-23T18:26:08.123153+0000","last_peered":"2024-04-23T18:26:08.123153+0000","last_clean":"2024-04-23T18:26:08.123153+0000","last_became_active":"2024-04-23T18:23:56.098299+0000","last_became_peered":"2024-04-23T18:23:56.098299+0000","last_unstale":"2024-04-23T18:26:08.123153+0000","last_undegraded":"2024-04-23T18:26:08.123153+0000","last_fullsized":"2024-04-23T18:26:08.123153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:05.711410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218497+0000","last_change":"2024-04-23T18:23:56.097437+0000","last_active":"2024-04-23T18:26:08.218497+0000","last_peered":"2024-04-23T18:26:08.218497+0000","last_clean":"2024-04-23T18:26:08.218497+0000","last_became_active":"2024-04-23T18:23:56.095702+0000","last_became_peered":"2024-04-23T18:23:56.095702+0000","last_unstale":"2024-04-23T18:26:08.218497+0000","last_undegraded":"2024-04-23T18:26:08.218497+0000","last_fullsized":"2024-04-23T18:26:08.218497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:35.754832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293226+0000","last_change":"2024-04-23T18:23:56.096060+0000","last_active":"2024-04-23T18:26:08.293226+0000","last_peered":"2024-04-23T18:26:08.293226+0000","last_clean":"2024-04-23T18:26:08.293226+0000","last_became_active":"2024-04-23T18:23:56.095597+0000","last_became_peered":"2024-04-23T18:23:56.095597+0000","last_unstale":"2024-04-23T18:26:08.293226+0000","last_undegraded":"2024-04-23T18:26:08.293226+0000","last_fullsized":"2024-04-23T18:26:08.293226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:59.735347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229983+0000","last_change":"2024-04-23T18:23:56.109033+0000","last_active":"2024-04-23T18:26:08.229983+0000","last_peered":"2024-04-23T18:26:08.229983+0000","last_clean":"2024-04-23T18:26:08.229983+0000","last_became_active":"2024-04-23T18:23:56.108472+0000","last_became_peered":"2024-04-23T18:23:56.108472+0000","last_unstale":"2024-04-23T18:26:08.229983+0000","last_undegraded":"2024-04-23T18:26:08.229983+0000","last_fullsized":"2024-04-23T18:26:08.229983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:32.491181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123484+0000","last_change":"2024-04-23T18:23:56.097771+0000","last_active":"2024-04-23T18:26:08.123484+0000","last_peered":"2024-04-23T18:26:08.123484+0000","last_clean":"2024-04-23T18:26:08.123484+0000","last_became_active":"2024-04-23T18:23:56.097403+0000","last_became_peered":"2024-04-23T18:23:56.097403+0000","last_unstale":"2024-04-23T18:26:08.123484+0000","last_undegraded":"2024-04-23T18:26:08.123484+0000","last_fullsized":"2024-04-23T18:26:08.123484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:48:36.261543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293091+0000","last_change":"2024-04-23T18:23:56.108550+0000","last_active":"2024-04-23T18:26:08.293091+0000","last_peered":"2024-04-23T18:26:08.293091+0000","last_clean":"2024-04-23T18:26:08.293091+0000","last_became_active":"2024-04-23T18:23:56.108191+0000","last_became_peered":"2024-04-23T18:23:56.108191+0000","last_unstale":"2024-04-23T18:26:08.293091+0000","last_undegraded":"2024-04-23T18:26:08.293091+0000","last_fullsized":"2024-04-23T18:26:08.293091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:25.142466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293119+0000","last_change":"2024-04-23T18:23:56.092573+0000","last_active":"2024-04-23T18:26:08.293119+0000","last_peered":"2024-04-23T18:26:08.293119+0000","last_clean":"2024-04-23T18:26:08.293119+0000","last_became_active":"2024-04-23T18:23:56.092095+0000","last_became_peered":"2024-04-23T18:23:56.092095+0000","last_unstale":"2024-04-23T18:26:08.293119+0000","last_undegraded":"2024-04-23T18:26:08.293119+0000","last_fullsized":"2024-04-23T18:26:08.293119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:01.601206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292816+0000","last_change":"2024-04-23T18:23:56.102073+0000","last_active":"2024-04-23T18:26:08.292816+0000","last_peered":"2024-04-23T18:26:08.292816+0000","last_clean":"2024-04-23T18:26:08.292816+0000","last_became_active":"2024-04-23T18:23:56.101059+0000","last_became_peered":"2024-04-23T18:23:56.101059+0000","last_unstale":"2024-04-23T18:26:08.292816+0000","last_undegraded":"2024-04-23T18:26:08.292816+0000","last_fullsized":"2024-04-23T18:26:08.292816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:37.904099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292355+0000","last_change":"2024-04-23T18:23:56.108482+0000","last_active":"2024-04-23T18:26:08.292355+0000","last_peered":"2024-04-23T18:26:08.292355+0000","last_clean":"2024-04-23T18:26:08.292355+0000","last_became_active":"2024-04-23T18:23:56.108128+0000","last_became_peered":"2024-04-23T18:23:56.108128+0000","last_unstale":"2024-04-23T18:26:08.292355+0000","last_undegraded":"2024-04-23T18:26:08.292355+0000","last_fullsized":"2024-04-23T18:26:08.292355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:51.925881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292324+0000","last_change":"2024-04-23T18:23:56.099691+0000","last_active":"2024-04-23T18:26:08.292324+0000","last_peered":"2024-04-23T18:26:08.292324+0000","last_clean":"2024-04-23T18:26:08.292324+0000","last_became_active":"2024-04-23T18:23:56.099272+0000","last_became_peered":"2024-04-23T18:23:56.099272+0000","last_unstale":"2024-04-23T18:26:08.292324+0000","last_undegraded":"2024-04-23T18:26:08.292324+0000","last_fullsized":"2024-04-23T18:26:08.292324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:41.834104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292954+0000","last_change":"2024-04-23T18:23:56.097965+0000","last_active":"2024-04-23T18:26:08.292954+0000","last_peered":"2024-04-23T18:26:08.292954+0000","last_clean":"2024-04-23T18:26:08.292954+0000","last_became_active":"2024-04-23T18:23:56.096766+0000","last_became_peered":"2024-04-23T18:23:56.096766+0000","last_unstale":"2024-04-23T18:26:08.292954+0000","last_undegraded":"2024-04-23T18:26:08.292954+0000","last_fullsized":"2024-04-23T18:26:08.292954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:40:00.076939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122351+0000","last_change":"2024-04-23T18:23:56.095158+0000","last_active":"2024-04-23T18:26:08.122351+0000","last_peered":"2024-04-23T18:26:08.122351+0000","last_clean":"2024-04-23T18:26:08.122351+0000","last_became_active":"2024-04-23T18:23:56.094812+0000","last_became_peered":"2024-04-23T18:23:56.094812+0000","last_unstale":"2024-04-23T18:26:08.122351+0000","last_undegraded":"2024-04-23T18:26:08.122351+0000","last_fullsized":"2024-04-23T18:26:08.122351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:40.874698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.126943+0000","last_change":"2024-04-23T18:23:56.099358+0000","last_active":"2024-04-23T18:26:08.126943+0000","last_peered":"2024-04-23T18:26:08.126943+0000","last_clean":"2024-04-23T18:26:08.126943+0000","last_became_active":"2024-04-23T18:23:56.099031+0000","last_became_peered":"2024-04-23T18:23:56.099031+0000","last_unstale":"2024-04-23T18:26:08.126943+0000","last_undegraded":"2024-04-23T18:26:08.126943+0000","last_fullsized":"2024-04-23T18:26:08.126943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:29.634932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.805802+0000","last_change":"2024-04-23T18:23:55.262939+0000","last_active":"2024-04-23T18:26:08.805802+0000","last_peered":"2024-04-23T18:26:08.805802+0000","last_clean":"2024-04-23T18:26:08.805802+0000","last_became_active":"2024-04-23T18:23:55.262609+0000","last_became_peered":"2024-04-23T18:23:55.262609+0000","last_unstale":"2024-04-23T18:26:08.805802+0000","last_undegraded":"2024-04-23T18:26:08.805802+0000","last_fullsized":"2024-04-23T18:26:08.805802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:12.418349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218972+0000","last_change":"2024-04-23T18:23:56.092182+0000","last_active":"2024-04-23T18:26:08.218972+0000","last_peered":"2024-04-23T18:26:08.218972+0000","last_clean":"2024-04-23T18:26:08.218972+0000","last_became_active":"2024-04-23T18:23:56.091444+0000","last_became_peered":"2024-04-23T18:23:56.091444+0000","last_unstale":"2024-04-23T18:26:08.218972+0000","last_undegraded":"2024-04-23T18:26:08.218972+0000","last_fullsized":"2024-04-23T18:26:08.218972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:00.775498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219056+0000","last_change":"2024-04-23T18:23:55.259880+0000","last_active":"2024-04-23T18:26:08.219056+0000","last_peered":"2024-04-23T18:26:08.219056+0000","last_clean":"2024-04-23T18:26:08.219056+0000","last_became_active":"2024-04-23T18:23:55.259576+0000","last_became_peered":"2024-04-23T18:23:55.259576+0000","last_unstale":"2024-04-23T18:26:08.219056+0000","last_undegraded":"2024-04-23T18:26:08.219056+0000","last_fullsized":"2024-04-23T18:26:08.219056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:13.783776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293588+0000","last_change":"2024-04-23T18:23:56.094307+0000","last_active":"2024-04-23T18:26:08.293588+0000","last_peered":"2024-04-23T18:26:08.293588+0000","last_clean":"2024-04-23T18:26:08.293588+0000","last_became_active":"2024-04-23T18:23:56.093893+0000","last_became_peered":"2024-04-23T18:23:56.093893+0000","last_unstale":"2024-04-23T18:26:08.293588+0000","last_undegraded":"2024-04-23T18:26:08.293588+0000","last_fullsized":"2024-04-23T18:26:08.293588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:12:35.385497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219326+0000","last_change":"2024-04-23T18:23:55.263137+0000","last_active":"2024-04-23T18:26:08.219326+0000","last_peered":"2024-04-23T18:26:08.219326+0000","last_clean":"2024-04-23T18:26:08.219326+0000","last_became_active":"2024-04-23T18:23:55.262553+0000","last_became_peered":"2024-04-23T18:23:55.262553+0000","last_unstale":"2024-04-23T18:26:08.219326+0000","last_undegraded":"2024-04-23T18:26:08.219326+0000","last_fullsized":"2024-04-23T18:26:08.219326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:33:42.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292576+0000","last_change":"2024-04-23T18:23:56.091098+0000","last_active":"2024-04-23T18:26:08.292576+0000","last_peered":"2024-04-23T18:26:08.292576+0000","last_clean":"2024-04-23T18:26:08.292576+0000","last_became_active":"2024-04-23T18:23:56.090635+0000","last_became_peered":"2024-04-23T18:23:56.090635+0000","last_unstale":"2024-04-23T18:26:08.292576+0000","last_undegraded":"2024-04-23T18:26:08.292576+0000","last_fullsized":"2024-04-23T18:26:08.292576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:17.787937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219108+0000","last_change":"2024-04-23T18:23:55.262146+0000","last_active":"2024-04-23T18:26:08.219108+0000","last_peered":"2024-04-23T18:26:08.219108+0000","last_clean":"2024-04-23T18:26:08.219108+0000","last_became_active":"2024-04-23T18:23:55.261838+0000","last_became_peered":"2024-04-23T18:23:55.261838+0000","last_unstale":"2024-04-23T18:26:08.219108+0000","last_undegraded":"2024-04-23T18:26:08.219108+0000","last_fullsized":"2024-04-23T18:26:08.219108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:42:39.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122228+0000","last_change":"2024-04-23T18:24:36.915394+0000","last_active":"2024-04-23T18:26:08.122228+0000","last_peered":"2024-04-23T18:26:08.122228+0000","last_clean":"2024-04-23T18:26:08.122228+0000","last_became_active":"2024-04-23T18:24:36.915072+0000","last_became_peered":"2024-04-23T18:24:36.915072+0000","last_unstale":"2024-04-23T18:26:08.122228+0000","last_undegraded":"2024-04-23T18:26:08.122228+0000","last_fullsized":"2024-04-23T18:26:08.122228+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.132410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226797+0000","last_change":"2024-04-23T18:23:55.261399+0000","last_active":"2024-04-23T18:26:08.226797+0000","last_peered":"2024-04-23T18:26:08.226797+0000","last_clean":"2024-04-23T18:26:08.226797+0000","last_became_active":"2024-04-23T18:23:55.261012+0000","last_became_peered":"2024-04-23T18:23:55.261012+0000","last_unstale":"2024-04-23T18:26:08.226797+0000","last_undegraded":"2024-04-23T18:26:08.226797+0000","last_fullsized":"2024-04-23T18:26:08.226797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:31.377776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218869+0000","last_change":"2024-04-23T18:23:56.096785+0000","last_active":"2024-04-23T18:26:08.218869+0000","last_peered":"2024-04-23T18:26:08.218869+0000","last_clean":"2024-04-23T18:26:08.218869+0000","last_became_active":"2024-04-23T18:23:56.096420+0000","last_became_peered":"2024-04-23T18:23:56.096420+0000","last_unstale":"2024-04-23T18:26:08.218869+0000","last_undegraded":"2024-04-23T18:26:08.218869+0000","last_fullsized":"2024-04-23T18:26:08.218869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:20:49.215071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229615+0000","last_change":"2024-04-23T18:23:55.260645+0000","last_active":"2024-04-23T18:26:08.229615+0000","last_peered":"2024-04-23T18:26:08.229615+0000","last_clean":"2024-04-23T18:26:08.229615+0000","last_became_active":"2024-04-23T18:23:55.260118+0000","last_became_peered":"2024-04-23T18:23:55.260118+0000","last_unstale":"2024-04-23T18:26:08.229615+0000","last_undegraded":"2024-04-23T18:26:08.229615+0000","last_fullsized":"2024-04-23T18:26:08.229615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:14:12.066297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226773+0000","last_change":"2024-04-23T18:24:36.702766+0000","last_active":"2024-04-23T18:26:08.226773+0000","last_peered":"2024-04-23T18:26:08.226773+0000","last_clean":"2024-04-23T18:26:08.226773+0000","last_became_active":"2024-04-23T18:24:36.702309+0000","last_became_peered":"2024-04-23T18:24:36.702309+0000","last_unstale":"2024-04-23T18:26:08.226773+0000","last_undegraded":"2024-04-23T18:26:08.226773+0000","last_fullsized":"2024-04-23T18:26:08.226773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:10.803924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292712+0000","last_change":"2024-04-23T18:23:55.258636+0000","last_active":"2024-04-23T18:26:08.292712+0000","last_peered":"2024-04-23T18:26:08.292712+0000","last_clean":"2024-04-23T18:26:08.292712+0000","last_became_active":"2024-04-23T18:23:55.257744+0000","last_became_peered":"2024-04-23T18:23:55.257744+0000","last_unstale":"2024-04-23T18:26:08.292712+0000","last_undegraded":"2024-04-23T18:26:08.292712+0000","last_fullsized":"2024-04-23T18:26:08.292712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:46:07.454725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218571+0000","last_change":"2024-04-23T18:23:56.094822+0000","last_active":"2024-04-23T18:26:08.218571+0000","last_peered":"2024-04-23T18:26:08.218571+0000","last_clean":"2024-04-23T18:26:08.218571+0000","last_became_active":"2024-04-23T18:23:56.094489+0000","last_became_peered":"2024-04-23T18:23:56.094489+0000","last_unstale":"2024-04-23T18:26:08.218571+0000","last_undegraded":"2024-04-23T18:26:08.218571+0000","last_fullsized":"2024-04-23T18:26:08.218571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:21.234363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226519+0000","last_change":"2024-04-23T18:23:55.261322+0000","last_active":"2024-04-23T18:26:08.226519+0000","last_peered":"2024-04-23T18:26:08.226519+0000","last_clean":"2024-04-23T18:26:08.226519+0000","last_became_active":"2024-04-23T18:23:55.260990+0000","last_became_peered":"2024-04-23T18:23:55.260990+0000","last_unstale":"2024-04-23T18:26:08.226519+0000","last_undegraded":"2024-04-23T18:26:08.226519+0000","last_fullsized":"2024-04-23T18:26:08.226519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:31.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122279+0000","last_change":"2024-04-23T18:23:56.093105+0000","last_active":"2024-04-23T18:26:08.122279+0000","last_peered":"2024-04-23T18:26:08.122279+0000","last_clean":"2024-04-23T18:26:08.122279+0000","last_became_active":"2024-04-23T18:23:56.092717+0000","last_became_peered":"2024-04-23T18:23:56.092717+0000","last_unstale":"2024-04-23T18:26:08.122279+0000","last_undegraded":"2024-04-23T18:26:08.122279+0000","last_fullsized":"2024-04-23T18:26:08.122279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:13:59.574057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":281,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.807789+0000","last_change":"2024-04-23T18:23:55.256307+0000","last_active":"2024-04-23T18:26:08.807789+0000","last_peered":"2024-04-23T18:26:08.807789+0000","last_clean":"2024-04-23T18:26:08.807789+0000","last_became_active":"2024-04-23T18:23:55.255959+0000","last_became_peered":"2024-04-23T18:23:55.255959+0000","last_unstale":"2024-04-23T18:26:08.807789+0000","last_undegraded":"2024-04-23T18:26:08.807789+0000","last_fullsized":"2024-04-23T18:26:08.807789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:04.945919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293141+0000","last_change":"2024-04-23T18:23:56.095177+0000","last_active":"2024-04-23T18:26:08.293141+0000","last_peered":"2024-04-23T18:26:08.293141+0000","last_clean":"2024-04-23T18:26:08.293141+0000","last_became_active":"2024-04-23T18:23:56.094825+0000","last_became_peered":"2024-04-23T18:23:56.094825+0000","last_unstale":"2024-04-23T18:26:08.293141+0000","last_undegraded":"2024-04-23T18:26:08.293141+0000","last_fullsized":"2024-04-23T18:26:08.293141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:07:33.499817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123551+0000","last_change":"2024-04-23T18:23:55.264258+0000","last_active":"2024-04-23T18:26:08.123551+0000","last_peered":"2024-04-23T18:26:08.123551+0000","last_clean":"2024-04-23T18:26:08.123551+0000","last_became_active":"2024-04-23T18:23:55.263698+0000","last_became_peered":"2024-04-23T18:23:55.263698+0000","last_unstale":"2024-04-23T18:26:08.123551+0000","last_undegraded":"2024-04-23T18:26:08.123551+0000","last_fullsized":"2024-04-23T18:26:08.123551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:00.566948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293683+0000","last_change":"2024-04-23T18:23:56.109085+0000","last_active":"2024-04-23T18:26:08.293683+0000","last_peered":"2024-04-23T18:26:08.293683+0000","last_clean":"2024-04-23T18:26:08.293683+0000","last_became_active":"2024-04-23T18:23:56.108033+0000","last_became_peered":"2024-04-23T18:23:56.108033+0000","last_unstale":"2024-04-23T18:26:08.293683+0000","last_undegraded":"2024-04-23T18:26:08.293683+0000","last_fullsized":"2024-04-23T18:26:08.293683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:59.980512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226931+0000","last_change":"2024-04-23T18:23:55.256728+0000","last_active":"2024-04-23T18:26:08.226931+0000","last_peered":"2024-04-23T18:26:08.226931+0000","last_clean":"2024-04-23T18:26:08.226931+0000","last_became_active":"2024-04-23T18:23:55.256367+0000","last_became_peered":"2024-04-23T18:23:55.256367+0000","last_unstale":"2024-04-23T18:26:08.226931+0000","last_undegraded":"2024-04-23T18:26:08.226931+0000","last_fullsized":"2024-04-23T18:26:08.226931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:08.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292666+0000","last_change":"2024-04-23T18:23:56.102898+0000","last_active":"2024-04-23T18:26:08.292666+0000","last_peered":"2024-04-23T18:26:08.292666+0000","last_clean":"2024-04-23T18:26:08.292666+0000","last_became_active":"2024-04-23T18:23:56.102373+0000","last_became_peered":"2024-04-23T18:23:56.102373+0000","last_unstale":"2024-04-23T18:26:08.292666+0000","last_undegraded":"2024-04-23T18:26:08.292666+0000","last_fullsized":"2024-04-23T18:26:08.292666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:49.316219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'37","reported_seq":322,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.808335+0000","last_change":"2024-04-23T18:23:55.255094+0000","last_active":"2024-04-23T18:26:08.808335+0000","last_peered":"2024-04-23T18:26:08.808335+0000","last_clean":"2024-04-23T18:26:08.808335+0000","last_became_active":"2024-04-23T18:23:55.254726+0000","last_became_peered":"2024-04-23T18:23:55.254726+0000","last_unstale":"2024-04-23T18:26:08.808335+0000","last_undegraded":"2024-04-23T18:26:08.808335+0000","last_fullsized":"2024-04-23T18:26:08.808335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:26.988184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":264,"num_read_kb":200,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219090+0000","last_change":"2024-04-23T18:23:56.099866+0000","last_active":"2024-04-23T18:26:08.219090+0000","last_peered":"2024-04-23T18:26:08.219090+0000","last_clean":"2024-04-23T18:26:08.219090+0000","last_became_active":"2024-04-23T18:23:56.098606+0000","last_became_peered":"2024-04-23T18:23:56.098606+0000","last_unstale":"2024-04-23T18:26:08.219090+0000","last_undegraded":"2024-04-23T18:26:08.219090+0000","last_fullsized":"2024-04-23T18:26:08.219090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:26.661857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292679+0000","last_change":"2024-04-23T18:24:36.710791+0000","last_active":"2024-04-23T18:26:08.292679+0000","last_peered":"2024-04-23T18:26:08.292679+0000","last_clean":"2024-04-23T18:26:08.292679+0000","last_became_active":"2024-04-23T18:24:36.697731+0000","last_became_peered":"2024-04-23T18:24:36.697731+0000","last_unstale":"2024-04-23T18:26:08.292679+0000","last_undegraded":"2024-04-23T18:26:08.292679+0000","last_fullsized":"2024-04-23T18:26:08.292679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:16:46.722132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292908+0000","last_change":"2024-04-23T18:23:56.099109+0000","last_active":"2024-04-23T18:26:08.292908+0000","last_peered":"2024-04-23T18:26:08.292908+0000","last_clean":"2024-04-23T18:26:08.292908+0000","last_became_active":"2024-04-23T18:23:56.098608+0000","last_became_peered":"2024-04-23T18:23:56.098608+0000","last_unstale":"2024-04-23T18:26:08.292908+0000","last_undegraded":"2024-04-23T18:26:08.292908+0000","last_fullsized":"2024-04-23T18:26:08.292908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:43.720160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226641+0000","last_change":"2024-04-23T18:23:55.262523+0000","last_active":"2024-04-23T18:26:08.226641+0000","last_peered":"2024-04-23T18:26:08.226641+0000","last_clean":"2024-04-23T18:26:08.226641+0000","last_became_active":"2024-04-23T18:23:55.262216+0000","last_became_peered":"2024-04-23T18:23:55.262216+0000","last_unstale":"2024-04-23T18:26:08.226641+0000","last_undegraded":"2024-04-23T18:26:08.226641+0000","last_fullsized":"2024-04-23T18:26:08.226641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:35.268435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123437+0000","last_change":"2024-04-23T18:23:56.106573+0000","last_active":"2024-04-23T18:26:08.123437+0000","last_peered":"2024-04-23T18:26:08.123437+0000","last_clean":"2024-04-23T18:26:08.123437+0000","last_became_active":"2024-04-23T18:23:56.106181+0000","last_became_peered":"2024-04-23T18:23:56.106181+0000","last_unstale":"2024-04-23T18:26:08.123437+0000","last_undegraded":"2024-04-23T18:26:08.123437+0000","last_fullsized":"2024-04-23T18:26:08.123437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:15.422647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'9","reported_seq":162,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.811881+0000","last_change":"2024-04-23T18:23:55.262813+0000","last_active":"2024-04-23T18:26:08.811881+0000","last_peered":"2024-04-23T18:26:08.811881+0000","last_clean":"2024-04-23T18:26:08.811881+0000","last_became_active":"2024-04-23T18:23:55.262365+0000","last_became_peered":"2024-04-23T18:23:55.262365+0000","last_unstale":"2024-04-23T18:26:08.811881+0000","last_undegraded":"2024-04-23T18:26:08.811881+0000","last_fullsized":"2024-04-23T18:26:08.811881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:15.234355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1819,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123074+0000","last_change":"2024-04-23T18:23:56.101491+0000","last_active":"2024-04-23T18:26:08.123074+0000","last_peered":"2024-04-23T18:26:08.123074+0000","last_clean":"2024-04-23T18:26:08.123074+0000","last_became_active":"2024-04-23T18:23:56.101069+0000","last_became_peered":"2024-04-23T18:23:56.101069+0000","last_unstale":"2024-04-23T18:26:08.123074+0000","last_undegraded":"2024-04-23T18:26:08.123074+0000","last_fullsized":"2024-04-23T18:26:08.123074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:42.484653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218843+0000","last_change":"2024-04-23T18:24:36.750105+0000","last_active":"2024-04-23T18:26:08.218843+0000","last_peered":"2024-04-23T18:26:08.218843+0000","last_clean":"2024-04-23T18:26:08.218843+0000","last_became_active":"2024-04-23T18:24:36.738326+0000","last_became_peered":"2024-04-23T18:24:36.738326+0000","last_unstale":"2024-04-23T18:26:08.218843+0000","last_undegraded":"2024-04-23T18:26:08.218843+0000","last_fullsized":"2024-04-23T18:26:08.218843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:07.414513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226554+0000","last_change":"2024-04-23T18:23:56.105691+0000","last_active":"2024-04-23T18:26:08.226554+0000","last_peered":"2024-04-23T18:26:08.226554+0000","last_clean":"2024-04-23T18:26:08.226554+0000","last_became_active":"2024-04-23T18:23:56.105205+0000","last_became_peered":"2024-04-23T18:23:56.105205+0000","last_unstale":"2024-04-23T18:26:08.226554+0000","last_undegraded":"2024-04-23T18:26:08.226554+0000","last_fullsized":"2024-04-23T18:26:08.226554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:30.779821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219253+0000","last_change":"2024-04-23T18:23:55.261198+0000","last_active":"2024-04-23T18:26:08.219253+0000","last_peered":"2024-04-23T18:26:08.219253+0000","last_clean":"2024-04-23T18:26:08.219253+0000","last_became_active":"2024-04-23T18:23:55.260884+0000","last_became_peered":"2024-04-23T18:23:55.260884+0000","last_unstale":"2024-04-23T18:26:08.219253+0000","last_undegraded":"2024-04-23T18:26:08.219253+0000","last_fullsized":"2024-04-23T18:26:08.219253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:00.841671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292792+0000","last_change":"2024-04-23T18:23:56.099450+0000","last_active":"2024-04-23T18:26:08.292792+0000","last_peered":"2024-04-23T18:26:08.292792+0000","last_clean":"2024-04-23T18:26:08.292792+0000","last_became_active":"2024-04-23T18:23:56.098723+0000","last_became_peered":"2024-04-23T18:23:56.098723+0000","last_unstale":"2024-04-23T18:26:08.292792+0000","last_undegraded":"2024-04-23T18:26:08.292792+0000","last_fullsized":"2024-04-23T18:26:08.292792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:53.027495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122351+0000","last_change":"2024-04-23T18:23:55.262030+0000","last_active":"2024-04-23T18:26:08.122351+0000","last_peered":"2024-04-23T18:26:08.122351+0000","last_clean":"2024-04-23T18:26:08.122351+0000","last_became_active":"2024-04-23T18:23:55.261659+0000","last_became_peered":"2024-04-23T18:23:55.261659+0000","last_unstale":"2024-04-23T18:26:08.122351+0000","last_undegraded":"2024-04-23T18:26:08.122351+0000","last_fullsized":"2024-04-23T18:26:08.122351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:17.036709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.230040+0000","last_change":"2024-04-23T18:23:56.096511+0000","last_active":"2024-04-23T18:26:08.230040+0000","last_peered":"2024-04-23T18:26:08.230040+0000","last_clean":"2024-04-23T18:26:08.230040+0000","last_became_active":"2024-04-23T18:23:56.096154+0000","last_became_peered":"2024-04-23T18:23:56.096154+0000","last_unstale":"2024-04-23T18:26:08.230040+0000","last_undegraded":"2024-04-23T18:26:08.230040+0000","last_fullsized":"2024-04-23T18:26:08.230040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:03.926178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.806819+0000","last_change":"2024-04-23T18:23:55.263618+0000","last_active":"2024-04-23T18:26:08.806819+0000","last_peered":"2024-04-23T18:26:08.806819+0000","last_clean":"2024-04-23T18:26:08.806819+0000","last_became_active":"2024-04-23T18:23:55.261758+0000","last_became_peered":"2024-04-23T18:23:55.261758+0000","last_unstale":"2024-04-23T18:26:08.806819+0000","last_undegraded":"2024-04-23T18:26:08.806819+0000","last_fullsized":"2024-04-23T18:26:08.806819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:23:27.745668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229933+0000","last_change":"2024-04-23T18:23:56.109292+0000","last_active":"2024-04-23T18:26:08.229933+0000","last_peered":"2024-04-23T18:26:08.229933+0000","last_clean":"2024-04-23T18:26:08.229933+0000","last_became_active":"2024-04-23T18:23:56.108414+0000","last_became_peered":"2024-04-23T18:23:56.108414+0000","last_unstale":"2024-04-23T18:26:08.229933+0000","last_undegraded":"2024-04-23T18:26:08.229933+0000","last_fullsized":"2024-04-23T18:26:08.229933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:13.428782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226673+0000","last_change":"2024-04-23T18:23:56.097122+0000","last_active":"2024-04-23T18:26:08.226673+0000","last_peered":"2024-04-23T18:26:08.226673+0000","last_clean":"2024-04-23T18:26:08.226673+0000","last_became_active":"2024-04-23T18:23:56.096751+0000","last_became_peered":"2024-04-23T18:23:56.096751+0000","last_unstale":"2024-04-23T18:26:08.226673+0000","last_undegraded":"2024-04-23T18:26:08.226673+0000","last_fullsized":"2024-04-23T18:26:08.226673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:00:55.090370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":162,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:12.816687+0000","last_change":"2024-04-23T18:23:55.256458+0000","last_active":"2024-04-23T18:26:12.816687+0000","last_peered":"2024-04-23T18:26:12.816687+0000","last_clean":"2024-04-23T18:26:12.816687+0000","last_became_active":"2024-04-23T18:23:55.256127+0000","last_became_peered":"2024-04-23T18:23:55.256127+0000","last_unstale":"2024-04-23T18:26:12.816687+0000","last_undegraded":"2024-04-23T18:26:12.816687+0000","last_fullsized":"2024-04-23T18:26:12.816687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:03.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218803+0000","last_change":"2024-04-23T18:23:56.103743+0000","last_active":"2024-04-23T18:26:08.218803+0000","last_peered":"2024-04-23T18:26:08.218803+0000","last_clean":"2024-04-23T18:26:08.218803+0000","last_became_active":"2024-04-23T18:23:56.103041+0000","last_became_peered":"2024-04-23T18:23:56.103041+0000","last_unstale":"2024-04-23T18:26:08.218803+0000","last_undegraded":"2024-04-23T18:26:08.218803+0000","last_fullsized":"2024-04-23T18:26:08.218803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:22.419555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292990+0000","last_change":"2024-04-23T18:23:55.257008+0000","last_active":"2024-04-23T18:26:08.292990+0000","last_peered":"2024-04-23T18:26:08.292990+0000","last_clean":"2024-04-23T18:26:08.292990+0000","last_became_active":"2024-04-23T18:23:55.256648+0000","last_became_peered":"2024-04-23T18:23:55.256648+0000","last_unstale":"2024-04-23T18:26:08.292990+0000","last_undegraded":"2024-04-23T18:26:08.292990+0000","last_fullsized":"2024-04-23T18:26:08.292990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:00.867083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123176+0000","last_change":"2024-04-23T18:22:38.610948+0000","last_active":"2024-04-23T18:26:08.123176+0000","last_peered":"2024-04-23T18:26:08.123176+0000","last_clean":"2024-04-23T18:26:08.123176+0000","last_became_active":"2024-04-23T18:22:38.610475+0000","last_became_peered":"2024-04-23T18:22:38.610475+0000","last_unstale":"2024-04-23T18:26:08.123176+0000","last_undegraded":"2024-04-23T18:26:08.123176+0000","last_fullsized":"2024-04-23T18:26:08.123176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_clean_scrub_stamp":"2024-04-23T18:19:07.980519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:08.209087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292611+0000","last_change":"2024-04-23T18:24:36.699127+0000","last_active":"2024-04-23T18:26:08.292611+0000","last_peered":"2024-04-23T18:26:08.292611+0000","last_clean":"2024-04-23T18:26:08.292611+0000","last_became_active":"2024-04-23T18:24:36.698734+0000","last_became_peered":"2024-04-23T18:24:36.698734+0000","last_unstale":"2024-04-23T18:26:08.292611+0000","last_undegraded":"2024-04-23T18:26:08.292611+0000","last_fullsized":"2024-04-23T18:26:08.292611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:28.239466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123284+0000","last_change":"2024-04-23T18:24:36.918722+0000","last_active":"2024-04-23T18:26:08.123284+0000","last_peered":"2024-04-23T18:26:08.123284+0000","last_clean":"2024-04-23T18:26:08.123284+0000","last_became_active":"2024-04-23T18:24:36.915658+0000","last_became_peered":"2024-04-23T18:24:36.915658+0000","last_unstale":"2024-04-23T18:26:08.123284+0000","last_undegraded":"2024-04-23T18:26:08.123284+0000","last_fullsized":"2024-04-23T18:26:08.123284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:34.706193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219189+0000","last_change":"2024-04-23T18:23:55.265853+0000","last_active":"2024-04-23T18:26:08.219189+0000","last_peered":"2024-04-23T18:26:08.219189+0000","last_clean":"2024-04-23T18:26:08.219189+0000","last_became_active":"2024-04-23T18:23:55.265435+0000","last_became_peered":"2024-04-23T18:23:55.265435+0000","last_unstale":"2024-04-23T18:26:08.219189+0000","last_undegraded":"2024-04-23T18:26:08.219189+0000","last_fullsized":"2024-04-23T18:26:08.219189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.969720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293278+0000","last_change":"2024-04-23T18:23:56.085609+0000","last_active":"2024-04-23T18:26:08.293278+0000","last_peered":"2024-04-23T18:26:08.293278+0000","last_clean":"2024-04-23T18:26:08.293278+0000","last_became_active":"2024-04-23T18:23:56.085244+0000","last_became_peered":"2024-04-23T18:23:56.085244+0000","last_unstale":"2024-04-23T18:26:08.293278+0000","last_undegraded":"2024-04-23T18:26:08.293278+0000","last_fullsized":"2024-04-23T18:26:08.293278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.211717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'48","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.341583+0000","last_change":"2024-04-23T18:23:55.260260+0000","last_active":"2024-04-23T18:26:08.341583+0000","last_peered":"2024-04-23T18:26:08.341583+0000","last_clean":"2024-04-23T18:26:08.341583+0000","last_became_active":"2024-04-23T18:23:55.259862+0000","last_became_peered":"2024-04-23T18:23:55.259862+0000","last_unstale":"2024-04-23T18:26:08.341583+0000","last_undegraded":"2024-04-23T18:26:08.341583+0000","last_fullsized":"2024-04-23T18:26:08.341583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:57.833293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219052+0000","last_change":"2024-04-23T18:23:56.098826+0000","last_active":"2024-04-23T18:26:08.219052+0000","last_peered":"2024-04-23T18:26:08.219052+0000","last_clean":"2024-04-23T18:26:08.219052+0000","last_became_active":"2024-04-23T18:23:56.098494+0000","last_became_peered":"2024-04-23T18:23:56.098494+0000","last_unstale":"2024-04-23T18:26:08.219052+0000","last_undegraded":"2024-04-23T18:26:08.219052+0000","last_fullsized":"2024-04-23T18:26:08.219052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:12:24.953330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293238+0000","last_change":"2024-04-23T18:23:55.258174+0000","last_active":"2024-04-23T18:26:08.293238+0000","last_peered":"2024-04-23T18:26:08.293238+0000","last_clean":"2024-04-23T18:26:08.293238+0000","last_became_active":"2024-04-23T18:23:55.257846+0000","last_became_peered":"2024-04-23T18:23:55.257846+0000","last_unstale":"2024-04-23T18:26:08.293238+0000","last_undegraded":"2024-04-23T18:26:08.293238+0000","last_fullsized":"2024-04-23T18:26:08.293238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:35.131566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293829+0000","last_change":"2024-04-23T18:23:56.110758+0000","last_active":"2024-04-23T18:26:08.293829+0000","last_peered":"2024-04-23T18:26:08.293829+0000","last_clean":"2024-04-23T18:26:08.293829+0000","last_became_active":"2024-04-23T18:23:56.110372+0000","last_became_peered":"2024-04-23T18:23:56.110372+0000","last_unstale":"2024-04-23T18:26:08.293829+0000","last_undegraded":"2024-04-23T18:26:08.293829+0000","last_fullsized":"2024-04-23T18:26:08.293829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:40.193088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292460+0000","last_change":"2024-04-23T18:23:55.265714+0000","last_active":"2024-04-23T18:26:08.292460+0000","last_peered":"2024-04-23T18:26:08.292460+0000","last_clean":"2024-04-23T18:26:08.292460+0000","last_became_active":"2024-04-23T18:23:55.265198+0000","last_became_peered":"2024-04-23T18:23:55.265198+0000","last_unstale":"2024-04-23T18:26:08.292460+0000","last_undegraded":"2024-04-23T18:26:08.292460+0000","last_fullsized":"2024-04-23T18:26:08.292460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:10.770748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218909+0000","last_change":"2024-04-23T18:23:56.102347+0000","last_active":"2024-04-23T18:26:08.218909+0000","last_peered":"2024-04-23T18:26:08.218909+0000","last_clean":"2024-04-23T18:26:08.218909+0000","last_became_active":"2024-04-23T18:23:56.100687+0000","last_became_peered":"2024-04-23T18:23:56.100687+0000","last_unstale":"2024-04-23T18:26:08.218909+0000","last_undegraded":"2024-04-23T18:26:08.218909+0000","last_fullsized":"2024-04-23T18:26:08.218909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:20.378730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218670+0000","last_change":"2024-04-23T18:23:55.255943+0000","last_active":"2024-04-23T18:26:08.218670+0000","last_peered":"2024-04-23T18:26:08.218670+0000","last_clean":"2024-04-23T18:26:08.218670+0000","last_became_active":"2024-04-23T18:23:55.255618+0000","last_became_peered":"2024-04-23T18:23:55.255618+0000","last_unstale":"2024-04-23T18:26:08.218670+0000","last_undegraded":"2024-04-23T18:26:08.218670+0000","last_fullsized":"2024-04-23T18:26:08.218670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:01.408533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292542+0000","last_change":"2024-04-23T18:23:56.091664+0000","last_active":"2024-04-23T18:26:08.292542+0000","last_peered":"2024-04-23T18:26:08.292542+0000","last_clean":"2024-04-23T18:26:08.292542+0000","last_became_active":"2024-04-23T18:23:56.091308+0000","last_became_peered":"2024-04-23T18:23:56.091308+0000","last_unstale":"2024-04-23T18:26:08.292542+0000","last_undegraded":"2024-04-23T18:26:08.292542+0000","last_fullsized":"2024-04-23T18:26:08.292542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:31.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.405535+0000","last_change":"2024-04-23T18:23:55.260992+0000","last_active":"2024-04-23T18:26:08.405535+0000","last_peered":"2024-04-23T18:26:08.405535+0000","last_clean":"2024-04-23T18:26:08.405535+0000","last_became_active":"2024-04-23T18:23:55.260679+0000","last_became_peered":"2024-04-23T18:23:55.260679+0000","last_unstale":"2024-04-23T18:26:08.405535+0000","last_undegraded":"2024-04-23T18:26:08.405535+0000","last_fullsized":"2024-04-23T18:26:08.405535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:02:50.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219168+0000","last_change":"2024-04-23T18:23:56.097956+0000","last_active":"2024-04-23T18:26:08.219168+0000","last_peered":"2024-04-23T18:26:08.219168+0000","last_clean":"2024-04-23T18:26:08.219168+0000","last_became_active":"2024-04-23T18:23:56.097231+0000","last_became_peered":"2024-04-23T18:23:56.097231+0000","last_unstale":"2024-04-23T18:26:08.219168+0000","last_undegraded":"2024-04-23T18:26:08.219168+0000","last_fullsized":"2024-04-23T18:26:08.219168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:10.945857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292582+0000","last_change":"2024-04-23T18:24:36.714223+0000","last_active":"2024-04-23T18:26:08.292582+0000","last_peered":"2024-04-23T18:26:08.292582+0000","last_clean":"2024-04-23T18:26:08.292582+0000","last_became_active":"2024-04-23T18:24:36.700975+0000","last_became_peered":"2024-04-23T18:24:36.700975+0000","last_unstale":"2024-04-23T18:26:08.292582+0000","last_undegraded":"2024-04-23T18:26:08.292582+0000","last_fullsized":"2024-04-23T18:26:08.292582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:58:50.227699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123311+0000","last_change":"2024-04-23T18:23:56.096889+0000","last_active":"2024-04-23T18:26:08.123311+0000","last_peered":"2024-04-23T18:26:08.123311+0000","last_clean":"2024-04-23T18:26:08.123311+0000","last_became_active":"2024-04-23T18:23:56.096520+0000","last_became_peered":"2024-04-23T18:23:56.096520+0000","last_unstale":"2024-04-23T18:26:08.123311+0000","last_undegraded":"2024-04-23T18:26:08.123311+0000","last_fullsized":"2024-04-23T18:26:08.123311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:34:53.785481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":159,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.808804+0000","last_change":"2024-04-23T18:23:55.258710+0000","last_active":"2024-04-23T18:26:08.808804+0000","last_peered":"2024-04-23T18:26:08.808804+0000","last_clean":"2024-04-23T18:26:08.808804+0000","last_became_active":"2024-04-23T18:23:55.256247+0000","last_became_peered":"2024-04-23T18:23:55.256247+0000","last_unstale":"2024-04-23T18:26:08.808804+0000","last_undegraded":"2024-04-23T18:26:08.808804+0000","last_fullsized":"2024-04-23T18:26:08.808804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:20.551614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123532+0000","last_change":"2024-04-23T18:23:56.101862+0000","last_active":"2024-04-23T18:26:08.123532+0000","last_peered":"2024-04-23T18:26:08.123532+0000","last_clean":"2024-04-23T18:26:08.123532+0000","last_became_active":"2024-04-23T18:23:56.101458+0000","last_became_peered":"2024-04-23T18:23:56.101458+0000","last_unstale":"2024-04-23T18:26:08.123532+0000","last_undegraded":"2024-04-23T18:26:08.123532+0000","last_fullsized":"2024-04-23T18:26:08.123532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:59.292077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'41","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.333872+0000","last_change":"2024-04-23T18:23:55.258292+0000","last_active":"2024-04-23T18:26:08.333872+0000","last_peered":"2024-04-23T18:26:08.333872+0000","last_clean":"2024-04-23T18:26:08.333872+0000","last_became_active":"2024-04-23T18:23:55.257954+0000","last_became_peered":"2024-04-23T18:23:55.257954+0000","last_unstale":"2024-04-23T18:26:08.333872+0000","last_undegraded":"2024-04-23T18:26:08.333872+0000","last_fullsized":"2024-04-23T18:26:08.333872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:39.105967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123377+0000","last_change":"2024-04-23T18:23:56.105977+0000","last_active":"2024-04-23T18:26:08.123377+0000","last_peered":"2024-04-23T18:26:08.123377+0000","last_clean":"2024-04-23T18:26:08.123377+0000","last_became_active":"2024-04-23T18:23:56.105582+0000","last_became_peered":"2024-04-23T18:23:56.105582+0000","last_unstale":"2024-04-23T18:26:08.123377+0000","last_undegraded":"2024-04-23T18:26:08.123377+0000","last_fullsized":"2024-04-23T18:26:08.123377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:28:10.700684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293550+0000","last_change":"2024-04-23T18:23:55.252095+0000","last_active":"2024-04-23T18:26:08.293550+0000","last_peered":"2024-04-23T18:26:08.293550+0000","last_clean":"2024-04-23T18:26:08.293550+0000","last_became_active":"2024-04-23T18:23:55.251702+0000","last_became_peered":"2024-04-23T18:23:55.251702+0000","last_unstale":"2024-04-23T18:26:08.293550+0000","last_undegraded":"2024-04-23T18:26:08.293550+0000","last_fullsized":"2024-04-23T18:26:08.293550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:21.625905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123737+0000","last_change":"2024-04-23T18:23:56.101997+0000","last_active":"2024-04-23T18:26:08.123737+0000","last_peered":"2024-04-23T18:26:08.123737+0000","last_clean":"2024-04-23T18:26:08.123737+0000","last_became_active":"2024-04-23T18:23:56.101629+0000","last_became_peered":"2024-04-23T18:23:56.101629+0000","last_unstale":"2024-04-23T18:26:08.123737+0000","last_undegraded":"2024-04-23T18:26:08.123737+0000","last_fullsized":"2024-04-23T18:26:08.123737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:42.484413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292621+0000","last_change":"2024-04-23T18:23:56.090543+0000","last_active":"2024-04-23T18:26:08.292621+0000","last_peered":"2024-04-23T18:26:08.292621+0000","last_clean":"2024-04-23T18:26:08.292621+0000","last_became_active":"2024-04-23T18:23:56.090191+0000","last_became_peered":"2024-04-23T18:23:56.090191+0000","last_unstale":"2024-04-23T18:26:08.292621+0000","last_undegraded":"2024-04-23T18:26:08.292621+0000","last_fullsized":"2024-04-23T18:26:08.292621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:01.401822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.127145+0000","last_change":"2024-04-23T18:23:56.105537+0000","last_active":"2024-04-23T18:26:08.127145+0000","last_peered":"2024-04-23T18:26:08.127145+0000","last_clean":"2024-04-23T18:26:08.127145+0000","last_became_active":"2024-04-23T18:23:56.105130+0000","last_became_peered":"2024-04-23T18:23:56.105130+0000","last_unstale":"2024-04-23T18:26:08.127145+0000","last_undegraded":"2024-04-23T18:26:08.127145+0000","last_fullsized":"2024-04-23T18:26:08.127145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:59.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.126910+0000","last_change":"2024-04-23T18:23:56.097765+0000","last_active":"2024-04-23T18:26:08.126910+0000","last_peered":"2024-04-23T18:26:08.126910+0000","last_clean":"2024-04-23T18:26:08.126910+0000","last_became_active":"2024-04-23T18:23:56.097439+0000","last_became_peered":"2024-04-23T18:23:56.097439+0000","last_unstale":"2024-04-23T18:26:08.126910+0000","last_undegraded":"2024-04-23T18:26:08.126910+0000","last_fullsized":"2024-04-23T18:26:08.126910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:31.529751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293019+0000","last_change":"2024-04-23T18:23:56.108548+0000","last_active":"2024-04-23T18:26:08.293019+0000","last_peered":"2024-04-23T18:26:08.293019+0000","last_clean":"2024-04-23T18:26:08.293019+0000","last_became_active":"2024-04-23T18:23:56.108214+0000","last_became_peered":"2024-04-23T18:23:56.108214+0000","last_unstale":"2024-04-23T18:26:08.293019+0000","last_undegraded":"2024-04-23T18:26:08.293019+0000","last_fullsized":"2024-04-23T18:26:08.293019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:41.738754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293157+0000","last_change":"2024-04-23T18:24:36.701809+0000","last_active":"2024-04-23T18:26:08.293157+0000","last_peered":"2024-04-23T18:26:08.293157+0000","last_clean":"2024-04-23T18:26:08.293157+0000","last_became_active":"2024-04-23T18:24:36.701405+0000","last_became_peered":"2024-04-23T18:24:36.701405+0000","last_unstale":"2024-04-23T18:26:08.293157+0000","last_undegraded":"2024-04-23T18:26:08.293157+0000","last_fullsized":"2024-04-23T18:26:08.293157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:03.215622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292776+0000","last_change":"2024-04-23T18:23:56.093969+0000","last_active":"2024-04-23T18:26:08.292776+0000","last_peered":"2024-04-23T18:26:08.292776+0000","last_clean":"2024-04-23T18:26:08.292776+0000","last_became_active":"2024-04-23T18:23:56.093568+0000","last_became_peered":"2024-04-23T18:23:56.093568+0000","last_unstale":"2024-04-23T18:26:08.292776+0000","last_undegraded":"2024-04-23T18:26:08.292776+0000","last_fullsized":"2024-04-23T18:26:08.292776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:08.361305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.127150+0000","last_change":"2024-04-23T18:23:56.096106+0000","last_active":"2024-04-23T18:26:08.127150+0000","last_peered":"2024-04-23T18:26:08.127150+0000","last_clean":"2024-04-23T18:26:08.127150+0000","last_became_active":"2024-04-23T18:23:56.095745+0000","last_became_peered":"2024-04-23T18:23:56.095745+0000","last_unstale":"2024-04-23T18:26:08.127150+0000","last_undegraded":"2024-04-23T18:26:08.127150+0000","last_fullsized":"2024-04-23T18:26:08.127150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:01.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218970+0000","last_change":"2024-04-23T18:23:56.095474+0000","last_active":"2024-04-23T18:26:08.218970+0000","last_peered":"2024-04-23T18:26:08.218970+0000","last_clean":"2024-04-23T18:26:08.218970+0000","last_became_active":"2024-04-23T18:23:56.095116+0000","last_became_peered":"2024-04-23T18:23:56.095116+0000","last_unstale":"2024-04-23T18:26:08.218970+0000","last_undegraded":"2024-04-23T18:26:08.218970+0000","last_fullsized":"2024-04-23T18:26:08.218970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:50:00.339558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226608+0000","last_change":"2024-04-23T18:23:56.096022+0000","last_active":"2024-04-23T18:26:08.226608+0000","last_peered":"2024-04-23T18:26:08.226608+0000","last_clean":"2024-04-23T18:26:08.226608+0000","last_became_active":"2024-04-23T18:23:56.095493+0000","last_became_peered":"2024-04-23T18:23:56.095493+0000","last_unstale":"2024-04-23T18:26:08.226608+0000","last_undegraded":"2024-04-23T18:26:08.226608+0000","last_fullsized":"2024-04-23T18:26:08.226608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:42.209072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218607+0000","last_change":"2024-04-23T18:23:56.101697+0000","last_active":"2024-04-23T18:26:08.218607+0000","last_peered":"2024-04-23T18:26:08.218607+0000","last_clean":"2024-04-23T18:26:08.218607+0000","last_became_active":"2024-04-23T18:23:56.098708+0000","last_became_peered":"2024-04-23T18:23:56.098708+0000","last_unstale":"2024-04-23T18:26:08.218607+0000","last_undegraded":"2024-04-23T18:26:08.218607+0000","last_fullsized":"2024-04-23T18:26:08.218607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:14.170700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292721+0000","last_change":"2024-04-23T18:23:56.096982+0000","last_active":"2024-04-23T18:26:08.292721+0000","last_peered":"2024-04-23T18:26:08.292721+0000","last_clean":"2024-04-23T18:26:08.292721+0000","last_became_active":"2024-04-23T18:23:56.096608+0000","last_became_peered":"2024-04-23T18:23:56.096608+0000","last_unstale":"2024-04-23T18:26:08.292721+0000","last_undegraded":"2024-04-23T18:26:08.292721+0000","last_fullsized":"2024-04-23T18:26:08.292721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:50.329337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293339+0000","last_change":"2024-04-23T18:23:56.100345+0000","last_active":"2024-04-23T18:26:08.293339+0000","last_peered":"2024-04-23T18:26:08.293339+0000","last_clean":"2024-04-23T18:26:08.293339+0000","last_became_active":"2024-04-23T18:23:56.100043+0000","last_became_peered":"2024-04-23T18:23:56.100043+0000","last_unstale":"2024-04-23T18:26:08.293339+0000","last_undegraded":"2024-04-23T18:26:08.293339+0000","last_fullsized":"2024-04-23T18:26:08.293339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:36.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292603+0000","last_change":"2024-04-23T18:23:56.101947+0000","last_active":"2024-04-23T18:26:08.292603+0000","last_peered":"2024-04-23T18:26:08.292603+0000","last_clean":"2024-04-23T18:26:08.292603+0000","last_became_active":"2024-04-23T18:23:56.101627+0000","last_became_peered":"2024-04-23T18:23:56.101627+0000","last_unstale":"2024-04-23T18:26:08.292603+0000","last_undegraded":"2024-04-23T18:26:08.292603+0000","last_fullsized":"2024-04-23T18:26:08.292603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:21:43.703653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106755,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602323,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1411,"num_read_kb":826,"num_write":373,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":737280,"data_stored":2003992,"data_compressed":90690,"data_compressed_allocated":122880,"data_compressed_original":1757184,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":461377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":1261854,"data_compressed":46511,"data_compressed_allocated":61440,"data_compressed_original":729088,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":863382,"data_compressed":22051,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":863760,"data_compressed":22051,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":1065216,"data_compressed":38490,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":471056,"data_compressed":2905,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":539336,"data_compressed":4770,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":472004,"data_compressed":2905,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":539908,"data_compressed":4770,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":1191156,"data_compressed":37396,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":471087,"data_compressed":2905,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1194481,"data_compressed":44646,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70292,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70864,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134952,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":202325,"data_compressed":8021,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:26:14.299 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-23T18:26:14.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:26:14.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.445+0000 7fc906337640 1 -- 172.21.15.19:0/3401709428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc900101110 msgr2=0x7fc9001014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.445+0000 7fc906337640 1 --2- 172.21.15.19:0/3401709428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc900101110 0x7fc9001014f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009900 tx=0x7fc8f002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:14.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.445+0000 7fc906337640 1 -- 172.21.15.19:0/3401709428 shutdown_connections 2024-04-23T18:26:14.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.445+0000 7fc906337640 1 --2- 172.21.15.19:0/3401709428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc900106190 0x7fc90010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.445+0000 7fc906337640 1 --2- 172.21.15.19:0/3401709428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc900101ac0 0x7fc900105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.445+0000 7fc906337640 1 --2- 172.21.15.19:0/3401709428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc900101110 0x7fc9001014f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.445+0000 7fc906337640 1 -- 172.21.15.19:0/3401709428 >> 172.21.15.19:0/3401709428 conn(0x7fc900077fb0 msgr2=0x7fc9000783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.445+0000 7fc906337640 1 -- 172.21.15.19:0/3401709428 shutdown_connections 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.445+0000 7fc906337640 1 -- 172.21.15.19:0/3401709428 wait complete. 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc906337640 1 Processor -- start 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc906337640 1 -- start start 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc906337640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc900101110 0x7fc90019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc906337640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc900106190 0x7fc90019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc906337640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9001985e0 0x7fc900198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc906337640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001110d0 con 0x7fc9001985e0 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc906337640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc900110f50 con 0x7fc900101110 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc906337640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc900111250 con 0x7fc900106190 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc9048d4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9001985e0 0x7fc900198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc9048d4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9001985e0 0x7fc900198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53434/0 (socket says 172.21.15.19:53434) 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc9048d4640 1 -- 172.21.15.19:0/510858317 learned_addr learned my addr 172.21.15.19:0/510858317 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc8fffff640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc900101110 0x7fc90019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc8ff7fe640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc900106190 0x7fc90019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc9048d4640 1 -- 172.21.15.19:0/510858317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc900106190 msgr2=0x7fc90019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc9048d4640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc900106190 0x7fc90019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc9048d4640 1 -- 172.21.15.19:0/510858317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc900101110 msgr2=0x7fc90019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc9048d4640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc900101110 0x7fc90019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.446+0000 7fc9048d4640 1 -- 172.21.15.19:0/510858317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc900199100 con 0x7fc9001985e0 2024-04-23T18:26:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.447+0000 7fc9048d4640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9001985e0 0x7fc900198a90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800ea60 tx=0x7fc8f800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.447+0000 7fc8fd7fa640 1 -- 172.21.15.19:0/510858317 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800ce40 con 0x7fc9001985e0 2024-04-23T18:26:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.447+0000 7fc906337640 1 -- 172.21.15.19:0/510858317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9001993f0 con 0x7fc9001985e0 2024-04-23T18:26:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.447+0000 7fc8fd7fa640 1 -- 172.21.15.19:0/510858317 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f8005020 con 0x7fc9001985e0 2024-04-23T18:26:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.447+0000 7fc8fd7fa640 1 -- 172.21.15.19:0/510858317 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8010670 con 0x7fc9001985e0 2024-04-23T18:26:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.447+0000 7fc906337640 1 -- 172.21.15.19:0/510858317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a55f0 con 0x7fc9001985e0 2024-04-23T18:26:14.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.448+0000 7fc906337640 1 -- 172.21.15.19:0/510858317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9000691c0 con 0x7fc9001985e0 2024-04-23T18:26:14.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.449+0000 7fc8fd7fa640 1 -- 172.21.15.19:0/510858317 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc8f80054b0 con 0x7fc9001985e0 2024-04-23T18:26:14.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.450+0000 7fc8fd7fa640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8e80a3e40 0x7fc8e80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.450+0000 7fc8fffff640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8e80a3e40 0x7fc8e80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.450+0000 7fc8fd7fa640 1 -- 172.21.15.19:0/510858317 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc8f80d0ff0 con 0x7fc9001985e0 2024-04-23T18:26:14.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.450+0000 7fc8fffff640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8e80a3e40 0x7fc8e80a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0031440 tx=0x7fc8f0005b50 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:14.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.454+0000 7fc8fd7fa640 1 -- 172.21.15.19:0/510858317 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8f809d410 con 0x7fc9001985e0 2024-04-23T18:26:14.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:14 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1647890227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:14.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:14 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/828083611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:14.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:14 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1647890227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:14.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:14 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/828083611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.640+0000 7fc906337640 1 -- 172.21.15.19:0/510858317 --> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc900101ac0 con 0x7fc8e80a3e40 2024-04-23T18:26:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.647+0000 7fc8fd7fa640 1 -- 172.21.15.19:0/510858317 <== mgr.25004 v2:172.21.15.19:6800/1177101685 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258074 (secure 0 0 0) 0x7fc900101ac0 con 0x7fc8e80a3e40 2024-04-23T18:26:14.643 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:14.644 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-23T18:26:14.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 -- 172.21.15.19:0/510858317 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8e80a3e40 msgr2=0x7fc8e80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8e80a3e40 0x7fc8e80a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0031440 tx=0x7fc8f0005b50 comp rx=0 tx=0).stop 2024-04-23T18:26:14.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 -- 172.21.15.19:0/510858317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9001985e0 msgr2=0x7fc900198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9001985e0 0x7fc900198a90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800ea60 tx=0x7fc8f800ef20 comp rx=0 tx=0).stop 2024-04-23T18:26:14.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 -- 172.21.15.19:0/510858317 shutdown_connections 2024-04-23T18:26:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8e80a3e40 0x7fc8e80a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9001985e0 0x7fc900198a90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc900106190 0x7fc90019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 --2- 172.21.15.19:0/510858317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc900101110 0x7fc90019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 -- 172.21.15.19:0/510858317 >> 172.21.15.19:0/510858317 conn(0x7fc900077fb0 msgr2=0x7fc90006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 -- 172.21.15.19:0/510858317 shutdown_connections 2024-04-23T18:26:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.651+0000 7fc906337640 1 -- 172.21.15.19:0/510858317 wait complete. 2024-04-23T18:26:14.655 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-23T18:26:14.135885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192925,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1547,"num_read_kb":945,"num_write":556,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":941,"ondisk_log_size":941,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382144,"kb_used_data":12156,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559680,"statfs":{"total":1151940427776,"available":1151549112320,"internally_reserved":0,"allocated":12447744,"data_stored":9394617,"data_compressed":229400,"data_compressed_allocated":303104,"data_compressed_original":3616768,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.986055"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292994+0000","last_change":"2024-04-23T18:23:56.098292+0000","last_active":"2024-04-23T18:26:08.292994+0000","last_peered":"2024-04-23T18:26:08.292994+0000","last_clean":"2024-04-23T18:26:08.292994+0000","last_became_active":"2024-04-23T18:23:56.096623+0000","last_became_peered":"2024-04-23T18:23:56.096623+0000","last_unstale":"2024-04-23T18:26:08.292994+0000","last_undegraded":"2024-04-23T18:26:08.292994+0000","last_fullsized":"2024-04-23T18:26:08.292994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:06.186581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293701+0000","last_change":"2024-04-23T18:23:56.096807+0000","last_active":"2024-04-23T18:26:08.293701+0000","last_peered":"2024-04-23T18:26:08.293701+0000","last_clean":"2024-04-23T18:26:08.293701+0000","last_became_active":"2024-04-23T18:23:56.096327+0000","last_became_peered":"2024-04-23T18:23:56.096327+0000","last_unstale":"2024-04-23T18:26:08.293701+0000","last_undegraded":"2024-04-23T18:26:08.293701+0000","last_fullsized":"2024-04-23T18:26:08.293701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:56:16.391821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293324+0000","last_change":"2024-04-23T18:23:56.101615+0000","last_active":"2024-04-23T18:26:08.293324+0000","last_peered":"2024-04-23T18:26:08.293324+0000","last_clean":"2024-04-23T18:26:08.293324+0000","last_became_active":"2024-04-23T18:23:56.100929+0000","last_became_peered":"2024-04-23T18:23:56.100929+0000","last_unstale":"2024-04-23T18:26:08.293324+0000","last_undegraded":"2024-04-23T18:26:08.293324+0000","last_fullsized":"2024-04-23T18:26:08.293324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:25.547029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293249+0000","last_change":"2024-04-23T18:23:56.103627+0000","last_active":"2024-04-23T18:26:08.293249+0000","last_peered":"2024-04-23T18:26:08.293249+0000","last_clean":"2024-04-23T18:26:08.293249+0000","last_became_active":"2024-04-23T18:23:56.102752+0000","last_became_peered":"2024-04-23T18:23:56.102752+0000","last_unstale":"2024-04-23T18:26:08.293249+0000","last_undegraded":"2024-04-23T18:26:08.293249+0000","last_fullsized":"2024-04-23T18:26:08.293249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:28.460342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218930+0000","last_change":"2024-04-23T18:23:56.094889+0000","last_active":"2024-04-23T18:26:08.218930+0000","last_peered":"2024-04-23T18:26:08.218930+0000","last_clean":"2024-04-23T18:26:08.218930+0000","last_became_active":"2024-04-23T18:23:56.094262+0000","last_became_peered":"2024-04-23T18:23:56.094262+0000","last_unstale":"2024-04-23T18:26:08.218930+0000","last_undegraded":"2024-04-23T18:26:08.218930+0000","last_fullsized":"2024-04-23T18:26:08.218930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:22.332114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229648+0000","last_change":"2024-04-23T18:23:56.108893+0000","last_active":"2024-04-23T18:26:08.229648+0000","last_peered":"2024-04-23T18:26:08.229648+0000","last_clean":"2024-04-23T18:26:08.229648+0000","last_became_active":"2024-04-23T18:23:56.108229+0000","last_became_peered":"2024-04-23T18:23:56.108229+0000","last_unstale":"2024-04-23T18:26:08.229648+0000","last_undegraded":"2024-04-23T18:26:08.229648+0000","last_fullsized":"2024-04-23T18:26:08.229648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:50.930321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123153+0000","last_change":"2024-04-23T18:23:56.098674+0000","last_active":"2024-04-23T18:26:08.123153+0000","last_peered":"2024-04-23T18:26:08.123153+0000","last_clean":"2024-04-23T18:26:08.123153+0000","last_became_active":"2024-04-23T18:23:56.098299+0000","last_became_peered":"2024-04-23T18:23:56.098299+0000","last_unstale":"2024-04-23T18:26:08.123153+0000","last_undegraded":"2024-04-23T18:26:08.123153+0000","last_fullsized":"2024-04-23T18:26:08.123153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:05.711410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218497+0000","last_change":"2024-04-23T18:23:56.097437+0000","last_active":"2024-04-23T18:26:08.218497+0000","last_peered":"2024-04-23T18:26:08.218497+0000","last_clean":"2024-04-23T18:26:08.218497+0000","last_became_active":"2024-04-23T18:23:56.095702+0000","last_became_peered":"2024-04-23T18:23:56.095702+0000","last_unstale":"2024-04-23T18:26:08.218497+0000","last_undegraded":"2024-04-23T18:26:08.218497+0000","last_fullsized":"2024-04-23T18:26:08.218497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:35.754832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293226+0000","last_change":"2024-04-23T18:23:56.096060+0000","last_active":"2024-04-23T18:26:08.293226+0000","last_peered":"2024-04-23T18:26:08.293226+0000","last_clean":"2024-04-23T18:26:08.293226+0000","last_became_active":"2024-04-23T18:23:56.095597+0000","last_became_peered":"2024-04-23T18:23:56.095597+0000","last_unstale":"2024-04-23T18:26:08.293226+0000","last_undegraded":"2024-04-23T18:26:08.293226+0000","last_fullsized":"2024-04-23T18:26:08.293226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:59.735347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229983+0000","last_change":"2024-04-23T18:23:56.109033+0000","last_active":"2024-04-23T18:26:08.229983+0000","last_peered":"2024-04-23T18:26:08.229983+0000","last_clean":"2024-04-23T18:26:08.229983+0000","last_became_active":"2024-04-23T18:23:56.108472+0000","last_became_peered":"2024-04-23T18:23:56.108472+0000","last_unstale":"2024-04-23T18:26:08.229983+0000","last_undegraded":"2024-04-23T18:26:08.229983+0000","last_fullsized":"2024-04-23T18:26:08.229983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:32.491181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123484+0000","last_change":"2024-04-23T18:23:56.097771+0000","last_active":"2024-04-23T18:26:08.123484+0000","last_peered":"2024-04-23T18:26:08.123484+0000","last_clean":"2024-04-23T18:26:08.123484+0000","last_became_active":"2024-04-23T18:23:56.097403+0000","last_became_peered":"2024-04-23T18:23:56.097403+0000","last_unstale":"2024-04-23T18:26:08.123484+0000","last_undegraded":"2024-04-23T18:26:08.123484+0000","last_fullsized":"2024-04-23T18:26:08.123484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:48:36.261543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293091+0000","last_change":"2024-04-23T18:23:56.108550+0000","last_active":"2024-04-23T18:26:08.293091+0000","last_peered":"2024-04-23T18:26:08.293091+0000","last_clean":"2024-04-23T18:26:08.293091+0000","last_became_active":"2024-04-23T18:23:56.108191+0000","last_became_peered":"2024-04-23T18:23:56.108191+0000","last_unstale":"2024-04-23T18:26:08.293091+0000","last_undegraded":"2024-04-23T18:26:08.293091+0000","last_fullsized":"2024-04-23T18:26:08.293091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:25.142466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293119+0000","last_change":"2024-04-23T18:23:56.092573+0000","last_active":"2024-04-23T18:26:08.293119+0000","last_peered":"2024-04-23T18:26:08.293119+0000","last_clean":"2024-04-23T18:26:08.293119+0000","last_became_active":"2024-04-23T18:23:56.092095+0000","last_became_peered":"2024-04-23T18:23:56.092095+0000","last_unstale":"2024-04-23T18:26:08.293119+0000","last_undegraded":"2024-04-23T18:26:08.293119+0000","last_fullsized":"2024-04-23T18:26:08.293119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:01.601206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292816+0000","last_change":"2024-04-23T18:23:56.102073+0000","last_active":"2024-04-23T18:26:08.292816+0000","last_peered":"2024-04-23T18:26:08.292816+0000","last_clean":"2024-04-23T18:26:08.292816+0000","last_became_active":"2024-04-23T18:23:56.101059+0000","last_became_peered":"2024-04-23T18:23:56.101059+0000","last_unstale":"2024-04-23T18:26:08.292816+0000","last_undegraded":"2024-04-23T18:26:08.292816+0000","last_fullsized":"2024-04-23T18:26:08.292816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:37.904099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292355+0000","last_change":"2024-04-23T18:23:56.108482+0000","last_active":"2024-04-23T18:26:08.292355+0000","last_peered":"2024-04-23T18:26:08.292355+0000","last_clean":"2024-04-23T18:26:08.292355+0000","last_became_active":"2024-04-23T18:23:56.108128+0000","last_became_peered":"2024-04-23T18:23:56.108128+0000","last_unstale":"2024-04-23T18:26:08.292355+0000","last_undegraded":"2024-04-23T18:26:08.292355+0000","last_fullsized":"2024-04-23T18:26:08.292355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:51.925881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292324+0000","last_change":"2024-04-23T18:23:56.099691+0000","last_active":"2024-04-23T18:26:08.292324+0000","last_peered":"2024-04-23T18:26:08.292324+0000","last_clean":"2024-04-23T18:26:08.292324+0000","last_became_active":"2024-04-23T18:23:56.099272+0000","last_became_peered":"2024-04-23T18:23:56.099272+0000","last_unstale":"2024-04-23T18:26:08.292324+0000","last_undegraded":"2024-04-23T18:26:08.292324+0000","last_fullsized":"2024-04-23T18:26:08.292324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:41.834104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292954+0000","last_change":"2024-04-23T18:23:56.097965+0000","last_active":"2024-04-23T18:26:08.292954+0000","last_peered":"2024-04-23T18:26:08.292954+0000","last_clean":"2024-04-23T18:26:08.292954+0000","last_became_active":"2024-04-23T18:23:56.096766+0000","last_became_peered":"2024-04-23T18:23:56.096766+0000","last_unstale":"2024-04-23T18:26:08.292954+0000","last_undegraded":"2024-04-23T18:26:08.292954+0000","last_fullsized":"2024-04-23T18:26:08.292954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:40:00.076939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122351+0000","last_change":"2024-04-23T18:23:56.095158+0000","last_active":"2024-04-23T18:26:08.122351+0000","last_peered":"2024-04-23T18:26:08.122351+0000","last_clean":"2024-04-23T18:26:08.122351+0000","last_became_active":"2024-04-23T18:23:56.094812+0000","last_became_peered":"2024-04-23T18:23:56.094812+0000","last_unstale":"2024-04-23T18:26:08.122351+0000","last_undegraded":"2024-04-23T18:26:08.122351+0000","last_fullsized":"2024-04-23T18:26:08.122351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:40.874698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.126943+0000","last_change":"2024-04-23T18:23:56.099358+0000","last_active":"2024-04-23T18:26:08.126943+0000","last_peered":"2024-04-23T18:26:08.126943+0000","last_clean":"2024-04-23T18:26:08.126943+0000","last_became_active":"2024-04-23T18:23:56.099031+0000","last_became_peered":"2024-04-23T18:23:56.099031+0000","last_unstale":"2024-04-23T18:26:08.126943+0000","last_undegraded":"2024-04-23T18:26:08.126943+0000","last_fullsized":"2024-04-23T18:26:08.126943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:29.634932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.805802+0000","last_change":"2024-04-23T18:23:55.262939+0000","last_active":"2024-04-23T18:26:08.805802+0000","last_peered":"2024-04-23T18:26:08.805802+0000","last_clean":"2024-04-23T18:26:08.805802+0000","last_became_active":"2024-04-23T18:23:55.262609+0000","last_became_peered":"2024-04-23T18:23:55.262609+0000","last_unstale":"2024-04-23T18:26:08.805802+0000","last_undegraded":"2024-04-23T18:26:08.805802+0000","last_fullsized":"2024-04-23T18:26:08.805802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:12.418349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218972+0000","last_change":"2024-04-23T18:23:56.092182+0000","last_active":"2024-04-23T18:26:08.218972+0000","last_peered":"2024-04-23T18:26:08.218972+0000","last_clean":"2024-04-23T18:26:08.218972+0000","last_became_active":"2024-04-23T18:23:56.091444+0000","last_became_peered":"2024-04-23T18:23:56.091444+0000","last_unstale":"2024-04-23T18:26:08.218972+0000","last_undegraded":"2024-04-23T18:26:08.218972+0000","last_fullsized":"2024-04-23T18:26:08.218972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:00.775498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219056+0000","last_change":"2024-04-23T18:23:55.259880+0000","last_active":"2024-04-23T18:26:08.219056+0000","last_peered":"2024-04-23T18:26:08.219056+0000","last_clean":"2024-04-23T18:26:08.219056+0000","last_became_active":"2024-04-23T18:23:55.259576+0000","last_became_peered":"2024-04-23T18:23:55.259576+0000","last_unstale":"2024-04-23T18:26:08.219056+0000","last_undegraded":"2024-04-23T18:26:08.219056+0000","last_fullsized":"2024-04-23T18:26:08.219056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:13.783776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293588+0000","last_change":"2024-04-23T18:23:56.094307+0000","last_active":"2024-04-23T18:26:08.293588+0000","last_peered":"2024-04-23T18:26:08.293588+0000","last_clean":"2024-04-23T18:26:08.293588+0000","last_became_active":"2024-04-23T18:23:56.093893+0000","last_became_peered":"2024-04-23T18:23:56.093893+0000","last_unstale":"2024-04-23T18:26:08.293588+0000","last_undegraded":"2024-04-23T18:26:08.293588+0000","last_fullsized":"2024-04-23T18:26:08.293588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:12:35.385497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219326+0000","last_change":"2024-04-23T18:23:55.263137+0000","last_active":"2024-04-23T18:26:08.219326+0000","last_peered":"2024-04-23T18:26:08.219326+0000","last_clean":"2024-04-23T18:26:08.219326+0000","last_became_active":"2024-04-23T18:23:55.262553+0000","last_became_peered":"2024-04-23T18:23:55.262553+0000","last_unstale":"2024-04-23T18:26:08.219326+0000","last_undegraded":"2024-04-23T18:26:08.219326+0000","last_fullsized":"2024-04-23T18:26:08.219326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:33:42.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292576+0000","last_change":"2024-04-23T18:23:56.091098+0000","last_active":"2024-04-23T18:26:08.292576+0000","last_peered":"2024-04-23T18:26:08.292576+0000","last_clean":"2024-04-23T18:26:08.292576+0000","last_became_active":"2024-04-23T18:23:56.090635+0000","last_became_peered":"2024-04-23T18:23:56.090635+0000","last_unstale":"2024-04-23T18:26:08.292576+0000","last_undegraded":"2024-04-23T18:26:08.292576+0000","last_fullsized":"2024-04-23T18:26:08.292576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:17.787937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219108+0000","last_change":"2024-04-23T18:23:55.262146+0000","last_active":"2024-04-23T18:26:08.219108+0000","last_peered":"2024-04-23T18:26:08.219108+0000","last_clean":"2024-04-23T18:26:08.219108+0000","last_became_active":"2024-04-23T18:23:55.261838+0000","last_became_peered":"2024-04-23T18:23:55.261838+0000","last_unstale":"2024-04-23T18:26:08.219108+0000","last_undegraded":"2024-04-23T18:26:08.219108+0000","last_fullsized":"2024-04-23T18:26:08.219108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:42:39.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122228+0000","last_change":"2024-04-23T18:24:36.915394+0000","last_active":"2024-04-23T18:26:08.122228+0000","last_peered":"2024-04-23T18:26:08.122228+0000","last_clean":"2024-04-23T18:26:08.122228+0000","last_became_active":"2024-04-23T18:24:36.915072+0000","last_became_peered":"2024-04-23T18:24:36.915072+0000","last_unstale":"2024-04-23T18:26:08.122228+0000","last_undegraded":"2024-04-23T18:26:08.122228+0000","last_fullsized":"2024-04-23T18:26:08.122228+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.132410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226797+0000","last_change":"2024-04-23T18:23:55.261399+0000","last_active":"2024-04-23T18:26:08.226797+0000","last_peered":"2024-04-23T18:26:08.226797+0000","last_clean":"2024-04-23T18:26:08.226797+0000","last_became_active":"2024-04-23T18:23:55.261012+0000","last_became_peered":"2024-04-23T18:23:55.261012+0000","last_unstale":"2024-04-23T18:26:08.226797+0000","last_undegraded":"2024-04-23T18:26:08.226797+0000","last_fullsized":"2024-04-23T18:26:08.226797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:31.377776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218869+0000","last_change":"2024-04-23T18:23:56.096785+0000","last_active":"2024-04-23T18:26:08.218869+0000","last_peered":"2024-04-23T18:26:08.218869+0000","last_clean":"2024-04-23T18:26:08.218869+0000","last_became_active":"2024-04-23T18:23:56.096420+0000","last_became_peered":"2024-04-23T18:23:56.096420+0000","last_unstale":"2024-04-23T18:26:08.218869+0000","last_undegraded":"2024-04-23T18:26:08.218869+0000","last_fullsized":"2024-04-23T18:26:08.218869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:20:49.215071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229615+0000","last_change":"2024-04-23T18:23:55.260645+0000","last_active":"2024-04-23T18:26:08.229615+0000","last_peered":"2024-04-23T18:26:08.229615+0000","last_clean":"2024-04-23T18:26:08.229615+0000","last_became_active":"2024-04-23T18:23:55.260118+0000","last_became_peered":"2024-04-23T18:23:55.260118+0000","last_unstale":"2024-04-23T18:26:08.229615+0000","last_undegraded":"2024-04-23T18:26:08.229615+0000","last_fullsized":"2024-04-23T18:26:08.229615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:14:12.066297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226773+0000","last_change":"2024-04-23T18:24:36.702766+0000","last_active":"2024-04-23T18:26:08.226773+0000","last_peered":"2024-04-23T18:26:08.226773+0000","last_clean":"2024-04-23T18:26:08.226773+0000","last_became_active":"2024-04-23T18:24:36.702309+0000","last_became_peered":"2024-04-23T18:24:36.702309+0000","last_unstale":"2024-04-23T18:26:08.226773+0000","last_undegraded":"2024-04-23T18:26:08.226773+0000","last_fullsized":"2024-04-23T18:26:08.226773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:10.803924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292712+0000","last_change":"2024-04-23T18:23:55.258636+0000","last_active":"2024-04-23T18:26:08.292712+0000","last_peered":"2024-04-23T18:26:08.292712+0000","last_clean":"2024-04-23T18:26:08.292712+0000","last_became_active":"2024-04-23T18:23:55.257744+0000","last_became_peered":"2024-04-23T18:23:55.257744+0000","last_unstale":"2024-04-23T18:26:08.292712+0000","last_undegraded":"2024-04-23T18:26:08.292712+0000","last_fullsized":"2024-04-23T18:26:08.292712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:46:07.454725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218571+0000","last_change":"2024-04-23T18:23:56.094822+0000","last_active":"2024-04-23T18:26:08.218571+0000","last_peered":"2024-04-23T18:26:08.218571+0000","last_clean":"2024-04-23T18:26:08.218571+0000","last_became_active":"2024-04-23T18:23:56.094489+0000","last_became_peered":"2024-04-23T18:23:56.094489+0000","last_unstale":"2024-04-23T18:26:08.218571+0000","last_undegraded":"2024-04-23T18:26:08.218571+0000","last_fullsized":"2024-04-23T18:26:08.218571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:21.234363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226519+0000","last_change":"2024-04-23T18:23:55.261322+0000","last_active":"2024-04-23T18:26:08.226519+0000","last_peered":"2024-04-23T18:26:08.226519+0000","last_clean":"2024-04-23T18:26:08.226519+0000","last_became_active":"2024-04-23T18:23:55.260990+0000","last_became_peered":"2024-04-23T18:23:55.260990+0000","last_unstale":"2024-04-23T18:26:08.226519+0000","last_undegraded":"2024-04-23T18:26:08.226519+0000","last_fullsized":"2024-04-23T18:26:08.226519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:31.829605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122279+0000","last_change":"2024-04-23T18:23:56.093105+0000","last_active":"2024-04-23T18:26:08.122279+0000","last_peered":"2024-04-23T18:26:08.122279+0000","last_clean":"2024-04-23T18:26:08.122279+0000","last_became_active":"2024-04-23T18:23:56.092717+0000","last_became_peered":"2024-04-23T18:23:56.092717+0000","last_unstale":"2024-04-23T18:26:08.122279+0000","last_undegraded":"2024-04-23T18:26:08.122279+0000","last_fullsized":"2024-04-23T18:26:08.122279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:13:59.574057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":281,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.807789+0000","last_change":"2024-04-23T18:23:55.256307+0000","last_active":"2024-04-23T18:26:08.807789+0000","last_peered":"2024-04-23T18:26:08.807789+0000","last_clean":"2024-04-23T18:26:08.807789+0000","last_became_active":"2024-04-23T18:23:55.255959+0000","last_became_peered":"2024-04-23T18:23:55.255959+0000","last_unstale":"2024-04-23T18:26:08.807789+0000","last_undegraded":"2024-04-23T18:26:08.807789+0000","last_fullsized":"2024-04-23T18:26:08.807789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:04.945919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293141+0000","last_change":"2024-04-23T18:23:56.095177+0000","last_active":"2024-04-23T18:26:08.293141+0000","last_peered":"2024-04-23T18:26:08.293141+0000","last_clean":"2024-04-23T18:26:08.293141+0000","last_became_active":"2024-04-23T18:23:56.094825+0000","last_became_peered":"2024-04-23T18:23:56.094825+0000","last_unstale":"2024-04-23T18:26:08.293141+0000","last_undegraded":"2024-04-23T18:26:08.293141+0000","last_fullsized":"2024-04-23T18:26:08.293141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:07:33.499817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123551+0000","last_change":"2024-04-23T18:23:55.264258+0000","last_active":"2024-04-23T18:26:08.123551+0000","last_peered":"2024-04-23T18:26:08.123551+0000","last_clean":"2024-04-23T18:26:08.123551+0000","last_became_active":"2024-04-23T18:23:55.263698+0000","last_became_peered":"2024-04-23T18:23:55.263698+0000","last_unstale":"2024-04-23T18:26:08.123551+0000","last_undegraded":"2024-04-23T18:26:08.123551+0000","last_fullsized":"2024-04-23T18:26:08.123551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:00.566948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293683+0000","last_change":"2024-04-23T18:23:56.109085+0000","last_active":"2024-04-23T18:26:08.293683+0000","last_peered":"2024-04-23T18:26:08.293683+0000","last_clean":"2024-04-23T18:26:08.293683+0000","last_became_active":"2024-04-23T18:23:56.108033+0000","last_became_peered":"2024-04-23T18:23:56.108033+0000","last_unstale":"2024-04-23T18:26:08.293683+0000","last_undegraded":"2024-04-23T18:26:08.293683+0000","last_fullsized":"2024-04-23T18:26:08.293683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:59.980512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226931+0000","last_change":"2024-04-23T18:23:55.256728+0000","last_active":"2024-04-23T18:26:08.226931+0000","last_peered":"2024-04-23T18:26:08.226931+0000","last_clean":"2024-04-23T18:26:08.226931+0000","last_became_active":"2024-04-23T18:23:55.256367+0000","last_became_peered":"2024-04-23T18:23:55.256367+0000","last_unstale":"2024-04-23T18:26:08.226931+0000","last_undegraded":"2024-04-23T18:26:08.226931+0000","last_fullsized":"2024-04-23T18:26:08.226931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:08.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292666+0000","last_change":"2024-04-23T18:23:56.102898+0000","last_active":"2024-04-23T18:26:08.292666+0000","last_peered":"2024-04-23T18:26:08.292666+0000","last_clean":"2024-04-23T18:26:08.292666+0000","last_became_active":"2024-04-23T18:23:56.102373+0000","last_became_peered":"2024-04-23T18:23:56.102373+0000","last_unstale":"2024-04-23T18:26:08.292666+0000","last_undegraded":"2024-04-23T18:26:08.292666+0000","last_fullsized":"2024-04-23T18:26:08.292666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:49.316219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'37","reported_seq":322,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.808335+0000","last_change":"2024-04-23T18:23:55.255094+0000","last_active":"2024-04-23T18:26:08.808335+0000","last_peered":"2024-04-23T18:26:08.808335+0000","last_clean":"2024-04-23T18:26:08.808335+0000","last_became_active":"2024-04-23T18:23:55.254726+0000","last_became_peered":"2024-04-23T18:23:55.254726+0000","last_unstale":"2024-04-23T18:26:08.808335+0000","last_undegraded":"2024-04-23T18:26:08.808335+0000","last_fullsized":"2024-04-23T18:26:08.808335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:26.988184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":264,"num_read_kb":200,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219090+0000","last_change":"2024-04-23T18:23:56.099866+0000","last_active":"2024-04-23T18:26:08.219090+0000","last_peered":"2024-04-23T18:26:08.219090+0000","last_clean":"2024-04-23T18:26:08.219090+0000","last_became_active":"2024-04-23T18:23:56.098606+0000","last_became_peered":"2024-04-23T18:23:56.098606+0000","last_unstale":"2024-04-23T18:26:08.219090+0000","last_undegraded":"2024-04-23T18:26:08.219090+0000","last_fullsized":"2024-04-23T18:26:08.219090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:26.661857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292679+0000","last_change":"2024-04-23T18:24:36.710791+0000","last_active":"2024-04-23T18:26:08.292679+0000","last_peered":"2024-04-23T18:26:08.292679+0000","last_clean":"2024-04-23T18:26:08.292679+0000","last_became_active":"2024-04-23T18:24:36.697731+0000","last_became_peered":"2024-04-23T18:24:36.697731+0000","last_unstale":"2024-04-23T18:26:08.292679+0000","last_undegraded":"2024-04-23T18:26:08.292679+0000","last_fullsized":"2024-04-23T18:26:08.292679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:16:46.722132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292908+0000","last_change":"2024-04-23T18:23:56.099109+0000","last_active":"2024-04-23T18:26:08.292908+0000","last_peered":"2024-04-23T18:26:08.292908+0000","last_clean":"2024-04-23T18:26:08.292908+0000","last_became_active":"2024-04-23T18:23:56.098608+0000","last_became_peered":"2024-04-23T18:23:56.098608+0000","last_unstale":"2024-04-23T18:26:08.292908+0000","last_undegraded":"2024-04-23T18:26:08.292908+0000","last_fullsized":"2024-04-23T18:26:08.292908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:43.720160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226641+0000","last_change":"2024-04-23T18:23:55.262523+0000","last_active":"2024-04-23T18:26:08.226641+0000","last_peered":"2024-04-23T18:26:08.226641+0000","last_clean":"2024-04-23T18:26:08.226641+0000","last_became_active":"2024-04-23T18:23:55.262216+0000","last_became_peered":"2024-04-23T18:23:55.262216+0000","last_unstale":"2024-04-23T18:26:08.226641+0000","last_undegraded":"2024-04-23T18:26:08.226641+0000","last_fullsized":"2024-04-23T18:26:08.226641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:35.268435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123437+0000","last_change":"2024-04-23T18:23:56.106573+0000","last_active":"2024-04-23T18:26:08.123437+0000","last_peered":"2024-04-23T18:26:08.123437+0000","last_clean":"2024-04-23T18:26:08.123437+0000","last_became_active":"2024-04-23T18:23:56.106181+0000","last_became_peered":"2024-04-23T18:23:56.106181+0000","last_unstale":"2024-04-23T18:26:08.123437+0000","last_undegraded":"2024-04-23T18:26:08.123437+0000","last_fullsized":"2024-04-23T18:26:08.123437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:15.422647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'9","reported_seq":162,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.811881+0000","last_change":"2024-04-23T18:23:55.262813+0000","last_active":"2024-04-23T18:26:08.811881+0000","last_peered":"2024-04-23T18:26:08.811881+0000","last_clean":"2024-04-23T18:26:08.811881+0000","last_became_active":"2024-04-23T18:23:55.262365+0000","last_became_peered":"2024-04-23T18:23:55.262365+0000","last_unstale":"2024-04-23T18:26:08.811881+0000","last_undegraded":"2024-04-23T18:26:08.811881+0000","last_fullsized":"2024-04-23T18:26:08.811881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:15.234355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1819,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123074+0000","last_change":"2024-04-23T18:23:56.101491+0000","last_active":"2024-04-23T18:26:08.123074+0000","last_peered":"2024-04-23T18:26:08.123074+0000","last_clean":"2024-04-23T18:26:08.123074+0000","last_became_active":"2024-04-23T18:23:56.101069+0000","last_became_peered":"2024-04-23T18:23:56.101069+0000","last_unstale":"2024-04-23T18:26:08.123074+0000","last_undegraded":"2024-04-23T18:26:08.123074+0000","last_fullsized":"2024-04-23T18:26:08.123074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:42.484653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218843+0000","last_change":"2024-04-23T18:24:36.750105+0000","last_active":"2024-04-23T18:26:08.218843+0000","last_peered":"2024-04-23T18:26:08.218843+0000","last_clean":"2024-04-23T18:26:08.218843+0000","last_became_active":"2024-04-23T18:24:36.738326+0000","last_became_peered":"2024-04-23T18:24:36.738326+0000","last_unstale":"2024-04-23T18:26:08.218843+0000","last_undegraded":"2024-04-23T18:26:08.218843+0000","last_fullsized":"2024-04-23T18:26:08.218843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:07.414513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226554+0000","last_change":"2024-04-23T18:23:56.105691+0000","last_active":"2024-04-23T18:26:08.226554+0000","last_peered":"2024-04-23T18:26:08.226554+0000","last_clean":"2024-04-23T18:26:08.226554+0000","last_became_active":"2024-04-23T18:23:56.105205+0000","last_became_peered":"2024-04-23T18:23:56.105205+0000","last_unstale":"2024-04-23T18:26:08.226554+0000","last_undegraded":"2024-04-23T18:26:08.226554+0000","last_fullsized":"2024-04-23T18:26:08.226554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:30.779821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219253+0000","last_change":"2024-04-23T18:23:55.261198+0000","last_active":"2024-04-23T18:26:08.219253+0000","last_peered":"2024-04-23T18:26:08.219253+0000","last_clean":"2024-04-23T18:26:08.219253+0000","last_became_active":"2024-04-23T18:23:55.260884+0000","last_became_peered":"2024-04-23T18:23:55.260884+0000","last_unstale":"2024-04-23T18:26:08.219253+0000","last_undegraded":"2024-04-23T18:26:08.219253+0000","last_fullsized":"2024-04-23T18:26:08.219253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:00.841671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292792+0000","last_change":"2024-04-23T18:23:56.099450+0000","last_active":"2024-04-23T18:26:08.292792+0000","last_peered":"2024-04-23T18:26:08.292792+0000","last_clean":"2024-04-23T18:26:08.292792+0000","last_became_active":"2024-04-23T18:23:56.098723+0000","last_became_peered":"2024-04-23T18:23:56.098723+0000","last_unstale":"2024-04-23T18:26:08.292792+0000","last_undegraded":"2024-04-23T18:26:08.292792+0000","last_fullsized":"2024-04-23T18:26:08.292792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:53.027495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.122351+0000","last_change":"2024-04-23T18:23:55.262030+0000","last_active":"2024-04-23T18:26:08.122351+0000","last_peered":"2024-04-23T18:26:08.122351+0000","last_clean":"2024-04-23T18:26:08.122351+0000","last_became_active":"2024-04-23T18:23:55.261659+0000","last_became_peered":"2024-04-23T18:23:55.261659+0000","last_unstale":"2024-04-23T18:26:08.122351+0000","last_undegraded":"2024-04-23T18:26:08.122351+0000","last_fullsized":"2024-04-23T18:26:08.122351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:17.036709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.230040+0000","last_change":"2024-04-23T18:23:56.096511+0000","last_active":"2024-04-23T18:26:08.230040+0000","last_peered":"2024-04-23T18:26:08.230040+0000","last_clean":"2024-04-23T18:26:08.230040+0000","last_became_active":"2024-04-23T18:23:56.096154+0000","last_became_peered":"2024-04-23T18:23:56.096154+0000","last_unstale":"2024-04-23T18:26:08.230040+0000","last_undegraded":"2024-04-23T18:26:08.230040+0000","last_fullsized":"2024-04-23T18:26:08.230040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:03.926178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.806819+0000","last_change":"2024-04-23T18:23:55.263618+0000","last_active":"2024-04-23T18:26:08.806819+0000","last_peered":"2024-04-23T18:26:08.806819+0000","last_clean":"2024-04-23T18:26:08.806819+0000","last_became_active":"2024-04-23T18:23:55.261758+0000","last_became_peered":"2024-04-23T18:23:55.261758+0000","last_unstale":"2024-04-23T18:26:08.806819+0000","last_undegraded":"2024-04-23T18:26:08.806819+0000","last_fullsized":"2024-04-23T18:26:08.806819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:23:27.745668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.229933+0000","last_change":"2024-04-23T18:23:56.109292+0000","last_active":"2024-04-23T18:26:08.229933+0000","last_peered":"2024-04-23T18:26:08.229933+0000","last_clean":"2024-04-23T18:26:08.229933+0000","last_became_active":"2024-04-23T18:23:56.108414+0000","last_became_peered":"2024-04-23T18:23:56.108414+0000","last_unstale":"2024-04-23T18:26:08.229933+0000","last_undegraded":"2024-04-23T18:26:08.229933+0000","last_fullsized":"2024-04-23T18:26:08.229933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:13.428782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226673+0000","last_change":"2024-04-23T18:23:56.097122+0000","last_active":"2024-04-23T18:26:08.226673+0000","last_peered":"2024-04-23T18:26:08.226673+0000","last_clean":"2024-04-23T18:26:08.226673+0000","last_became_active":"2024-04-23T18:23:56.096751+0000","last_became_peered":"2024-04-23T18:23:56.096751+0000","last_unstale":"2024-04-23T18:26:08.226673+0000","last_undegraded":"2024-04-23T18:26:08.226673+0000","last_fullsized":"2024-04-23T18:26:08.226673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:00:55.090370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":162,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:12.816687+0000","last_change":"2024-04-23T18:23:55.256458+0000","last_active":"2024-04-23T18:26:12.816687+0000","last_peered":"2024-04-23T18:26:12.816687+0000","last_clean":"2024-04-23T18:26:12.816687+0000","last_became_active":"2024-04-23T18:23:55.256127+0000","last_became_peered":"2024-04-23T18:23:55.256127+0000","last_unstale":"2024-04-23T18:26:12.816687+0000","last_undegraded":"2024-04-23T18:26:12.816687+0000","last_fullsized":"2024-04-23T18:26:12.816687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:03.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218803+0000","last_change":"2024-04-23T18:23:56.103743+0000","last_active":"2024-04-23T18:26:08.218803+0000","last_peered":"2024-04-23T18:26:08.218803+0000","last_clean":"2024-04-23T18:26:08.218803+0000","last_became_active":"2024-04-23T18:23:56.103041+0000","last_became_peered":"2024-04-23T18:23:56.103041+0000","last_unstale":"2024-04-23T18:26:08.218803+0000","last_undegraded":"2024-04-23T18:26:08.218803+0000","last_fullsized":"2024-04-23T18:26:08.218803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:22.419555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292990+0000","last_change":"2024-04-23T18:23:55.257008+0000","last_active":"2024-04-23T18:26:08.292990+0000","last_peered":"2024-04-23T18:26:08.292990+0000","last_clean":"2024-04-23T18:26:08.292990+0000","last_became_active":"2024-04-23T18:23:55.256648+0000","last_became_peered":"2024-04-23T18:23:55.256648+0000","last_unstale":"2024-04-23T18:26:08.292990+0000","last_undegraded":"2024-04-23T18:26:08.292990+0000","last_fullsized":"2024-04-23T18:26:08.292990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:00.867083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123176+0000","last_change":"2024-04-23T18:22:38.610948+0000","last_active":"2024-04-23T18:26:08.123176+0000","last_peered":"2024-04-23T18:26:08.123176+0000","last_clean":"2024-04-23T18:26:08.123176+0000","last_became_active":"2024-04-23T18:22:38.610475+0000","last_became_peered":"2024-04-23T18:22:38.610475+0000","last_unstale":"2024-04-23T18:26:08.123176+0000","last_undegraded":"2024-04-23T18:26:08.123176+0000","last_fullsized":"2024-04-23T18:26:08.123176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:07.980519+0000","last_clean_scrub_stamp":"2024-04-23T18:19:07.980519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:03:08.209087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292611+0000","last_change":"2024-04-23T18:24:36.699127+0000","last_active":"2024-04-23T18:26:08.292611+0000","last_peered":"2024-04-23T18:26:08.292611+0000","last_clean":"2024-04-23T18:26:08.292611+0000","last_became_active":"2024-04-23T18:24:36.698734+0000","last_became_peered":"2024-04-23T18:24:36.698734+0000","last_unstale":"2024-04-23T18:26:08.292611+0000","last_undegraded":"2024-04-23T18:26:08.292611+0000","last_fullsized":"2024-04-23T18:26:08.292611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:12:28.239466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123284+0000","last_change":"2024-04-23T18:24:36.918722+0000","last_active":"2024-04-23T18:26:08.123284+0000","last_peered":"2024-04-23T18:26:08.123284+0000","last_clean":"2024-04-23T18:26:08.123284+0000","last_became_active":"2024-04-23T18:24:36.915658+0000","last_became_peered":"2024-04-23T18:24:36.915658+0000","last_unstale":"2024-04-23T18:26:08.123284+0000","last_undegraded":"2024-04-23T18:26:08.123284+0000","last_fullsized":"2024-04-23T18:26:08.123284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:34.706193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219189+0000","last_change":"2024-04-23T18:23:55.265853+0000","last_active":"2024-04-23T18:26:08.219189+0000","last_peered":"2024-04-23T18:26:08.219189+0000","last_clean":"2024-04-23T18:26:08.219189+0000","last_became_active":"2024-04-23T18:23:55.265435+0000","last_became_peered":"2024-04-23T18:23:55.265435+0000","last_unstale":"2024-04-23T18:26:08.219189+0000","last_undegraded":"2024-04-23T18:26:08.219189+0000","last_fullsized":"2024-04-23T18:26:08.219189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.969720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293278+0000","last_change":"2024-04-23T18:23:56.085609+0000","last_active":"2024-04-23T18:26:08.293278+0000","last_peered":"2024-04-23T18:26:08.293278+0000","last_clean":"2024-04-23T18:26:08.293278+0000","last_became_active":"2024-04-23T18:23:56.085244+0000","last_became_peered":"2024-04-23T18:23:56.085244+0000","last_unstale":"2024-04-23T18:26:08.293278+0000","last_undegraded":"2024-04-23T18:26:08.293278+0000","last_fullsized":"2024-04-23T18:26:08.293278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.211717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'48","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.341583+0000","last_change":"2024-04-23T18:23:55.260260+0000","last_active":"2024-04-23T18:26:08.341583+0000","last_peered":"2024-04-23T18:26:08.341583+0000","last_clean":"2024-04-23T18:26:08.341583+0000","last_became_active":"2024-04-23T18:23:55.259862+0000","last_became_peered":"2024-04-23T18:23:55.259862+0000","last_unstale":"2024-04-23T18:26:08.341583+0000","last_undegraded":"2024-04-23T18:26:08.341583+0000","last_fullsized":"2024-04-23T18:26:08.341583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:57.833293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219052+0000","last_change":"2024-04-23T18:23:56.098826+0000","last_active":"2024-04-23T18:26:08.219052+0000","last_peered":"2024-04-23T18:26:08.219052+0000","last_clean":"2024-04-23T18:26:08.219052+0000","last_became_active":"2024-04-23T18:23:56.098494+0000","last_became_peered":"2024-04-23T18:23:56.098494+0000","last_unstale":"2024-04-23T18:26:08.219052+0000","last_undegraded":"2024-04-23T18:26:08.219052+0000","last_fullsized":"2024-04-23T18:26:08.219052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:12:24.953330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293238+0000","last_change":"2024-04-23T18:23:55.258174+0000","last_active":"2024-04-23T18:26:08.293238+0000","last_peered":"2024-04-23T18:26:08.293238+0000","last_clean":"2024-04-23T18:26:08.293238+0000","last_became_active":"2024-04-23T18:23:55.257846+0000","last_became_peered":"2024-04-23T18:23:55.257846+0000","last_unstale":"2024-04-23T18:26:08.293238+0000","last_undegraded":"2024-04-23T18:26:08.293238+0000","last_fullsized":"2024-04-23T18:26:08.293238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:35.131566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293829+0000","last_change":"2024-04-23T18:23:56.110758+0000","last_active":"2024-04-23T18:26:08.293829+0000","last_peered":"2024-04-23T18:26:08.293829+0000","last_clean":"2024-04-23T18:26:08.293829+0000","last_became_active":"2024-04-23T18:23:56.110372+0000","last_became_peered":"2024-04-23T18:23:56.110372+0000","last_unstale":"2024-04-23T18:26:08.293829+0000","last_undegraded":"2024-04-23T18:26:08.293829+0000","last_fullsized":"2024-04-23T18:26:08.293829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:40.193088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292460+0000","last_change":"2024-04-23T18:23:55.265714+0000","last_active":"2024-04-23T18:26:08.292460+0000","last_peered":"2024-04-23T18:26:08.292460+0000","last_clean":"2024-04-23T18:26:08.292460+0000","last_became_active":"2024-04-23T18:23:55.265198+0000","last_became_peered":"2024-04-23T18:23:55.265198+0000","last_unstale":"2024-04-23T18:26:08.292460+0000","last_undegraded":"2024-04-23T18:26:08.292460+0000","last_fullsized":"2024-04-23T18:26:08.292460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:10.770748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218909+0000","last_change":"2024-04-23T18:23:56.102347+0000","last_active":"2024-04-23T18:26:08.218909+0000","last_peered":"2024-04-23T18:26:08.218909+0000","last_clean":"2024-04-23T18:26:08.218909+0000","last_became_active":"2024-04-23T18:23:56.100687+0000","last_became_peered":"2024-04-23T18:23:56.100687+0000","last_unstale":"2024-04-23T18:26:08.218909+0000","last_undegraded":"2024-04-23T18:26:08.218909+0000","last_fullsized":"2024-04-23T18:26:08.218909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:20.378730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218670+0000","last_change":"2024-04-23T18:23:55.255943+0000","last_active":"2024-04-23T18:26:08.218670+0000","last_peered":"2024-04-23T18:26:08.218670+0000","last_clean":"2024-04-23T18:26:08.218670+0000","last_became_active":"2024-04-23T18:23:55.255618+0000","last_became_peered":"2024-04-23T18:23:55.255618+0000","last_unstale":"2024-04-23T18:26:08.218670+0000","last_undegraded":"2024-04-23T18:26:08.218670+0000","last_fullsized":"2024-04-23T18:26:08.218670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:01.408533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292542+0000","last_change":"2024-04-23T18:23:56.091664+0000","last_active":"2024-04-23T18:26:08.292542+0000","last_peered":"2024-04-23T18:26:08.292542+0000","last_clean":"2024-04-23T18:26:08.292542+0000","last_became_active":"2024-04-23T18:23:56.091308+0000","last_became_peered":"2024-04-23T18:23:56.091308+0000","last_unstale":"2024-04-23T18:26:08.292542+0000","last_undegraded":"2024-04-23T18:26:08.292542+0000","last_fullsized":"2024-04-23T18:26:08.292542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:31.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.405535+0000","last_change":"2024-04-23T18:23:55.260992+0000","last_active":"2024-04-23T18:26:08.405535+0000","last_peered":"2024-04-23T18:26:08.405535+0000","last_clean":"2024-04-23T18:26:08.405535+0000","last_became_active":"2024-04-23T18:23:55.260679+0000","last_became_peered":"2024-04-23T18:23:55.260679+0000","last_unstale":"2024-04-23T18:26:08.405535+0000","last_undegraded":"2024-04-23T18:26:08.405535+0000","last_fullsized":"2024-04-23T18:26:08.405535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:02:50.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.219168+0000","last_change":"2024-04-23T18:23:56.097956+0000","last_active":"2024-04-23T18:26:08.219168+0000","last_peered":"2024-04-23T18:26:08.219168+0000","last_clean":"2024-04-23T18:26:08.219168+0000","last_became_active":"2024-04-23T18:23:56.097231+0000","last_became_peered":"2024-04-23T18:23:56.097231+0000","last_unstale":"2024-04-23T18:26:08.219168+0000","last_undegraded":"2024-04-23T18:26:08.219168+0000","last_fullsized":"2024-04-23T18:26:08.219168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:10.945857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292582+0000","last_change":"2024-04-23T18:24:36.714223+0000","last_active":"2024-04-23T18:26:08.292582+0000","last_peered":"2024-04-23T18:26:08.292582+0000","last_clean":"2024-04-23T18:26:08.292582+0000","last_became_active":"2024-04-23T18:24:36.700975+0000","last_became_peered":"2024-04-23T18:24:36.700975+0000","last_unstale":"2024-04-23T18:26:08.292582+0000","last_undegraded":"2024-04-23T18:26:08.292582+0000","last_fullsized":"2024-04-23T18:26:08.292582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:58:50.227699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123311+0000","last_change":"2024-04-23T18:23:56.096889+0000","last_active":"2024-04-23T18:26:08.123311+0000","last_peered":"2024-04-23T18:26:08.123311+0000","last_clean":"2024-04-23T18:26:08.123311+0000","last_became_active":"2024-04-23T18:23:56.096520+0000","last_became_peered":"2024-04-23T18:23:56.096520+0000","last_unstale":"2024-04-23T18:26:08.123311+0000","last_undegraded":"2024-04-23T18:26:08.123311+0000","last_fullsized":"2024-04-23T18:26:08.123311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:34:53.785481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":159,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.808804+0000","last_change":"2024-04-23T18:23:55.258710+0000","last_active":"2024-04-23T18:26:08.808804+0000","last_peered":"2024-04-23T18:26:08.808804+0000","last_clean":"2024-04-23T18:26:08.808804+0000","last_became_active":"2024-04-23T18:23:55.256247+0000","last_became_peered":"2024-04-23T18:23:55.256247+0000","last_unstale":"2024-04-23T18:26:08.808804+0000","last_undegraded":"2024-04-23T18:26:08.808804+0000","last_fullsized":"2024-04-23T18:26:08.808804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:20.551614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123532+0000","last_change":"2024-04-23T18:23:56.101862+0000","last_active":"2024-04-23T18:26:08.123532+0000","last_peered":"2024-04-23T18:26:08.123532+0000","last_clean":"2024-04-23T18:26:08.123532+0000","last_became_active":"2024-04-23T18:23:56.101458+0000","last_became_peered":"2024-04-23T18:23:56.101458+0000","last_unstale":"2024-04-23T18:26:08.123532+0000","last_undegraded":"2024-04-23T18:26:08.123532+0000","last_fullsized":"2024-04-23T18:26:08.123532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:59.292077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'41","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.333872+0000","last_change":"2024-04-23T18:23:55.258292+0000","last_active":"2024-04-23T18:26:08.333872+0000","last_peered":"2024-04-23T18:26:08.333872+0000","last_clean":"2024-04-23T18:26:08.333872+0000","last_became_active":"2024-04-23T18:23:55.257954+0000","last_became_peered":"2024-04-23T18:23:55.257954+0000","last_unstale":"2024-04-23T18:26:08.333872+0000","last_undegraded":"2024-04-23T18:26:08.333872+0000","last_fullsized":"2024-04-23T18:26:08.333872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:39.105967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123377+0000","last_change":"2024-04-23T18:23:56.105977+0000","last_active":"2024-04-23T18:26:08.123377+0000","last_peered":"2024-04-23T18:26:08.123377+0000","last_clean":"2024-04-23T18:26:08.123377+0000","last_became_active":"2024-04-23T18:23:56.105582+0000","last_became_peered":"2024-04-23T18:23:56.105582+0000","last_unstale":"2024-04-23T18:26:08.123377+0000","last_undegraded":"2024-04-23T18:26:08.123377+0000","last_fullsized":"2024-04-23T18:26:08.123377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:28:10.700684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293550+0000","last_change":"2024-04-23T18:23:55.252095+0000","last_active":"2024-04-23T18:26:08.293550+0000","last_peered":"2024-04-23T18:26:08.293550+0000","last_clean":"2024-04-23T18:26:08.293550+0000","last_became_active":"2024-04-23T18:23:55.251702+0000","last_became_peered":"2024-04-23T18:23:55.251702+0000","last_unstale":"2024-04-23T18:26:08.293550+0000","last_undegraded":"2024-04-23T18:26:08.293550+0000","last_fullsized":"2024-04-23T18:26:08.293550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:54.234542+0000","last_clean_scrub_stamp":"2024-04-23T18:23:54.234542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:21.625905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.123737+0000","last_change":"2024-04-23T18:23:56.101997+0000","last_active":"2024-04-23T18:26:08.123737+0000","last_peered":"2024-04-23T18:26:08.123737+0000","last_clean":"2024-04-23T18:26:08.123737+0000","last_became_active":"2024-04-23T18:23:56.101629+0000","last_became_peered":"2024-04-23T18:23:56.101629+0000","last_unstale":"2024-04-23T18:26:08.123737+0000","last_undegraded":"2024-04-23T18:26:08.123737+0000","last_fullsized":"2024-04-23T18:26:08.123737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:42.484413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292621+0000","last_change":"2024-04-23T18:23:56.090543+0000","last_active":"2024-04-23T18:26:08.292621+0000","last_peered":"2024-04-23T18:26:08.292621+0000","last_clean":"2024-04-23T18:26:08.292621+0000","last_became_active":"2024-04-23T18:23:56.090191+0000","last_became_peered":"2024-04-23T18:23:56.090191+0000","last_unstale":"2024-04-23T18:26:08.292621+0000","last_undegraded":"2024-04-23T18:26:08.292621+0000","last_fullsized":"2024-04-23T18:26:08.292621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:01.401822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.127145+0000","last_change":"2024-04-23T18:23:56.105537+0000","last_active":"2024-04-23T18:26:08.127145+0000","last_peered":"2024-04-23T18:26:08.127145+0000","last_clean":"2024-04-23T18:26:08.127145+0000","last_became_active":"2024-04-23T18:23:56.105130+0000","last_became_peered":"2024-04-23T18:23:56.105130+0000","last_unstale":"2024-04-23T18:26:08.127145+0000","last_undegraded":"2024-04-23T18:26:08.127145+0000","last_fullsized":"2024-04-23T18:26:08.127145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:59.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.126910+0000","last_change":"2024-04-23T18:23:56.097765+0000","last_active":"2024-04-23T18:26:08.126910+0000","last_peered":"2024-04-23T18:26:08.126910+0000","last_clean":"2024-04-23T18:26:08.126910+0000","last_became_active":"2024-04-23T18:23:56.097439+0000","last_became_peered":"2024-04-23T18:23:56.097439+0000","last_unstale":"2024-04-23T18:26:08.126910+0000","last_undegraded":"2024-04-23T18:26:08.126910+0000","last_fullsized":"2024-04-23T18:26:08.126910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:31.529751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293019+0000","last_change":"2024-04-23T18:23:56.108548+0000","last_active":"2024-04-23T18:26:08.293019+0000","last_peered":"2024-04-23T18:26:08.293019+0000","last_clean":"2024-04-23T18:26:08.293019+0000","last_became_active":"2024-04-23T18:23:56.108214+0000","last_became_peered":"2024-04-23T18:23:56.108214+0000","last_unstale":"2024-04-23T18:26:08.293019+0000","last_undegraded":"2024-04-23T18:26:08.293019+0000","last_fullsized":"2024-04-23T18:26:08.293019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:41.738754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293157+0000","last_change":"2024-04-23T18:24:36.701809+0000","last_active":"2024-04-23T18:26:08.293157+0000","last_peered":"2024-04-23T18:26:08.293157+0000","last_clean":"2024-04-23T18:26:08.293157+0000","last_became_active":"2024-04-23T18:24:36.701405+0000","last_became_peered":"2024-04-23T18:24:36.701405+0000","last_unstale":"2024-04-23T18:26:08.293157+0000","last_undegraded":"2024-04-23T18:26:08.293157+0000","last_fullsized":"2024-04-23T18:26:08.293157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:03.215622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292776+0000","last_change":"2024-04-23T18:23:56.093969+0000","last_active":"2024-04-23T18:26:08.292776+0000","last_peered":"2024-04-23T18:26:08.292776+0000","last_clean":"2024-04-23T18:26:08.292776+0000","last_became_active":"2024-04-23T18:23:56.093568+0000","last_became_peered":"2024-04-23T18:23:56.093568+0000","last_unstale":"2024-04-23T18:26:08.292776+0000","last_undegraded":"2024-04-23T18:26:08.292776+0000","last_fullsized":"2024-04-23T18:26:08.292776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:08.361305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.127150+0000","last_change":"2024-04-23T18:23:56.096106+0000","last_active":"2024-04-23T18:26:08.127150+0000","last_peered":"2024-04-23T18:26:08.127150+0000","last_clean":"2024-04-23T18:26:08.127150+0000","last_became_active":"2024-04-23T18:23:56.095745+0000","last_became_peered":"2024-04-23T18:23:56.095745+0000","last_unstale":"2024-04-23T18:26:08.127150+0000","last_undegraded":"2024-04-23T18:26:08.127150+0000","last_fullsized":"2024-04-23T18:26:08.127150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:01.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218970+0000","last_change":"2024-04-23T18:23:56.095474+0000","last_active":"2024-04-23T18:26:08.218970+0000","last_peered":"2024-04-23T18:26:08.218970+0000","last_clean":"2024-04-23T18:26:08.218970+0000","last_became_active":"2024-04-23T18:23:56.095116+0000","last_became_peered":"2024-04-23T18:23:56.095116+0000","last_unstale":"2024-04-23T18:26:08.218970+0000","last_undegraded":"2024-04-23T18:26:08.218970+0000","last_fullsized":"2024-04-23T18:26:08.218970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:50:00.339558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.226608+0000","last_change":"2024-04-23T18:23:56.096022+0000","last_active":"2024-04-23T18:26:08.226608+0000","last_peered":"2024-04-23T18:26:08.226608+0000","last_clean":"2024-04-23T18:26:08.226608+0000","last_became_active":"2024-04-23T18:23:56.095493+0000","last_became_peered":"2024-04-23T18:23:56.095493+0000","last_unstale":"2024-04-23T18:26:08.226608+0000","last_undegraded":"2024-04-23T18:26:08.226608+0000","last_fullsized":"2024-04-23T18:26:08.226608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:42.209072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.218607+0000","last_change":"2024-04-23T18:23:56.101697+0000","last_active":"2024-04-23T18:26:08.218607+0000","last_peered":"2024-04-23T18:26:08.218607+0000","last_clean":"2024-04-23T18:26:08.218607+0000","last_became_active":"2024-04-23T18:23:56.098708+0000","last_became_peered":"2024-04-23T18:23:56.098708+0000","last_unstale":"2024-04-23T18:26:08.218607+0000","last_undegraded":"2024-04-23T18:26:08.218607+0000","last_fullsized":"2024-04-23T18:26:08.218607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:14.170700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292721+0000","last_change":"2024-04-23T18:23:56.096982+0000","last_active":"2024-04-23T18:26:08.292721+0000","last_peered":"2024-04-23T18:26:08.292721+0000","last_clean":"2024-04-23T18:26:08.292721+0000","last_became_active":"2024-04-23T18:23:56.096608+0000","last_became_peered":"2024-04-23T18:23:56.096608+0000","last_unstale":"2024-04-23T18:26:08.292721+0000","last_undegraded":"2024-04-23T18:26:08.292721+0000","last_fullsized":"2024-04-23T18:26:08.292721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:50.329337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.293339+0000","last_change":"2024-04-23T18:23:56.100345+0000","last_active":"2024-04-23T18:26:08.293339+0000","last_peered":"2024-04-23T18:26:08.293339+0000","last_clean":"2024-04-23T18:26:08.293339+0000","last_became_active":"2024-04-23T18:23:56.100043+0000","last_became_peered":"2024-04-23T18:23:56.100043+0000","last_unstale":"2024-04-23T18:26:08.293339+0000","last_undegraded":"2024-04-23T18:26:08.293339+0000","last_fullsized":"2024-04-23T18:26:08.293339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:36.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T18:26:08.292603+0000","last_change":"2024-04-23T18:23:56.101947+0000","last_active":"2024-04-23T18:26:08.292603+0000","last_peered":"2024-04-23T18:26:08.292603+0000","last_clean":"2024-04-23T18:26:08.292603+0000","last_became_active":"2024-04-23T18:23:56.101627+0000","last_became_peered":"2024-04-23T18:23:56.101627+0000","last_unstale":"2024-04-23T18:26:08.292603+0000","last_undegraded":"2024-04-23T18:26:08.292603+0000","last_fullsized":"2024-04-23T18:26:08.292603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:23:55.242117+0000","last_clean_scrub_stamp":"2024-04-23T18:23:55.242117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:21:43.703653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106755,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602323,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1411,"num_read_kb":826,"num_write":373,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":737280,"data_stored":2003992,"data_compressed":90690,"data_compressed_allocated":122880,"data_compressed_original":1757184,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":461377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":1261854,"data_compressed":46511,"data_compressed_allocated":61440,"data_compressed_original":729088,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":863382,"data_compressed":22051,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":863760,"data_compressed":22051,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":1065216,"data_compressed":38490,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":471056,"data_compressed":2905,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":539336,"data_compressed":4770,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":472004,"data_compressed":2905,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":539908,"data_compressed":4770,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":1191156,"data_compressed":37396,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":471087,"data_compressed":2905,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1194481,"data_compressed":44646,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35585,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70292,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70864,"data_compressed":1865,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134952,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":202325,"data_compressed":8021,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:26:14.658 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.803+0000 7fa5345a1640 1 -- 172.21.15.19:0/1638069127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa52c10c580 msgr2=0x7fa52c10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.803+0000 7fa5345a1640 1 --2- 172.21.15.19:0/1638069127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa52c10c580 0x7fa52c10e9b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa524009ca0 tx=0x7fa52402a950 comp rx=0 tx=0).stop 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.803+0000 7fa5345a1640 1 -- 172.21.15.19:0/1638069127 shutdown_connections 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.803+0000 7fa5345a1640 1 --2- 172.21.15.19:0/1638069127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa52c10c580 0x7fa52c10e9b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.803+0000 7fa5345a1640 1 --2- 172.21.15.19:0/1638069127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa52c103ad0 0x7fa52c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.803+0000 7fa5345a1640 1 --2- 172.21.15.19:0/1638069127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa52c1031b0 0x7fa52c103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.803+0000 7fa5345a1640 1 -- 172.21.15.19:0/1638069127 >> 172.21.15.19:0/1638069127 conn(0x7fa52c0771d0 msgr2=0x7fa52c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.803+0000 7fa5345a1640 1 -- 172.21.15.19:0/1638069127 shutdown_connections 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.803+0000 7fa5345a1640 1 -- 172.21.15.19:0/1638069127 wait complete. 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.803+0000 7fa5345a1640 1 Processor -- start 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa5345a1640 1 -- start start 2024-04-23T18:26:14.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa5345a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa52c1031b0 0x7fa52c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa5345a1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa52c103ad0 0x7fa52c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa5345a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa52c10c580 0x7fa52c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa5345a1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c1121a0 con 0x7fa52c1031b0 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa5345a1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c112020 con 0x7fa52c10c580 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa5345a1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c112320 con 0x7fa52c103ad0 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa53233d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa52c1031b0 0x7fa52c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa53233d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa52c1031b0 0x7fa52c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53454/0 (socket says 172.21.15.19:53454) 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa53233d640 1 -- 172.21.15.19:0/4189611477 learned_addr learned my addr 172.21.15.19:0/4189611477 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa532b3e640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa52c10c580 0x7fa52c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa53233d640 1 -- 172.21.15.19:0/4189611477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa52c103ad0 msgr2=0x7fa52c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa53233d640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa52c103ad0 0x7fa52c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa53233d640 1 -- 172.21.15.19:0/4189611477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa52c10c580 msgr2=0x7fa52c19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa53233d640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa52c10c580 0x7fa52c19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa53233d640 1 -- 172.21.15.19:0/4189611477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c19d6f0 con 0x7fa52c1031b0 2024-04-23T18:26:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa532b3e640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa52c10c580 0x7fa52c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa53233d640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa52c1031b0 0x7fa52c198700 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa52800e9f0 tx=0x7fa52800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa5237fe640 1 -- 172.21.15.19:0/4189611477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52800cd80 con 0x7fa52c1031b0 2024-04-23T18:26:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa5237fe640 1 -- 172.21.15.19:0/4189611477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5280108e0 con 0x7fa52c1031b0 2024-04-23T18:26:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.804+0000 7fa5345a1640 1 -- 172.21.15.19:0/4189611477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c19d9e0 con 0x7fa52c1031b0 2024-04-23T18:26:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.805+0000 7fa5237fe640 1 -- 172.21.15.19:0/4189611477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528018980 con 0x7fa52c1031b0 2024-04-23T18:26:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.805+0000 7fa5345a1640 1 -- 172.21.15.19:0/4189611477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c1a5270 con 0x7fa52c1031b0 2024-04-23T18:26:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.806+0000 7fa5345a1640 1 -- 172.21.15.19:0/4189611477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c06add0 con 0x7fa52c1031b0 2024-04-23T18:26:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.807+0000 7fa5237fe640 1 -- 172.21.15.19:0/4189611477 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa528010430 con 0x7fa52c1031b0 2024-04-23T18:26:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.807+0000 7fa5237fe640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5180a3ec0 0x7fa5180a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.807+0000 7fa531b3c640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5180a3ec0 0x7fa5180a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.807+0000 7fa5237fe640 1 -- 172.21.15.19:0/4189611477 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa5280d0e50 con 0x7fa52c1031b0 2024-04-23T18:26:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.808+0000 7fa531b3c640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5180a3ec0 0x7fa5180a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa52c10e3e0 tx=0x7fa51c005b60 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:14.812+0000 7fa5237fe640 1 -- 172.21.15.19:0/4189611477 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa52809d2d0 con 0x7fa52c1031b0 2024-04-23T18:26:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.039+0000 7fa5345a1640 1 -- 172.21.15.19:0/4189611477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa52c0691c0 con 0x7fa52c1031b0 2024-04-23T18:26:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.040+0000 7fa5237fe640 1 -- 172.21.15.19:0/4189611477 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7fa52809cbd0 con 0x7fa52c1031b0 2024-04-23T18:26:15.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:15.034 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:26:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:26:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 -- 172.21.15.19:0/4189611477 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5180a3ec0 msgr2=0x7fa5180a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5180a3ec0 0x7fa5180a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa52c10e3e0 tx=0x7fa51c005b60 comp rx=0 tx=0).stop 2024-04-23T18:26:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 -- 172.21.15.19:0/4189611477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa52c1031b0 msgr2=0x7fa52c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa52c1031b0 0x7fa52c198700 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa52800e9f0 tx=0x7fa52800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 -- 172.21.15.19:0/4189611477 shutdown_connections 2024-04-23T18:26:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5180a3ec0 0x7fa5180a6380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa52c10c580 0x7fa52c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa52c103ad0 0x7fa52c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 --2- 172.21.15.19:0/4189611477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa52c1031b0 0x7fa52c198700 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 -- 172.21.15.19:0/4189611477 >> 172.21.15.19:0/4189611477 conn(0x7fa52c0771d0 msgr2=0x7fa52c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 -- 172.21.15.19:0/4189611477 shutdown_connections 2024-04-23T18:26:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.042+0000 7fa5345a1640 1 -- 172.21.15.19:0/4189611477 wait complete. 2024-04-23T18:26:15.047 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:15.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.158+0000 7f1be11cd640 1 -- 172.21.15.19:0/1305795644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc106170 msgr2=0x7f1bdc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.158+0000 7f1be11cd640 1 --2- 172.21.15.19:0/1305795644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc106170 0x7f1bdc10dd40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009ca0 tx=0x7f1bd402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.158+0000 7f1be11cd640 1 -- 172.21.15.19:0/1305795644 shutdown_connections 2024-04-23T18:26:15.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.158+0000 7f1be11cd640 1 --2- 172.21.15.19:0/1305795644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc106170 0x7f1bdc10dd40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.158+0000 7f1be11cd640 1 --2- 172.21.15.19:0/1305795644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bdc101a50 0x7f1bdc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.158+0000 7f1be11cd640 1 --2- 172.21.15.19:0/1305795644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bdc1010a0 0x7f1bdc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.158+0000 7f1be11cd640 1 -- 172.21.15.19:0/1305795644 >> 172.21.15.19:0/1305795644 conn(0x7f1bdc077f50 msgr2=0x7f1bdc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:15.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.158+0000 7f1be11cd640 1 -- 172.21.15.19:0/1305795644 shutdown_connections 2024-04-23T18:26:15.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.158+0000 7f1be11cd640 1 -- 172.21.15.19:0/1305795644 wait complete. 2024-04-23T18:26:15.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.158+0000 7f1be11cd640 1 Processor -- start 2024-04-23T18:26:15.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1be11cd640 1 -- start start 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1be11cd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bdc1010a0 0x7f1bdc19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1be11cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc106170 0x7f1bdc19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1be11cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bdc1985b0 0x7f1bdc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1be11cd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc111130 con 0x7f1bdc106170 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1be11cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc110fb0 con 0x7f1bdc1985b0 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1be11cd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1112b0 con 0x7f1bdc1010a0 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bda59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc106170 0x7f1bdc19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bda59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc106170 0x7f1bdc19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53470/0 (socket says 172.21.15.19:53470) 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bda59c640 1 -- 172.21.15.19:0/3756888835 learned_addr learned my addr 172.21.15.19:0/3756888835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bdad9d640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bdc1010a0 0x7f1bdc19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bdb59e640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bdc1985b0 0x7f1bdc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bda59c640 1 -- 172.21.15.19:0/3756888835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bdc1010a0 msgr2=0x7f1bdc19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bda59c640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bdc1010a0 0x7f1bdc19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bda59c640 1 -- 172.21.15.19:0/3756888835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bdc1985b0 msgr2=0x7f1bdc198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bda59c640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bdc1985b0 0x7f1bdc198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bda59c640 1 -- 172.21.15.19:0/3756888835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc1990d0 con 0x7f1bdc106170 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.159+0000 7f1bdb59e640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bdc1985b0 0x7f1bdc198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.160+0000 7f1bda59c640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc106170 0x7f1bdc19e940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000b450 tx=0x7f1bd000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.160+0000 7f1bc3fff640 1 -- 172.21.15.19:0/3756888835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0015020 con 0x7f1bdc106170 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.160+0000 7f1bc3fff640 1 -- 172.21.15.19:0/3756888835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd0011d40 con 0x7f1bdc106170 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.160+0000 7f1be11cd640 1 -- 172.21.15.19:0/3756888835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc1993c0 con 0x7f1bdc106170 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.160+0000 7f1bc3fff640 1 -- 172.21.15.19:0/3756888835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd000f340 con 0x7f1bdc106170 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.160+0000 7f1be11cd640 1 -- 172.21.15.19:0/3756888835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc1a55c0 con 0x7f1bdc106170 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.161+0000 7f1be11cd640 1 -- 172.21.15.19:0/3756888835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bdc068d70 con 0x7f1bdc106170 2024-04-23T18:26:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.162+0000 7f1bc3fff640 1 -- 172.21.15.19:0/3756888835 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f1bd0018020 con 0x7f1bdc106170 2024-04-23T18:26:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.163+0000 7f1bc3fff640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bc40a3e40 0x7f1bc40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.163+0000 7f1bc3fff640 1 -- 172.21.15.19:0/3756888835 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f1bd00d0480 con 0x7f1bdc106170 2024-04-23T18:26:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.163+0000 7f1bdad9d640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bc40a3e40 0x7f1bc40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.163+0000 7f1bdad9d640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bc40a3e40 0x7f1bc40a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00bbe0 tx=0x7f1bcc00b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.167+0000 7f1bc3fff640 1 -- 172.21.15.19:0/3756888835 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bd001f020 con 0x7f1bdc106170 2024-04-23T18:26:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.356+0000 7f1be11cd640 1 -- 172.21.15.19:0/3756888835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bdc199ae0 con 0x7f1bdc106170 2024-04-23T18:26:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.357+0000 7f1bc3fff640 1 -- 172.21.15.19:0/3756888835 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23018 (secure 0 0 0) 0x7f1bdc199ae0 con 0x7f1bdc106170 2024-04-23T18:26:15.351 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:15.351 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:26:08.116951+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/304874449":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1108384152":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6801/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6800/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/983971095":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2556512605":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 -- 172.21.15.19:0/3756888835 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bc40a3e40 msgr2=0x7f1bc40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bc40a3e40 0x7f1bc40a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00bbe0 tx=0x7f1bcc00b390 comp rx=0 tx=0).stop 2024-04-23T18:26:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 -- 172.21.15.19:0/3756888835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc106170 msgr2=0x7f1bdc19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc106170 0x7f1bdc19e940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000b450 tx=0x7f1bd000b910 comp rx=0 tx=0).stop 2024-04-23T18:26:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 -- 172.21.15.19:0/3756888835 shutdown_connections 2024-04-23T18:26:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bc40a3e40 0x7f1bc40a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bdc1985b0 0x7f1bdc198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc106170 0x7f1bdc19e940 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 --2- 172.21.15.19:0/3756888835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bdc1010a0 0x7f1bdc19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 -- 172.21.15.19:0/3756888835 >> 172.21.15.19:0/3756888835 conn(0x7f1bdc077f50 msgr2=0x7f1bdc100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 -- 172.21.15.19:0/3756888835 shutdown_connections 2024-04-23T18:26:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.359+0000 7f1be11cd640 1 -- 172.21.15.19:0/3756888835 wait complete. 2024-04-23T18:26:15.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:15 smithi019 ceph-mon[29215]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:15.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:15 smithi019 ceph-mon[29215]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-23T18:26:15.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:15 smithi019 ceph-mon[29215]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:15.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:15 smithi019 ceph-mon[29215]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:15.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:15 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4189611477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:15.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:23:53.296504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:23:54.573484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:26:15.365 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:26:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.476+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/3940351099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb58107260 msgr2=0x7fbb5810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.476+0000 7fbb5ca8a640 1 --2- 172.21.15.19:0/3940351099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb58107260 0x7fbb5810ee30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009ca0 tx=0x7fbb5002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.476+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/3940351099 shutdown_connections 2024-04-23T18:26:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.476+0000 7fbb5ca8a640 1 --2- 172.21.15.19:0/3940351099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb58107260 0x7fbb5810ee30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.476+0000 7fbb5ca8a640 1 --2- 172.21.15.19:0/3940351099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb58102b40 0x7fbb58106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.476+0000 7fbb5ca8a640 1 --2- 172.21.15.19:0/3940351099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb58102190 0x7fbb58102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.476+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/3940351099 >> 172.21.15.19:0/3940351099 conn(0x7fbb580793a0 msgr2=0x7fbb580797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.476+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/3940351099 shutdown_connections 2024-04-23T18:26:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.476+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/3940351099 wait complete. 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb5ca8a640 1 Processor -- start 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb5ca8a640 1 -- start start 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb5ca8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb58102190 0x7fbb58198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb5ca8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb58107260 0x7fbb58198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb5ca8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb5819d010 0x7fbb5819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb5ca8a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58112270 con 0x7fbb58107260 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb5ca8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581120f0 con 0x7fbb58102190 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb5ca8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581123f0 con 0x7fbb5819d010 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb55d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb58107260 0x7fbb58198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb55d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb58107260 0x7fbb58198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53502/0 (socket says 172.21.15.19:53502) 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb55d9b640 1 -- 172.21.15.19:0/1452999664 learned_addr learned my addr 172.21.15.19:0/1452999664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:15.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb5659c640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb58102190 0x7fbb58198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb55d9b640 1 -- 172.21.15.19:0/1452999664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb5819d010 msgr2=0x7fbb5819d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb55d9b640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb5819d010 0x7fbb5819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb55d9b640 1 -- 172.21.15.19:0/1452999664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb58102190 msgr2=0x7fbb58198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb55d9b640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb58102190 0x7fbb58198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb55d9b640 1 -- 172.21.15.19:0/1452999664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5819db00 con 0x7fbb58107260 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.477+0000 7fbb5659c640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb58102190 0x7fbb58198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.478+0000 7fbb55d9b640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb58107260 0x7fbb58198c90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00b450 tx=0x7fbb4c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.478+0000 7fbb477fe640 1 -- 172.21.15.19:0/1452999664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c015020 con 0x7fbb58107260 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.478+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/1452999664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5819ddf0 con 0x7fbb58107260 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.478+0000 7fbb477fe640 1 -- 172.21.15.19:0/1452999664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4c011d40 con 0x7fbb58107260 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.478+0000 7fbb477fe640 1 -- 172.21.15.19:0/1452999664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00f340 con 0x7fbb58107260 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.478+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/1452999664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb581a5590 con 0x7fbb58107260 2024-04-23T18:26:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.479+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/1452999664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb580691c0 con 0x7fbb58107260 2024-04-23T18:26:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.480+0000 7fbb477fe640 1 -- 172.21.15.19:0/1452999664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fbb4c018020 con 0x7fbb58107260 2024-04-23T18:26:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.481+0000 7fbb477fe640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbb400a3e40 0x7fbb400a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.481+0000 7fbb5659c640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbb400a3e40 0x7fbb400a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.481+0000 7fbb477fe640 1 -- 172.21.15.19:0/1452999664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbb4c0d04e0 con 0x7fbb58107260 2024-04-23T18:26:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.481+0000 7fbb5659c640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbb400a3e40 0x7fbb400a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb480058c0 tx=0x7fbb48005d30 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:15.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.486+0000 7fbb477fe640 1 -- 172.21.15.19:0/1452999664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb4c01f020 con 0x7fbb58107260 2024-04-23T18:26:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:15 smithi044 ceph-mon[32458]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:15 smithi044 ceph-mon[32458]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-23T18:26:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:15 smithi044 ceph-mon[32458]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:15 smithi044 ceph-mon[32458]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:15 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4189611477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:15 smithi155 ceph-mon[33468]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:15 smithi155 ceph-mon[33468]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-23T18:26:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:15 smithi155 ceph-mon[33468]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:15 smithi155 ceph-mon[33468]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:15 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4189611477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:15.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.670+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/1452999664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb58102b40 con 0x7fbb58107260 2024-04-23T18:26:15.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.670+0000 7fbb477fe640 1 -- 172.21.15.19:0/1452999664 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fbb4c00d040 con 0x7fbb58107260 2024-04-23T18:26:15.664 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:26:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/1452999664 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbb400a3e40 msgr2=0x7fbb400a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbb400a3e40 0x7fbb400a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb480058c0 tx=0x7fbb48005d30 comp rx=0 tx=0).stop 2024-04-23T18:26:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/1452999664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb58107260 msgr2=0x7fbb58198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb58107260 0x7fbb58198c90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00b450 tx=0x7fbb4c00b910 comp rx=0 tx=0).stop 2024-04-23T18:26:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/1452999664 shutdown_connections 2024-04-23T18:26:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbb400a3e40 0x7fbb400a6300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb5819d010 0x7fbb5819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb58107260 0x7fbb58198c90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 --2- 172.21.15.19:0/1452999664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb58102190 0x7fbb58198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/1452999664 >> 172.21.15.19:0/1452999664 conn(0x7fbb580793a0 msgr2=0x7fbb580ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.672+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/1452999664 shutdown_connections 2024-04-23T18:26:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.673+0000 7fbb5ca8a640 1 -- 172.21.15.19:0/1452999664 wait complete. 2024-04-23T18:26:15.677 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 -- 172.21.15.19:0/1466676131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4105dd0 msgr2=0x7f6cb41061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 --2- 172.21.15.19:0/1466676131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4105dd0 0x7f6cb41061b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009900 tx=0x7f6ca402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 -- 172.21.15.19:0/1466676131 shutdown_connections 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 --2- 172.21.15.19:0/1466676131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb41010a0 0x7f6cb4101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 --2- 172.21.15.19:0/1466676131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4101a40 0x7f6cb410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 --2- 172.21.15.19:0/1466676131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb4105dd0 0x7f6cb41061b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 -- 172.21.15.19:0/1466676131 >> 172.21.15.19:0/1466676131 conn(0x7f6cb4077c40 msgr2=0x7f6cb4078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 -- 172.21.15.19:0/1466676131 shutdown_connections 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 -- 172.21.15.19:0/1466676131 wait complete. 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 Processor -- start 2024-04-23T18:26:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 -- start start 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb41010a0 0x7f6cb40feba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4101a40 0x7f6cb40ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4105dd0 0x7f6cb40ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4111170 con 0x7f6cb41010a0 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4110ff0 con 0x7f6cb4101a40 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.789+0000 7f6cbab4b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41112f0 con 0x7f6cb4105dd0 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb88e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb41010a0 0x7f6cb40feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb88e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb41010a0 0x7f6cb40feba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53530/0 (socket says 172.21.15.19:53530) 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb88e7640 1 -- 172.21.15.19:0/2691234658 learned_addr learned my addr 172.21.15.19:0/2691234658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb3fff640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4101a40 0x7f6cb40ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb90e8640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4105dd0 0x7f6cb40ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb88e7640 1 -- 172.21.15.19:0/2691234658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4105dd0 msgr2=0x7f6cb40ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb88e7640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4105dd0 0x7f6cb40ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb88e7640 1 -- 172.21.15.19:0/2691234658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4101a40 msgr2=0x7f6cb40ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb88e7640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4101a40 0x7f6cb40ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb88e7640 1 -- 172.21.15.19:0/2691234658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb41a6c80 con 0x7f6cb41010a0 2024-04-23T18:26:15.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb90e8640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4105dd0 0x7f6cb40ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:15.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb88e7640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb41010a0 0x7f6cb40feba0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4005940 tx=0x7f6ca40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:15.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/2691234658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca403f030 con 0x7f6cb41010a0 2024-04-23T18:26:15.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/2691234658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca4003cb0 con 0x7f6cb41010a0 2024-04-23T18:26:15.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cbab4b640 1 -- 172.21.15.19:0/2691234658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb41a6f10 con 0x7f6cb41010a0 2024-04-23T18:26:15.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/2691234658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4035070 con 0x7f6cb41010a0 2024-04-23T18:26:15.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.790+0000 7f6cbab4b640 1 -- 172.21.15.19:0/2691234658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb41a7370 con 0x7f6cb41010a0 2024-04-23T18:26:15.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.792+0000 7f6cbab4b640 1 -- 172.21.15.19:0/2691234658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb4102540 con 0x7f6cb41010a0 2024-04-23T18:26:15.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.793+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/2691234658 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6ca4042030 con 0x7f6cb41010a0 2024-04-23T18:26:15.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.793+0000 7f6cb1ffb640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6c9c0a3e40 0x7f6c9c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:15.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.793+0000 7f6cb3fff640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6c9c0a3e40 0x7f6c9c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:15.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.793+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/2691234658 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f6ca40f1200 con 0x7f6cb41010a0 2024-04-23T18:26:15.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.794+0000 7f6cb3fff640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6c9c0a3e40 0x7f6c9c0a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4100630 tx=0x7f6ca800b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:15.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.798+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/2691234658 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ca40bd750 con 0x7f6cb41010a0 2024-04-23T18:26:15.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.986+0000 7f6cbab4b640 1 -- 172.21.15.19:0/2691234658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6cb4109de0 con 0x7f6cb41010a0 2024-04-23T18:26:15.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.986+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/2691234658 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f6ca40bd050 con 0x7f6cb41010a0 2024-04-23T18:26:15.980 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-23T18:26:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 -- 172.21.15.19:0/2691234658 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6c9c0a3e40 msgr2=0x7f6c9c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6c9c0a3e40 0x7f6c9c0a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4100630 tx=0x7f6ca800b390 comp rx=0 tx=0).stop 2024-04-23T18:26:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 -- 172.21.15.19:0/2691234658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb41010a0 msgr2=0x7f6cb40feba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb41010a0 0x7f6cb40feba0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4005940 tx=0x7f6ca40042f0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 -- 172.21.15.19:0/2691234658 shutdown_connections 2024-04-23T18:26:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6c9c0a3e40 0x7f6c9c0a6300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb4101a40 0x7f6cb40ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb4105dd0 0x7f6cb40ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 --2- 172.21.15.19:0/2691234658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cb41010a0 0x7f6cb40feba0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 -- 172.21.15.19:0/2691234658 >> 172.21.15.19:0/2691234658 conn(0x7f6cb4077c40 msgr2=0x7f6cb406a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 -- 172.21.15.19:0/2691234658 shutdown_connections 2024-04-23T18:26:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:15.988+0000 7f6cbab4b640 1 -- 172.21.15.19:0/2691234658 wait complete. 2024-04-23T18:26:15.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:26:16.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.105+0000 7fbff0376640 1 -- 172.21.15.19:0/3272757542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe806b1f0 msgr2=0x7fbfe8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.105+0000 7fbff0376640 1 --2- 172.21.15.19:0/3272757542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe806b1f0 0x7fbfe8106b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009900 tx=0x7fbfd802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:16.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.105+0000 7fbff0376640 1 -- 172.21.15.19:0/3272757542 shutdown_connections 2024-04-23T18:26:16.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.105+0000 7fbff0376640 1 --2- 172.21.15.19:0/3272757542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfe8107270 0x7fbfe810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.105+0000 7fbff0376640 1 --2- 172.21.15.19:0/3272757542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe806b1f0 0x7fbfe8106b90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.105+0000 7fbff0376640 1 --2- 172.21.15.19:0/3272757542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe806a840 0x7fbfe806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.105+0000 7fbff0376640 1 -- 172.21.15.19:0/3272757542 >> 172.21.15.19:0/3272757542 conn(0x7fbfe8077eb0 msgr2=0x7fbfe80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:16.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.105+0000 7fbff0376640 1 -- 172.21.15.19:0/3272757542 shutdown_connections 2024-04-23T18:26:16.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.105+0000 7fbff0376640 1 -- 172.21.15.19:0/3272757542 wait complete. 2024-04-23T18:26:16.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbff0376640 1 Processor -- start 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbff0376640 1 -- start start 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbff0376640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe806a840 0x7fbfe81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbff0376640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfe8107270 0x7fbfe8198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbff0376640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe819cfb0 0x7fbfe819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbff0376640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe81121b0 con 0x7fbfe819cfb0 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbff0376640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe8112030 con 0x7fbfe806a840 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbff0376640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe8112330 con 0x7fbfe8107270 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfee913640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe819cfb0 0x7fbfe819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfee913640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe819cfb0 0x7fbfe819d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53540/0 (socket says 172.21.15.19:53540) 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfee913640 1 -- 172.21.15.19:0/3602353249 learned_addr learned my addr 172.21.15.19:0/3602353249 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfee112640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe806a840 0x7fbfe81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfed911640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfe8107270 0x7fbfe8198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfee913640 1 -- 172.21.15.19:0/3602353249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfe8107270 msgr2=0x7fbfe8198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfee913640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfe8107270 0x7fbfe8198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfee913640 1 -- 172.21.15.19:0/3602353249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe806a840 msgr2=0x7fbfe81986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfee913640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe806a840 0x7fbfe81986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfee913640 1 -- 172.21.15.19:0/3602353249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe819daa0 con 0x7fbfe819cfb0 2024-04-23T18:26:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.106+0000 7fbfee112640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe806a840 0x7fbfe81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.107+0000 7fbfee913640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe819cfb0 0x7fbfe819d460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000eaa0 tx=0x7fbfe000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.107+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3602353249 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000cdb0 con 0x7fbfe819cfb0 2024-04-23T18:26:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.107+0000 7fbff0376640 1 -- 172.21.15.19:0/3602353249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe819dd90 con 0x7fbfe819cfb0 2024-04-23T18:26:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.107+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3602353249 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfe0005020 con 0x7fbfe819cfb0 2024-04-23T18:26:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.107+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3602353249 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00116d0 con 0x7fbfe819cfb0 2024-04-23T18:26:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.107+0000 7fbff0376640 1 -- 172.21.15.19:0/3602353249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe81a5580 con 0x7fbfe819cfb0 2024-04-23T18:26:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.108+0000 7fbff0376640 1 -- 172.21.15.19:0/3602353249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe80691c0 con 0x7fbfe819cfb0 2024-04-23T18:26:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.109+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3602353249 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fbfe00054a0 con 0x7fbfe819cfb0 2024-04-23T18:26:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.109+0000 7fbfdf7fe640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfd40a3e40 0x7fbfd40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.110+0000 7fbfee112640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfd40a3e40 0x7fbfd40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.110+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3602353249 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbfe00d0c60 con 0x7fbfe819cfb0 2024-04-23T18:26:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.110+0000 7fbfee112640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfd40a3e40 0x7fbfd40a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009b50 tx=0x7fbfe4005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.114+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3602353249 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfe0016030 con 0x7fbfe819cfb0 2024-04-23T18:26:16.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:16 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3756888835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:16.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:16 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1452999664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:16.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:16 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2691234658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:16.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.301+0000 7fbff0376640 1 -- 172.21.15.19:0/3602353249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbfe806b1f0 con 0x7fbfe819cfb0 2024-04-23T18:26:16.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.301+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3602353249 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fbfe009d0e0 con 0x7fbfe819cfb0 2024-04-23T18:26:16.295 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-23T18:26:16.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 -- 172.21.15.19:0/3602353249 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfd40a3e40 msgr2=0x7fbfd40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfd40a3e40 0x7fbfd40a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009b50 tx=0x7fbfe4005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:16.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 -- 172.21.15.19:0/3602353249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe819cfb0 msgr2=0x7fbfe819d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe819cfb0 0x7fbfe819d460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000eaa0 tx=0x7fbfe000ef60 comp rx=0 tx=0).stop 2024-04-23T18:26:16.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 -- 172.21.15.19:0/3602353249 shutdown_connections 2024-04-23T18:26:16.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfd40a3e40 0x7fbfd40a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe819cfb0 0x7fbfe819d460 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfe8107270 0x7fbfe8198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 --2- 172.21.15.19:0/3602353249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe806a840 0x7fbfe81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 -- 172.21.15.19:0/3602353249 >> 172.21.15.19:0/3602353249 conn(0x7fbfe8077eb0 msgr2=0x7fbfe8108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:16.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 -- 172.21.15.19:0/3602353249 shutdown_connections 2024-04-23T18:26:16.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.303+0000 7fbff0376640 1 -- 172.21.15.19:0/3602353249 wait complete. 2024-04-23T18:26:16.308 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:16.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.417+0000 7f0707259640 1 -- 172.21.15.19:0/1074691159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0700101a50 msgr2=0x7f0700105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.417+0000 7f0707259640 1 --2- 172.21.15.19:0/1074691159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0700101a50 0x7f0700105a90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009900 tx=0x7f06f402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:16.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.417+0000 7f0707259640 1 -- 172.21.15.19:0/1074691159 shutdown_connections 2024-04-23T18:26:16.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.417+0000 7f0707259640 1 --2- 172.21.15.19:0/1074691159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0700106170 0x7f070010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.417+0000 7f0707259640 1 --2- 172.21.15.19:0/1074691159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0700101a50 0x7f0700105a90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.417+0000 7f0707259640 1 --2- 172.21.15.19:0/1074691159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07001010a0 0x7f0700101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.417+0000 7f0707259640 1 -- 172.21.15.19:0/1074691159 >> 172.21.15.19:0/1074691159 conn(0x7f0700077f50 msgr2=0x7f0700078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.417+0000 7f0707259640 1 -- 172.21.15.19:0/1074691159 shutdown_connections 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.417+0000 7f0707259640 1 -- 172.21.15.19:0/1074691159 wait complete. 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.417+0000 7f0707259640 1 Processor -- start 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0707259640 1 -- start start 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0707259640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07001010a0 0x7f0700102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0707259640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0700106170 0x7f0700103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0707259640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07001035b0 0x7f07001a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0707259640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07001110b0 con 0x7f07001010a0 2024-04-23T18:26:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0707259640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0700110f30 con 0x7f07001035b0 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0707259640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0700111230 con 0x7f0700106170 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0704ff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07001010a0 0x7f0700102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0704ff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07001010a0 0x7f0700102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53564/0 (socket says 172.21.15.19:53564) 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0704ff5640 1 -- 172.21.15.19:0/4080853443 learned_addr learned my addr 172.21.15.19:0/4080853443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f07057f6640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07001035b0 0x7f07001a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f06fffff640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0700106170 0x7f0700103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0704ff5640 1 -- 172.21.15.19:0/4080853443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0700106170 msgr2=0x7f0700103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0704ff5640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0700106170 0x7f0700103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0704ff5640 1 -- 172.21.15.19:0/4080853443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07001035b0 msgr2=0x7f07001a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0704ff5640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07001035b0 0x7f07001a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0704ff5640 1 -- 172.21.15.19:0/4080853443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07001a7010 con 0x7f07001010a0 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f07057f6640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07001035b0 0x7f07001a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f06fffff640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0700106170 0x7f0700103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.418+0000 7f0704ff5640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07001010a0 0x7f0700102b30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f06f800eab0 tx=0x7f06f800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.419+0000 7f06fdffb640 1 -- 172.21.15.19:0/4080853443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f800ce40 con 0x7f07001010a0 2024-04-23T18:26:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.419+0000 7f06fdffb640 1 -- 172.21.15.19:0/4080853443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f8011920 con 0x7f07001010a0 2024-04-23T18:26:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.419+0000 7f0707259640 1 -- 172.21.15.19:0/4080853443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07001a7300 con 0x7f07001010a0 2024-04-23T18:26:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.419+0000 7f06fdffb640 1 -- 172.21.15.19:0/4080853443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8010a30 con 0x7f07001010a0 2024-04-23T18:26:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.419+0000 7f0707259640 1 -- 172.21.15.19:0/4080853443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07001a7850 con 0x7f07001010a0 2024-04-23T18:26:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.420+0000 7f0707259640 1 -- 172.21.15.19:0/4080853443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0700068d70 con 0x7f07001010a0 2024-04-23T18:26:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.421+0000 7f06fdffb640 1 -- 172.21.15.19:0/4080853443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f06f8010bd0 con 0x7f07001010a0 2024-04-23T18:26:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.421+0000 7f06fdffb640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f06e80a3e70 0x7f06e80a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.421+0000 7f06fdffb640 1 -- 172.21.15.19:0/4080853443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f06f80d1090 con 0x7f07001010a0 2024-04-23T18:26:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.421+0000 7f06fffff640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f06e80a3e70 0x7f06e80a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.422+0000 7f06fffff640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f06e80a3e70 0x7f06e80a6330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f06f4000c00 tx=0x7f06f4031400 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:16.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.426+0000 7f06fdffb640 1 -- 172.21.15.19:0/4080853443 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06f8016030 con 0x7f07001010a0 2024-04-23T18:26:16.474 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:16 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3756888835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:16.474 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:16 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1452999664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:16.474 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:16 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2691234658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:16.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:16 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3756888835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:16.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:16 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1452999664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:16.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:16 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2691234658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:16.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.650+0000 7f0707259640 1 -- 172.21.15.19:0/4080853443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f070010a410 con 0x7f07001010a0 2024-04-23T18:26:16.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.651+0000 7f06fdffb640 1 -- 172.21.15.19:0/4080853443 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f06f809d5b0 con 0x7f07001010a0 2024-04-23T18:26:16.645 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:16.645 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:26:16.645 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:26:16.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 -- 172.21.15.19:0/4080853443 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f06e80a3e70 msgr2=0x7f06e80a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f06e80a3e70 0x7f06e80a6330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f06f4000c00 tx=0x7f06f4031400 comp rx=0 tx=0).stop 2024-04-23T18:26:16.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 -- 172.21.15.19:0/4080853443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07001010a0 msgr2=0x7f0700102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07001010a0 0x7f0700102b30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f06f800eab0 tx=0x7f06f800ef70 comp rx=0 tx=0).stop 2024-04-23T18:26:16.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 -- 172.21.15.19:0/4080853443 shutdown_connections 2024-04-23T18:26:16.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f06e80a3e70 0x7f06e80a6330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07001035b0 0x7f07001a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0700106170 0x7f0700103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 --2- 172.21.15.19:0/4080853443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07001010a0 0x7f0700102b30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 -- 172.21.15.19:0/4080853443 >> 172.21.15.19:0/4080853443 conn(0x7f0700077f50 msgr2=0x7f07001001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:16.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 -- 172.21.15.19:0/4080853443 shutdown_connections 2024-04-23T18:26:16.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.653+0000 7f0707259640 1 -- 172.21.15.19:0/4080853443 wait complete. 2024-04-23T18:26:16.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:16.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.767+0000 7f10d4891640 1 -- 172.21.15.19:0/1150615485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d0101110 msgr2=0x7f10d01014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.767+0000 7f10d4891640 1 --2- 172.21.15.19:0/1150615485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d0101110 0x7f10d01014f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009900 tx=0x7f10c002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:16.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.767+0000 7f10d4891640 1 -- 172.21.15.19:0/1150615485 shutdown_connections 2024-04-23T18:26:16.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.767+0000 7f10d4891640 1 --2- 172.21.15.19:0/1150615485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0106190 0x7f10d010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.767+0000 7f10d4891640 1 --2- 172.21.15.19:0/1150615485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0101ac0 0x7f10d0105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.767+0000 7f10d4891640 1 --2- 172.21.15.19:0/1150615485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d0101110 0x7f10d01014f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.767+0000 7f10d4891640 1 -- 172.21.15.19:0/1150615485 >> 172.21.15.19:0/1150615485 conn(0x7f10d0077fb0 msgr2=0x7f10d00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:16.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.767+0000 7f10d4891640 1 -- 172.21.15.19:0/1150615485 shutdown_connections 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.767+0000 7f10d4891640 1 -- 172.21.15.19:0/1150615485 wait complete. 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10d4891640 1 Processor -- start 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10d4891640 1 -- start start 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10d4891640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101110 0x7f10d0199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10d4891640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0106190 0x7f10d0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10d4891640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d019a030 0x7f10d01a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10d4891640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10d0111140 con 0x7f10d019a030 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10d4891640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10d0110fc0 con 0x7f10d0101110 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10d4891640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f10d01112c0 con 0x7f10d0106190 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10ced9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d019a030 0x7f10d01a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10ced9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d019a030 0x7f10d01a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53582/0 (socket says 172.21.15.19:53582) 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10ced9d640 1 -- 172.21.15.19:0/4293256742 learned_addr learned my addr 172.21.15.19:0/4293256742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10c7fff640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0106190 0x7f10d0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.768+0000 7f10ce59c640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101110 0x7f10d0199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10ced9d640 1 -- 172.21.15.19:0/4293256742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0106190 msgr2=0x7f10d0199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10ced9d640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0106190 0x7f10d0199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10ced9d640 1 -- 172.21.15.19:0/4293256742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101110 msgr2=0x7f10d0199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10ced9d640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101110 0x7f10d0199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10ced9d640 1 -- 172.21.15.19:0/4293256742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d01a4db0 con 0x7f10d019a030 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10ce59c640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101110 0x7f10d0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10c7fff640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0106190 0x7f10d0199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10ced9d640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d019a030 0x7f10d01a45e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f10c800cab0 tx=0x7f10c800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10c77fe640 1 -- 172.21.15.19:0/4293256742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c800ae20 con 0x7f10d019a030 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10c77fe640 1 -- 172.21.15.19:0/4293256742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c800f900 con 0x7f10d019a030 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10d4891640 1 -- 172.21.15.19:0/4293256742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d01a50a0 con 0x7f10d019a030 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10c77fe640 1 -- 172.21.15.19:0/4293256742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c80189f0 con 0x7f10d019a030 2024-04-23T18:26:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.769+0000 7f10d4891640 1 -- 172.21.15.19:0/4293256742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d01a55f0 con 0x7f10d019a030 2024-04-23T18:26:16.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.770+0000 7f10d4891640 1 -- 172.21.15.19:0/4293256742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d00691c0 con 0x7f10d019a030 2024-04-23T18:26:16.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.771+0000 7f10c77fe640 1 -- 172.21.15.19:0/4293256742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f10c800fab0 con 0x7f10d019a030 2024-04-23T18:26:16.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.772+0000 7f10c77fe640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f10bc0a3ec0 0x7f10bc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.772+0000 7f10ce59c640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f10bc0a3ec0 0x7f10bc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.772+0000 7f10c77fe640 1 -- 172.21.15.19:0/4293256742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f10c80d10c0 con 0x7f10d019a030 2024-04-23T18:26:16.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.773+0000 7f10ce59c640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f10bc0a3ec0 0x7f10bc0a6380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10c0000c00 tx=0x7f10c0005a20 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.777+0000 7f10c77fe640 1 -- 172.21.15.19:0/4293256742 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10c8014030 con 0x7f10d019a030 2024-04-23T18:26:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.964+0000 7f10d4891640 1 -- 172.21.15.19:0/4293256742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10d0101ac0 con 0x7f10d019a030 2024-04-23T18:26:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.966+0000 7f10c77fe640 1 -- 172.21.15.19:0/4293256742 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23018 (secure 0 0 0) 0x7f10c809d5e0 con 0x7f10d019a030 2024-04-23T18:26:16.960 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:16.960 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:26:08.116951+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/304874449":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1108384152":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6801/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6800/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/983971095":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2556512605":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 -- 172.21.15.19:0/4293256742 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f10bc0a3ec0 msgr2=0x7f10bc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f10bc0a3ec0 0x7f10bc0a6380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10c0000c00 tx=0x7f10c0005a20 comp rx=0 tx=0).stop 2024-04-23T18:26:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 -- 172.21.15.19:0/4293256742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d019a030 msgr2=0x7f10d01a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d019a030 0x7f10d01a45e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f10c800cab0 tx=0x7f10c800cf70 comp rx=0 tx=0).stop 2024-04-23T18:26:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 -- 172.21.15.19:0/4293256742 shutdown_connections 2024-04-23T18:26:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f10bc0a3ec0 0x7f10bc0a6380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d019a030 0x7f10d01a45e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0106190 0x7f10d0199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 --2- 172.21.15.19:0/4293256742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101110 0x7f10d0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 -- 172.21.15.19:0/4293256742 >> 172.21.15.19:0/4293256742 conn(0x7f10d0077fb0 msgr2=0x7f10d0107360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 -- 172.21.15.19:0/4293256742 shutdown_connections 2024-04-23T18:26:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:16.968+0000 7f10d4891640 1 -- 172.21.15.19:0/4293256742 wait complete. 2024-04-23T18:26:16.973 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 146960 2024-04-23T18:26:16.973 DEBUG:tasks.quiescer:Yielding 2024-04-23T18:26:16.974 INFO:teuthology.run_tasks:Running task workunit... 2024-04-23T18:26:16.987 INFO:tasks.workunit:Pulling workunits from ref 57e0caf76fdb7ab8b1358588f292d08519163844 2024-04-23T18:26:16.988 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-23T18:26:16.988 INFO:tasks.workunit:timeout=3h 2024-04-23T18:26:16.988 INFO:tasks.workunit:cleanup=True 2024-04-23T18:26:16.988 DEBUG:teuthology.orchestra.run.smithi019:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:16.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:17.005 INFO:teuthology.orchestra.run.smithi019.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:17.005 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-23T18:26:17.005 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-23T18:26:17.005 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:26:17.005 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-23T18:26:17.005 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-23 18:24:03.484900777 +0000 2024-04-23T18:26:17.005 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-23 18:24:03.484900777 +0000 2024-04-23T18:26:17.005 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-23 18:26:06.168089987 +0000 2024-04-23T18:26:17.005 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-23 18:24:03.484900777 +0000 2024-04-23T18:26:17.005 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:17.006 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-23T18:26:17.077 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 57e0caf76fdb7ab8b1358588f292d08519163844 2024-04-23T18:26:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.100+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/865465101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7d4102b40 msgr2=0x7ff7d4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.100+0000 7ff7dc0ae640 1 --2- 172.21.15.19:0/865465101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7d4102b40 0x7ff7d4106b80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009900 tx=0x7ff7c802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.100+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/865465101 shutdown_connections 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.100+0000 7ff7dc0ae640 1 --2- 172.21.15.19:0/865465101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4107260 0x7ff7d410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.100+0000 7ff7dc0ae640 1 --2- 172.21.15.19:0/865465101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7d4102b40 0x7ff7d4106b80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.100+0000 7ff7dc0ae640 1 --2- 172.21.15.19:0/865465101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d4102190 0x7ff7d4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.100+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/865465101 >> 172.21.15.19:0/865465101 conn(0x7ff7d4079310 msgr2=0x7ff7d4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.100+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/865465101 shutdown_connections 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.100+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/865465101 wait complete. 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7dc0ae640 1 Processor -- start 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7dc0ae640 1 -- start start 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7dc0ae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7d4102190 0x7ff7d419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7dc0ae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4107260 0x7ff7d419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7dc0ae640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d41985c0 0x7ff7d4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7dc0ae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4112200 con 0x7ff7d4102190 2024-04-23T18:26:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7dc0ae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4112080 con 0x7ff7d4107260 2024-04-23T18:26:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7dc0ae640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4112380 con 0x7ff7d41985c0 2024-04-23T18:26:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7d9e4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7d4102190 0x7ff7d419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7d9e4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7d4102190 0x7ff7d419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53596/0 (socket says 172.21.15.19:53596) 2024-04-23T18:26:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7d9e4a640 1 -- 172.21.15.19:0/2101208235 learned_addr learned my addr 172.21.15.19:0/2101208235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7da64b640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d41985c0 0x7ff7d4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7d9649640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4107260 0x7ff7d419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7d9e4a640 1 -- 172.21.15.19:0/2101208235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d41985c0 msgr2=0x7ff7d4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7d9e4a640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d41985c0 0x7ff7d4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7d9e4a640 1 -- 172.21.15.19:0/2101208235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4107260 msgr2=0x7ff7d419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.101+0000 7ff7d9e4a640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4107260 0x7ff7d419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.102+0000 7ff7d9e4a640 1 -- 172.21.15.19:0/2101208235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d41990e0 con 0x7ff7d4102190 2024-04-23T18:26:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.102+0000 7ff7d9e4a640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7d4102190 0x7ff7d419e410 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000d950 tx=0x7ff7d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.102+0000 7ff7c6ffd640 1 -- 172.21.15.19:0/2101208235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0017030 con 0x7ff7d4102190 2024-04-23T18:26:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.102+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/2101208235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d41993d0 con 0x7ff7d4102190 2024-04-23T18:26:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.102+0000 7ff7c6ffd640 1 -- 172.21.15.19:0/2101208235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d0004020 con 0x7ff7d4102190 2024-04-23T18:26:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.102+0000 7ff7c6ffd640 1 -- 172.21.15.19:0/2101208235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00047a0 con 0x7ff7d4102190 2024-04-23T18:26:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.102+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/2101208235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d41a5620 con 0x7ff7d4102190 2024-04-23T18:26:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.103+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/2101208235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d40691c0 con 0x7ff7d4102190 2024-04-23T18:26:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.105+0000 7ff7c6ffd640 1 -- 172.21.15.19:0/2101208235 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff7d0009a30 con 0x7ff7d4102190 2024-04-23T18:26:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.105+0000 7ff7c6ffd640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff7c00a3e40 0x7ff7c00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.106+0000 7ff7d9649640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff7c00a3e40 0x7ff7c00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.106+0000 7ff7c6ffd640 1 -- 172.21.15.19:0/2101208235 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff7d00d0ad0 con 0x7ff7d4102190 2024-04-23T18:26:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.106+0000 7ff7d9649640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff7c00a3e40 0x7ff7c00a6300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8000c00 tx=0x7ff7c8031400 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.109+0000 7ff7c6ffd640 1 -- 172.21.15.19:0/2101208235 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7d009cef0 con 0x7ff7d4102190 2024-04-23T18:26:17.110 INFO:tasks.workunit.client.0.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-23T18:26:17.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:17 smithi019 ceph-mon[29215]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-23T18:26:17.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:17 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3602353249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:17.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:17 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4080853443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:17.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:17 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4293256742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:17.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.337+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/2101208235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7d4199af0 con 0x7ff7d4102190 2024-04-23T18:26:17.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.338+0000 7ff7c6ffd640 1 -- 172.21.15.19:0/2101208235 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7ff7d009c7f0 con 0x7ff7d4102190 2024-04-23T18:26:17.332 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:17.332 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:26:17.332 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/2101208235 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff7c00a3e40 msgr2=0x7ff7c00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff7c00a3e40 0x7ff7c00a6300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8000c00 tx=0x7ff7c8031400 comp rx=0 tx=0).stop 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/2101208235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7d4102190 msgr2=0x7ff7d419e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7d4102190 0x7ff7d419e410 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000d950 tx=0x7ff7d000de10 comp rx=0 tx=0).stop 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/2101208235 shutdown_connections 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff7c00a3e40 0x7ff7c00a6300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d4107260 0x7ff7d419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d41985c0 0x7ff7d4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 --2- 172.21.15.19:0/2101208235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7d4102190 0x7ff7d419e410 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/2101208235 >> 172.21.15.19:0/2101208235 conn(0x7ff7d4079310 msgr2=0x7ff7d41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/2101208235 shutdown_connections 2024-04-23T18:26:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.340+0000 7ff7dc0ae640 1 -- 172.21.15.19:0/2101208235 wait complete. 2024-04-23T18:26:17.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:23:55.611471+0000', 'modified': '2024-04-23T18:26:06.874631+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24521, 'mds_1': 24508, 'mds_2': 24491, 'mds_3': 24487, 'mds_4': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/716138605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 716138605}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 716138605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14604': {'gid': 14604, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/2593314342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 2593314342}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 2593314342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6833/3128313210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 3128313210}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 3128313210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6833/3545031610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 3545031610}, {'type': 'v1', 'addr': '172.21.15.44:6833', 'nonce': 3545031610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6835/659103876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 659103876}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 659103876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.155:6835/698922157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 698922157}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 698922157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.155:6837/980782961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 980782961}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 980782961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.44:6837/728759484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 728759484}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 728759484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.155:6839/3934638415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 3934638415}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 3934638415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6839/2221834529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 2221834529}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 2221834529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24491, 24521, 24469, 24487, 24508]} 2024-04-23T18:26:17.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T18:26:17.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-23T18:26:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.496+0000 7f2ff5336640 1 -- 172.21.15.19:0/1018827022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ff0102b50 msgr2=0x7f2ff0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.496+0000 7f2ff5336640 1 --2- 172.21.15.19:0/1018827022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ff0102b50 0x7f2ff0106b90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009900 tx=0x7f2fd802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.496+0000 7f2ff5336640 1 -- 172.21.15.19:0/1018827022 shutdown_connections 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.496+0000 7f2ff5336640 1 --2- 172.21.15.19:0/1018827022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff0107270 0x7f2ff010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.496+0000 7f2ff5336640 1 --2- 172.21.15.19:0/1018827022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ff0102b50 0x7f2ff0106b90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.496+0000 7f2ff5336640 1 --2- 172.21.15.19:0/1018827022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff01021a0 0x7f2ff0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.496+0000 7f2ff5336640 1 -- 172.21.15.19:0/1018827022 >> 172.21.15.19:0/1018827022 conn(0x7f2ff00fda20 msgr2=0x7f2ff00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.496+0000 7f2ff5336640 1 -- 172.21.15.19:0/1018827022 shutdown_connections 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.496+0000 7f2ff5336640 1 -- 172.21.15.19:0/1018827022 wait complete. 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.496+0000 7f2ff5336640 1 Processor -- start 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2ff5336640 1 -- start start 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2ff5336640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff01021a0 0x7f2ff019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2ff5336640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff0107270 0x7f2ff019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2ff5336640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ff01985a0 0x7f2ff0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2ff5336640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0112200 con 0x7f2ff01985a0 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2ff5336640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0112080 con 0x7f2ff01021a0 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2ff5336640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0112380 con 0x7f2ff0107270 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ff01985a0 0x7f2ff0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ff01985a0 0x7f2ff0198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53608/0 (socket says 172.21.15.19:53608) 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fef7fe640 1 -- 172.21.15.19:0/4264544482 learned_addr learned my addr 172.21.15.19:0/4264544482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2feeffd640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff01021a0 0x7f2ff019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff0107270 0x7f2ff019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fef7fe640 1 -- 172.21.15.19:0/4264544482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff0107270 msgr2=0x7f2ff019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fef7fe640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff0107270 0x7f2ff019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fef7fe640 1 -- 172.21.15.19:0/4264544482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff01021a0 msgr2=0x7f2ff019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fef7fe640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff01021a0 0x7f2ff019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fef7fe640 1 -- 172.21.15.19:0/4264544482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff01990c0 con 0x7f2ff01985a0 2024-04-23T18:26:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fef7fe640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ff01985a0 0x7f2ff0198a50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800eb10 tx=0x7f2fe800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.497+0000 7f2fecff9640 1 -- 172.21.15.19:0/4264544482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8004b20 con 0x7f2ff01985a0 2024-04-23T18:26:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.498+0000 7f2ff5336640 1 -- 172.21.15.19:0/4264544482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff01993b0 con 0x7f2ff01985a0 2024-04-23T18:26:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.498+0000 7f2fecff9640 1 -- 172.21.15.19:0/4264544482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fe8004cc0 con 0x7f2ff01985a0 2024-04-23T18:26:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.498+0000 7f2fecff9640 1 -- 172.21.15.19:0/4264544482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8010690 con 0x7f2ff01985a0 2024-04-23T18:26:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.498+0000 7f2ff5336640 1 -- 172.21.15.19:0/4264544482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff01a5650 con 0x7f2ff01985a0 2024-04-23T18:26:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.500+0000 7f2fecff9640 1 -- 172.21.15.19:0/4264544482 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f2fe8005020 con 0x7f2ff01985a0 2024-04-23T18:26:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.500+0000 7f2ff8055740 1 Processor -- start 2024-04-23T18:26:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.500+0000 7f2ff8055740 1 -- start start 2024-04-23T18:26:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.500+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb40190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.500+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb4a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.500+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb4d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.500+0000 7f2ff8055740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d8dcaa5c50 con 0x55d8dcb4add0 2024-04-23T18:26:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.500+0000 7f2ff8055740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d8dca42830 con 0x55d8dcb3fdb0 2024-04-23T18:26:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.500+0000 7f2ff8055740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8dcaa1c10 con 0x55d8dcb40760 2024-04-23T18:26:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.500+0000 7f2fecff9640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2fdc0a3e40 0x7f2fdc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb4a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2fe7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb4d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb4a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:33536/0 (socket says 172.21.15.19:33536) 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2feeffd640 1 -- 172.21.15.19:0/1627250051 learned_addr learned my addr 172.21.15.19:0/1627250051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/1627250051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb40190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2fef7fe640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2fdc0a3e40 0x7f2fdc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2fecff9640 1 -- 172.21.15.19:0/4264544482 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2fe80d0f50 con 0x7f2ff01985a0 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2feeffd640 1 -- 172.21.15.19:0/1627250051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 msgr2=0x55d8dcb40190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2feeffd640 1 --2- 172.21.15.19:0/1627250051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb40190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2feeffd640 1 -- 172.21.15.19:0/1627250051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 msgr2=0x55d8dcb4d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2feeffd640 1 --2- 172.21.15.19:0/1627250051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb4d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2feeffd640 1 -- 172.21.15.19:0/1627250051 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8dc7b82d0 con 0x55d8dcb40760 2024-04-23T18:26:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2feeffd640 1 --2- 172.21.15.19:0/1627250051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb4a800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009cf0 tx=0x7f2fe0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d65e11c01d840e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2fef7fe640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2fdc0a3e40 0x7f2fdc0a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80ce5b0 tx=0x7f2fe8092160 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2fe57fa640 1 -- 172.21.15.19:0/1627250051 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe001e070 con 0x55d8dcb40760 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2fe57fa640 1 -- 172.21.15.19:0/1627250051 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fe0009de0 con 0x55d8dcb40760 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.501+0000 7f2fe57fa640 1 -- 172.21.15.19:0/1627250051 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0019450 con 0x55d8dcb40760 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.502+0000 7f2ff8055740 1 -- 172.21.15.19:0/1627250051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 msgr2=0x55d8dcb4a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.502+0000 7f2ff8055740 1 --2- 172.21.15.19:0/1627250051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb4a800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009cf0 tx=0x7f2fe0011810 comp rx=0 tx=0).stop 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.502+0000 7f2ff8055740 1 -- 172.21.15.19:0/1627250051 shutdown_connections 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.502+0000 7f2ff8055740 1 --2- 172.21.15.19:0/1627250051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb4d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.502+0000 7f2ff8055740 1 --2- 172.21.15.19:0/1627250051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb4a800 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.502+0000 7f2ff8055740 1 --2- 172.21.15.19:0/1627250051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb40190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.502+0000 7f2ff8055740 1 -- 172.21.15.19:0/1627250051 >> 172.21.15.19:0/1627250051 conn(0x55d8dcb3c070 msgr2=0x55d8dcb3c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.502+0000 7f2ff8055740 1 -- 172.21.15.19:0/1627250051 shutdown_connections 2024-04-23T18:26:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.502+0000 7f2ff8055740 1 -- 172.21.15.19:0/1627250051 wait complete. 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.502+0000 7f2ff8055740 1 -- start start 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2ff8055740 1 Processor -- start 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb47d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb48270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2ff8055740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d8dcb4dd40 con 0x55d8dcb4add0 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2ff8055740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d8dca40470 con 0x55d8dcb3fdb0 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2ff8055740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8dcb4dec0 con 0x55d8dcb40760 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2ff8055740 10 client.0 _ll_register_callbacks cb 0x55d8dca65f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fe7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fe7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53640/0 (socket says 172.21.15.19:53640) 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fe7fff640 1 -- 172.21.15.19:0/3364094568 learned_addr learned my addr 172.21.15.19:0/3364094568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2feeffd640 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb48270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb47d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fe7fff640 1 -- 172.21.15.19:0/3364094568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 msgr2=0x55d8dcb48270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb48270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fe7fff640 1 -- 172.21.15.19:0/3364094568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 msgr2=0x55d8dcb47d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb47d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fe7fff640 1 -- 172.21.15.19:0/3364094568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8dc8826b0 con 0x55d8dcb4add0 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb487b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8000c00 tx=0x7f2fd802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46e602ea32365cd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.503+0000 7f2fd67fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2fd67fc640 1 -- 172.21.15.19:0/3364094568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8035070 con 0x55d8dcb4add0 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2fd67fc640 1 -- 172.21.15.19:0/3364094568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2fd8003e40 con 0x55d8dcb4add0 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2ff8055740 1 -- 172.21.15.19:0/3364094568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fe0009de0 con 0x55d8dcb4add0 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2fd67fc640 1 -- 172.21.15.19:0/3364094568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2fd80314a0 con 0x55d8dcb4add0 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2ff8055740 10 client.15543 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2ff8055740 1 -- 172.21.15.19:0/3364094568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8dcb5a040 con 0x55d8dcb4add0 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2fd67fc640 1 -- 172.21.15.19:0/3364094568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fe0009de0 con 0x55d8dcb4add0 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2fd67fc640 1 -- 172.21.15.19:0/3364094568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2fd803d5f0 con 0x55d8dcb4add0 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2ff8055740 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2ff8055740 10 client.15543 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2ff8055740 20 client.15543 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2ff8055740 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55d8dcb5c600 0x55d8dcb5da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2ff8055740 4 client.15543 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.504+0000 7f2ff8055740 1 -- 172.21.15.19:0/3364094568 --> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8dcab3670 con 0x55d8dcb5c600 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.505+0000 7f2feeffd640 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55d8dcb5c600 0x55d8dcb5da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.505+0000 7f2feeffd640 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55d8dcb5c600 0x55d8dcb5da70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8a195077df8103e3 server_cookie=b1abba03bbd69498 in_seq=0 out_seq=0 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.505+0000 7f2fd67fc640 10 client.15543 ms_handle_connect on v2:172.21.15.155:6834/698922157 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2fd67fc640 1 -- 172.21.15.19:0/3364094568 <== mds.4 v2:172.21.15.155:6834/698922157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8dcab3670 con 0x55d8dcb5c600 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2fd67fc640 10 client.15543 handle_command_reply: tid=0 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 client.15543 shutdown 2024-04-23T18:26:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 -- 172.21.15.19:0/3364094568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 msgr2=0x55d8dcb487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb487b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8000c00 tx=0x7f2fd802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 -- 172.21.15.19:0/3364094568 shutdown_connections 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55d8dcb5c600 0x55d8dcb5da70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb487b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb48270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2fd67fc640 0 client.15543 ms_handle_reset on v2:172.21.15.155:6834/698922157 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 --2- 172.21.15.19:0/3364094568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb47d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 -- 172.21.15.19:0/3364094568 >> 172.21.15.19:0/3364094568 conn(0x55d8dcb3c070 msgr2=0x55d8dcb41ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 -- 172.21.15.19:0/3364094568 shutdown_connections 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 1 -- 172.21.15.19:0/3364094568 wait complete. 2024-04-23T18:26:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.506+0000 7f2ff8055740 20 client.15543 trim_cache size 0 max 16384 2024-04-23T18:26:17.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.524+0000 7f2ff8055740 1 Processor -- start 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.524+0000 7f2ff8055740 1 -- start start 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.524+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb42470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.524+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb40760 0x55d8dcb46620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.524+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb4add0 0x55d8dcb46b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.524+0000 7f2ff8055740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d8dcb5bdb0 con 0x55d8dcb40760 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.524+0000 7f2ff8055740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d8dcb5c480 con 0x55d8dcb4add0 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.524+0000 7f2ff8055740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8dcb5cb50 con 0x55d8dcb3fdb0 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb40760 0x55d8dcb46620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe7fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb42470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb40760 0x55d8dcb46620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53654/0 (socket says 172.21.15.19:53654) 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2feeffd640 1 -- 172.21.15.19:0/3610747116 learned_addr learned my addr 172.21.15.19:0/3610747116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/3610747116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb4add0 0x55d8dcb46b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe7fff640 1 -- 172.21.15.19:0/3610747116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb4add0 msgr2=0x55d8dcb46b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/3610747116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb4add0 0x55d8dcb46b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe7fff640 1 -- 172.21.15.19:0/3610747116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb40760 msgr2=0x55d8dcb46620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/3610747116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb40760 0x55d8dcb46620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe7fff640 1 -- 172.21.15.19:0/3610747116 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8dcb3bea0 con 0x55d8dcb3fdb0 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2feeffd640 1 --2- 172.21.15.19:0/3610747116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb40760 0x55d8dcb46620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/3610747116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb42470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8003740 tx=0x7f2fd8041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae4aca14a70e86c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe4ff9640 1 -- 172.21.15.19:0/3610747116 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8035070 con 0x55d8dcb3fdb0 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe4ff9640 1 -- 172.21.15.19:0/3610747116 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd803d450 con 0x55d8dcb3fdb0 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.525+0000 7f2fe4ff9640 1 -- 172.21.15.19:0/3610747116 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8031b80 con 0x55d8dcb3fdb0 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 -- 172.21.15.19:0/3610747116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb3fdb0 msgr2=0x55d8dcb42470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 --2- 172.21.15.19:0/3610747116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb42470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8003740 tx=0x7f2fd8041040 comp rx=0 tx=0).stop 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 -- 172.21.15.19:0/3610747116 shutdown_connections 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 --2- 172.21.15.19:0/3610747116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb4add0 0x55d8dcb46b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 --2- 172.21.15.19:0/3610747116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb40760 0x55d8dcb46620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 --2- 172.21.15.19:0/3610747116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb42470 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 -- 172.21.15.19:0/3610747116 >> 172.21.15.19:0/3610747116 conn(0x55d8dcb3c070 msgr2=0x55d8dcb430c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 -- 172.21.15.19:0/3610747116 shutdown_connections 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 -- 172.21.15.19:0/3610747116 wait complete. 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 -- start start 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.526+0000 7f2ff8055740 1 Processor -- start 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb44770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb44cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2ff8055740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb64d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2ff8055740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d8dcb5d8f0 con 0x55d8dcb4add0 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2ff8055740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d8dcb5d220 con 0x55d8dcb3fdb0 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2ff8055740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8dcb5dfc0 con 0x55d8dcb40760 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2ff8055740 10 client.0 _ll_register_callbacks cb 0x7f2fd8003800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:17.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2fe7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb64d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb44cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2fe7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb64d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53670/0 (socket says 172.21.15.19:53670) 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2fe7fff640 1 -- 172.21.15.19:0/1467055710 learned_addr learned my addr 172.21.15.19:0/1467055710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb44770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2feeffd640 1 -- 172.21.15.19:0/1467055710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 msgr2=0x55d8dcb44770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2feeffd640 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb44770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2feeffd640 1 -- 172.21.15.19:0/1467055710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 msgr2=0x55d8dcb64d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2feeffd640 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb64d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2feeffd640 1 -- 172.21.15.19:0/1467055710 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8dcb65270 con 0x55d8dcb40760 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2feeffd640 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb44cb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000a010 tx=0x7f2fe0005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e42f192012c7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2fd67fc640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2fd67fc640 1 -- 172.21.15.19:0/1467055710 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe002e020 con 0x55d8dcb40760 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2ff8055740 1 -- 172.21.15.19:0/1467055710 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fd803d450 con 0x55d8dcb40760 2024-04-23T18:26:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.527+0000 7f2fd67fc640 1 -- 172.21.15.19:0/1467055710 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2fe0003770 con 0x55d8dcb40760 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2fd67fc640 1 -- 172.21.15.19:0/1467055710 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2fe0039050 con 0x55d8dcb40760 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2ff8055740 10 client.25192 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2ff8055740 1 -- 172.21.15.19:0/1467055710 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8dcb66570 con 0x55d8dcb40760 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2fd67fc640 1 -- 172.21.15.19:0/1467055710 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd803d450 con 0x55d8dcb40760 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2fd67fc640 1 -- 172.21.15.19:0/1467055710 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2fe002a260 con 0x55d8dcb40760 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2ff8055740 10 client.25192 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2ff8055740 10 client.25192 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2ff8055740 20 client.25192 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2ff8055740 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55d8dcbc6ca0 0x55d8dcbc59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2ff8055740 4 client.25192 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2ff8055740 1 -- 172.21.15.19:0/1467055710 --> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] -- command(tid 0: {"prefix": "status"}) -- 0x55d8dcaad4e0 con 0x55d8dcbc6ca0 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.528+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55d8dcbc6ca0 0x55d8dcbc59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2fd67fc640 10 client.25192 ms_handle_connect on v2:172.21.15.155:6834/698922157 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2fe7fff640 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55d8dcbc6ca0 0x55d8dcbc59b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a195a86bca746482 server_cookie=3d85cfbd41fcc7af in_seq=0 out_seq=0 2024-04-23T18:26:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2fd67fc640 1 -- 172.21.15.19:0/1467055710 <== mds.4 v2:172.21.15.155:6834/698922157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d8dcaad4e0 con 0x55d8dcbc6ca0 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2fd67fc640 10 client.25192 handle_command_reply: tid=0 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 client.25192 shutdown 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 -- 172.21.15.19:0/1467055710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 msgr2=0x55d8dcb44cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb44cb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000a010 tx=0x7f2fe0005ce0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 -- 172.21.15.19:0/1467055710 shutdown_connections 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.155:6834/698922157,v1:172.21.15.155:6835/698922157] conn(0x55d8dcbc6ca0 0x55d8dcbc59b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8dcb4add0 0x55d8dcb64d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2fd67fc640 0 client.25192 ms_handle_reset on v2:172.21.15.155:6834/698922157 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8dcb40760 0x55d8dcb44cb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 --2- 172.21.15.19:0/1467055710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8dcb3fdb0 0x55d8dcb44770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 -- 172.21.15.19:0/1467055710 >> 172.21.15.19:0/1467055710 conn(0x55d8dcb3c070 msgr2=0x55d8dcb3a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 -- 172.21.15.19:0/1467055710 shutdown_connections 2024-04-23T18:26:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 1 -- 172.21.15.19:0/1467055710 wait complete. 2024-04-23T18:26:17.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff8055740 20 client.25192 trim_cache size 0 max 16384 2024-04-23T18:26:17.530 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:26:17.530 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:17.530 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-04-23T18:26:17.530 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24469, 2024-04-23T18:26:17.530 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:26:17.530 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:26:17.530 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:26:17.530 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 135.14161687800001, 2024-04-23T18:26:17.530 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-04-23T18:26:17.531 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-04-23T18:26:17.531 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:26:17.531 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 164.994547224 2024-04-23T18:26:17.531 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:26:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff5336640 1 -- 172.21.15.19:0/4264544482 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2fdc0a3e40 msgr2=0x7f2fdc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff5336640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2fdc0a3e40 0x7f2fdc0a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80ce5b0 tx=0x7f2fe8092160 comp rx=0 tx=0).stop 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff5336640 1 -- 172.21.15.19:0/4264544482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ff01985a0 msgr2=0x7f2ff0198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.529+0000 7f2ff5336640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ff01985a0 0x7f2ff0198a50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800eb10 tx=0x7f2fe800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.530+0000 7f2ff5336640 1 -- 172.21.15.19:0/4264544482 shutdown_connections 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.530+0000 7f2ff5336640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2fdc0a3e40 0x7f2fdc0a6300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.530+0000 7f2ff5336640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ff01985a0 0x7f2ff0198a50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.530+0000 7f2ff5336640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff0107270 0x7f2ff019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.530+0000 7f2ff5336640 1 --2- 172.21.15.19:0/4264544482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff01021a0 0x7f2ff019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.530+0000 7f2ff5336640 1 -- 172.21.15.19:0/4264544482 >> 172.21.15.19:0/4264544482 conn(0x7f2ff00fda20 msgr2=0x7f2ff00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.530+0000 7f2ff5336640 1 -- 172.21.15.19:0/4264544482 shutdown_connections 2024-04-23T18:26:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.530+0000 7f2ff5336640 1 -- 172.21.15.19:0/4264544482 wait complete. 2024-04-23T18:26:17.534 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T18:26:17.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:17 smithi044 ceph-mon[32458]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-23T18:26:17.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:17 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3602353249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:17.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:17 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4080853443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:17.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:17 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4293256742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:17.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:17 smithi155 ceph-mon[33468]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-23T18:26:17.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:17 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3602353249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:17.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:17 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4080853443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:17.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:17 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4293256742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.643+0000 7f08a773d640 1 -- 172.21.15.19:0/1861042759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a010c600 msgr2=0x7f08a010ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.643+0000 7f08a773d640 1 --2- 172.21.15.19:0/1861042759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a010c600 0x7f08a010ea50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0898009be0 tx=0x7f089802a750 comp rx=0 tx=0).stop 2024-04-23T18:26:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.643+0000 7f08a773d640 1 -- 172.21.15.19:0/1861042759 shutdown_connections 2024-04-23T18:26:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.643+0000 7f08a773d640 1 --2- 172.21.15.19:0/1861042759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08a0103b50 0x7f08a010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.643+0000 7f08a773d640 1 --2- 172.21.15.19:0/1861042759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a010c600 0x7f08a010ea50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.643+0000 7f08a773d640 1 --2- 172.21.15.19:0/1861042759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08a0103230 0x7f08a0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.643+0000 7f08a773d640 1 -- 172.21.15.19:0/1861042759 >> 172.21.15.19:0/1861042759 conn(0x7f08a0077f80 msgr2=0x7f08a0078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.643+0000 7f08a773d640 1 -- 172.21.15.19:0/1861042759 shutdown_connections 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.643+0000 7f08a773d640 1 -- 172.21.15.19:0/1861042759 wait complete. 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a773d640 1 Processor -- start 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a773d640 1 -- start start 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a773d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08a0103230 0x7f08a0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a773d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08a0103b50 0x7f08a01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a773d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a010c600 0x7f08a0101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a773d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08a0112230 con 0x7f08a0103230 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a773d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f08a01120b0 con 0x7f08a0103b50 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a773d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08a01123b0 con 0x7f08a010c600 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a54d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08a0103230 0x7f08a0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a54d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08a0103230 0x7f08a0100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53698/0 (socket says 172.21.15.19:53698) 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a54d9640 1 -- 172.21.15.19:0/427024427 learned_addr learned my addr 172.21.15.19:0/427024427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.644+0000 7f08a5cda640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a010c600 0x7f08a0101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08a54d9640 1 -- 172.21.15.19:0/427024427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a010c600 msgr2=0x7f08a0101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08a54d9640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a010c600 0x7f08a0101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08a54d9640 1 -- 172.21.15.19:0/427024427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08a0103b50 msgr2=0x7f08a01013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08a0103b50 0x7f08a01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08a54d9640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08a0103b50 0x7f08a01013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08a54d9640 1 -- 172.21.15.19:0/427024427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a01a6c70 con 0x7f08a0103230 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08a54d9640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08a0103230 0x7f08a0100e80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f089c00e9c0 tx=0x7f089c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08967fc640 1 -- 172.21.15.19:0/427024427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c00cde0 con 0x7f08a0103230 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08a773d640 1 -- 172.21.15.19:0/427024427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a01a6f60 con 0x7f08a0103230 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08967fc640 1 -- 172.21.15.19:0/427024427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f089c0035f0 con 0x7f08a0103230 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08967fc640 1 -- 172.21.15.19:0/427024427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c010690 con 0x7f08a0103230 2024-04-23T18:26:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.645+0000 7f08a773d640 1 -- 172.21.15.19:0/427024427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a01a7470 con 0x7f08a0103230 2024-04-23T18:26:17.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.647+0000 7f08967fc640 1 -- 172.21.15.19:0/427024427 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f089c004000 con 0x7f08a0103230 2024-04-23T18:26:17.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.647+0000 7f08aa458740 1 Processor -- start 2024-04-23T18:26:17.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08aa458740 1 -- start start 2024-04-23T18:26:17.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08aa458740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 0x55f0de738190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08aa458740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de742800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08aa458740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de745220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08aa458740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f0de69dc50 con 0x55f0de742dd0 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08aa458740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f0de63a830 con 0x55f0de738760 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08aa458740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f0de699c10 con 0x55f0de737db0 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08967fc640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f088c0a3e40 0x7f088c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a4cd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de745220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a5cda640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 0x55f0de738190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a4cd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de745220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53700/0 (socket says 172.21.15.19:53700) 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a4cd8640 1 -- 172.21.15.19:0/74740191 learned_addr learned my addr 172.21.15.19:0/74740191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a5cda640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 0x55f0de738190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:33586/0 (socket says 172.21.15.19:33586) 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08967fc640 1 -- 172.21.15.19:0/427024427 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f089c0d0e20 con 0x7f08a0103230 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a5cda640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f088c0a3e40 0x7f088c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a54d9640 1 --2- 172.21.15.19:0/74740191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de742800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a4cd8640 1 -- 172.21.15.19:0/74740191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 msgr2=0x55f0de738190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/74740191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 0x55f0de738190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a4cd8640 1 -- 172.21.15.19:0/74740191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 msgr2=0x55f0de742800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/74740191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de742800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.648+0000 7f08a4cd8640 1 -- 172.21.15.19:0/74740191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0de3b02d0 con 0x55f0de742dd0 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/74740191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de745220 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0890009b80 tx=0x7f0890010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f042b66b96bb93f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08a5cda640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f088c0a3e40 0x7f088c0a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0898006010 tx=0x7f0898004a40 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f088b7fe640 1 -- 172.21.15.19:0/74740191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089001b480 con 0x55f0de742dd0 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f088b7fe640 1 -- 172.21.15.19:0/74740191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f089001bad0 con 0x55f0de742dd0 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f088b7fe640 1 -- 172.21.15.19:0/74740191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08900129d0 con 0x55f0de742dd0 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08aa458740 1 -- 172.21.15.19:0/74740191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 msgr2=0x55f0de745220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08aa458740 1 --2- 172.21.15.19:0/74740191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de745220 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0890009b80 tx=0x7f0890010740 comp rx=0 tx=0).stop 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08aa458740 1 -- 172.21.15.19:0/74740191 shutdown_connections 2024-04-23T18:26:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08aa458740 1 --2- 172.21.15.19:0/74740191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de742800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08aa458740 1 --2- 172.21.15.19:0/74740191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de745220 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08aa458740 1 --2- 172.21.15.19:0/74740191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 0x55f0de738190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08aa458740 1 -- 172.21.15.19:0/74740191 >> 172.21.15.19:0/74740191 conn(0x55f0de734070 msgr2=0x55f0de734450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08aa458740 1 -- 172.21.15.19:0/74740191 shutdown_connections 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.649+0000 7f08aa458740 1 -- 172.21.15.19:0/74740191 wait complete. 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08aa458740 1 -- start start 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08aa458740 1 Processor -- start 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08aa458740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de737db0 0x55f0de73fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08aa458740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de740230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08aa458740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de742dd0 0x55f0de740770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08aa458740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f0de745d40 con 0x55f0de737db0 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08aa458740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f0de638470 con 0x55f0de738760 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08aa458740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f0de745ec0 con 0x55f0de742dd0 2024-04-23T18:26:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08aa458740 10 client.0 _ll_register_callbacks cb 0x55f0de65df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08a54d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de737db0 0x55f0de73fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08a54d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de737db0 0x55f0de73fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53708/0 (socket says 172.21.15.19:53708) 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08a54d9640 1 -- 172.21.15.19:0/598297832 learned_addr learned my addr 172.21.15.19:0/598297832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.650+0000 7f08a5cda640 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de742dd0 0x55f0de740770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de740230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08a54d9640 1 -- 172.21.15.19:0/598297832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de742dd0 msgr2=0x55f0de740770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08a54d9640 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de742dd0 0x55f0de740770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08a54d9640 1 -- 172.21.15.19:0/598297832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 msgr2=0x55f0de740230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08a54d9640 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de740230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08a54d9640 1 -- 172.21.15.19:0/598297832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0de47a6b0 con 0x55f0de737db0 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de740230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08a54d9640 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de737db0 0x55f0de73fcf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f089c003e20 tx=0x7f089c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca6076d7dccf641 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f0888ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f0888ff9640 1 -- 172.21.15.19:0/598297832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c0a0f50 con 0x55f0de737db0 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f0888ff9640 1 -- 172.21.15.19:0/598297832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f089c0bd070 con 0x55f0de737db0 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08aa458740 1 -- 172.21.15.19:0/598297832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f089001bad0 con 0x55f0de737db0 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f0888ff9640 1 -- 172.21.15.19:0/598297832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f089c0a5810 con 0x55f0de737db0 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08aa458740 10 client.15561 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f08aa458740 1 -- 172.21.15.19:0/598297832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0de752000 con 0x55f0de737db0 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.651+0000 7f0888ff9640 1 -- 172.21.15.19:0/598297832 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f089001bad0 con 0x55f0de737db0 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.652+0000 7f0888ff9640 1 -- 172.21.15.19:0/598297832 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f089c0988f0 con 0x55f0de737db0 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.652+0000 7f08aa458740 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.652+0000 7f08aa458740 10 client.15561 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.652+0000 7f08aa458740 20 client.15561 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.652+0000 7f08aa458740 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x55f0de7545e0 0x55f0de755a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.652+0000 7f08aa458740 4 client.15561 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.652+0000 7f08aa458740 1 -- 172.21.15.19:0/598297832 --> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0de6ab670 con 0x55f0de7545e0 2024-04-23T18:26:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.652+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x55f0de7545e0 0x55f0de755a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.652+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x55f0de7545e0 0x55f0de755a50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3a4c66b1f2cd0fa server_cookie=54685b41ccde8442 in_seq=0 out_seq=0 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.652+0000 7f0888ff9640 10 client.15561 ms_handle_connect on v2:172.21.15.155:6836/980782961 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f0888ff9640 1 -- 172.21.15.19:0/598297832 <== mds.3 v2:172.21.15.155:6836/980782961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0de6ab670 con 0x55f0de7545e0 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f0888ff9640 10 client.15561 handle_command_reply: tid=0 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 client.15561 shutdown 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 -- 172.21.15.19:0/598297832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de737db0 msgr2=0x55f0de73fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de737db0 0x55f0de73fcf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f089c003e20 tx=0x7f089c009510 comp rx=0 tx=0).stop 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 -- 172.21.15.19:0/598297832 shutdown_connections 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x55f0de7545e0 0x55f0de755a50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de740230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de742dd0 0x55f0de740770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f0888ff9640 0 client.15561 ms_handle_reset on v2:172.21.15.155:6836/980782961 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 --2- 172.21.15.19:0/598297832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de737db0 0x55f0de73fcf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 -- 172.21.15.19:0/598297832 >> 172.21.15.19:0/598297832 conn(0x55f0de734070 msgr2=0x55f0de739ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 -- 172.21.15.19:0/598297832 shutdown_connections 2024-04-23T18:26:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 1 -- 172.21.15.19:0/598297832 wait complete. 2024-04-23T18:26:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.653+0000 7f08aa458740 20 client.15561 trim_cache size 0 max 16384 2024-04-23T18:26:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08aa458740 1 Processor -- start 2024-04-23T18:26:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08aa458740 1 -- start start 2024-04-23T18:26:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08aa458740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 0x55f0de73f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08aa458740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de733050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08aa458740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de73a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08aa458740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f0de753d90 con 0x55f0de742dd0 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08aa458740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f0de754460 con 0x55f0de738760 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08aa458740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f0de754b30 con 0x55f0de737db0 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08a5cda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de73a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08a5cda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de73a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53712/0 (socket says 172.21.15.19:53712) 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08a5cda640 1 -- 172.21.15.19:0/1667859852 learned_addr learned my addr 172.21.15.19:0/1667859852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.672+0000 7f08a54d9640 1 --2- 172.21.15.19:0/1667859852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 0x55f0de73f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/1667859852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de733050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f08a5cda640 1 -- 172.21.15.19:0/1667859852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 msgr2=0x55f0de73f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f08a5cda640 1 --2- 172.21.15.19:0/1667859852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 0x55f0de73f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f08a5cda640 1 -- 172.21.15.19:0/1667859852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 msgr2=0x55f0de733050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f08a5cda640 1 --2- 172.21.15.19:0/1667859852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de733050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f08a5cda640 1 -- 172.21.15.19:0/1667859852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0de745e50 con 0x55f0de742dd0 2024-04-23T18:26:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/1667859852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de733050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f08a5cda640 1 --2- 172.21.15.19:0/1667859852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de73a360 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0898033000 tx=0x7f0898005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c31f4fe36ebe2a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f088affd640 1 -- 172.21.15.19:0/1667859852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898036040 con 0x55f0de742dd0 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f088affd640 1 -- 172.21.15.19:0/1667859852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0898030eb0 con 0x55f0de742dd0 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.673+0000 7f088affd640 1 -- 172.21.15.19:0/1667859852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898031b40 con 0x55f0de742dd0 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 -- 172.21.15.19:0/1667859852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 msgr2=0x55f0de73a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 --2- 172.21.15.19:0/1667859852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de73a360 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0898033000 tx=0x7f0898005c30 comp rx=0 tx=0).stop 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 -- 172.21.15.19:0/1667859852 shutdown_connections 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 --2- 172.21.15.19:0/1667859852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de738760 0x55f0de733050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 --2- 172.21.15.19:0/1667859852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de73a360 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 --2- 172.21.15.19:0/1667859852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de737db0 0x55f0de73f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 -- 172.21.15.19:0/1667859852 >> 172.21.15.19:0/1667859852 conn(0x55f0de734070 msgr2=0x55f0de732940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 -- 172.21.15.19:0/1667859852 shutdown_connections 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 -- 172.21.15.19:0/1667859852 wait complete. 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 -- start start 2024-04-23T18:26:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 Processor -- start 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de737db0 0x55f0de73c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de738760 0x55f0de73cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de75cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f0de7558d0 con 0x55f0de742dd0 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f0de755200 con 0x55f0de737db0 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f0de755fa0 con 0x55f0de738760 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.674+0000 7f08aa458740 10 client.0 _ll_register_callbacks cb 0x7f089c09c500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a5cda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de75cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a4cd8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de738760 0x55f0de73cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a5cda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de75cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53720/0 (socket says 172.21.15.19:53720) 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a5cda640 1 -- 172.21.15.19:0/1148821916 learned_addr learned my addr 172.21.15.19:0/1148821916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a54d9640 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de737db0 0x55f0de73c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a5cda640 1 -- 172.21.15.19:0/1148821916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de738760 msgr2=0x55f0de73cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a5cda640 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de738760 0x55f0de73cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a5cda640 1 -- 172.21.15.19:0/1148821916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de737db0 msgr2=0x55f0de73c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a5cda640 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de737db0 0x55f0de73c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a5cda640 1 -- 172.21.15.19:0/1148821916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0de75d2e0 con 0x55f0de742dd0 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a4cd8640 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de738760 0x55f0de73cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08a5cda640 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de75cca0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f089802ac70 tx=0x7f0898031440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff714ba6d07a2b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f0888ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f0888ff9640 1 -- 172.21.15.19:0/1148821916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898036040 con 0x55f0de742dd0 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f0888ff9640 1 -- 172.21.15.19:0/1148821916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0898004240 con 0x55f0de742dd0 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f08aa458740 1 -- 172.21.15.19:0/1148821916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f0de6a54e0 con 0x55f0de742dd0 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.675+0000 7f0888ff9640 1 -- 172.21.15.19:0/1148821916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0898031e60 con 0x55f0de742dd0 2024-04-23T18:26:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f08aa458740 10 client.15567 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f08aa458740 1 -- 172.21.15.19:0/1148821916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0de75e550 con 0x55f0de742dd0 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f0888ff9640 1 -- 172.21.15.19:0/1148821916 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f0de6a54e0 con 0x55f0de742dd0 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f0888ff9640 1 -- 172.21.15.19:0/1148821916 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f089803c030 con 0x55f0de742dd0 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f08aa458740 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f08aa458740 10 client.15567 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f08aa458740 20 client.15567 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f08aa458740 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x55f0de7bea30 0x55f0de7bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f08aa458740 4 client.15567 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f08aa458740 1 -- 172.21.15.19:0/1148821916 --> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] -- command(tid 0: {"prefix": "status"}) -- 0x55f0de6219a0 con 0x55f0de7bea30 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.676+0000 7f08a54d9640 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x55f0de7bea30 0x55f0de7bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08a54d9640 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x55f0de7bea30 0x55f0de7bd740 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ecce8e142681d86 server_cookie=e72a703e09b7eb2d in_seq=0 out_seq=0 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f0888ff9640 10 client.15567 ms_handle_connect on v2:172.21.15.155:6836/980782961 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f0888ff9640 1 -- 172.21.15.19:0/1148821916 <== mds.3 v2:172.21.15.155:6836/980782961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55f0de6219a0 con 0x55f0de7bea30 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f0888ff9640 10 client.15567 handle_command_reply: tid=0 2024-04-23T18:26:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 client.15567 shutdown 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 -- 172.21.15.19:0/1148821916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 msgr2=0x55f0de75cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de75cca0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f089802ac70 tx=0x7f0898031440 comp rx=0 tx=0).stop 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 -- 172.21.15.19:0/1148821916 shutdown_connections 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0de742dd0 0x55f0de75cca0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0de738760 0x55f0de73cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.155:6836/980782961,v1:172.21.15.155:6837/980782961] conn(0x55f0de7bea30 0x55f0de7bd740 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 --2- 172.21.15.19:0/1148821916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0de737db0 0x55f0de73c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f0888ff9640 0 client.15567 ms_handle_reset on v2:172.21.15.155:6836/980782961 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 -- 172.21.15.19:0/1148821916 >> 172.21.15.19:0/1148821916 conn(0x55f0de734070 msgr2=0x55f0de7331d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 -- 172.21.15.19:0/1148821916 shutdown_connections 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 1 -- 172.21.15.19:0/1148821916 wait complete. 2024-04-23T18:26:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.677+0000 7f08aa458740 20 client.15567 trim_cache size 0 max 16384 2024-04-23T18:26:17.678 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:26:17.678 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:17.678 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-04-23T18:26:17.678 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24487, 2024-04-23T18:26:17.678 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:26:17.678 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:26:17.678 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:26:17.678 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 136.294987539, 2024-04-23T18:26:17.678 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-04-23T18:26:17.678 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-04-23T18:26:17.679 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:26:17.679 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 162.195990964 2024-04-23T18:26:17.679 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:26:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 -- 172.21.15.19:0/427024427 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f088c0a3e40 msgr2=0x7f088c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f088c0a3e40 0x7f088c0a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0898006010 tx=0x7f0898004a40 comp rx=0 tx=0).stop 2024-04-23T18:26:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 -- 172.21.15.19:0/427024427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08a0103230 msgr2=0x7f08a0100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08a0103230 0x7f08a0100e80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f089c00e9c0 tx=0x7f089c00ee80 comp rx=0 tx=0).stop 2024-04-23T18:26:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 -- 172.21.15.19:0/427024427 shutdown_connections 2024-04-23T18:26:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f088c0a3e40 0x7f088c0a6300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08a0103b50 0x7f08a01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a010c600 0x7f08a0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 --2- 172.21.15.19:0/427024427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08a0103230 0x7f08a0100e80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 -- 172.21.15.19:0/427024427 >> 172.21.15.19:0/427024427 conn(0x7f08a0077f80 msgr2=0x7f08a00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 -- 172.21.15.19:0/427024427 shutdown_connections 2024-04-23T18:26:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.678+0000 7f08a773d640 1 -- 172.21.15.19:0/427024427 wait complete. 2024-04-23T18:26:17.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:26:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.791+0000 7ff81707c640 1 -- 172.21.15.19:0/116946390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff810106200 msgr2=0x7ff81010ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.791+0000 7ff81707c640 1 --2- 172.21.15.19:0/116946390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff810106200 0x7ff81010ddd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff808009ca0 tx=0x7ff80802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.791+0000 7ff81707c640 1 -- 172.21.15.19:0/116946390 shutdown_connections 2024-04-23T18:26:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.791+0000 7ff81707c640 1 --2- 172.21.15.19:0/116946390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff810101ae0 0x7ff810105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.791+0000 7ff81707c640 1 --2- 172.21.15.19:0/116946390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff810106200 0x7ff81010ddd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.791+0000 7ff81707c640 1 --2- 172.21.15.19:0/116946390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff810101130 0x7ff810101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.791+0000 7ff81707c640 1 -- 172.21.15.19:0/116946390 >> 172.21.15.19:0/116946390 conn(0x7ff810077f80 msgr2=0x7ff810078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.791+0000 7ff81707c640 1 -- 172.21.15.19:0/116946390 shutdown_connections 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.791+0000 7ff81707c640 1 -- 172.21.15.19:0/116946390 wait complete. 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff81707c640 1 Processor -- start 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff81707c640 1 -- start start 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff81707c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff810101130 0x7ff810197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff81707c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff810106200 0x7ff810197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff81707c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81019bed0 0x7ff81019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff81707c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff810111170 con 0x7ff81019bed0 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff81707c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff810110ff0 con 0x7ff810101130 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff81707c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101112f0 con 0x7ff810106200 2024-04-23T18:26:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff815619640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81019bed0 0x7ff81019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff815619640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81019bed0 0x7ff81019c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53736/0 (socket says 172.21.15.19:53736) 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff815619640 1 -- 172.21.15.19:0/2398785413 learned_addr learned my addr 172.21.15.19:0/2398785413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff814e18640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff810101130 0x7ff810197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff810106200 0x7ff810197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff815619640 1 -- 172.21.15.19:0/2398785413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff810106200 msgr2=0x7ff810197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff815619640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff810106200 0x7ff810197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff815619640 1 -- 172.21.15.19:0/2398785413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff810101130 msgr2=0x7ff810197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff815619640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff810101130 0x7ff810197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.792+0000 7ff815619640 1 -- 172.21.15.19:0/2398785413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81019c9c0 con 0x7ff81019bed0 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.793+0000 7ff815619640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81019bed0 0x7ff81019c380 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff808005cc0 tx=0x7ff80802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.793+0000 7ff80dffb640 1 -- 172.21.15.19:0/2398785413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808003cf0 con 0x7ff81019bed0 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.793+0000 7ff80dffb640 1 -- 172.21.15.19:0/2398785413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8080044f0 con 0x7ff81019bed0 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.793+0000 7ff81707c640 1 -- 172.21.15.19:0/2398785413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81019cc50 con 0x7ff81019bed0 2024-04-23T18:26:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.793+0000 7ff80dffb640 1 -- 172.21.15.19:0/2398785413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80802fd10 con 0x7ff81019bed0 2024-04-23T18:26:17.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.793+0000 7ff81707c640 1 -- 172.21.15.19:0/2398785413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81006a8b0 con 0x7ff81019bed0 2024-04-23T18:26:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.795+0000 7ff80dffb640 1 -- 172.21.15.19:0/2398785413 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff808004040 con 0x7ff81019bed0 2024-04-23T18:26:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff819d91740 1 Processor -- start 2024-04-23T18:26:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff819d91740 1 -- start start 2024-04-23T18:26:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff819d91740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564156db0 0x556564157190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff819d91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564157760 0x556564161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff819d91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564164220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff819d91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5565640bcc50 con 0x556564161dd0 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff819d91740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556564059830 con 0x556564156db0 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff819d91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5565640b8c10 con 0x556564157760 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564164220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564164220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53742/0 (socket says 172.21.15.19:53742) 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff80ffff640 1 -- 172.21.15.19:0/3838441958 learned_addr learned my addr 172.21.15.19:0/3838441958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff814e18640 1 --2- 172.21.15.19:0/3838441958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564157760 0x556564161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff80dffb640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8000a3ec0 0x7ff8000a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff80ffff640 1 --2- 172.21.15.19:0/3838441958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564156db0 0x556564157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff815619640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8000a3ec0 0x7ff8000a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff80ffff640 1 -- 172.21.15.19:0/3838441958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564157760 msgr2=0x556564161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff80ffff640 1 --2- 172.21.15.19:0/3838441958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564157760 0x556564161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff80ffff640 1 -- 172.21.15.19:0/3838441958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564156db0 msgr2=0x556564157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff80ffff640 1 --2- 172.21.15.19:0/3838441958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564156db0 0x556564157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff80ffff640 1 -- 172.21.15.19:0/3838441958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556563dcf2d0 con 0x556564161dd0 2024-04-23T18:26:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.796+0000 7ff814e18640 1 --2- 172.21.15.19:0/3838441958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564157760 0x556564161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.797+0000 7ff80dffb640 1 -- 172.21.15.19:0/2398785413 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff8080f2130 con 0x7ff81019bed0 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.797+0000 7ff80ffff640 1 --2- 172.21.15.19:0/3838441958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564164220 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00efa0 tx=0x7ff7fc00e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00349f26fcce746 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.797+0000 7ff815619640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8000a3ec0 0x7ff8000a6380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff8080ef8e0 tx=0x7ff808090af0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.797+0000 7ff7faffd640 1 -- 172.21.15.19:0/3838441958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc018070 con 0x556564161dd0 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.797+0000 7ff7faffd640 1 -- 172.21.15.19:0/3838441958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc004220 con 0x556564161dd0 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.797+0000 7ff7faffd640 1 -- 172.21.15.19:0/3838441958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0038c0 con 0x556564161dd0 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.797+0000 7ff819d91740 1 -- 172.21.15.19:0/3838441958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 msgr2=0x556564164220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.797+0000 7ff819d91740 1 --2- 172.21.15.19:0/3838441958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564164220 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00efa0 tx=0x7ff7fc00e610 comp rx=0 tx=0).stop 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 -- 172.21.15.19:0/3838441958 shutdown_connections 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 --2- 172.21.15.19:0/3838441958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564164220 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 --2- 172.21.15.19:0/3838441958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564157760 0x556564161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 --2- 172.21.15.19:0/3838441958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564156db0 0x556564157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 -- 172.21.15.19:0/3838441958 >> 172.21.15.19:0/3838441958 conn(0x556564153070 msgr2=0x556564153450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 -- 172.21.15.19:0/3838441958 shutdown_connections 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 -- 172.21.15.19:0/3838441958 wait complete. 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 -- start start 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 Processor -- start 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564156db0 0x55656415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564157760 0x55656415f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564161dd0 0x55656415f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556564164d40 con 0x556564156db0 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556564057470 con 0x556564157760 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556564164ec0 con 0x556564161dd0 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.798+0000 7ff819d91740 10 client.0 _ll_register_callbacks cb 0x55656407cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564156db0 0x55656415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564156db0 0x55656415ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53756/0 (socket says 172.21.15.19:53756) 2024-04-23T18:26:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff80ffff640 1 -- 172.21.15.19:0/2817152903 learned_addr learned my addr 172.21.15.19:0/2817152903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff814e18640 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564157760 0x55656415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff80ffff640 1 -- 172.21.15.19:0/2817152903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564161dd0 msgr2=0x55656415f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564161dd0 0x55656415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff80ffff640 1 -- 172.21.15.19:0/2817152903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564157760 msgr2=0x55656415f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564157760 0x55656415f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff80ffff640 1 -- 172.21.15.19:0/2817152903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556563e996b0 con 0x556564156db0 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564156db0 0x55656415ece0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00e790 tx=0x7ff7fc004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65a8dfe4cd9b36e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff7effff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff7effff640 1 -- 172.21.15.19:0/2817152903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc018070 con 0x556564156db0 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff7effff640 1 -- 172.21.15.19:0/2817152903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff7fc005240 con 0x556564156db0 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff819d91740 1 -- 172.21.15.19:0/2817152903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7fc004220 con 0x556564156db0 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff7effff640 1 -- 172.21.15.19:0/2817152903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7fc0044d0 con 0x556564156db0 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff819d91740 10 client.15579 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.799+0000 7ff819d91740 1 -- 172.21.15.19:0/2817152903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556564171060 con 0x556564156db0 2024-04-23T18:26:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.800+0000 7ff7effff640 1 -- 172.21.15.19:0/2817152903 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc004220 con 0x556564156db0 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.800+0000 7ff7effff640 1 -- 172.21.15.19:0/2817152903 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff7fc003660 con 0x556564156db0 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.800+0000 7ff819d91740 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.800+0000 7ff819d91740 10 client.15579 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.800+0000 7ff819d91740 20 client.15579 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.800+0000 7ff819d91740 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x556564173480 0x5565641748f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.800+0000 7ff819d91740 4 client.15579 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.800+0000 7ff819d91740 1 -- 172.21.15.19:0/2817152903 --> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565640ca670 con 0x556564173480 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.800+0000 7ff814e18640 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x556564173480 0x5565641748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.801+0000 7ff814e18640 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x556564173480 0x5565641748f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a9663e2e497ce9b server_cookie=ed9fe49582345998 in_seq=0 out_seq=0 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.801+0000 7ff7effff640 10 client.15579 ms_handle_connect on v2:172.21.15.44:6836/728759484 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.801+0000 7ff7effff640 1 -- 172.21.15.19:0/2817152903 <== mds.2 v2:172.21.15.44:6836/728759484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565640ca670 con 0x556564173480 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.801+0000 7ff7effff640 10 client.15579 handle_command_reply: tid=0 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.801+0000 7ff819d91740 1 client.15579 shutdown 2024-04-23T18:26:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 1 -- 172.21.15.19:0/2817152903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564156db0 msgr2=0x55656415ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564156db0 0x55656415ece0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00e790 tx=0x7ff7fc004130 comp rx=0 tx=0).stop 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 1 -- 172.21.15.19:0/2817152903 shutdown_connections 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x556564173480 0x5565641748f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564157760 0x55656415f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564161dd0 0x55656415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff7effff640 0 client.15579 ms_handle_reset on v2:172.21.15.44:6836/728759484 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 1 --2- 172.21.15.19:0/2817152903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564156db0 0x55656415ece0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 1 -- 172.21.15.19:0/2817152903 >> 172.21.15.19:0/2817152903 conn(0x556564153070 msgr2=0x556564158ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 1 -- 172.21.15.19:0/2817152903 shutdown_connections 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 1 -- 172.21.15.19:0/2817152903 wait complete. 2024-04-23T18:26:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.802+0000 7ff819d91740 20 client.15579 trim_cache size 0 max 16384 2024-04-23T18:26:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.820+0000 7ff819d91740 1 Processor -- start 2024-04-23T18:26:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.820+0000 7ff819d91740 1 -- start start 2024-04-23T18:26:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.820+0000 7ff819d91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 0x55656415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.820+0000 7ff819d91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564157760 0x55656415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.820+0000 7ff819d91740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564161dd0 0x55656415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.820+0000 7ff819d91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556564172c30 con 0x556564157760 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.820+0000 7ff819d91740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556564173300 con 0x556564161dd0 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.820+0000 7ff819d91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5565641739d0 con 0x556564156db0 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff814e18640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564157760 0x55656415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 0x55656415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 0x55656415d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:33670/0 (socket says 172.21.15.19:33670) 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff80ffff640 1 -- 172.21.15.19:0/2980567646 learned_addr learned my addr 172.21.15.19:0/2980567646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2980567646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564161dd0 0x55656415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff80ffff640 1 -- 172.21.15.19:0/2980567646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564161dd0 msgr2=0x55656415bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2980567646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564161dd0 0x55656415bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff80ffff640 1 -- 172.21.15.19:0/2980567646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564157760 msgr2=0x55656415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2980567646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564157760 0x55656415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff80ffff640 1 -- 172.21.15.19:0/2980567646 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556564152ea0 con 0x556564156db0 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff814e18640 1 --2- 172.21.15.19:0/2980567646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564157760 0x55656415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2980567646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 0x55656415d5b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc013fd0 tx=0x7ff7fc005b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b2376795e0c50f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff7fa7fc640 1 -- 172.21.15.19:0/2980567646 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc037070 con 0x556564156db0 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff7fa7fc640 1 -- 172.21.15.19:0/2980567646 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc0054f0 con 0x556564156db0 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.821+0000 7ff7fa7fc640 1 -- 172.21.15.19:0/2980567646 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc030740 con 0x556564156db0 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 -- 172.21.15.19:0/2980567646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 msgr2=0x55656415d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 --2- 172.21.15.19:0/2980567646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 0x55656415d5b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc013fd0 tx=0x7ff7fc005b10 comp rx=0 tx=0).stop 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 -- 172.21.15.19:0/2980567646 shutdown_connections 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 --2- 172.21.15.19:0/2980567646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564161dd0 0x55656415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 --2- 172.21.15.19:0/2980567646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564157760 0x55656415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 --2- 172.21.15.19:0/2980567646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 0x55656415d5b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 -- 172.21.15.19:0/2980567646 >> 172.21.15.19:0/2980567646 conn(0x556564153070 msgr2=0x556564172470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 -- 172.21.15.19:0/2980567646 shutdown_connections 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 -- 172.21.15.19:0/2980567646 wait complete. 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 -- start start 2024-04-23T18:26:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.822+0000 7ff819d91740 1 Processor -- start 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff819d91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 0x55656417cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff819d91740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564157760 0x556564176110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff819d91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564176650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff819d91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556564174770 con 0x556564161dd0 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff819d91740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5565641740a0 con 0x556564157760 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff819d91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556564174e40 con 0x556564156db0 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff819d91740 10 client.0 _ll_register_callbacks cb 0x7ff80400c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564176650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564176650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53772/0 (socket says 172.21.15.19:53772) 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff80ffff640 1 -- 172.21.15.19:0/2605124823 learned_addr learned my addr 172.21.15.19:0/2605124823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff814e18640 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564157760 0x556564176110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 0x55656417cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff80ffff640 1 -- 172.21.15.19:0/2605124823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 msgr2=0x55656417cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 0x55656417cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff80ffff640 1 -- 172.21.15.19:0/2605124823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564157760 msgr2=0x556564176110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564157760 0x556564176110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff80ffff640 1 -- 172.21.15.19:0/2605124823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556564176c90 con 0x556564161dd0 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff80ffff640 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564176650 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc013fa0 tx=0x7ff7fc0038e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e33fef116811d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.823+0000 7ff7effff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff7effff640 1 -- 172.21.15.19:0/2605124823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc037070 con 0x556564161dd0 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff819d91740 1 -- 172.21.15.19:0/2605124823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7fc0054f0 con 0x556564161dd0 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff7effff640 1 -- 172.21.15.19:0/2605124823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff7fc030450 con 0x556564161dd0 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff7effff640 1 -- 172.21.15.19:0/2605124823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7fc030650 con 0x556564161dd0 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff819d91740 10 client.15585 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff819d91740 1 -- 172.21.15.19:0/2605124823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55656417b700 con 0x556564161dd0 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff7effff640 1 -- 172.21.15.19:0/2605124823 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc0054f0 con 0x556564161dd0 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff7effff640 1 -- 172.21.15.19:0/2605124823 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff7fc046450 con 0x556564161dd0 2024-04-23T18:26:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff819d91740 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff819d91740 10 client.15585 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:26:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff819d91740 20 client.15585 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff819d91740 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x5565641ddb60 0x5565641dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff819d91740 4 client.15585 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.824+0000 7ff819d91740 1 -- 172.21.15.19:0/2605124823 --> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] -- command(tid 0: {"prefix": "status"}) -- 0x5565640c44e0 con 0x5565641ddb60 2024-04-23T18:26:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff814e18640 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x5565641ddb60 0x5565641dc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff814e18640 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x5565641ddb60 0x5565641dc870 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46d7ca7ce6ffce66 server_cookie=587ad2506d215e20 in_seq=0 out_seq=0 2024-04-23T18:26:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff7effff640 10 client.15585 ms_handle_connect on v2:172.21.15.44:6836/728759484 2024-04-23T18:26:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff7effff640 1 -- 172.21.15.19:0/2605124823 <== mds.2 v2:172.21.15.44:6836/728759484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5565640c44e0 con 0x5565641ddb60 2024-04-23T18:26:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff7effff640 10 client.15585 handle_command_reply: tid=0 2024-04-23T18:26:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 client.15585 shutdown 2024-04-23T18:26:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 -- 172.21.15.19:0/2605124823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 msgr2=0x556564176650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564176650 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc013fa0 tx=0x7ff7fc0038e0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 -- 172.21.15.19:0/2605124823 shutdown_connections 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.44:6836/728759484,v1:172.21.15.44:6837/728759484] conn(0x5565641ddb60 0x5565641dc870 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556564157760 0x556564176110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556564161dd0 0x556564176650 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff7effff640 0 client.15585 ms_handle_reset on v2:172.21.15.44:6836/728759484 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 --2- 172.21.15.19:0/2605124823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556564156db0 0x55656417cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 -- 172.21.15.19:0/2605124823 >> 172.21.15.19:0/2605124823 conn(0x556564153070 msgr2=0x55656415d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 -- 172.21.15.19:0/2605124823 shutdown_connections 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 1 -- 172.21.15.19:0/2605124823 wait complete. 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.825+0000 7ff819d91740 20 client.15585 trim_cache size 0 max 16384 2024-04-23T18:26:17.825 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24491, 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 137.45306164300001, 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 159.394333383 2024-04-23T18:26:17.826 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 -- 172.21.15.19:0/2398785413 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8000a3ec0 msgr2=0x7ff8000a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8000a3ec0 0x7ff8000a6380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff8080ef8e0 tx=0x7ff808090af0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 -- 172.21.15.19:0/2398785413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81019bed0 msgr2=0x7ff81019c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81019bed0 0x7ff81019c380 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff808005cc0 tx=0x7ff80802f470 comp rx=0 tx=0).stop 2024-04-23T18:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 -- 172.21.15.19:0/2398785413 shutdown_connections 2024-04-23T18:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8000a3ec0 0x7ff8000a6380 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81019bed0 0x7ff81019c380 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff810106200 0x7ff810197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 --2- 172.21.15.19:0/2398785413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff810101130 0x7ff810197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 -- 172.21.15.19:0/2398785413 >> 172.21.15.19:0/2398785413 conn(0x7ff810077f80 msgr2=0x7ff8100fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 -- 172.21.15.19:0/2398785413 shutdown_connections 2024-04-23T18:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.826+0000 7ff81707c640 1 -- 172.21.15.19:0/2398785413 wait complete. 2024-04-23T18:26:17.831 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:26:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.940+0000 7f4671e79640 1 -- 172.21.15.19:0/4213203647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f466c102df0 msgr2=0x7f466c0690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.940+0000 7f4671e79640 1 --2- 172.21.15.19:0/4213203647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f466c102df0 0x7f466c0690e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f465c009900 tx=0x7f465c02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 -- 172.21.15.19:0/4213203647 shutdown_connections 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 --2- 172.21.15.19:0/4213203647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f466c10e750 0x7f466c110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 --2- 172.21.15.19:0/4213203647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f466c069620 0x7f466c069a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 --2- 172.21.15.19:0/4213203647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f466c102df0 0x7f466c0690e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 -- 172.21.15.19:0/4213203647 >> 172.21.15.19:0/4213203647 conn(0x7f466c077cb0 msgr2=0x7f466c0780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 -- 172.21.15.19:0/4213203647 shutdown_connections 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 -- 172.21.15.19:0/4213203647 wait complete. 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 Processor -- start 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 -- start start 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f466c069620 0x7f466c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.941+0000 7f4671e79640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f466c102df0 0x7f466c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f4671e79640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f466c10e750 0x7f466c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f4671e79640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f466c114360 con 0x7f466c102df0 2024-04-23T18:26:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f4671e79640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1141e0 con 0x7f466c10e750 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f4671e79640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1144e0 con 0x7f466c069620 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f466c102df0 0x7f466c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f466c102df0 0x7f466c1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53800/0 (socket says 172.21.15.19:53800) 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466affd640 1 -- 172.21.15.19:0/1307110626 learned_addr learned my addr 172.21.15.19:0/1307110626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466b7fe640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f466c069620 0x7f466c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466bfff640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f466c10e750 0x7f466c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466affd640 1 -- 172.21.15.19:0/1307110626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f466c069620 msgr2=0x7f466c1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466affd640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f466c069620 0x7f466c1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466affd640 1 -- 172.21.15.19:0/1307110626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f466c10e750 msgr2=0x7f466c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466affd640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f466c10e750 0x7f466c19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466affd640 1 -- 172.21.15.19:0/1307110626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c19b000 con 0x7f466c102df0 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466b7fe640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f466c069620 0x7f466c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:17.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f466affd640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f466c102df0 0x7f466c1a0bc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f466000d770 tx=0x7f466000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f4668ff9640 1 -- 172.21.15.19:0/1307110626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660017070 con 0x7f466c102df0 2024-04-23T18:26:17.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f4671e79640 1 -- 172.21.15.19:0/1307110626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c19b2f0 con 0x7f466c102df0 2024-04-23T18:26:17.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f4668ff9640 1 -- 172.21.15.19:0/1307110626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46600054d0 con 0x7f466c102df0 2024-04-23T18:26:17.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f4668ff9640 1 -- 172.21.15.19:0/1307110626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660004c30 con 0x7f466c102df0 2024-04-23T18:26:17.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.942+0000 7f4671e79640 1 -- 172.21.15.19:0/1307110626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c1a7600 con 0x7f466c102df0 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f4668ff9640 1 -- 172.21.15.19:0/1307110626 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4660004e10 con 0x7f466c102df0 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f4674b8e740 1 Processor -- start 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f4674b8e740 1 -- start start 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa37db0 0x555fcfa38190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa38760 0x555fcfa42800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 0x555fcfa45220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f4674b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555fcf99dc50 con 0x555fcfa38760 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f4674b8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555fcf93a830 con 0x555fcfa37db0 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f4674b8e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555fcf999c10 con 0x555fcfa42dd0 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f466b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa38760 0x555fcfa42800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f4668ff9640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46540a3e40 0x7f46540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f466b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa38760 0x555fcfa42800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53808/0 (socket says 172.21.15.19:53808) 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f466b7fe640 1 -- 172.21.15.19:0/1989546926 learned_addr learned my addr 172.21.15.19:0/1989546926 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f466affd640 1 --2- 172.21.15.19:0/1989546926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 0x555fcfa45220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.945+0000 7f466bfff640 1 --2- 172.21.15.19:0/1989546926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa37db0 0x555fcfa38190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f466bfff640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46540a3e40 0x7f46540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f466b7fe640 1 -- 172.21.15.19:0/1989546926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 msgr2=0x555fcfa45220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f466b7fe640 1 --2- 172.21.15.19:0/1989546926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 0x555fcfa45220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f466b7fe640 1 -- 172.21.15.19:0/1989546926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa37db0 msgr2=0x555fcfa38190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f466b7fe640 1 --2- 172.21.15.19:0/1989546926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa37db0 0x555fcfa38190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f466b7fe640 1 -- 172.21.15.19:0/1989546926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fcf6b02d0 con 0x555fcfa38760 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f4668ff9640 1 -- 172.21.15.19:0/1307110626 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f46600d0600 con 0x7f466c102df0 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f466affd640 1 --2- 172.21.15.19:0/1989546926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 0x555fcfa45220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f466b7fe640 1 --2- 172.21.15.19:0/1989546926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa38760 0x555fcfa42800 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f465c000c00 tx=0x7f465c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27839cfc456100ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f466bfff640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46540a3e40 0x7f46540a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4664005c40 tx=0x7f4664004a40 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f4651ffb640 1 -- 172.21.15.19:0/1989546926 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c03f070 con 0x555fcfa38760 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f4651ffb640 1 -- 172.21.15.19:0/1989546926 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c0316a0 con 0x555fcfa38760 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.946+0000 7f4651ffb640 1 -- 172.21.15.19:0/1989546926 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c035070 con 0x555fcfa38760 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 -- 172.21.15.19:0/1989546926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa38760 msgr2=0x555fcfa42800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 --2- 172.21.15.19:0/1989546926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa38760 0x555fcfa42800 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f465c000c00 tx=0x7f465c032ce0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 -- 172.21.15.19:0/1989546926 shutdown_connections 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 --2- 172.21.15.19:0/1989546926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 0x555fcfa45220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 --2- 172.21.15.19:0/1989546926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa38760 0x555fcfa42800 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 --2- 172.21.15.19:0/1989546926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa37db0 0x555fcfa38190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 -- 172.21.15.19:0/1989546926 >> 172.21.15.19:0/1989546926 conn(0x555fcfa34070 msgr2=0x555fcfa34450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 -- 172.21.15.19:0/1989546926 shutdown_connections 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 -- 172.21.15.19:0/1989546926 wait complete. 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 -- start start 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.947+0000 7f4674b8e740 1 Processor -- start 2024-04-23T18:26:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa38760 0x555fcfa40140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 0x555fcfa40680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f4674b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555fcfa45ce0 con 0x555fcfa37db0 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f4674b8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555fcf938470 con 0x555fcfa38760 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f4674b8e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555fcfa45e60 con 0x555fcfa42dd0 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f4674b8e740 10 client.0 _ll_register_callbacks cb 0x555fcf95df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53824/0 (socket says 172.21.15.19:53824) 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466b7fe640 1 -- 172.21.15.19:0/1140651587 learned_addr learned my addr 172.21.15.19:0/1140651587 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466affd640 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa38760 0x555fcfa40140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466bfff640 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 0x555fcfa40680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466b7fe640 1 -- 172.21.15.19:0/1140651587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 msgr2=0x555fcfa40680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466b7fe640 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 0x555fcfa40680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466b7fe640 1 -- 172.21.15.19:0/1140651587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa38760 msgr2=0x555fcfa40140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466b7fe640 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa38760 0x555fcfa40140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466b7fe640 1 -- 172.21.15.19:0/1140651587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fcf77a6b0 con 0x555fcfa37db0 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f466b7fe640 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3fc00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f465c02a450 tx=0x7f465c031590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3eca09cb8aea1c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f464affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f464affd640 1 -- 172.21.15.19:0/1140651587 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c03f070 con 0x555fcfa37db0 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f464affd640 1 -- 172.21.15.19:0/1140651587 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f465c004050 con 0x555fcfa37db0 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.948+0000 7f4674b8e740 1 -- 172.21.15.19:0/1140651587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465c0316a0 con 0x555fcfa37db0 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f464affd640 1 -- 172.21.15.19:0/1140651587 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f465c053050 con 0x555fcfa37db0 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f4674b8e740 10 client.15597 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f4674b8e740 1 -- 172.21.15.19:0/1140651587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555fcfa51ee0 con 0x555fcfa37db0 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f464affd640 1 -- 172.21.15.19:0/1140651587 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c0316a0 con 0x555fcfa37db0 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f464affd640 1 -- 172.21.15.19:0/1140651587 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f465c0046c0 con 0x555fcfa37db0 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f4674b8e740 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f4674b8e740 10 client.15597 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f4674b8e740 20 client.15597 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f4674b8e740 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x555fcfa541c0 0x555fcfa55630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f4674b8e740 4 client.15597 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f4674b8e740 1 -- 172.21.15.19:0/1140651587 --> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fcf9ab670 con 0x555fcfa541c0 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.949+0000 7f466affd640 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x555fcfa541c0 0x555fcfa55630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f464affd640 10 client.15597 ms_handle_connect on v2:172.21.15.155:6838/3934638415 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f466affd640 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x555fcfa541c0 0x555fcfa55630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5df1f07ff6026bf6 server_cookie=d3db422e704256be in_seq=0 out_seq=0 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f464affd640 1 -- 172.21.15.19:0/1140651587 <== mds.1 v2:172.21.15.155:6838/3934638415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fcf9ab670 con 0x555fcfa541c0 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f464affd640 10 client.15597 handle_command_reply: tid=0 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f4674b8e740 1 client.15597 shutdown 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f4674b8e740 1 -- 172.21.15.19:0/1140651587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 msgr2=0x555fcfa3fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f4674b8e740 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3fc00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f465c02a450 tx=0x7f465c031590 comp rx=0 tx=0).stop 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f4674b8e740 1 -- 172.21.15.19:0/1140651587 shutdown_connections 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f4674b8e740 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x555fcfa541c0 0x555fcfa55630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f4674b8e740 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa38760 0x555fcfa40140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f4674b8e740 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa42dd0 0x555fcfa40680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f4674b8e740 1 --2- 172.21.15.19:0/1140651587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3fc00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f464affd640 0 client.15597 ms_handle_reset on v2:172.21.15.155:6838/3934638415 2024-04-23T18:26:17.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.950+0000 7f4674b8e740 1 -- 172.21.15.19:0/1140651587 >> 172.21.15.19:0/1140651587 conn(0x555fcfa34070 msgr2=0x555fcfa39ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.951+0000 7f4674b8e740 1 -- 172.21.15.19:0/1140651587 shutdown_connections 2024-04-23T18:26:17.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.951+0000 7f4674b8e740 1 -- 172.21.15.19:0/1140651587 wait complete. 2024-04-23T18:26:17.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.951+0000 7f4674b8e740 20 client.15597 trim_cache size 0 max 16384 2024-04-23T18:26:17.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f4674b8e740 1 Processor -- start 2024-04-23T18:26:17.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f4674b8e740 1 -- start start 2024-04-23T18:26:17.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 0x555fcfa4e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa42dd0 0x555fcfa5d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f4674b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555fcfa53970 con 0x555fcfa37db0 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f4674b8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555fcfa54040 con 0x555fcfa42dd0 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f4674b8e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555fcfa54710 con 0x555fcfa38760 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53830/0 (socket says 172.21.15.19:53830) 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466b7fe640 1 -- 172.21.15.19:0/3523150961 learned_addr learned my addr 172.21.15.19:0/3523150961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466affd640 1 --2- 172.21.15.19:0/3523150961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 0x555fcfa4e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466bfff640 1 --2- 172.21.15.19:0/3523150961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa42dd0 0x555fcfa5d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466b7fe640 1 -- 172.21.15.19:0/3523150961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 msgr2=0x555fcfa4e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466b7fe640 1 --2- 172.21.15.19:0/3523150961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 0x555fcfa4e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466b7fe640 1 -- 172.21.15.19:0/3523150961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa42dd0 msgr2=0x555fcfa5d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466b7fe640 1 --2- 172.21.15.19:0/3523150961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa42dd0 0x555fcfa5d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466b7fe640 1 -- 172.21.15.19:0/3523150961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fcfa37360 con 0x555fcfa37db0 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.969+0000 7f466bfff640 1 --2- 172.21.15.19:0/3523150961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa42dd0 0x555fcfa5d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f466b7fe640 1 --2- 172.21.15.19:0/3523150961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3a4b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f465c004a90 tx=0x7f465c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2996be2e28e53141 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f46517fa640 1 -- 172.21.15.19:0/3523150961 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c03f040 con 0x555fcfa37db0 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f46517fa640 1 -- 172.21.15.19:0/3523150961 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c048020 con 0x555fcfa37db0 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f46517fa640 1 -- 172.21.15.19:0/3523150961 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c035040 con 0x555fcfa37db0 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f4674b8e740 1 -- 172.21.15.19:0/3523150961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 msgr2=0x555fcfa3a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f4674b8e740 1 --2- 172.21.15.19:0/3523150961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3a4b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f465c004a90 tx=0x7f465c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f4674b8e740 1 -- 172.21.15.19:0/3523150961 shutdown_connections 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f4674b8e740 1 --2- 172.21.15.19:0/3523150961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa42dd0 0x555fcfa5d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f4674b8e740 1 --2- 172.21.15.19:0/3523150961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 0x555fcfa4e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f4674b8e740 1 --2- 172.21.15.19:0/3523150961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa37db0 0x555fcfa3a4b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f4674b8e740 1 -- 172.21.15.19:0/3523150961 >> 172.21.15.19:0/3523150961 conn(0x555fcfa34070 msgr2=0x555fcfa4e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f4674b8e740 1 -- 172.21.15.19:0/3523150961 shutdown_connections 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.970+0000 7f4674b8e740 1 -- 172.21.15.19:0/3523150961 wait complete. 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f4674b8e740 1 -- start start 2024-04-23T18:26:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f4674b8e740 1 Processor -- start 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa37db0 0x555fcfa57480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 0x555fcfa579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f4674b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa42dd0 0x555fcfa57f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f4674b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555fcfa554b0 con 0x555fcfa42dd0 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f4674b8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555fcfa54de0 con 0x555fcfa37db0 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f4674b8e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555fcfa55b80 con 0x555fcfa38760 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f4674b8e740 10 client.0 _ll_register_callbacks cb 0x7f465c032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f466bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa42dd0 0x555fcfa57f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f466bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa42dd0 0x555fcfa57f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53838/0 (socket says 172.21.15.19:53838) 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f466bfff640 1 -- 172.21.15.19:0/3686128624 learned_addr learned my addr 172.21.15.19:0/3686128624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f466affd640 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 0x555fcfa579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.971+0000 7f466b7fe640 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa37db0 0x555fcfa57480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f466bfff640 1 -- 172.21.15.19:0/3686128624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 msgr2=0x555fcfa579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f466bfff640 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 0x555fcfa579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f466bfff640 1 -- 172.21.15.19:0/3686128624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa37db0 msgr2=0x555fcfa57480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f466bfff640 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa37db0 0x555fcfa57480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f466bfff640 1 -- 172.21.15.19:0/3686128624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fcfa5beb0 con 0x555fcfa42dd0 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f466affd640 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 0x555fcfa579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f466bfff640 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa42dd0 0x555fcfa57f00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f466400f920 tx=0x7f4664004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fc50c22c455794 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f464affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f464affd640 1 -- 172.21.15.19:0/3686128624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664023030 con 0x555fcfa42dd0 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f464affd640 1 -- 172.21.15.19:0/3686128624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f4664011990 con 0x555fcfa42dd0 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f4674b8e740 1 -- 172.21.15.19:0/3686128624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465c048020 con 0x555fcfa42dd0 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f464affd640 1 -- 172.21.15.19:0/3686128624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4664011b90 con 0x555fcfa42dd0 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f4674b8e740 10 client.15603 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f4674b8e740 1 -- 172.21.15.19:0/3686128624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555fcfaba5e0 con 0x555fcfa42dd0 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f464affd640 1 -- 172.21.15.19:0/3686128624 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c048020 con 0x555fcfa42dd0 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.972+0000 7f464affd640 1 -- 172.21.15.19:0/3686128624 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4664028410 con 0x555fcfa42dd0 2024-04-23T18:26:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f4674b8e740 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f4674b8e740 10 client.15603 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f4674b8e740 20 client.15603 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f4674b8e740 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x555fcfabf1f0 0x555fcfabdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f4674b8e740 4 client.15603 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f4674b8e740 1 -- 172.21.15.19:0/3686128624 --> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] -- command(tid 0: {"prefix": "status"}) -- 0x555fcf9a54e0 con 0x555fcfabf1f0 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f466b7fe640 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x555fcfabf1f0 0x555fcfabdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f464affd640 10 client.15603 ms_handle_connect on v2:172.21.15.155:6838/3934638415 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f466b7fe640 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x555fcfabf1f0 0x555fcfabdf00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16870c589e5b33d7 server_cookie=283aaadca664e5cd in_seq=0 out_seq=0 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f464affd640 1 -- 172.21.15.19:0/3686128624 <== mds.1 v2:172.21.15.155:6838/3934638415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555fcf9a54e0 con 0x555fcfabf1f0 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f464affd640 10 client.15603 handle_command_reply: tid=0 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.973+0000 7f4674b8e740 1 client.15603 shutdown 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 1 -- 172.21.15.19:0/3686128624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa42dd0 msgr2=0x555fcfa57f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa42dd0 0x555fcfa57f00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f466400f920 tx=0x7f4664004ab0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 1 -- 172.21.15.19:0/3686128624 shutdown_connections 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.155:6838/3934638415,v1:172.21.15.155:6839/3934638415] conn(0x555fcfabf1f0 0x555fcfabdf00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fcfa42dd0 0x555fcfa57f00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fcfa38760 0x555fcfa579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 1 --2- 172.21.15.19:0/3686128624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555fcfa37db0 0x555fcfa57480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f464affd640 0 client.15603 ms_handle_reset on v2:172.21.15.155:6838/3934638415 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 1 -- 172.21.15.19:0/3686128624 >> 172.21.15.19:0/3686128624 conn(0x555fcfa34070 msgr2=0x555fcfa4e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 1 -- 172.21.15.19:0/3686128624 shutdown_connections 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 1 -- 172.21.15.19:0/3686128624 wait complete. 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4674b8e740 20 client.15603 trim_cache size 0 max 16384 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:26:17.974 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24508, 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 138.60881820899999, 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 156.60995772300001 2024-04-23T18:26:17.975 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:26:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 -- 172.21.15.19:0/1307110626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46540a3e40 msgr2=0x7f46540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46540a3e40 0x7f46540a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4664005c40 tx=0x7f4664004a40 comp rx=0 tx=0).stop 2024-04-23T18:26:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 -- 172.21.15.19:0/1307110626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f466c102df0 msgr2=0x7f466c1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f466c102df0 0x7f466c1a0bc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f466000d770 tx=0x7f466000dc30 comp rx=0 tx=0).stop 2024-04-23T18:26:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 -- 172.21.15.19:0/1307110626 shutdown_connections 2024-04-23T18:26:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46540a3e40 0x7f46540a6300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f466c10e750 0x7f466c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f466c102df0 0x7f466c1a0bc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 --2- 172.21.15.19:0/1307110626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f466c069620 0x7f466c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 -- 172.21.15.19:0/1307110626 >> 172.21.15.19:0/1307110626 conn(0x7f466c077cb0 msgr2=0x7f466c10f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 -- 172.21.15.19:0/1307110626 shutdown_connections 2024-04-23T18:26:17.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:17.974+0000 7f4671e79640 1 -- 172.21.15.19:0/1307110626 wait complete. 2024-04-23T18:26:17.979 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.088+0000 7fbfc7336640 1 -- 172.21.15.19:0/3353402909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc0103230 msgr2=0x7fbfc0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.088+0000 7fbfc7336640 1 --2- 172.21.15.19:0/3353402909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc0103230 0x7fbfc0103610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009860 tx=0x7fbfbc02a3d0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 -- 172.21.15.19:0/3353402909 shutdown_connections 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 --2- 172.21.15.19:0/3353402909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc0103b50 0x7fbfc010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 --2- 172.21.15.19:0/3353402909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010c600 0x7fbfc010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 --2- 172.21.15.19:0/3353402909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc0103230 0x7fbfc0103610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 -- 172.21.15.19:0/3353402909 >> 172.21.15.19:0/3353402909 conn(0x7fbfc0077f80 msgr2=0x7fbfc0078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 -- 172.21.15.19:0/3353402909 shutdown_connections 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 -- 172.21.15.19:0/3353402909 wait complete. 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 Processor -- start 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 -- start start 2024-04-23T18:26:18.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc0103230 0x7fbfc0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc0103b50 0x7fbfc01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.089+0000 7fbfc7336640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010c600 0x7fbfc0101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc7336640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0112280 con 0x7fbfc010c600 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc7336640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0112100 con 0x7fbfc0103230 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc7336640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0112400 con 0x7fbfc0103b50 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc58d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010c600 0x7fbfc0101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc58d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010c600 0x7fbfc0101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53846/0 (socket says 172.21.15.19:53846) 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc58d3640 1 -- 172.21.15.19:0/462717088 learned_addr learned my addr 172.21.15.19:0/462717088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc0103230 0x7fbfc0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc48d1640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc0103b50 0x7fbfc01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc58d3640 1 -- 172.21.15.19:0/462717088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc0103b50 msgr2=0x7fbfc01013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc0103b50 0x7fbfc01013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc58d3640 1 -- 172.21.15.19:0/462717088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc0103230 msgr2=0x7fbfc0100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc0103230 0x7fbfc0100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc58d3640 1 -- 172.21.15.19:0/462717088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc01a6d20 con 0x7fbfc010c600 2024-04-23T18:26:18.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010c600 0x7fbfc0101900 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009fd0 tx=0x7fbfb800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfb67fc640 1 -- 172.21.15.19:0/462717088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8004b20 con 0x7fbfc010c600 2024-04-23T18:26:18.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfb67fc640 1 -- 172.21.15.19:0/462717088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb8003e20 con 0x7fbfc010c600 2024-04-23T18:26:18.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc7336640 1 -- 172.21.15.19:0/462717088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc01a7010 con 0x7fbfc010c600 2024-04-23T18:26:18.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfb67fc640 1 -- 172.21.15.19:0/462717088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8004820 con 0x7fbfc010c600 2024-04-23T18:26:18.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.090+0000 7fbfc7336640 1 -- 172.21.15.19:0/462717088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc01a74e0 con 0x7fbfc010c600 2024-04-23T18:26:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.092+0000 7fbfb67fc640 1 -- 172.21.15.19:0/462717088 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fbfb801d020 con 0x7fbfc010c600 2024-04-23T18:26:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfca053740 1 Processor -- start 2024-04-23T18:26:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfca053740 1 -- start start 2024-04-23T18:26:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfca053740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7bdb0 0x5638f8f7c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfca053740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7c760 0x5638f8f86800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfb67fc640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfac0a3e40 0x7fbfac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfca053740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8f89220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfca053740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638f8ee1c50 con 0x5638f8f7c760 2024-04-23T18:26:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfca053740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638f8e7e830 con 0x5638f8f7bdb0 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfca053740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638f8eddc10 con 0x5638f8f86dd0 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfc50d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7c760 0x5638f8f86800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfc48d1640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfac0a3e40 0x7fbfac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfc50d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7c760 0x5638f8f86800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53850/0 (socket says 172.21.15.19:53850) 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/2879786647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8f89220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfc50d2640 1 -- 172.21.15.19:0/2879786647 learned_addr learned my addr 172.21.15.19:0/2879786647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfb67fc640 1 -- 172.21.15.19:0/462717088 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbfb80d0990 con 0x7fbfc010c600 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.093+0000 7fbfc48d1640 1 --2- 172.21.15.19:0/2879786647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7bdb0 0x5638f8f7c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfc58d3640 1 -- 172.21.15.19:0/2879786647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7bdb0 msgr2=0x5638f8f7c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/2879786647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7bdb0 0x5638f8f7c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfc58d3640 1 -- 172.21.15.19:0/2879786647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7c760 msgr2=0x5638f8f86800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/2879786647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7c760 0x5638f8f86800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfc58d3640 1 -- 172.21.15.19:0/2879786647 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638f8bf42d0 con 0x5638f8f86dd0 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/2879786647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7c760 0x5638f8f86800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfc48d1640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfac0a3e40 0x7fbfac0a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0102910 tx=0x7fbfb0009b50 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/2879786647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8f89220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80cdf90 tx=0x7fbfb80196e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf7090fc10be421d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfab7fe640 1 -- 172.21.15.19:0/2879786647 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb809c070 con 0x5638f8f86dd0 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfab7fe640 1 -- 172.21.15.19:0/2879786647 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb80870e0 con 0x5638f8f86dd0 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.094+0000 7fbfab7fe640 1 -- 172.21.15.19:0/2879786647 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb809d840 con 0x5638f8f86dd0 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 -- 172.21.15.19:0/2879786647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 msgr2=0x5638f8f89220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 --2- 172.21.15.19:0/2879786647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8f89220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80cdf90 tx=0x7fbfb80196e0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 -- 172.21.15.19:0/2879786647 shutdown_connections 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 --2- 172.21.15.19:0/2879786647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8f89220 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 --2- 172.21.15.19:0/2879786647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7c760 0x5638f8f86800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 --2- 172.21.15.19:0/2879786647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7bdb0 0x5638f8f7c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 -- 172.21.15.19:0/2879786647 >> 172.21.15.19:0/2879786647 conn(0x5638f8f78070 msgr2=0x5638f8f78450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 -- 172.21.15.19:0/2879786647 shutdown_connections 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 -- 172.21.15.19:0/2879786647 wait complete. 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 -- start start 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.095+0000 7fbfca053740 1 Processor -- start 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfca053740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7bdb0 0x5638f8f83d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfca053740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f7c760 0x5638f8f84240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfca053740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f86dd0 0x5638f8f84780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfca053740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638f8f89d40 con 0x5638f8f86dd0 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfca053740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638f8e7c470 con 0x5638f8f7bdb0 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfca053740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638f8f89ec0 con 0x5638f8f7c760 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfca053740 10 client.0 _ll_register_callbacks cb 0x5638f8ea1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc58d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f86dd0 0x5638f8f84780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc58d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f86dd0 0x5638f8f84780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53860/0 (socket says 172.21.15.19:53860) 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc58d3640 1 -- 172.21.15.19:0/273964818 learned_addr learned my addr 172.21.15.19:0/273964818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc48d1640 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f7c760 0x5638f8f84240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7bdb0 0x5638f8f83d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc58d3640 1 -- 172.21.15.19:0/273964818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f7c760 msgr2=0x5638f8f84240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f7c760 0x5638f8f84240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc58d3640 1 -- 172.21.15.19:0/273964818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7bdb0 msgr2=0x5638f8f83d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7bdb0 0x5638f8f83d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc58d3640 1 -- 172.21.15.19:0/273964818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638f8cbe6b0 con 0x5638f8f86dd0 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f86dd0 0x5638f8f84780 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800cd00 tx=0x7fbfb809d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1addde564ce8e5b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/273964818 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb809ce10 con 0x5638f8f86dd0 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/273964818 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbfb80a5100 con 0x5638f8f86dd0 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.096+0000 7fbfca053740 1 -- 172.21.15.19:0/273964818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb80870e0 con 0x5638f8f86dd0 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/273964818 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfb80a19f0 con 0x5638f8f86dd0 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfca053740 10 client.15615 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfca053740 1 -- 172.21.15.19:0/273964818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638f8f89d40 con 0x5638f8f86dd0 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/273964818 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb80870e0 con 0x5638f8f86dd0 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/273964818 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbfb800b040 con 0x5638f8f86dd0 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfca053740 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfca053740 10 client.15615 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfca053740 20 client.15615 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfca053740 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5638f8f98200 0x5638f8f99670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfca053740 4 client.15615 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfca053740 1 -- 172.21.15.19:0/273964818 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638f8eef670 con 0x5638f8f98200 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.097+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5638f8f98200 0x5638f8f99670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.098+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5638f8f98200 0x5638f8f99670 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a6f60a5e2b36d1 server_cookie=6b563da5287af70a in_seq=0 out_seq=0 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.098+0000 7fbfa8ff9640 10 client.15615 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.098+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/273964818 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638f8eef670 con 0x5638f8f98200 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.098+0000 7fbfa8ff9640 10 client.15615 handle_command_reply: tid=0 2024-04-23T18:26:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 client.15615 shutdown 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 -- 172.21.15.19:0/273964818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f86dd0 msgr2=0x5638f8f84780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f86dd0 0x5638f8f84780 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800cd00 tx=0x7fbfb809d840 comp rx=0 tx=0).stop 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 -- 172.21.15.19:0/273964818 shutdown_connections 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5638f8f98200 0x5638f8f99670 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfa8ff9640 0 client.15615 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f86dd0 0x5638f8f84780 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f7c760 0x5638f8f84240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 --2- 172.21.15.19:0/273964818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7bdb0 0x5638f8f83d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 -- 172.21.15.19:0/273964818 >> 172.21.15.19:0/273964818 conn(0x5638f8f78070 msgr2=0x5638f8f7dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 -- 172.21.15.19:0/273964818 shutdown_connections 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 1 -- 172.21.15.19:0/273964818 wait complete. 2024-04-23T18:26:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.099+0000 7fbfca053740 20 client.15615 trim_cache size 0 max 16384 2024-04-23T18:26:18.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfca053740 1 Processor -- start 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfca053740 1 -- start start 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfca053740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f7e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfca053740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7c760 0x5638f8f80f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfca053740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8f814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfca053740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638f8f979b0 con 0x5638f8f7bdb0 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfca053740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638f8f98080 con 0x5638f8f7c760 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfca053740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638f8f98750 con 0x5638f8f86dd0 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfc50d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f7e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfc50d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f7e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53870/0 (socket says 172.21.15.19:53870) 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.117+0000 7fbfc50d2640 1 -- 172.21.15.19:0/88823593 learned_addr learned my addr 172.21.15.19:0/88823593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/88823593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8f814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfc48d1640 1 --2- 172.21.15.19:0/88823593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7c760 0x5638f8f80f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfc50d2640 1 -- 172.21.15.19:0/88823593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 msgr2=0x5638f8f814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/88823593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8f814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfc50d2640 1 -- 172.21.15.19:0/88823593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7c760 msgr2=0x5638f8f80f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/88823593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7c760 0x5638f8f80f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfc50d2640 1 -- 172.21.15.19:0/88823593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638f8f7f8f0 con 0x5638f8f7bdb0 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/88823593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f7e3b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009830 tx=0x7fbfbc0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598a4de827d9e127 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfaaffd640 1 -- 172.21.15.19:0/88823593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc048630 con 0x5638f8f7bdb0 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfaaffd640 1 -- 172.21.15.19:0/88823593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfbc030e30 con 0x5638f8f7bdb0 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.118+0000 7fbfaaffd640 1 -- 172.21.15.19:0/88823593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0046c0 con 0x5638f8f7bdb0 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 -- 172.21.15.19:0/88823593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 msgr2=0x5638f8f7e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 --2- 172.21.15.19:0/88823593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f7e3b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009830 tx=0x7fbfbc0058b0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 -- 172.21.15.19:0/88823593 shutdown_connections 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 --2- 172.21.15.19:0/88823593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7c760 0x5638f8f80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 --2- 172.21.15.19:0/88823593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8f814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 --2- 172.21.15.19:0/88823593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f7e3b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 -- 172.21.15.19:0/88823593 >> 172.21.15.19:0/88823593 conn(0x5638f8f78070 msgr2=0x5638f8f971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 -- 172.21.15.19:0/88823593 shutdown_connections 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 -- 172.21.15.19:0/88823593 wait complete. 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 -- start start 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 Processor -- start 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f80380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7c760 0x5638f8f808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8fa08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638f8f994f0 con 0x5638f8f7bdb0 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638f8f98e20 con 0x5638f8f7c760 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638f8f99bc0 con 0x5638f8f86dd0 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.119+0000 7fbfca053740 10 client.0 _ll_register_callbacks cb 0x7fbfb0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc50d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f80380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc50d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f80380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53882/0 (socket says 172.21.15.19:53882) 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc50d2640 1 -- 172.21.15.19:0/683003406 learned_addr learned my addr 172.21.15.19:0/683003406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc58d3640 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8fa08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc48d1640 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7c760 0x5638f8f808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc50d2640 1 -- 172.21.15.19:0/683003406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 msgr2=0x5638f8fa08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8fa08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc50d2640 1 -- 172.21.15.19:0/683003406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7c760 msgr2=0x5638f8f808c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7c760 0x5638f8f808c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc50d2640 1 -- 172.21.15.19:0/683003406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638f8fa0e00 con 0x5638f8f7bdb0 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfc50d2640 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f80380 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc005940 tx=0x7fbfbc0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8431f06fc52fee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/683003406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc056070 con 0x5638f8f7bdb0 2024-04-23T18:26:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/683003406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbfbc048630 con 0x5638f8f7bdb0 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfca053740 1 -- 172.21.15.19:0/683003406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfbc030e30 con 0x5638f8f7bdb0 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/683003406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfbc062050 con 0x5638f8f7bdb0 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfca053740 10 client.15621 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.120+0000 7fbfca053740 1 -- 172.21.15.19:0/683003406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638f8fa2060 con 0x5638f8f7bdb0 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.121+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/683003406 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfbc030e30 con 0x5638f8f7bdb0 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.121+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/683003406 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbfbc004a90 con 0x5638f8f7bdb0 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.121+0000 7fbfca053740 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.121+0000 7fbfca053740 10 client.15621 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.121+0000 7fbfca053740 20 client.15621 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.121+0000 7fbfca053740 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5638f8f98200 0x5638f9001180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.121+0000 7fbfca053740 4 client.15621 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.121+0000 7fbfca053740 1 -- 172.21.15.19:0/683003406 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "status"}) -- 0x5638f8ee94e0 con 0x5638f8f98200 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.121+0000 7fbfc48d1640 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5638f8f98200 0x5638f9001180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfa8ff9640 10 client.15621 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfc48d1640 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5638f8f98200 0x5638f9001180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4051d0c87752c0 server_cookie=a15e0172bfecb54f in_seq=0 out_seq=0 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfa8ff9640 1 -- 172.21.15.19:0/683003406 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5638f8ee94e0 con 0x5638f8f98200 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfa8ff9640 10 client.15621 handle_command_reply: tid=0 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 client.15621 shutdown 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 -- 172.21.15.19:0/683003406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 msgr2=0x5638f8f80380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f80380 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc005940 tx=0x7fbfbc0305d0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 -- 172.21.15.19:0/683003406 shutdown_connections 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5638f8f98200 0x5638f9001180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f8f7c760 0x5638f8f808c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfa8ff9640 0 client.15621 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638f8f86dd0 0x5638f8fa08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 --2- 172.21.15.19:0/683003406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638f8f7bdb0 0x5638f8f80380 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 -- 172.21.15.19:0/683003406 >> 172.21.15.19:0/683003406 conn(0x5638f8f78070 msgr2=0x5638f8f825d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 -- 172.21.15.19:0/683003406 shutdown_connections 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 1 -- 172.21.15.19:0/683003406 wait complete. 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.122+0000 7fbfca053740 20 client.15621 trim_cache size 0 max 16384 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "46706838-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24521, 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-23T18:26:18.123 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 141.77561725800001, 2024-04-23T18:26:18.124 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-04-23T18:26:18.124 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-04-23T18:26:18.124 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:26:18.124 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 147.842942271 2024-04-23T18:26:18.124 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:26:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 -- 172.21.15.19:0/462717088 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfac0a3e40 msgr2=0x7fbfac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfac0a3e40 0x7fbfac0a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0102910 tx=0x7fbfb0009b50 comp rx=0 tx=0).stop 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 -- 172.21.15.19:0/462717088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010c600 msgr2=0x7fbfc0101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010c600 0x7fbfc0101900 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009fd0 tx=0x7fbfb800c800 comp rx=0 tx=0).stop 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 -- 172.21.15.19:0/462717088 shutdown_connections 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfac0a3e40 0x7fbfac0a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010c600 0x7fbfc0101900 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc0103b50 0x7fbfc01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 --2- 172.21.15.19:0/462717088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc0103230 0x7fbfc0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 -- 172.21.15.19:0/462717088 >> 172.21.15.19:0/462717088 conn(0x7fbfc0077f80 msgr2=0x7fbfc00ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 -- 172.21.15.19:0/462717088 shutdown_connections 2024-04-23T18:26:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:18.123+0000 7fbfc7336640 1 -- 172.21.15.19:0/462717088 wait complete. 2024-04-23T18:26:18.127 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-23T18:26:18.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:18 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2101208235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:18.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:18 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2101208235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:18.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:18 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2101208235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:19.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:19 smithi019 ceph-mon[29215]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-04-23T18:26:19.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:19 smithi044 ceph-mon[32458]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-04-23T18:26:19.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:19 smithi155 ceph-mon[33468]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-04-23T18:26:21.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:21 smithi019 ceph-mon[29215]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-23T18:26:21.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:21 smithi044 ceph-mon[32458]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-23T18:26:21.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:21 smithi155 ceph-mon[33468]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-23T18:26:23.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:23 smithi019 ceph-mon[29215]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-23T18:26:23.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:23 smithi044 ceph-mon[32458]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-23T18:26:23.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:23 smithi155 ceph-mon[33468]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-23T18:26:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.049+0000 7f8e60f19640 1 -- 172.21.15.19:0/4252982071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c103b00 msgr2=0x7f8e5c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.049+0000 7f8e60f19640 1 --2- 172.21.15.19:0/4252982071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c103b00 0x7f8e5c10bfe0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009900 tx=0x7f8e4c02a480 comp rx=0 tx=0).stop 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.049+0000 7f8e60f19640 1 -- 172.21.15.19:0/4252982071 shutdown_connections 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.049+0000 7f8e60f19640 1 --2- 172.21.15.19:0/4252982071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5c10c5b0 0x7f8e5c10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.049+0000 7f8e60f19640 1 --2- 172.21.15.19:0/4252982071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c103b00 0x7f8e5c10bfe0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.049+0000 7f8e60f19640 1 --2- 172.21.15.19:0/4252982071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c1031e0 0x7f8e5c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.049+0000 7f8e60f19640 1 -- 172.21.15.19:0/4252982071 >> 172.21.15.19:0/4252982071 conn(0x7f8e5c077fa0 msgr2=0x7f8e5c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.049+0000 7f8e60f19640 1 -- 172.21.15.19:0/4252982071 shutdown_connections 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.049+0000 7f8e60f19640 1 -- 172.21.15.19:0/4252982071 wait complete. 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e60f19640 1 Processor -- start 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e60f19640 1 -- start start 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e60f19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c1031e0 0x7f8e5c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e60f19640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5c103b00 0x7f8e5c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e60f19640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c10c5b0 0x7f8e5c1018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e60f19640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1121c0 con 0x7f8e5c1031e0 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e60f19640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c112040 con 0x7f8e5c103b00 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e60f19640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c112340 con 0x7f8e5c10c5b0 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e5a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c1031e0 0x7f8e5c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e5a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c1031e0 0x7f8e5c100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33190/0 (socket says 172.21.15.19:33190) 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e5a59c640 1 -- 172.21.15.19:0/1580539674 learned_addr learned my addr 172.21.15.19:0/1580539674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.050+0000 7f8e59d9b640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5c103b00 0x7f8e5c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c10c5b0 0x7f8e5c1018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e5a59c640 1 -- 172.21.15.19:0/1580539674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c10c5b0 msgr2=0x7f8e5c1018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c10c5b0 0x7f8e5c1018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e5a59c640 1 -- 172.21.15.19:0/1580539674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5c103b00 msgr2=0x7f8e5c101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5c103b00 0x7f8e5c101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e5a59c640 1 -- 172.21.15.19:0/1580539674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5c1a6b80 con 0x7f8e5c1031e0 2024-04-23T18:26:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e59d9b640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5c103b00 0x7f8e5c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c1031e0 0x7f8e5c100e30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000d950 tx=0x7f8e5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e4b7fe640 1 -- 172.21.15.19:0/1580539674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50017070 con 0x7f8e5c1031e0 2024-04-23T18:26:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e4b7fe640 1 -- 172.21.15.19:0/1580539674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e50009c30 con 0x7f8e5c1031e0 2024-04-23T18:26:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e4b7fe640 1 -- 172.21.15.19:0/1580539674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5000f620 con 0x7f8e5c1031e0 2024-04-23T18:26:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e60f19640 1 -- 172.21.15.19:0/1580539674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5c1a6e70 con 0x7f8e5c1031e0 2024-04-23T18:26:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.051+0000 7f8e60f19640 1 -- 172.21.15.19:0/1580539674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5c1a72e0 con 0x7f8e5c1031e0 2024-04-23T18:26:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e4b7fe640 1 -- 172.21.15.19:0/1580539674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8e50010460 con 0x7f8e5c1031e0 2024-04-23T18:26:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e63c2f740 1 Processor -- start 2024-04-23T18:26:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e63c2f740 1 -- start start 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 0x5599d9e71a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 0x5599d9e7ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599d9dd6970 con 0x5599d9e71f40 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5599d9d734a0 con 0x5599d9e71620 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5599d9dd2930 con 0x5599d9e7c800 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e5a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e4b7fe640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8e440a3ee0 0x7f8e440a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e5a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33192/0 (socket says 172.21.15.19:33192) 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e5a59c640 1 -- 172.21.15.19:0/583782493 learned_addr learned my addr 172.21.15.19:0/583782493 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.054+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/583782493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 0x5599d9e71a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e59d9b640 1 --2- 172.21.15.19:0/583782493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 0x5599d9e7ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8e440a3ee0 0x7f8e440a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e4b7fe640 1 -- 172.21.15.19:0/1580539674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8e500d0750 con 0x7f8e5c1031e0 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e5ad9d640 1 -- 172.21.15.19:0/583782493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 msgr2=0x5599d9e7ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/583782493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 0x5599d9e7ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e5ad9d640 1 -- 172.21.15.19:0/583782493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 msgr2=0x5599d9e7c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/583782493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e5ad9d640 1 -- 172.21.15.19:0/583782493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d9ae9210 con 0x5599d9e71620 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/583782493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8e440a3ee0 0x7f8e440a63a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e54010c40 tx=0x7f8e54017000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/583782493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 0x5599d9e71a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e54005fd0 tx=0x7f8e54010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f18dea77720d6f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.055+0000 7f8e48ff9640 1 -- 172.21.15.19:0/583782493 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54023070 con 0x5599d9e71620 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e48ff9640 1 -- 172.21.15.19:0/583782493 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e54009b90 con 0x5599d9e71620 2024-04-23T18:26:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e48ff9640 1 -- 172.21.15.19:0/583782493 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54005280 con 0x5599d9e71620 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e63c2f740 1 -- 172.21.15.19:0/583782493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 msgr2=0x5599d9e71a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/583782493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 0x5599d9e71a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e54005fd0 tx=0x7f8e54010740 comp rx=0 tx=0).stop 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e63c2f740 1 -- 172.21.15.19:0/583782493 shutdown_connections 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/583782493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 0x5599d9e7ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/583782493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/583782493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 0x5599d9e71a00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e63c2f740 1 -- 172.21.15.19:0/583782493 >> 172.21.15.19:0/583782493 conn(0x5599d9e6db30 msgr2=0x5599d9e6df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e63c2f740 1 -- 172.21.15.19:0/583782493 shutdown_connections 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.056+0000 7f8e63c2f740 1 -- 172.21.15.19:0/583782493 wait complete. 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e63c2f740 1 -- start start 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e63c2f740 1 Processor -- start 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e71620 0x5599d9e7c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71f40 0x5599d9e79440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e7c800 0x5599d9e799a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599d9e7f3d0 con 0x5599d9e7c800 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5599d9d710b0 con 0x5599d9e71f40 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5599d9e7f550 con 0x5599d9e71620 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e63c2f740 10 client.0 _ll_register_callbacks cb 0x5599d9d5ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e5ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e7c800 0x5599d9e799a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e5ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e7c800 0x5599d9e799a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33196/0 (socket says 172.21.15.19:33196) 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e5ad9d640 1 -- 172.21.15.19:0/2060466340 learned_addr learned my addr 172.21.15.19:0/2060466340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e59d9b640 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71f40 0x5599d9e79440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e71620 0x5599d9e7c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e5ad9d640 1 -- 172.21.15.19:0/2060466340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e71620 msgr2=0x5599d9e7c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e71620 0x5599d9e7c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e5ad9d640 1 -- 172.21.15.19:0/2060466340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71f40 msgr2=0x5599d9e79440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71f40 0x5599d9e79440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.057+0000 7f8e5ad9d640 1 -- 172.21.15.19:0/2060466340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d9bb5620 con 0x5599d9e7c800 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.058+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e7c800 0x5599d9e799a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8e54004c90 tx=0x7f8e54004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db32691a287f773 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.058+0000 7f8e41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.058+0000 7f8e41ffb640 1 -- 172.21.15.19:0/2060466340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54023050 con 0x5599d9e7c800 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.058+0000 7f8e63c2f740 1 -- 172.21.15.19:0/2060466340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5599d9de4390 con 0x5599d9e7c800 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.058+0000 7f8e41ffb640 1 -- 172.21.15.19:0/2060466340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8e54005250 con 0x5599d9e7c800 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.058+0000 7f8e41ffb640 1 -- 172.21.15.19:0/2060466340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e540126c0 con 0x5599d9e7c800 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.058+0000 7f8e63c2f740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.058+0000 7f8e63c2f740 1 -- 172.21.15.19:0/2060466340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599d9e8b300 con 0x5599d9e7c800 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.058+0000 7f8e41ffb640 1 -- 172.21.15.19:0/2060466340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5599d9de4390 con 0x5599d9e7c800 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.059+0000 7f8e41ffb640 1 -- 172.21.15.19:0/2060466340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8e5401f480 con 0x5599d9e7c800 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.059+0000 7f8e63c2f740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.059+0000 7f8e63c2f740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.059+0000 7f8e63c2f740 20 client.15633 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.059+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5599d9e8c770 0x5599d9e8eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.059+0000 7f8e63c2f740 4 client.15633 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.059+0000 7f8e63c2f740 1 -- 172.21.15.19:0/2060466340 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599d9dde200 con 0x5599d9e8c770 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.059+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5599d9e8c770 0x5599d9e8eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.059+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5599d9e8c770 0x5599d9e8eb60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57a58f8f0e8e080 server_cookie=e74e6ad2fd589964 in_seq=0 out_seq=0 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.059+0000 7f8e41ffb640 10 client.15633 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e41ffb640 1 -- 172.21.15.19:0/2060466340 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599d9dde200 con 0x5599d9e8c770 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e41ffb640 10 client.15633 handle_command_reply: tid=0 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 client.15633 shutdown 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 -- 172.21.15.19:0/2060466340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e7c800 msgr2=0x5599d9e799a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e7c800 0x5599d9e799a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8e54004c90 tx=0x7f8e54004cc0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 -- 172.21.15.19:0/2060466340 shutdown_connections 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5599d9e8c770 0x5599d9e8eb60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71f40 0x5599d9e79440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e7c800 0x5599d9e799a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e41ffb640 0 client.15633 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/2060466340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e71620 0x5599d9e7c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 -- 172.21.15.19:0/2060466340 >> 172.21.15.19:0/2060466340 conn(0x5599d9e6db30 msgr2=0x5599d9e73860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 -- 172.21.15.19:0/2060466340 shutdown_connections 2024-04-23T18:26:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 1 -- 172.21.15.19:0/2060466340 wait complete. 2024-04-23T18:26:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.060+0000 7f8e63c2f740 20 client.15633 trim_cache size 0 max 16384 2024-04-23T18:26:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.076+0000 7f8e63c2f740 1 Processor -- start 2024-04-23T18:26:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.076+0000 7f8e63c2f740 1 -- start start 2024-04-23T18:26:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71620 0x5599d9e88290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e71f40 0x5599d9e6cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e7c800 0x5599d9e7bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1afd40 con 0x5599d9e71620 2024-04-23T18:26:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1b1880 con 0x5599d9e7c800 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1b0410 con 0x5599d9e71f40 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71620 0x5599d9e88290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71620 0x5599d9e88290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33206/0 (socket says 172.21.15.19:33206) 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5a59c640 1 -- 172.21.15.19:0/946836094 learned_addr learned my addr 172.21.15.19:0/946836094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/946836094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e7c800 0x5599d9e7bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e59d9b640 1 --2- 172.21.15.19:0/946836094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e71f40 0x5599d9e6cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5a59c640 1 -- 172.21.15.19:0/946836094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e71f40 msgr2=0x5599d9e6cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/946836094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e71f40 0x5599d9e6cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5a59c640 1 -- 172.21.15.19:0/946836094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e7c800 msgr2=0x5599d9e7bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/946836094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e7c800 0x5599d9e7bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5a59c640 1 -- 172.21.15.19:0/946836094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d9e729b0 con 0x5599d9e71620 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/946836094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e7c800 0x5599d9e7bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/946836094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71620 0x5599d9e88290 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8e500cde60 tx=0x7f8e50098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95944d3ea067b48a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e43fff640 1 -- 172.21.15.19:0/946836094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50020040 con 0x5599d9e71620 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e43fff640 1 -- 172.21.15.19:0/946836094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e500a0880 con 0x5599d9e71620 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.077+0000 7f8e43fff640 1 -- 172.21.15.19:0/946836094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e500ce030 con 0x5599d9e71620 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.078+0000 7f8e63c2f740 1 -- 172.21.15.19:0/946836094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71620 msgr2=0x5599d9e88290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.078+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/946836094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71620 0x5599d9e88290 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8e500cde60 tx=0x7f8e50098040 comp rx=0 tx=0).stop 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.078+0000 7f8e63c2f740 1 -- 172.21.15.19:0/946836094 shutdown_connections 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.078+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/946836094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e7c800 0x5599d9e7bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.078+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/946836094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e71f40 0x5599d9e6cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.078+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/946836094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71620 0x5599d9e88290 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.078+0000 7f8e63c2f740 1 -- 172.21.15.19:0/946836094 >> 172.21.15.19:0/946836094 conn(0x5599d9e6db30 msgr2=0x5599d9e79560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.078+0000 7f8e63c2f740 1 -- 172.21.15.19:0/946836094 shutdown_connections 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.078+0000 7f8e63c2f740 1 -- 172.21.15.19:0/946836094 wait complete. 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.078+0000 7f8e63c2f740 1 -- start start 2024-04-23T18:26:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e63c2f740 1 Processor -- start 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 0x5599d9e849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e63c2f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 0x5599d9e80260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1b11b0 con 0x5599d9e71f40 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1b0ae0 con 0x5599d9e71620 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e63c2f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1add20 con 0x5599d9e7c800 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e63c2f740 10 client.0 _ll_register_callbacks cb 0x7f8e540129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33222/0 (socket says 172.21.15.19:33222) 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e59d9b640 1 -- 172.21.15.19:0/1301247969 learned_addr learned my addr 172.21.15.19:0/1301247969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 0x5599d9e849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 0x5599d9e80260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e59d9b640 1 -- 172.21.15.19:0/1301247969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 msgr2=0x5599d9e80260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e59d9b640 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 0x5599d9e80260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e59d9b640 1 -- 172.21.15.19:0/1301247969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 msgr2=0x5599d9e849b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e59d9b640 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 0x5599d9e849b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e59d9b640 1 -- 172.21.15.19:0/1301247969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d9e808a0 con 0x5599d9e71f40 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 0x5599d9e849b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.079+0000 7f8e5ad9d640 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 0x5599d9e80260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e59d9b640 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7fd20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c02a9a0 tx=0x7f8e4c031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf14875d3d36cab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e41ffb640 1 -- 172.21.15.19:0/1301247969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c035040 con 0x5599d9e71f40 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e41ffb640 1 -- 172.21.15.19:0/1301247969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8e4c004450 con 0x5599d9e71f40 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e63c2f740 1 -- 172.21.15.19:0/1301247969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e500a0880 con 0x5599d9e71f40 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e41ffb640 1 -- 172.21.15.19:0/1301247969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e4c005030 con 0x5599d9e71f40 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e63c2f740 10 client.15639 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e63c2f740 1 -- 172.21.15.19:0/1301247969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599d9ef27a0 con 0x5599d9e71f40 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e41ffb640 1 -- 172.21.15.19:0/1301247969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e500a0880 con 0x5599d9e71f40 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e41ffb640 1 -- 172.21.15.19:0/1301247969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8e4c0035f0 con 0x5599d9e71f40 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e63c2f740 10 client.15639 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e63c2f740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e63c2f740 20 client.15639 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5599d9ef2f80 0x5599d9ef53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e63c2f740 4 client.15639 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.080+0000 7f8e63c2f740 1 -- 172.21.15.19:0/1301247969 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599d9d5a5f0 con 0x5599d9ef2f80 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.081+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5599d9ef2f80 0x5599d9ef53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.081+0000 7f8e5a59c640 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5599d9ef2f80 0x5599d9ef53a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d1b2e0c52a47f5 server_cookie=b4df05d16874a9c7 in_seq=0 out_seq=0 2024-04-23T18:26:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.081+0000 7f8e41ffb640 10 client.15639 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.093+0000 7f8e41ffb640 1 -- 172.21.15.19:0/1301247969 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599d9d5a5f0 con 0x5599d9ef2f80 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.093+0000 7f8e41ffb640 10 client.15639 handle_command_reply: tid=0 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.093+0000 7f8e63c2f740 1 client.15639 shutdown 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.093+0000 7f8e63c2f740 1 -- 172.21.15.19:0/1301247969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 msgr2=0x5599d9e7fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.093+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7fd20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c02a9a0 tx=0x7f8e4c031810 comp rx=0 tx=0).stop 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e63c2f740 1 -- 172.21.15.19:0/1301247969 shutdown_connections 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5599d9ef2f80 0x5599d9ef53a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d9e7c800 0x5599d9e80260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599d9e71f40 0x5599d9e7fd20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e63c2f740 1 --2- 172.21.15.19:0/1301247969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599d9e71620 0x5599d9e849b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e63c2f740 1 -- 172.21.15.19:0/1301247969 >> 172.21.15.19:0/1301247969 conn(0x5599d9e6db30 msgr2=0x5599d9e87280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e41ffb640 0 client.15639 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e63c2f740 1 -- 172.21.15.19:0/1301247969 shutdown_connections 2024-04-23T18:26:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e63c2f740 1 -- 172.21.15.19:0/1301247969 wait complete. 2024-04-23T18:26:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e63c2f740 20 client.15639 trim_cache size 0 max 16384 2024-04-23T18:26:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:26:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:26:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:26:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:26:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e60f19640 1 -- 172.21.15.19:0/1580539674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8e440a3ee0 msgr2=0x7f8e440a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e60f19640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8e440a3ee0 0x7f8e440a63a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e54010c40 tx=0x7f8e54017000 comp rx=0 tx=0).stop 2024-04-23T18:26:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e60f19640 1 -- 172.21.15.19:0/1580539674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c1031e0 msgr2=0x7f8e5c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.094+0000 7f8e60f19640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c1031e0 0x7f8e5c100e30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000d950 tx=0x7f8e5000de10 comp rx=0 tx=0).stop 2024-04-23T18:26:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.095+0000 7f8e60f19640 1 -- 172.21.15.19:0/1580539674 shutdown_connections 2024-04-23T18:26:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.095+0000 7f8e60f19640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8e440a3ee0 0x7f8e440a63a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.095+0000 7f8e60f19640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e5c103b00 0x7f8e5c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.095+0000 7f8e60f19640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c10c5b0 0x7f8e5c1018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.095+0000 7f8e60f19640 1 --2- 172.21.15.19:0/1580539674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c1031e0 0x7f8e5c100e30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.095+0000 7f8e60f19640 1 -- 172.21.15.19:0/1580539674 >> 172.21.15.19:0/1580539674 conn(0x7f8e5c077fa0 msgr2=0x7f8e5c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.095+0000 7f8e60f19640 1 -- 172.21.15.19:0/1580539674 shutdown_connections 2024-04-23T18:26:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:24.095+0000 7f8e60f19640 1 -- 172.21.15.19:0/1580539674 wait complete. 2024-04-23T18:26:25.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:25 smithi019 ceph-mon[29215]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-23T18:26:25.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:25 smithi044 ceph-mon[32458]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-23T18:26:25.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:25 smithi155 ceph-mon[33468]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-23T18:26:27.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:27 smithi019 ceph-mon[29215]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-04-23T18:26:27.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:27 smithi044 ceph-mon[32458]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-04-23T18:26:27.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:27 smithi155 ceph-mon[33468]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-04-23T18:26:29.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:29 smithi019 ceph-mon[29215]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-23T18:26:29.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:29 smithi044 ceph-mon[32458]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-23T18:26:29.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:29 smithi155 ceph-mon[33468]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-23T18:26:31.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:31 smithi019 ceph-mon[29215]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-04-23T18:26:31.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:31 smithi044 ceph-mon[32458]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-04-23T18:26:31.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:31 smithi155 ceph-mon[33468]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-04-23T18:26:33.128 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:26:33.128 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.5 2024-04-23T18:26:33.129 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.247+0000 7f27d69f7640 1 -- 172.21.15.19:0/1288968607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0106170 msgr2=0x7f27d010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.247+0000 7f27d69f7640 1 --2- 172.21.15.19:0/1288968607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0106170 0x7f27d010dd40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009ca0 tx=0x7f27c802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.247+0000 7f27d69f7640 1 -- 172.21.15.19:0/1288968607 shutdown_connections 2024-04-23T18:26:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.247+0000 7f27d69f7640 1 --2- 172.21.15.19:0/1288968607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d0101a50 0x7f27d0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.247+0000 7f27d69f7640 1 --2- 172.21.15.19:0/1288968607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0106170 0x7f27d010dd40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.247+0000 7f27d69f7640 1 --2- 172.21.15.19:0/1288968607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d01010a0 0x7f27d0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.247+0000 7f27d69f7640 1 -- 172.21.15.19:0/1288968607 >> 172.21.15.19:0/1288968607 conn(0x7f27d0077f50 msgr2=0x7f27d0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.247+0000 7f27d69f7640 1 -- 172.21.15.19:0/1288968607 shutdown_connections 2024-04-23T18:26:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.247+0000 7f27d69f7640 1 -- 172.21.15.19:0/1288968607 wait complete. 2024-04-23T18:26:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.248+0000 7f27d69f7640 1 Processor -- start 2024-04-23T18:26:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.248+0000 7f27d69f7640 1 -- start start 2024-04-23T18:26:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.248+0000 7f27d69f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d01010a0 0x7f27d0197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.248+0000 7f27d69f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0106170 0x7f27d0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.248+0000 7f27d69f7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d019be20 0x7f27d019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.248+0000 7f27d69f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01110c0 con 0x7f27d0106170 2024-04-23T18:26:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.248+0000 7f27d69f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0110f40 con 0x7f27d01010a0 2024-04-23T18:26:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.248+0000 7f27d69f7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0111240 con 0x7f27d019be20 2024-04-23T18:26:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0106170 0x7f27d0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0106170 0x7f27d0197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34212/0 (socket says 172.21.15.19:34212) 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cf7fe640 1 -- 172.21.15.19:0/28011031 learned_addr learned my addr 172.21.15.19:0/28011031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cffff640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d01010a0 0x7f27d0197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27d4f94640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d019be20 0x7f27d019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cf7fe640 1 -- 172.21.15.19:0/28011031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d019be20 msgr2=0x7f27d019c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cf7fe640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d019be20 0x7f27d019c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cf7fe640 1 -- 172.21.15.19:0/28011031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d01010a0 msgr2=0x7f27d0197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cf7fe640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d01010a0 0x7f27d0197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cf7fe640 1 -- 172.21.15.19:0/28011031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d019c910 con 0x7f27d0106170 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27d4f94640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d019be20 0x7f27d019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cf7fe640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0106170 0x7f27d0197aa0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f27c400c970 tx=0x7f27c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27cd7fa640 1 -- 172.21.15.19:0/28011031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4014070 con 0x7f27d0106170 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.249+0000 7f27d69f7640 1 -- 172.21.15.19:0/28011031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d019cc00 con 0x7f27d0106170 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.250+0000 7f27cd7fa640 1 -- 172.21.15.19:0/28011031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c4004440 con 0x7f27d0106170 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.250+0000 7f27d69f7640 1 -- 172.21.15.19:0/28011031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d01a4440 con 0x7f27d0106170 2024-04-23T18:26:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.250+0000 7f27cd7fa640 1 -- 172.21.15.19:0/28011031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4003e50 con 0x7f27d0106170 2024-04-23T18:26:33.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.251+0000 7f27d69f7640 1 -- 172.21.15.19:0/28011031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d0068d70 con 0x7f27d0106170 2024-04-23T18:26:33.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.252+0000 7f27cd7fa640 1 -- 172.21.15.19:0/28011031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f27c4017070 con 0x7f27d0106170 2024-04-23T18:26:33.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.253+0000 7f27cd7fa640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27b80a3fe0 0x7f27b80a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.253+0000 7f27cffff640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27b80a3fe0 0x7f27b80a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.253+0000 7f27cd7fa640 1 -- 172.21.15.19:0/28011031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f27c40d0c90 con 0x7f27d0106170 2024-04-23T18:26:33.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.253+0000 7f27cffff640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27b80a3fe0 0x7f27b80a64a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27c000bbe0 tx=0x7f27c000b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.262+0000 7f27cd7fa640 1 -- 172.21.15.19:0/28011031 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27c401e020 con 0x7f27d0106170 2024-04-23T18:26:33.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:33 smithi019 ceph-mon[29215]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-23T18:26:33.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.517+0000 7f27d69f7640 1 -- 172.21.15.19:0/28011031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27d01988b0 con 0x7f27d0106170 2024-04-23T18:26:33.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.518+0000 7f27cd7fa640 1 -- 172.21.15.19:0/28011031 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f27d01988b0 con 0x7f27d0106170 2024-04-23T18:26:33.511 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:33.511 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:26:33.512 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:26:33.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.520+0000 7f27d69f7640 1 -- 172.21.15.19:0/28011031 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27b80a3fe0 msgr2=0x7f27b80a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.520+0000 7f27d69f7640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27b80a3fe0 0x7f27b80a64a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27c000bbe0 tx=0x7f27c000b390 comp rx=0 tx=0).stop 2024-04-23T18:26:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.520+0000 7f27d69f7640 1 -- 172.21.15.19:0/28011031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0106170 msgr2=0x7f27d0197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.520+0000 7f27d69f7640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0106170 0x7f27d0197aa0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f27c400c970 tx=0x7f27c400ce30 comp rx=0 tx=0).stop 2024-04-23T18:26:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.520+0000 7f27d69f7640 1 -- 172.21.15.19:0/28011031 shutdown_connections 2024-04-23T18:26:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.520+0000 7f27d69f7640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27b80a3fe0 0x7f27b80a64a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.520+0000 7f27d69f7640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d019be20 0x7f27d019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.520+0000 7f27d69f7640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0106170 0x7f27d0197aa0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.520+0000 7f27d69f7640 1 --2- 172.21.15.19:0/28011031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27d01010a0 0x7f27d0197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.520+0000 7f27d69f7640 1 -- 172.21.15.19:0/28011031 >> 172.21.15.19:0/28011031 conn(0x7f27d0077f50 msgr2=0x7f27d00fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.521+0000 7f27d69f7640 1 -- 172.21.15.19:0/28011031 shutdown_connections 2024-04-23T18:26:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.521+0000 7f27d69f7640 1 -- 172.21.15.19:0/28011031 wait complete. 2024-04-23T18:26:33.525 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 78.5 --await-for 42.0' 2024-04-23T18:26:33.525 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 78.5 --await-for 42.0 2024-04-23T18:26:33.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:33 smithi044 ceph-mon[32458]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-23T18:26:33.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:33 smithi155 ceph-mon[33468]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-23T18:26:33.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.651+0000 7fbe5af9b640 1 -- 172.21.15.19:0/220329366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe541010a0 msgr2=0x7fbe54101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.651+0000 7fbe5af9b640 1 --2- 172.21.15.19:0/220329366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe541010a0 0x7fbe54101480 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009900 tx=0x7fbe4402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:33.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.651+0000 7fbe5af9b640 1 -- 172.21.15.19:0/220329366 shutdown_connections 2024-04-23T18:26:33.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.651+0000 7fbe5af9b640 1 --2- 172.21.15.19:0/220329366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe54106170 0x7fbe5410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.651+0000 7fbe5af9b640 1 --2- 172.21.15.19:0/220329366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe54101a50 0x7fbe54105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.651+0000 7fbe5af9b640 1 --2- 172.21.15.19:0/220329366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe541010a0 0x7fbe54101480 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.651+0000 7fbe5af9b640 1 -- 172.21.15.19:0/220329366 >> 172.21.15.19:0/220329366 conn(0x7fbe54077f50 msgr2=0x7fbe54078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.651+0000 7fbe5af9b640 1 -- 172.21.15.19:0/220329366 shutdown_connections 2024-04-23T18:26:33.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.652+0000 7fbe5af9b640 1 -- 172.21.15.19:0/220329366 wait complete. 2024-04-23T18:26:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.652+0000 7fbe5af9b640 1 Processor -- start 2024-04-23T18:26:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.652+0000 7fbe5af9b640 1 -- start start 2024-04-23T18:26:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.652+0000 7fbe5af9b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe541010a0 0x7fbe54110660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.652+0000 7fbe5af9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe54106170 0x7fbe54110ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.652+0000 7fbe5af9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe541110e0 0x7fbe541a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.652+0000 7fbe5af9b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54069c70 con 0x7fbe541110e0 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.652+0000 7fbe5af9b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54069af0 con 0x7fbe54106170 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe5af9b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54069df0 con 0x7fbe541010a0 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe59538640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe541110e0 0x7fbe541a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe59538640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe541110e0 0x7fbe541a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34222/0 (socket says 172.21.15.19:34222) 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe59538640 1 -- 172.21.15.19:0/962053994 learned_addr learned my addr 172.21.15.19:0/962053994 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe53fff640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe54106170 0x7fbe54110ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe58d37640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe541010a0 0x7fbe54110660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe59538640 1 -- 172.21.15.19:0/962053994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe541010a0 msgr2=0x7fbe54110660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe59538640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe541010a0 0x7fbe54110660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe59538640 1 -- 172.21.15.19:0/962053994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe54106170 msgr2=0x7fbe54110ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe59538640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe54106170 0x7fbe54110ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe59538640 1 -- 172.21.15.19:0/962053994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe541a7010 con 0x7fbe541110e0 2024-04-23T18:26:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe59538640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe541110e0 0x7fbe541a6910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009fd0 tx=0x7fbe4c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe51ffb640 1 -- 172.21.15.19:0/962053994 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c017070 con 0x7fbe541110e0 2024-04-23T18:26:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe51ffb640 1 -- 172.21.15.19:0/962053994 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4c003e20 con 0x7fbe541110e0 2024-04-23T18:26:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe5af9b640 1 -- 172.21.15.19:0/962053994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe541a7300 con 0x7fbe541110e0 2024-04-23T18:26:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.653+0000 7fbe51ffb640 1 -- 172.21.15.19:0/962053994 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c010460 con 0x7fbe541110e0 2024-04-23T18:26:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.654+0000 7fbe5af9b640 1 -- 172.21.15.19:0/962053994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe541a7800 con 0x7fbe541110e0 2024-04-23T18:26:33.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.656+0000 7fbe51ffb640 1 -- 172.21.15.19:0/962053994 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fbe4c003970 con 0x7fbe541110e0 2024-04-23T18:26:33.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.656+0000 7fbe5dd81740 1 Processor -- start 2024-04-23T18:26:33.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe5dd81740 1 -- start start 2024-04-23T18:26:33.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe51ffb640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe3c0a3e70 0x7fbe3c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 0x55fb24a59480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe53fff640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe3c0a3e70 0x7fbe3c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a63b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a66480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe5dd81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb249be370 con 0x55fb24a599c0 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe5dd81740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fb2495b2e0 con 0x55fb24a64090 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe5dd81740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fb249ba330 con 0x55fb24a590a0 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a63b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a63b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34230/0 (socket says 172.21.15.19:34230) 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe51ffb640 1 -- 172.21.15.19:0/962053994 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbe4c0d0680 con 0x7fbe541110e0 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe53fff640 1 -- 172.21.15.19:0/2831060565 learned_addr learned my addr 172.21.15.19:0/2831060565 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe58d37640 1 --2- 172.21.15.19:0/2831060565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 0x55fb24a59480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.657+0000 7fbe59538640 1 --2- 172.21.15.19:0/2831060565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a66480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe53fff640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe3c0a3e70 0x7fbe3c0a6330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe48004650 tx=0x7fbe4800beb0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe53fff640 1 -- 172.21.15.19:0/2831060565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 msgr2=0x55fb24a59480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:33.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe53fff640 1 --2- 172.21.15.19:0/2831060565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 0x55fb24a59480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe53fff640 1 -- 172.21.15.19:0/2831060565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 msgr2=0x55fb24a66480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe53fff640 1 --2- 172.21.15.19:0/2831060565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a66480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe53fff640 1 -- 172.21.15.19:0/2831060565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb2479cd80 con 0x55fb24a599c0 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe58d37640 1 --2- 172.21.15.19:0/2831060565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 0x55fb24a59480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe53fff640 1 --2- 172.21.15.19:0/2831060565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a63b50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbe48003fd0 tx=0x7fbe4801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1538dcb83b9e900 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe3affd640 1 -- 172.21.15.19:0/2831060565 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48005220 con 0x55fb24a599c0 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe3affd640 1 -- 172.21.15.19:0/2831060565 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe48023030 con 0x55fb24a599c0 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.658+0000 7fbe3affd640 1 -- 172.21.15.19:0/2831060565 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4801f770 con 0x55fb24a599c0 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 -- 172.21.15.19:0/2831060565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 msgr2=0x55fb24a63b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/2831060565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a63b50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbe48003fd0 tx=0x7fbe4801e000 comp rx=0 tx=0).stop 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 -- 172.21.15.19:0/2831060565 shutdown_connections 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/2831060565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a66480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/2831060565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a63b50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/2831060565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 0x55fb24a59480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 -- 172.21.15.19:0/2831060565 >> 172.21.15.19:0/2831060565 conn(0x55fb24a553c0 msgr2=0x55fb24a557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 -- 172.21.15.19:0/2831060565 shutdown_connections 2024-04-23T18:26:33.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 -- 172.21.15.19:0/2831060565 wait complete. 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 -- start start 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.659+0000 7fbe5dd81740 1 Processor -- start 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a590a0 0x55fb24a61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 0x55fb24a61560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a61aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe58d37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a590a0 0x55fb24a61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe5dd81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb24a66e70 con 0x55fb24a590a0 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe58d37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a590a0 0x55fb24a61020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34236/0 (socket says 172.21.15.19:34236) 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe5dd81740 1 -- 172.21.15.19:0/1797112846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fb24958fb0 con 0x55fb24a64090 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe58d37640 1 -- 172.21.15.19:0/1797112846 learned_addr learned my addr 172.21.15.19:0/1797112846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe5dd81740 1 -- 172.21.15.19:0/1797112846 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fb24a66ff0 con 0x55fb24a599c0 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe5dd81740 10 client.0 _ll_register_callbacks cb 0x55fb249589a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe59538640 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a61aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe53fff640 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 0x55fb24a61560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe58d37640 1 -- 172.21.15.19:0/1797112846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 msgr2=0x55fb24a61560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe58d37640 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 0x55fb24a61560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe58d37640 1 -- 172.21.15.19:0/1797112846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 msgr2=0x55fb24a61aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe58d37640 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a61aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe58d37640 1 -- 172.21.15.19:0/1797112846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb247fd510 con 0x55fb24a590a0 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe53fff640 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 0x55fb24a61560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe58d37640 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a590a0 0x55fb24a61020 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009230 tx=0x7fbe440045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e222be9425948ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.660+0000 7fbe33fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe33fff640 1 -- 172.21.15.19:0/1797112846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44035040 con 0x55fb24a590a0 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe5dd81740 1 -- 172.21.15.19:0/1797112846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe48023030 con 0x55fb24a590a0 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe33fff640 1 -- 172.21.15.19:0/1797112846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbe44004770 con 0x55fb24a590a0 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe33fff640 1 -- 172.21.15.19:0/1797112846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe44004970 con 0x55fb24a590a0 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe5dd81740 10 client.15657 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe5dd81740 1 -- 172.21.15.19:0/1797112846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb24690670 con 0x55fb24a590a0 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe33fff640 1 -- 172.21.15.19:0/1797112846 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe48023030 con 0x55fb24a590a0 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe33fff640 1 -- 172.21.15.19:0/1797112846 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbe440314a0 con 0x55fb24a590a0 2024-04-23T18:26:33.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe5dd81740 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:33.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe5dd81740 10 client.15657 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:26:33.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe5dd81740 20 client.15657 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:33.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fb24a74360 0x55fb24a76770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe5dd81740 4 client.15657 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:33.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.661+0000 7fbe5dd81740 1 -- 172.21.15.19:0/1797112846 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb249cbd90 con 0x55fb24a74360 2024-04-23T18:26:33.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.662+0000 7fbe53fff640 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fb24a74360 0x55fb24a76770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.662+0000 7fbe33fff640 10 client.15657 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:33.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.662+0000 7fbe53fff640 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fb24a74360 0x55fb24a76770 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9db97d7f83027dc server_cookie=ad40b36e3087e6b3 in_seq=0 out_seq=0 2024-04-23T18:26:33.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe33fff640 1 -- 172.21.15.19:0/1797112846 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb249cbd90 con 0x55fb24a74360 2024-04-23T18:26:33.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe33fff640 10 client.15657 handle_command_reply: tid=0 2024-04-23T18:26:33.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 client.15657 shutdown 2024-04-23T18:26:33.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 -- 172.21.15.19:0/1797112846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a590a0 msgr2=0x55fb24a61020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a590a0 0x55fb24a61020 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009230 tx=0x7fbe440045c0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 -- 172.21.15.19:0/1797112846 shutdown_connections 2024-04-23T18:26:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fb24a74360 0x55fb24a76770 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a61aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe33fff640 0 client.15657 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 0x55fb24a61560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/1797112846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a590a0 0x55fb24a61020 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 -- 172.21.15.19:0/1797112846 >> 172.21.15.19:0/1797112846 conn(0x55fb24a553c0 msgr2=0x55fb24a5b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 -- 172.21.15.19:0/1797112846 shutdown_connections 2024-04-23T18:26:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 1 -- 172.21.15.19:0/1797112846 wait complete. 2024-04-23T18:26:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.663+0000 7fbe5dd81740 20 client.15657 trim_cache size 0 max 16384 2024-04-23T18:26:33.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.681+0000 7fbe5dd81740 1 Processor -- start 2024-04-23T18:26:33.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe5dd81740 1 -- start start 2024-04-23T18:26:33.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a590a0 0x55fb24a6fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 0x55fb24a5b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a64090 0x55fb24a616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe5dd81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe541afc40 con 0x55fb24a64090 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe5dd81740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe541b1780 con 0x55fb24a590a0 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe5dd81740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe541b0310 con 0x55fb24a599c0 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe59538640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a64090 0x55fb24a616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe59538640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a64090 0x55fb24a616f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34252/0 (socket says 172.21.15.19:34252) 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe59538640 1 -- 172.21.15.19:0/953887810 learned_addr learned my addr 172.21.15.19:0/953887810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe58d37640 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a590a0 0x55fb24a6fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe53fff640 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 0x55fb24a5b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe59538640 1 -- 172.21.15.19:0/953887810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 msgr2=0x55fb24a5b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe59538640 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 0x55fb24a5b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe59538640 1 -- 172.21.15.19:0/953887810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a590a0 msgr2=0x55fb24a6fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe59538640 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a590a0 0x55fb24a6fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe59538640 1 -- 172.21.15.19:0/953887810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb24a5fff0 con 0x55fb24a64090 2024-04-23T18:26:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe53fff640 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 0x55fb24a5b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe58d37640 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a590a0 0x55fb24a6fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.682+0000 7fbe59538640 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a64090 0x55fb24a616f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c098910 tx=0x7fbe4c0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8f38a769668621 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe3a7fc640 1 -- 172.21.15.19:0/953887810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c020040 con 0x55fb24a64090 2024-04-23T18:26:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe3a7fc640 1 -- 172.21.15.19:0/953887810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4c003c60 con 0x55fb24a64090 2024-04-23T18:26:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe3a7fc640 1 -- 172.21.15.19:0/953887810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c04aab0 con 0x55fb24a64090 2024-04-23T18:26:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe5dd81740 1 -- 172.21.15.19:0/953887810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a64090 msgr2=0x55fb24a616f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a64090 0x55fb24a616f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c098910 tx=0x7fbe4c0092a0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe5dd81740 1 -- 172.21.15.19:0/953887810 shutdown_connections 2024-04-23T18:26:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a64090 0x55fb24a616f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a599c0 0x55fb24a5b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/953887810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a590a0 0x55fb24a6fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe5dd81740 1 -- 172.21.15.19:0/953887810 >> 172.21.15.19:0/953887810 conn(0x55fb24a553c0 msgr2=0x55fb24a60fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.683+0000 7fbe5dd81740 1 -- 172.21.15.19:0/953887810 shutdown_connections 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe5dd81740 1 -- 172.21.15.19:0/953887810 wait complete. 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe5dd81740 1 -- start start 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe5dd81740 1 Processor -- start 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 0x55fb24a6c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a6c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe5dd81740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a69210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe5dd81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe541b10b0 con 0x55fb24a599c0 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe5dd81740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe541b09e0 con 0x55fb24a64090 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe5dd81740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe541adc40 con 0x55fb24a590a0 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe5dd81740 10 client.0 _ll_register_callbacks cb 0x7fbe44004250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a6c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a6c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34268/0 (socket says 172.21.15.19:34268) 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe58d37640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 0x55fb24a6c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.684+0000 7fbe53fff640 1 -- 172.21.15.19:0/933060448 learned_addr learned my addr 172.21.15.19:0/933060448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe59538640 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a69210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe53fff640 1 -- 172.21.15.19:0/933060448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 msgr2=0x55fb24a6c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe53fff640 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 0x55fb24a6c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe53fff640 1 -- 172.21.15.19:0/933060448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 msgr2=0x55fb24a69210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe53fff640 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a69210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe53fff640 1 -- 172.21.15.19:0/933060448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb24a69a10 con 0x55fb24a599c0 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe53fff640 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a6c780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbe48003a60 tx=0x7fbe48005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77391db330b732ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe33fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe33fff640 1 -- 172.21.15.19:0/933060448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48037020 con 0x55fb24a599c0 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe5dd81740 1 -- 172.21.15.19:0/933060448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fb249c5c00 con 0x55fb24a599c0 2024-04-23T18:26:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe33fff640 1 -- 172.21.15.19:0/933060448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbe480335b0 con 0x55fb24a599c0 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe33fff640 1 -- 172.21.15.19:0/933060448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe48042050 con 0x55fb24a599c0 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe5dd81740 10 client.15663 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe5dd81740 1 -- 172.21.15.19:0/933060448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb24adb2f0 con 0x55fb24a599c0 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.685+0000 7fbe33fff640 1 -- 172.21.15.19:0/933060448 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fb249c5c00 con 0x55fb24a599c0 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.686+0000 7fbe33fff640 1 -- 172.21.15.19:0/933060448 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbe48028450 con 0x55fb24a599c0 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.686+0000 7fbe5dd81740 10 client.15663 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.686+0000 7fbe5dd81740 10 client.15663 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.686+0000 7fbe5dd81740 20 client.15663 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.686+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fb24a74360 0x55fb24add8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.686+0000 7fbe5dd81740 4 client.15663 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.5, "await_for": 42.0}] 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.686+0000 7fbe5dd81740 1 -- 172.21.15.19:0/933060448 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.5, "await_for": 42.0}) -- 0x55fb24941ca0 con 0x55fb24a74360 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.686+0000 7fbe58d37640 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fb24a74360 0x55fb24add8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.686+0000 7fbe58d37640 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fb24a74360 0x55fb24add8a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b49054be0d187f53 server_cookie=7813278482c4809 in_seq=0 out_seq=0 2024-04-23T18:26:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.686+0000 7fbe33fff640 10 client.15663 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:33.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe33fff640 1 -- 172.21.15.19:0/933060448 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55fb24941ca0 con 0x55fb24a74360 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe33fff640 10 client.15663 handle_command_reply: tid=0 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe5dd81740 1 client.15663 shutdown 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe5dd81740 1 -- 172.21.15.19:0/933060448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 msgr2=0x55fb24a6c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a6c780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbe48003a60 tx=0x7fbe48005ca0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe5dd81740 1 -- 172.21.15.19:0/933060448 shutdown_connections 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fb24a74360 0x55fb24add8a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb24a64090 0x55fb24a69210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb24a599c0 0x55fb24a6c780 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe33fff640 0 client.15663 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe5dd81740 1 --2- 172.21.15.19:0/933060448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb24a590a0 0x55fb24a6c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.706+0000 7fbe5dd81740 1 -- 172.21.15.19:0/933060448 >> 172.21.15.19:0/933060448 conn(0x55fb24a553c0 msgr2=0x55fb24a61850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5dd81740 1 -- 172.21.15.19:0/933060448 shutdown_connections 2024-04-23T18:26:33.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5dd81740 1 -- 172.21.15.19:0/933060448 wait complete. 2024-04-23T18:26:33.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5dd81740 20 client.15663 trim_cache size 0 max 16384 2024-04-23T18:26:33.701 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:26:33.701 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:26:33.701 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:26:33.701 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 2, 2024-04-23T18:26:33.701 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:26:33.701 INFO:teuthology.orchestra.run.smithi019.stdout: "92bdbb00": { 2024-04-23T18:26:33.701 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 2, 2024-04-23T18:26:33.701 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 78.5, 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:33.703 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:33.703 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:33.703 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:26:33.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 -- 172.21.15.19:0/962053994 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe3c0a3e70 msgr2=0x7fbe3c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe3c0a3e70 0x7fbe3c0a6330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe48004650 tx=0x7fbe4800beb0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 -- 172.21.15.19:0/962053994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe541110e0 msgr2=0x7fbe541a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe541110e0 0x7fbe541a6910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009fd0 tx=0x7fbe4c00b260 comp rx=0 tx=0).stop 2024-04-23T18:26:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 -- 172.21.15.19:0/962053994 shutdown_connections 2024-04-23T18:26:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe3c0a3e70 0x7fbe3c0a6330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe54106170 0x7fbe54110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe541110e0 0x7fbe541a6910 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 --2- 172.21.15.19:0/962053994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe541010a0 0x7fbe54110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 -- 172.21.15.19:0/962053994 >> 172.21.15.19:0/962053994 conn(0x7fbe54077f50 msgr2=0x7fbe540ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.707+0000 7fbe5af9b640 1 -- 172.21.15.19:0/962053994 shutdown_connections 2024-04-23T18:26:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:33.708+0000 7fbe5af9b640 1 -- 172.21.15.19:0/962053994 wait complete. 2024-04-23T18:26:33.712 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '92bdbb00', quiesce took 0.6 seconds. Will release after: 18.5 2024-04-23T18:26:34.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:34 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/28011031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:34.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:34 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/28011031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:34.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:34 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/28011031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:35.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:35 smithi019 ceph-mon[29215]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-23T18:26:35.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:35 smithi044 ceph-mon[32458]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-23T18:26:35.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:35 smithi155 ceph-mon[33468]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-23T18:26:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.226+0000 7fda98cae640 1 -- 172.21.15.19:0/1160262572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda94103b00 msgr2=0x7fda9410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.226+0000 7fda98cae640 1 --2- 172.21.15.19:0/1160262572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda94103b00 0x7fda9410bfe0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fda88009900 tx=0x7fda8802a510 comp rx=0 tx=0).stop 2024-04-23T18:26:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.227+0000 7fda98cae640 1 -- 172.21.15.19:0/1160262572 shutdown_connections 2024-04-23T18:26:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.227+0000 7fda98cae640 1 --2- 172.21.15.19:0/1160262572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda9410c520 0x7fda9410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.227+0000 7fda98cae640 1 --2- 172.21.15.19:0/1160262572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda94103b00 0x7fda9410bfe0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.227+0000 7fda98cae640 1 --2- 172.21.15.19:0/1160262572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda941031e0 0x7fda941035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.227+0000 7fda98cae640 1 -- 172.21.15.19:0/1160262572 >> 172.21.15.19:0/1160262572 conn(0x7fda94077fd0 msgr2=0x7fda940783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.227+0000 7fda98cae640 1 -- 172.21.15.19:0/1160262572 shutdown_connections 2024-04-23T18:26:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.227+0000 7fda98cae640 1 -- 172.21.15.19:0/1160262572 wait complete. 2024-04-23T18:26:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda98cae640 1 Processor -- start 2024-04-23T18:26:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda98cae640 1 -- start start 2024-04-23T18:26:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda98cae640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda941031e0 0x7fda94100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda98cae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda94103b00 0x7fda94101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda98cae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda9410c520 0x7fda941018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda98cae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda94112190 con 0x7fda9410c520 2024-04-23T18:26:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda98cae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda94112010 con 0x7fda94103b00 2024-04-23T18:26:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda98cae640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda94112310 con 0x7fda941031e0 2024-04-23T18:26:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda92d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda9410c520 0x7fda941018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda92d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda9410c520 0x7fda941018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34300/0 (socket says 172.21.15.19:34300) 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda92d9d640 1 -- 172.21.15.19:0/2127822834 learned_addr learned my addr 172.21.15.19:0/2127822834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda91d9b640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda94103b00 0x7fda94101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda9259c640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda941031e0 0x7fda94100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda92d9d640 1 -- 172.21.15.19:0/2127822834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda941031e0 msgr2=0x7fda94100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.228+0000 7fda92d9d640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda941031e0 0x7fda94100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.229+0000 7fda92d9d640 1 -- 172.21.15.19:0/2127822834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda94103b00 msgr2=0x7fda94101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.229+0000 7fda92d9d640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda94103b00 0x7fda94101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.229+0000 7fda92d9d640 1 -- 172.21.15.19:0/2127822834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda941a6b90 con 0x7fda9410c520 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.229+0000 7fda92d9d640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda9410c520 0x7fda941018b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00eb10 tx=0x7fda8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.229+0000 7fda91d9b640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda94103b00 0x7fda94101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.229+0000 7fda837fe640 1 -- 172.21.15.19:0/2127822834 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c004b20 con 0x7fda9410c520 2024-04-23T18:26:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.229+0000 7fda98cae640 1 -- 172.21.15.19:0/2127822834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda941a6e80 con 0x7fda9410c520 2024-04-23T18:26:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.229+0000 7fda837fe640 1 -- 172.21.15.19:0/2127822834 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8c004cc0 con 0x7fda9410c520 2024-04-23T18:26:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.229+0000 7fda837fe640 1 -- 172.21.15.19:0/2127822834 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c010690 con 0x7fda9410c520 2024-04-23T18:26:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.229+0000 7fda98cae640 1 -- 172.21.15.19:0/2127822834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda941a73a0 con 0x7fda9410c520 2024-04-23T18:26:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.232+0000 7fda9b9c8740 1 Processor -- start 2024-04-23T18:26:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.233+0000 7fda837fe640 1 -- 172.21.15.19:0/2127822834 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fda8c005020 con 0x7fda9410c520 2024-04-23T18:26:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.233+0000 7fda9b9c8740 1 -- start start 2024-04-23T18:26:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.233+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571620 0x560f6c571a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c57c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c57c800 0x560f6c57ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda837fe640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fda7c0a3f90 0x7fda7c0a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560f6c4d6970 con 0x560f6c571620 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560f6c4734a0 con 0x560f6c571f40 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560f6c4d2930 con 0x560f6c57c800 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda837fe640 1 -- 172.21.15.19:0/2127822834 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fda8c0d1020 con 0x7fda9410c520 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c57c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda92d9d640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fda7c0a3f90 0x7fda7c0a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda91d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571620 0x560f6c571a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda91d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571620 0x560f6c571a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34310/0 (socket says 172.21.15.19:34310) 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c57c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:60736/0 (socket says 172.21.15.19:60736) 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda91d9b640 1 -- 172.21.15.19:0/4022308105 learned_addr learned my addr 172.21.15.19:0/4022308105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda91d9b640 1 --2- 172.21.15.19:0/4022308105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c57c800 0x560f6c57ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9259c640 1 -- 172.21.15.19:0/4022308105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c57c800 msgr2=0x560f6c57ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9259c640 1 --2- 172.21.15.19:0/4022308105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c57c800 0x560f6c57ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9259c640 1 -- 172.21.15.19:0/4022308105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571620 msgr2=0x560f6c571a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9259c640 1 --2- 172.21.15.19:0/4022308105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571620 0x560f6c571a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.234+0000 7fda9259c640 1 -- 172.21.15.19:0/4022308105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6c1e9210 con 0x560f6c571f40 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.235+0000 7fda91d9b640 1 --2- 172.21.15.19:0/4022308105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571620 0x560f6c571a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.235+0000 7fda9259c640 1 --2- 172.21.15.19:0/4022308105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c57c2c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda84005fd0 tx=0x7fda840107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39fa156a3b406f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.235+0000 7fda92d9d640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fda7c0a3f90 0x7fda7c0a6450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0ce680 tx=0x7fda8c00cee0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.235+0000 7fda80ff9640 1 -- 172.21.15.19:0/4022308105 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8401c480 con 0x560f6c571f40 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.235+0000 7fda80ff9640 1 -- 172.21.15.19:0/4022308105 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8401cad0 con 0x560f6c571f40 2024-04-23T18:26:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.235+0000 7fda80ff9640 1 -- 172.21.15.19:0/4022308105 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84011b60 con 0x560f6c571f40 2024-04-23T18:26:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.236+0000 7fda9b9c8740 1 -- 172.21.15.19:0/4022308105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 msgr2=0x560f6c57c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.236+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/4022308105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c57c2c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda84005fd0 tx=0x7fda840107a0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.236+0000 7fda9b9c8740 1 -- 172.21.15.19:0/4022308105 shutdown_connections 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.236+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/4022308105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c57c2c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.236+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/4022308105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c57c800 0x560f6c57ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.236+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/4022308105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571620 0x560f6c571a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.236+0000 7fda9b9c8740 1 -- 172.21.15.19:0/4022308105 >> 172.21.15.19:0/4022308105 conn(0x560f6c56db30 msgr2=0x560f6c56df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.236+0000 7fda9b9c8740 1 -- 172.21.15.19:0/4022308105 shutdown_connections 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.236+0000 7fda9b9c8740 1 -- 172.21.15.19:0/4022308105 wait complete. 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9b9c8740 1 -- start start 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9b9c8740 1 Processor -- start 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c57c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571f40 0x560f6c579460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c57c800 0x560f6c5799c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560f6c57f430 con 0x560f6c571f40 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560f6c4710b0 con 0x560f6c57c800 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560f6c57f5b0 con 0x560f6c571620 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571f40 0x560f6c579460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571f40 0x560f6c579460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34312/0 (socket says 172.21.15.19:34312) 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9259c640 1 -- 172.21.15.19:0/2962244473 learned_addr learned my addr 172.21.15.19:0/2962244473 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda91d9b640 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c57c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda91d9b640 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c57c800 0x560f6c5799c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.237+0000 7fda9b9c8740 10 client.0 _ll_register_callbacks cb 0x560f6c45ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda9259c640 1 -- 172.21.15.19:0/2962244473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 msgr2=0x560f6c57c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda9259c640 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c57c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda9259c640 1 -- 172.21.15.19:0/2962244473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c57c800 msgr2=0x560f6c5799c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda9259c640 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c57c800 0x560f6c5799c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda9259c640 1 -- 172.21.15.19:0/2962244473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f6c2b5620 con 0x560f6c571f40 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda91d9b640 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c57c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda9259c640 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571f40 0x560f6c579460 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fda8401ceb0 tx=0x7fda8401cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e674c7ae600d675 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda79ffb640 1 -- 172.21.15.19:0/2962244473 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8402a060 con 0x560f6c571f40 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda79ffb640 1 -- 172.21.15.19:0/2962244473 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fda8401c690 con 0x560f6c571f40 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.238+0000 7fda9b9c8740 1 -- 172.21.15.19:0/2962244473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda8401cad0 con 0x560f6c571f40 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda79ffb640 1 -- 172.21.15.19:0/2962244473 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda8400bea0 con 0x560f6c571f40 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda9b9c8740 10 client.15675 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda9b9c8740 1 -- 172.21.15.19:0/2962244473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560f6c58b380 con 0x560f6c571f40 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda79ffb640 1 -- 172.21.15.19:0/2962244473 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8401c890 con 0x560f6c571f40 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda79ffb640 1 -- 172.21.15.19:0/2962244473 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fda840046c0 con 0x560f6c571f40 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda9b9c8740 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda9b9c8740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda9b9c8740 20 client.15675 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560f6c58c780 0x560f6c58eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda9b9c8740 4 client.15675 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.239+0000 7fda9b9c8740 1 -- 172.21.15.19:0/2962244473 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fda8401cad0 con 0x560f6c58c780 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.240+0000 7fda91d9b640 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560f6c58c780 0x560f6c58eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.240+0000 7fda91d9b640 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560f6c58c780 0x560f6c58eb70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ab4673608fdc13 server_cookie=743be8d3c17788c1 in_seq=0 out_seq=0 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.240+0000 7fda79ffb640 10 client.15675 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda79ffb640 1 -- 172.21.15.19:0/2962244473 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fda8401cad0 con 0x560f6c58c780 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda79ffb640 10 client.15675 handle_command_reply: tid=0 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 client.15675 shutdown 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 -- 172.21.15.19:0/2962244473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571f40 msgr2=0x560f6c579460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571f40 0x560f6c579460 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fda8401ceb0 tx=0x7fda8401cee0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 -- 172.21.15.19:0/2962244473 shutdown_connections 2024-04-23T18:26:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560f6c58c780 0x560f6c58eb70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c57c800 0x560f6c5799c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda79ffb640 0 client.15675 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c571f40 0x560f6c579460 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/2962244473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c57c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 -- 172.21.15.19:0/2962244473 >> 172.21.15.19:0/2962244473 conn(0x560f6c56db30 msgr2=0x560f6c5738b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 -- 172.21.15.19:0/2962244473 shutdown_connections 2024-04-23T18:26:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 1 -- 172.21.15.19:0/2962244473 wait complete. 2024-04-23T18:26:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.241+0000 7fda9b9c8740 20 client.15675 trim_cache size 0 max 16384 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.271+0000 7fda9b9c8740 1 Processor -- start 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.271+0000 7fda9b9c8740 1 -- start start 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.271+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c574970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.271+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c57bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.271+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c591b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda941afd80 con 0x560f6c57c800 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda941b18c0 con 0x560f6c571f40 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda941b0450 con 0x560f6c571620 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda91d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c591b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda91d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c591b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34316/0 (socket says 172.21.15.19:34316) 2024-04-23T18:26:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda91d9b640 1 -- 172.21.15.19:0/375391834 learned_addr learned my addr 172.21.15.19:0/375391834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda9259c640 1 --2- 172.21.15.19:0/375391834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c57bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda91d9b640 1 --2- 172.21.15.19:0/375391834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c574970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda91d9b640 1 -- 172.21.15.19:0/375391834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 msgr2=0x560f6c574970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda91d9b640 1 --2- 172.21.15.19:0/375391834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c574970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda91d9b640 1 -- 172.21.15.19:0/375391834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 msgr2=0x560f6c57bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda91d9b640 1 --2- 172.21.15.19:0/375391834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c57bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda91d9b640 1 -- 172.21.15.19:0/375391834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6c579d30 con 0x560f6c57c800 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.272+0000 7fda91d9b640 1 --2- 172.21.15.19:0/375391834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c591b90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fda8802ad00 tx=0x7fda88005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bdc74733d5a4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.273+0000 7fda7bfff640 1 -- 172.21.15.19:0/375391834 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88035040 con 0x560f6c57c800 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.273+0000 7fda7bfff640 1 -- 172.21.15.19:0/375391834 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda88031dc0 con 0x560f6c57c800 2024-04-23T18:26:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.273+0000 7fda7bfff640 1 -- 172.21.15.19:0/375391834 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda880046c0 con 0x560f6c57c800 2024-04-23T18:26:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.274+0000 7fda9b9c8740 1 -- 172.21.15.19:0/375391834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 msgr2=0x560f6c591b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.274+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/375391834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c591b90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fda8802ad00 tx=0x7fda88005b50 comp rx=0 tx=0).stop 2024-04-23T18:26:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.274+0000 7fda9b9c8740 1 -- 172.21.15.19:0/375391834 shutdown_connections 2024-04-23T18:26:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.274+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/375391834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c57bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.274+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/375391834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c591b90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.274+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/375391834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c574970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.274+0000 7fda9b9c8740 1 -- 172.21.15.19:0/375391834 >> 172.21.15.19:0/375391834 conn(0x560f6c56db30 msgr2=0x560f6c574260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.274+0000 7fda9b9c8740 1 -- 172.21.15.19:0/375391834 shutdown_connections 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.274+0000 7fda9b9c8740 1 -- 172.21.15.19:0/375391834 wait complete. 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda9b9c8740 1 -- start start 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda9b9c8740 1 Processor -- start 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c58edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c58f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda9b9c8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c5f0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda941b11f0 con 0x560f6c57c800 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda941b0b20 con 0x560f6c571f40 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda9b9c8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda941add60 con 0x560f6c571620 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda91d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c5f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda91d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c5f0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34322/0 (socket says 172.21.15.19:34322) 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.275+0000 7fda91d9b640 1 -- 172.21.15.19:0/1838429065 learned_addr learned my addr 172.21.15.19:0/1838429065 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda9259c640 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c58f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda91d9b640 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c58edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda9b9c8740 10 client.0 _ll_register_callbacks cb 0x7fda840114e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda91d9b640 1 -- 172.21.15.19:0/1838429065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 msgr2=0x560f6c58edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda91d9b640 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c58edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda91d9b640 1 -- 172.21.15.19:0/1838429065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 msgr2=0x560f6c58f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda91d9b640 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c58f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda91d9b640 1 -- 172.21.15.19:0/1838429065 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f6c5f0ee0 con 0x560f6c57c800 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda9b9c8740 1 -- 172.21.15.19:0/1838429065 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda88031dc0 con 0x560f6c57c800 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda91d9b640 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c5f0950 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fda88032d60 tx=0x7fda8802ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe0f4b47440d4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda79ffb640 10 client.15681 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda79ffb640 1 -- 172.21.15.19:0/1838429065 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88035070 con 0x560f6c57c800 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda79ffb640 1 -- 172.21.15.19:0/1838429065 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fda88004230 con 0x560f6c57c800 2024-04-23T18:26:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda79ffb640 1 -- 172.21.15.19:0/1838429065 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda88004430 con 0x560f6c57c800 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.276+0000 7fda9b9c8740 10 client.15681 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.277+0000 7fda9b9c8740 1 -- 172.21.15.19:0/1838429065 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=27}) -- 0x560f6c5f2110 con 0x560f6c57c800 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.277+0000 7fda79ffb640 1 -- 172.21.15.19:0/1838429065 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fda8803a040 con 0x560f6c57c800 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.277+0000 7fda9b9c8740 10 client.15681 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.277+0000 7fda9b9c8740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.277+0000 7fda9b9c8740 20 client.15681 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.277+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560f6c58c780 0x560f6c5f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.277+0000 7fda9b9c8740 4 client.15681 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.277+0000 7fda9b9c8740 1 -- 172.21.15.19:0/1838429065 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f6c4e4390 con 0x560f6c58c780 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.278+0000 7fda9259c640 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560f6c58c780 0x560f6c5f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.278+0000 7fda79ffb640 10 client.15681 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.278+0000 7fda9259c640 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560f6c58c780 0x560f6c5f45e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb82ca37014c5c9c server_cookie=8061b50c1be5c8b3 in_seq=0 out_seq=0 2024-04-23T18:26:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda79ffb640 1 -- 172.21.15.19:0/1838429065 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f6c4e4390 con 0x560f6c58c780 2024-04-23T18:26:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda79ffb640 10 client.15681 handle_command_reply: tid=0 2024-04-23T18:26:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 client.15681 shutdown 2024-04-23T18:26:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 -- 172.21.15.19:0/1838429065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 msgr2=0x560f6c5f0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c5f0950 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fda88032d60 tx=0x7fda8802ac90 comp rx=0 tx=0).stop 2024-04-23T18:26:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 -- 172.21.15.19:0/1838429065 shutdown_connections 2024-04-23T18:26:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560f6c58c780 0x560f6c5f45e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda79ffb640 0 client.15681 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6c571f40 0x560f6c58f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f6c57c800 0x560f6c5f0950 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 --2- 172.21.15.19:0/1838429065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f6c571620 0x560f6c58edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 -- 172.21.15.19:0/1838429065 >> 172.21.15.19:0/1838429065 conn(0x560f6c56db30 msgr2=0x560f6c56c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 -- 172.21.15.19:0/1838429065 shutdown_connections 2024-04-23T18:26:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 1 -- 172.21.15.19:0/1838429065 wait complete. 2024-04-23T18:26:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.286+0000 7fda9b9c8740 20 client.15681 trim_cache size 0 max 16384 2024-04-23T18:26:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:26:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:26:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:26:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:26:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 -- 172.21.15.19:0/2127822834 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fda7c0a3f90 msgr2=0x7fda7c0a6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fda7c0a3f90 0x7fda7c0a6450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0ce680 tx=0x7fda8c00cee0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 -- 172.21.15.19:0/2127822834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda9410c520 msgr2=0x7fda941018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda9410c520 0x7fda941018b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00eb10 tx=0x7fda8c00c2d0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 -- 172.21.15.19:0/2127822834 shutdown_connections 2024-04-23T18:26:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fda7c0a3f90 0x7fda7c0a6450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda94103b00 0x7fda94101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda9410c520 0x7fda941018b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 --2- 172.21.15.19:0/2127822834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda941031e0 0x7fda94100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 -- 172.21.15.19:0/2127822834 >> 172.21.15.19:0/2127822834 conn(0x7fda94077fd0 msgr2=0x7fda940fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 -- 172.21.15.19:0/2127822834 shutdown_connections 2024-04-23T18:26:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:37.287+0000 7fda98cae640 1 -- 172.21.15.19:0/2127822834 wait complete. 2024-04-23T18:26:37.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:37 smithi019 ceph-mon[29215]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2024-04-23T18:26:37.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:37 smithi044 ceph-mon[32458]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2024-04-23T18:26:37.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:37 smithi155 ceph-mon[33468]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2024-04-23T18:26:39.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:39 smithi019 ceph-mon[29215]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:26:39.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:39 smithi044 ceph-mon[32458]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:26:39.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:39 smithi155 ceph-mon[33468]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:26:41.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:41 smithi019 ceph-mon[29215]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T18:26:41.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:41 smithi044 ceph-mon[32458]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T18:26:41.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:41 smithi155 ceph-mon[33468]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T18:26:43.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:43 smithi019 ceph-mon[29215]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:26:43.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:43 smithi044 ceph-mon[32458]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:26:43.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:43 smithi155 ceph-mon[33468]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:26:45.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:45 smithi019 ceph-mon[29215]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-23T18:26:45.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:45 smithi044 ceph-mon[32458]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-23T18:26:45.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:45 smithi155 ceph-mon[33468]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-23T18:26:47.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:47 smithi019 ceph-mon[29215]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-23T18:26:47.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:47 smithi044 ceph-mon[32458]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-23T18:26:47.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:47 smithi155 ceph-mon[33468]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-23T18:26:49.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:49 smithi019 ceph-mon[29215]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-23T18:26:49.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:49 smithi044 ceph-mon[32458]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-23T18:26:49.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:49 smithi155 ceph-mon[33468]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-23T18:26:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.398+0000 7fa5f54a0640 1 -- 172.21.15.19:0/344927640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f01010f0 msgr2=0x7fa5f01014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.398+0000 7fa5f54a0640 1 --2- 172.21.15.19:0/344927640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f01010f0 0x7fa5f01014d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009900 tx=0x7fa5e002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.398+0000 7fa5f54a0640 1 -- 172.21.15.19:0/344927640 shutdown_connections 2024-04-23T18:26:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.398+0000 7fa5f54a0640 1 --2- 172.21.15.19:0/344927640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5f0101aa0 0x7fa5f0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.398+0000 7fa5f54a0640 1 --2- 172.21.15.19:0/344927640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f0106170 0x7fa5f010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.398+0000 7fa5f54a0640 1 --2- 172.21.15.19:0/344927640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f01010f0 0x7fa5f01014d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.398+0000 7fa5f54a0640 1 -- 172.21.15.19:0/344927640 >> 172.21.15.19:0/344927640 conn(0x7fa5f0077fd0 msgr2=0x7fa5f00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.398+0000 7fa5f54a0640 1 -- 172.21.15.19:0/344927640 shutdown_connections 2024-04-23T18:26:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.398+0000 7fa5f54a0640 1 -- 172.21.15.19:0/344927640 wait complete. 2024-04-23T18:26:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.398+0000 7fa5f54a0640 1 Processor -- start 2024-04-23T18:26:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5f54a0640 1 -- start start 2024-04-23T18:26:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5f54a0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5f01010f0 0x7fa5f0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5f54a0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f0106170 0x7fa5f0198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5f54a0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f019cfc0 0x7fa5f019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5f54a0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0112280 con 0x7fa5f019cfc0 2024-04-23T18:26:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5f54a0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0112100 con 0x7fa5f01010f0 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5f54a0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0112400 con 0x7fa5f0106170 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f019cfc0 0x7fa5f019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f019cfc0 0x7fa5f019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59070/0 (socket says 172.21.15.19:59070) 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5ef7fe640 1 -- 172.21.15.19:0/48405356 learned_addr learned my addr 172.21.15.19:0/48405356 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5f01010f0 0x7fa5f0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f0106170 0x7fa5f0198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5ef7fe640 1 -- 172.21.15.19:0/48405356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f0106170 msgr2=0x7fa5f0198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5ef7fe640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f0106170 0x7fa5f0198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5ef7fe640 1 -- 172.21.15.19:0/48405356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5f01010f0 msgr2=0x7fa5f0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5ef7fe640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5f01010f0 0x7fa5f0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.399+0000 7fa5ef7fe640 1 -- 172.21.15.19:0/48405356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f019dab0 con 0x7fa5f019cfc0 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.400+0000 7fa5ef7fe640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f019cfc0 0x7fa5f019d470 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800b970 tx=0x7fa5e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.400+0000 7fa5d7fff640 1 -- 172.21.15.19:0/48405356 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8019070 con 0x7fa5f019cfc0 2024-04-23T18:26:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.400+0000 7fa5d7fff640 1 -- 172.21.15.19:0/48405356 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e8014440 con 0x7fa5f019cfc0 2024-04-23T18:26:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.400+0000 7fa5f54a0640 1 -- 172.21.15.19:0/48405356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f019dda0 con 0x7fa5f019cfc0 2024-04-23T18:26:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.400+0000 7fa5d7fff640 1 -- 172.21.15.19:0/48405356 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80075d0 con 0x7fa5f019cfc0 2024-04-23T18:26:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.400+0000 7fa5f54a0640 1 -- 172.21.15.19:0/48405356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f01a5590 con 0x7fa5f019cfc0 2024-04-23T18:26:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.403+0000 7fa5f81b9740 1 Processor -- start 2024-04-23T18:26:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5d7fff640 1 -- 172.21.15.19:0/48405356 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa5e801c070 con 0x7fa5f019cfc0 2024-04-23T18:26:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5f81b9740 1 -- start start 2024-04-23T18:26:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a25f0 0x5646b34a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34ad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34ad7d0 0x5646b34afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646b3407970 con 0x5646b34a2f10 2024-04-23T18:26:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5646b33a44a0 con 0x5646b34a25f0 2024-04-23T18:26:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5646b3403930 con 0x5646b34ad7d0 2024-04-23T18:26:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59086/0 (socket says 172.21.15.19:59086) 2024-04-23T18:26:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5eeffd640 1 -- 172.21.15.19:0/2706680658 learned_addr learned my addr 172.21.15.19:0/2706680658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.404+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/2706680658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34ad7d0 0x5646b34afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/2706680658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a25f0 0x5646b34a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5eeffd640 1 -- 172.21.15.19:0/2706680658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34ad7d0 msgr2=0x5646b34afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/2706680658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34ad7d0 0x5646b34afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5eeffd640 1 -- 172.21.15.19:0/2706680658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a25f0 msgr2=0x5646b34a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/2706680658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a25f0 0x5646b34a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5eeffd640 1 -- 172.21.15.19:0/2706680658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b311a210 con 0x5646b34a2f10 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5d7fff640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5d80a3f60 0x7fa5d80a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/2706680658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a25f0 0x5646b34a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5d7fff640 1 -- 172.21.15.19:0/48405356 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa5e80cfea0 con 0x7fa5f019cfc0 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5d80a3f60 0x7fa5d80a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.405+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/2706680658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34ad290 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009230 tx=0x7fa5e0004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db702e4dd622be20 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.406+0000 7fa5d57fa640 1 -- 172.21.15.19:0/2706680658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0035040 con 0x5646b34a2f10 2024-04-23T18:26:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.406+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5d80a3f60 0x7fa5d80a6420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5646b349ff20 tx=0x7fa5e400b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.406+0000 7fa5d57fa640 1 -- 172.21.15.19:0/2706680658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e0004770 con 0x5646b34a2f10 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.406+0000 7fa5d57fa640 1 -- 172.21.15.19:0/2706680658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0003850 con 0x5646b34a2f10 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2706680658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 msgr2=0x5646b34ad290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/2706680658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34ad290 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009230 tx=0x7fa5e0004310 comp rx=0 tx=0).stop 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2706680658 shutdown_connections 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/2706680658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34ad7d0 0x5646b34afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/2706680658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34ad290 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/2706680658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a25f0 0x5646b34a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2706680658 >> 172.21.15.19:0/2706680658 conn(0x5646b349eb00 msgr2=0x5646b349eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2706680658 shutdown_connections 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2706680658 wait complete. 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 -- start start 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.407+0000 7fa5f81b9740 1 Processor -- start 2024-04-23T18:26:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a25f0 0x5646b34ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a2f10 0x5646b34aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34ad7d0 0x5646b34aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646b34b03a0 con 0x5646b34a25f0 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5646b33a20b0 con 0x5646b34ad7d0 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5646b34b0520 con 0x5646b34a2f10 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5f81b9740 10 client.0 _ll_register_callbacks cb 0x5646b338fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a25f0 0x5646b34ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a25f0 0x5646b34ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59098/0 (socket says 172.21.15.19:59098) 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5eeffd640 1 -- 172.21.15.19:0/286737616 learned_addr learned my addr 172.21.15.19:0/286737616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5ef7fe640 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34ad7d0 0x5646b34aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a2f10 0x5646b34aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5eeffd640 1 -- 172.21.15.19:0/286737616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a2f10 msgr2=0x5646b34aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a2f10 0x5646b34aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5eeffd640 1 -- 172.21.15.19:0/286737616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34ad7d0 msgr2=0x5646b34aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34ad7d0 0x5646b34aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5eeffd640 1 -- 172.21.15.19:0/286737616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b31e6620 con 0x5646b34a25f0 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.408+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a2f10 0x5646b34aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a25f0 0x5646b34ad1a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0031440 tx=0x7fa5e0004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1225e84ea4d5a675 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/286737616 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0052060 con 0x5646b34a25f0 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/286737616 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa5e0005210 con 0x5646b34a25f0 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5f81b9740 1 -- 172.21.15.19:0/286737616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e0004770 con 0x5646b34a25f0 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/286737616 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5e0035050 con 0x5646b34a25f0 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5f81b9740 10 client.15693 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5f81b9740 1 -- 172.21.15.19:0/286737616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646b34bc2a0 con 0x5646b34a25f0 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/286737616 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e0004770 con 0x5646b34a25f0 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/286737616 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa5e0031cc0 con 0x5646b34a25f0 2024-04-23T18:26:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5f81b9740 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.409+0000 7fa5f81b9740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.410+0000 7fa5f81b9740 20 client.15693 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.410+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5646b34bd820 0x5646b34bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.410+0000 7fa5f81b9740 4 client.15693 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.410+0000 7fa5f81b9740 1 -- 172.21.15.19:0/286737616 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646b3415390 con 0x5646b34bd820 2024-04-23T18:26:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.410+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5646b34bd820 0x5646b34bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.410+0000 7fa5d2ff5640 10 client.15693 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.410+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5646b34bd820 0x5646b34bfca0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aeb86c20e0c0068 server_cookie=978ee4f8ee7c01fb in_seq=0 out_seq=0 2024-04-23T18:26:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/286737616 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646b3415390 con 0x5646b34bd820 2024-04-23T18:26:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5d2ff5640 10 client.15693 handle_command_reply: tid=0 2024-04-23T18:26:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5f81b9740 1 client.15693 shutdown 2024-04-23T18:26:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5f81b9740 1 -- 172.21.15.19:0/286737616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a25f0 msgr2=0x5646b34ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a25f0 0x5646b34ad1a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0031440 tx=0x7fa5e0004c20 comp rx=0 tx=0).stop 2024-04-23T18:26:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5f81b9740 1 -- 172.21.15.19:0/286737616 shutdown_connections 2024-04-23T18:26:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5646b34bd820 0x5646b34bfca0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34ad7d0 0x5646b34aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5d2ff5640 0 client.15693 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a2f10 0x5646b34aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/286737616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a25f0 0x5646b34ad1a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.411+0000 7fa5f81b9740 1 -- 172.21.15.19:0/286737616 >> 172.21.15.19:0/286737616 conn(0x5646b349eb00 msgr2=0x5646b34a4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.412+0000 7fa5f81b9740 1 -- 172.21.15.19:0/286737616 shutdown_connections 2024-04-23T18:26:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.412+0000 7fa5f81b9740 1 -- 172.21.15.19:0/286737616 wait complete. 2024-04-23T18:26:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.412+0000 7fa5f81b9740 20 client.15693 trim_cache size 0 max 16384 2024-04-23T18:26:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.441+0000 7fa5f81b9740 1 Processor -- start 2024-04-23T18:26:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5f81b9740 1 -- start start 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 0x5646b34b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34c5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34ad7d0 0x5646b34c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01adae0 con 0x5646b34a2f10 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01af620 con 0x5646b34ad7d0 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01ae1b0 con 0x5646b34a25f0 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5ee7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5ee7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59102/0 (socket says 172.21.15.19:59102) 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5ee7fc640 1 -- 172.21.15.19:0/455062687 learned_addr learned my addr 172.21.15.19:0/455062687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/455062687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 0x5646b34b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5ef7fe640 1 --2- 172.21.15.19:0/455062687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34ad7d0 0x5646b34c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5ee7fc640 1 -- 172.21.15.19:0/455062687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 msgr2=0x5646b34b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/455062687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 0x5646b34b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5ee7fc640 1 -- 172.21.15.19:0/455062687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34ad7d0 msgr2=0x5646b34c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/455062687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34ad7d0 0x5646b34c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.442+0000 7fa5ee7fc640 1 -- 172.21.15.19:0/455062687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b34a2d40 con 0x5646b34a2f10 2024-04-23T18:26:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.443+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/455062687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34c5570 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4003a30 tx=0x7fa5e4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a49b921d30c4672 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.443+0000 7fa5d4ff9640 1 -- 172.21.15.19:0/455062687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40282d0 con 0x5646b34a2f10 2024-04-23T18:26:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.443+0000 7fa5d4ff9640 1 -- 172.21.15.19:0/455062687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e401bdc0 con 0x5646b34a2f10 2024-04-23T18:26:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.443+0000 7fa5d4ff9640 1 -- 172.21.15.19:0/455062687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e401c3f0 con 0x5646b34a2f10 2024-04-23T18:26:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.444+0000 7fa5f81b9740 1 -- 172.21.15.19:0/455062687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 msgr2=0x5646b34c5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.444+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/455062687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34c5570 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4003a30 tx=0x7fa5e4015000 comp rx=0 tx=0).stop 2024-04-23T18:26:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.444+0000 7fa5f81b9740 1 -- 172.21.15.19:0/455062687 shutdown_connections 2024-04-23T18:26:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.444+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/455062687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34ad7d0 0x5646b34c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.444+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/455062687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34a2f10 0x5646b34c5570 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.444+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/455062687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 0x5646b34b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.444+0000 7fa5f81b9740 1 -- 172.21.15.19:0/455062687 >> 172.21.15.19:0/455062687 conn(0x5646b349eb00 msgr2=0x5646b349dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.444+0000 7fa5f81b9740 1 -- 172.21.15.19:0/455062687 shutdown_connections 2024-04-23T18:26:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.444+0000 7fa5f81b9740 1 -- 172.21.15.19:0/455062687 wait complete. 2024-04-23T18:26:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.444+0000 7fa5f81b9740 1 -- start start 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5f81b9740 1 Processor -- start 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 0x5646b34b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a2f10 0x5646b34b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5f81b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34ad7d0 0x5646b34b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01aef50 con 0x5646b34ad7d0 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01ae880 con 0x5646b34a2f10 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5f81b9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01abac0 con 0x5646b34a25f0 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5f81b9740 10 client.0 _ll_register_callbacks cb 0x7fa5e0004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34ad7d0 0x5646b34b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34ad7d0 0x5646b34b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59108/0 (socket says 172.21.15.19:59108) 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5ef7fe640 1 -- 172.21.15.19:0/2455878043 learned_addr learned my addr 172.21.15.19:0/2455878043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a2f10 0x5646b34b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 0x5646b34b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5ef7fe640 1 -- 172.21.15.19:0/2455878043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 msgr2=0x5646b34b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5ef7fe640 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 0x5646b34b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5ef7fe640 1 -- 172.21.15.19:0/2455878043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a2f10 msgr2=0x5646b34b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5ef7fe640 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a2f10 0x5646b34b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5ef7fe640 1 -- 172.21.15.19:0/2455878043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b34b3d80 con 0x5646b34ad7d0 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.445+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 0x5646b34b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5ee7fc640 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a2f10 0x5646b34b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5ef7fe640 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34ad7d0 0x5646b34b35b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800b940 tx=0x7fa5e8014d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a302c8980b767856 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5d2ff5640 10 client.15699 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2455878043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e401bdc0 con 0x5646b34ad7d0 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/2455878043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80b4020 con 0x5646b34ad7d0 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/2455878043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa5e80a4510 con 0x5646b34ad7d0 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/2455878043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5e809a1a0 con 0x5646b34ad7d0 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5f81b9740 10 client.15699 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2455878043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5646b3523ec0 con 0x5646b34ad7d0 2024-04-23T18:26:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/2455878043 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa5e80cd780 con 0x5646b34ad7d0 2024-04-23T18:26:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5f81b9740 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5f81b9740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.446+0000 7fa5f81b9740 20 client.15699 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.447+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5646b34bd820 0x5646b35264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.447+0000 7fa5f81b9740 4 client.15699 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:26:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.447+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2455878043 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646b340f200 con 0x5646b34bd820 2024-04-23T18:26:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.447+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5646b34bd820 0x5646b35264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.447+0000 7fa5d2ff5640 10 client.15699 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.447+0000 7fa5eeffd640 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5646b34bd820 0x5646b35264c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=586677e37cf97389 server_cookie=4cfde6b8ae767067 in_seq=0 out_seq=0 2024-04-23T18:26:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.451+0000 7fa5d2ff5640 1 -- 172.21.15.19:0/2455878043 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646b340f200 con 0x5646b34bd820 2024-04-23T18:26:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.451+0000 7fa5d2ff5640 10 client.15699 handle_command_reply: tid=0 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 client.15699 shutdown 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2455878043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34ad7d0 msgr2=0x5646b34b35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34ad7d0 0x5646b34b35b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800b940 tx=0x7fa5e8014d00 comp rx=0 tx=0).stop 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2455878043 shutdown_connections 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5646b34bd820 0x5646b35264c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b34a2f10 0x5646b34b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646b34ad7d0 0x5646b34b35b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5d2ff5640 0 client.15699 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 --2- 172.21.15.19:0/2455878043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646b34a25f0 0x5646b34b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2455878043 >> 172.21.15.19:0/2455878043 conn(0x5646b349eb00 msgr2=0x5646b34b8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2455878043 shutdown_connections 2024-04-23T18:26:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 1 -- 172.21.15.19:0/2455878043 wait complete. 2024-04-23T18:26:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.452+0000 7fa5f81b9740 20 client.15699 trim_cache size 0 max 16384 2024-04-23T18:26:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:26:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:26:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:26:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 -- 172.21.15.19:0/48405356 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5d80a3f60 msgr2=0x7fa5d80a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5d80a3f60 0x7fa5d80a6420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5646b349ff20 tx=0x7fa5e400b390 comp rx=0 tx=0).stop 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 -- 172.21.15.19:0/48405356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f019cfc0 msgr2=0x7fa5f019d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f019cfc0 0x7fa5f019d470 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800b970 tx=0x7fa5e800be30 comp rx=0 tx=0).stop 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 -- 172.21.15.19:0/48405356 shutdown_connections 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa5d80a3f60 0x7fa5d80a6420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f019cfc0 0x7fa5f019d470 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f0106170 0x7fa5f0198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 --2- 172.21.15.19:0/48405356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5f01010f0 0x7fa5f0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 -- 172.21.15.19:0/48405356 >> 172.21.15.19:0/48405356 conn(0x7fa5f0077fd0 msgr2=0x7fa5f00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 -- 172.21.15.19:0/48405356 shutdown_connections 2024-04-23T18:26:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:26:50.453+0000 7fa5f54a0640 1 -- 172.21.15.19:0/48405356 wait complete. 2024-04-23T18:26:51.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:51 smithi019 ceph-mon[29215]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 14 op/s 2024-04-23T18:26:51.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:51 smithi044 ceph-mon[32458]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 14 op/s 2024-04-23T18:26:51.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:51 smithi155 ceph-mon[33468]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 14 op/s 2024-04-23T18:26:52.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:52.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.354+0000 7f84691bf640 1 -- 172.21.15.19:0/3763239790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84641031e0 msgr2=0x7f84641035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.354+0000 7f84691bf640 1 --2- 172.21.15.19:0/3763239790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84641031e0 0x7f84641035c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8454009900 tx=0x7f845402a510 comp rx=0 tx=0).stop 2024-04-23T18:26:52.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.354+0000 7f84691bf640 1 -- 172.21.15.19:0/3763239790 shutdown_connections 2024-04-23T18:26:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.354+0000 7f84691bf640 1 --2- 172.21.15.19:0/3763239790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f846410c520 0x7f846410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.354+0000 7f84691bf640 1 --2- 172.21.15.19:0/3763239790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8464103b00 0x7f846410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.354+0000 7f84691bf640 1 --2- 172.21.15.19:0/3763239790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84641031e0 0x7f84641035c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.354+0000 7f84691bf640 1 -- 172.21.15.19:0/3763239790 >> 172.21.15.19:0/3763239790 conn(0x7f8464077f50 msgr2=0x7f8464078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.354+0000 7f84691bf640 1 -- 172.21.15.19:0/3763239790 shutdown_connections 2024-04-23T18:26:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.354+0000 7f84691bf640 1 -- 172.21.15.19:0/3763239790 wait complete. 2024-04-23T18:26:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.355+0000 7f84691bf640 1 Processor -- start 2024-04-23T18:26:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f84691bf640 1 -- start start 2024-04-23T18:26:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f84691bf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84641031e0 0x7f8464198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f84691bf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8464103b00 0x7f8464198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f84691bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f846410c520 0x7f846419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f84691bf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8464112190 con 0x7f8464103b00 2024-04-23T18:26:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f84691bf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8464112010 con 0x7f846410c520 2024-04-23T18:26:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f84691bf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8464112310 con 0x7f84641031e0 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f846259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8464103b00 0x7f8464198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f8462d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84641031e0 0x7f8464198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f846259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8464103b00 0x7f8464198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59122/0 (socket says 172.21.15.19:59122) 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f846259c640 1 -- 172.21.15.19:0/3765189991 learned_addr learned my addr 172.21.15.19:0/3765189991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.356+0000 7f846359e640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f846410c520 0x7f846419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f846259c640 1 -- 172.21.15.19:0/3765189991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84641031e0 msgr2=0x7f8464198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f846259c640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84641031e0 0x7f8464198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f846259c640 1 -- 172.21.15.19:0/3765189991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f846410c520 msgr2=0x7f846419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f846259c640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f846410c520 0x7f846419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f846259c640 1 -- 172.21.15.19:0/3765189991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846419d720 con 0x7f8464103b00 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f8462d9d640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84641031e0 0x7f8464198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f846259c640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8464103b00 0x7f8464198c70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f845800d950 tx=0x7f845800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f844bfff640 1 -- 172.21.15.19:0/3765189991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458017070 con 0x7f8464103b00 2024-04-23T18:26:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f84691bf640 1 -- 172.21.15.19:0/3765189991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846419da10 con 0x7f8464103b00 2024-04-23T18:26:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f844bfff640 1 -- 172.21.15.19:0/3765189991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8458009c30 con 0x7f8464103b00 2024-04-23T18:26:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f844bfff640 1 -- 172.21.15.19:0/3765189991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845800f620 con 0x7f8464103b00 2024-04-23T18:26:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.357+0000 7f84691bf640 1 -- 172.21.15.19:0/3765189991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84641a52a0 con 0x7f8464103b00 2024-04-23T18:26:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.359+0000 7f84691bf640 1 -- 172.21.15.19:0/3765189991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8464068d70 con 0x7f8464103b00 2024-04-23T18:26:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.360+0000 7f844bfff640 1 -- 172.21.15.19:0/3765189991 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8458010460 con 0x7f8464103b00 2024-04-23T18:26:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.361+0000 7f844bfff640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f844c0a3e40 0x7f844c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.361+0000 7f8462d9d640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f844c0a3e40 0x7f844c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.361+0000 7f844bfff640 1 -- 172.21.15.19:0/3765189991 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f84580d0690 con 0x7f8464103b00 2024-04-23T18:26:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.361+0000 7f8462d9d640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f844c0a3e40 0x7f844c0a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8454031440 tx=0x7f8454005b50 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:52.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.369+0000 7f844bfff640 1 -- 172.21.15.19:0/3765189991 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8458009de0 con 0x7f8464103b00 2024-04-23T18:26:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.828+0000 7f84691bf640 1 -- 172.21.15.19:0/3765189991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8464107700 con 0x7f8464103b00 2024-04-23T18:26:52.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.829+0000 7f844bfff640 1 -- 172.21.15.19:0/3765189991 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f845809c460 con 0x7f8464103b00 2024-04-23T18:26:52.822 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:52.822 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:26:52.826 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:26:52.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 -- 172.21.15.19:0/3765189991 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f844c0a3e40 msgr2=0x7f844c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f844c0a3e40 0x7f844c0a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8454031440 tx=0x7f8454005b50 comp rx=0 tx=0).stop 2024-04-23T18:26:52.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 -- 172.21.15.19:0/3765189991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8464103b00 msgr2=0x7f8464198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8464103b00 0x7f8464198c70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f845800d950 tx=0x7f845800de10 comp rx=0 tx=0).stop 2024-04-23T18:26:52.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 -- 172.21.15.19:0/3765189991 shutdown_connections 2024-04-23T18:26:52.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f844c0a3e40 0x7f844c0a6300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f846410c520 0x7f846419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8464103b00 0x7f8464198c70 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 --2- 172.21.15.19:0/3765189991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84641031e0 0x7f8464198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 -- 172.21.15.19:0/3765189991 >> 172.21.15.19:0/3765189991 conn(0x7f8464077f50 msgr2=0x7f84640fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:52.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 -- 172.21.15.19:0/3765189991 shutdown_connections 2024-04-23T18:26:52.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.832+0000 7f84691bf640 1 -- 172.21.15.19:0/3765189991 wait complete. 2024-04-23T18:26:52.837 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id 92bdbb00 --release --await' 2024-04-23T18:26:52.837 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id 92bdbb00 --release --await 2024-04-23T18:26:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.958+0000 7ff9a1c05640 1 -- 172.21.15.19:0/4211260107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff99c1031d0 msgr2=0x7ff99c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.958+0000 7ff9a1c05640 1 --2- 172.21.15.19:0/4211260107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff99c1031d0 0x7ff99c1035b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009900 tx=0x7ff98c02a510 comp rx=0 tx=0).stop 2024-04-23T18:26:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.958+0000 7ff9a1c05640 1 -- 172.21.15.19:0/4211260107 shutdown_connections 2024-04-23T18:26:52.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.958+0000 7ff9a1c05640 1 --2- 172.21.15.19:0/4211260107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c10c510 0x7ff99c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.958+0000 7ff9a1c05640 1 --2- 172.21.15.19:0/4211260107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff99c103af0 0x7ff99c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.958+0000 7ff9a1c05640 1 --2- 172.21.15.19:0/4211260107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff99c1031d0 0x7ff99c1035b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.958+0000 7ff9a1c05640 1 -- 172.21.15.19:0/4211260107 >> 172.21.15.19:0/4211260107 conn(0x7ff99c077f50 msgr2=0x7ff99c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.958+0000 7ff9a1c05640 1 -- 172.21.15.19:0/4211260107 shutdown_connections 2024-04-23T18:26:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.958+0000 7ff9a1c05640 1 -- 172.21.15.19:0/4211260107 wait complete. 2024-04-23T18:26:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff9a1c05640 1 Processor -- start 2024-04-23T18:26:52.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff9a1c05640 1 -- start start 2024-04-23T18:26:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff9a1c05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff99c1031d0 0x7ff99c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff9a1c05640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c103af0 0x7ff99c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff9a1c05640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff99c10c510 0x7ff99c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff9a1c05640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1121a0 con 0x7ff99c1031d0 2024-04-23T18:26:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff9a1c05640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c112020 con 0x7ff99c103af0 2024-04-23T18:26:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff9a1c05640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c112320 con 0x7ff99c10c510 2024-04-23T18:26:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff99b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff99c1031d0 0x7ff99c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff99b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff99c1031d0 0x7ff99c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59136/0 (socket says 172.21.15.19:59136) 2024-04-23T18:26:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff99b7fe640 1 -- 172.21.15.19:0/1471383996 learned_addr learned my addr 172.21.15.19:0/1471383996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.959+0000 7ff99affd640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c103af0 0x7ff99c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.960+0000 7ff99bfff640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff99c10c510 0x7ff99c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.960+0000 7ff99b7fe640 1 -- 172.21.15.19:0/1471383996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff99c10c510 msgr2=0x7ff99c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.960+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff99c10c510 0x7ff99c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.960+0000 7ff99b7fe640 1 -- 172.21.15.19:0/1471383996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c103af0 msgr2=0x7ff99c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.960+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c103af0 0x7ff99c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.960+0000 7ff99b7fe640 1 -- 172.21.15.19:0/1471383996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c19af70 con 0x7ff99c1031d0 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.960+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff99c1031d0 0x7ff99c1a0630 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff98c005940 tx=0x7ff98c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.960+0000 7ff998ff9640 1 -- 172.21.15.19:0/1471383996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c03f030 con 0x7ff99c1031d0 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.960+0000 7ff998ff9640 1 -- 172.21.15.19:0/1471383996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff98c003aa0 con 0x7ff99c1031d0 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.960+0000 7ff9a1c05640 1 -- 172.21.15.19:0/1471383996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99c19b200 con 0x7ff99c1031d0 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.961+0000 7ff998ff9640 1 -- 172.21.15.19:0/1471383996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c035070 con 0x7ff99c1031d0 2024-04-23T18:26:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.961+0000 7ff9a1c05640 1 -- 172.21.15.19:0/1471383996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99c1a7480 con 0x7ff99c1031d0 2024-04-23T18:26:52.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.963+0000 7ff998ff9640 1 -- 172.21.15.19:0/1471383996 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff98c042030 con 0x7ff99c1031d0 2024-04-23T18:26:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.963+0000 7ff998ff9640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff9840a3f40 0x7ff9840a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.963+0000 7ff998ff9640 1 -- 172.21.15.19:0/1471383996 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff98c0f1070 con 0x7ff99c1031d0 2024-04-23T18:26:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.964+0000 7ff99affd640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff9840a3f40 0x7ff9840a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.964+0000 7ff99affd640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff9840a3f40 0x7ff9840a6400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff99c102d30 tx=0x7ff99000b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff9a49f7740 1 Processor -- start 2024-04-23T18:26:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff9a49f7740 1 -- start start 2024-04-23T18:26:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c603490 0x55e84c603870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c60df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c610870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e84c5686d0 con 0x55e84c60e480 2024-04-23T18:26:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e84c5051a0 con 0x55e84c603db0 2024-04-23T18:26:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e84c564690 con 0x55e84c603490 2024-04-23T18:26:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff99bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c610870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff99bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c610870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59150/0 (socket says 172.21.15.19:59150) 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff99bfff640 1 -- 172.21.15.19:0/1188973501 learned_addr learned my addr 172.21.15.19:0/1188973501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/1188973501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c603490 0x55e84c603870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.965+0000 7ff99affd640 1 --2- 172.21.15.19:0/1188973501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c60df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.966+0000 7ff99bfff640 1 -- 172.21.15.19:0/1188973501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c603490 msgr2=0x55e84c603870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.966+0000 7ff99bfff640 1 --2- 172.21.15.19:0/1188973501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c603490 0x55e84c603870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.966+0000 7ff99bfff640 1 -- 172.21.15.19:0/1188973501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 msgr2=0x55e84c60df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.966+0000 7ff99bfff640 1 --2- 172.21.15.19:0/1188973501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c60df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.966+0000 7ff99bfff640 1 -- 172.21.15.19:0/1188973501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e84c50f5b0 con 0x55e84c60e480 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.966+0000 7ff99bfff640 1 --2- 172.21.15.19:0/1188973501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c610870 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff994009f40 tx=0x7ff994010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6f0070e35262bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.966+0000 7ff981ffb640 1 -- 172.21.15.19:0/1188973501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994003b70 con 0x55e84c60e480 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.966+0000 7ff981ffb640 1 -- 172.21.15.19:0/1188973501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff994003d10 con 0x55e84c60e480 2024-04-23T18:26:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.966+0000 7ff981ffb640 1 -- 172.21.15.19:0/1188973501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940119c0 con 0x55e84c60e480 2024-04-23T18:26:52.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.967+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1188973501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 msgr2=0x55e84c610870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.967+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/1188973501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c610870 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff994009f40 tx=0x7ff994010860 comp rx=0 tx=0).stop 2024-04-23T18:26:52.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.967+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1188973501 shutdown_connections 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.967+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/1188973501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c60df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.967+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/1188973501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c610870 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.967+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/1188973501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c603490 0x55e84c603870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.967+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1188973501 >> 172.21.15.19:0/1188973501 conn(0x55e84c5ff7b0 msgr2=0x55e84c5ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.967+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1188973501 shutdown_connections 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.967+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1188973501 wait complete. 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.967+0000 7ff9a49f7740 1 -- start start 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff9a49f7740 1 Processor -- start 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603490 0x55e84c60ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603db0 0x55e84c60b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c60e480 0x55e84c60b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e84c611310 con 0x55e84c603db0 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e84c502b20 con 0x55e84c603490 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e84c611490 con 0x55e84c60e480 2024-04-23T18:26:52.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff99affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603db0 0x55e84c60b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff9a49f7740 10 client.0 _ll_register_callbacks cb 0x55e84c502530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff99affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603db0 0x55e84c60b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59160/0 (socket says 172.21.15.19:59160) 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff99affd640 1 -- 172.21.15.19:0/1703493593 learned_addr learned my addr 172.21.15.19:0/1703493593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603490 0x55e84c60ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff99bfff640 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c60e480 0x55e84c60b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff99affd640 1 -- 172.21.15.19:0/1703493593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c60e480 msgr2=0x55e84c60b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff99affd640 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c60e480 0x55e84c60b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff99affd640 1 -- 172.21.15.19:0/1703493593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603490 msgr2=0x55e84c60ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff99affd640 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603490 0x55e84c60ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.968+0000 7ff99affd640 1 -- 172.21.15.19:0/1703493593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e84c27a920 con 0x55e84c603db0 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.969+0000 7ff99affd640 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603db0 0x55e84c60b1c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff990003e80 tx=0x7ff990004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e824b749e46b99a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.969+0000 7ff97affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.969+0000 7ff97affd640 1 -- 172.21.15.19:0/1703493593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99001f840 con 0x55e84c603db0 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.969+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1703493593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff994003d10 con 0x55e84c603db0 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.969+0000 7ff97affd640 1 -- 172.21.15.19:0/1703493593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff99001f9e0 con 0x55e84c603db0 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.969+0000 7ff97affd640 1 -- 172.21.15.19:0/1703493593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff990011e50 con 0x55e84c603db0 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.969+0000 7ff9a49f7740 10 client.15717 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.969+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1703493593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e84c346d30 con 0x55e84c603db0 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.969+0000 7ff97affd640 1 -- 172.21.15.19:0/1703493593 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff994003d10 con 0x55e84c603db0 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.970+0000 7ff97affd640 1 -- 172.21.15.19:0/1703493593 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff99001e430 con 0x55e84c603db0 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.970+0000 7ff9a49f7740 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.970+0000 7ff9a49f7740 10 client.15717 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.970+0000 7ff9a49f7740 20 client.15717 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.970+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e84c61e680 0x55e84c620ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.970+0000 7ff9a49f7740 4 client.15717 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.970+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1703493593 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e84c526500 con 0x55e84c61e680 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.970+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e84c61e680 0x55e84c620ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.970+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e84c61e680 0x55e84c620ab0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2370f32e0b177cf3 server_cookie=73bc4a16ce58ee9c in_seq=0 out_seq=0 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.971+0000 7ff97affd640 10 client.15717 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.971+0000 7ff97affd640 1 -- 172.21.15.19:0/1703493593 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e84c526500 con 0x55e84c61e680 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.971+0000 7ff97affd640 10 client.15717 handle_command_reply: tid=0 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.971+0000 7ff9a49f7740 1 client.15717 shutdown 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.971+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1703493593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603db0 msgr2=0x55e84c60b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.971+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603db0 0x55e84c60b1c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff990003e80 tx=0x7ff990004730 comp rx=0 tx=0).stop 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.971+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1703493593 shutdown_connections 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.972+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e84c61e680 0x55e84c620ab0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.972+0000 7ff97affd640 0 client.15717 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.972+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c60e480 0x55e84c60b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.972+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603db0 0x55e84c60b1c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.972+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/1703493593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603490 0x55e84c60ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.972+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1703493593 >> 172.21.15.19:0/1703493593 conn(0x55e84c5ff7b0 msgr2=0x55e84c6055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:52.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.972+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1703493593 shutdown_connections 2024-04-23T18:26:52.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.972+0000 7ff9a49f7740 1 -- 172.21.15.19:0/1703493593 wait complete. 2024-04-23T18:26:52.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.972+0000 7ff9a49f7740 20 client.15717 trim_cache size 0 max 16384 2024-04-23T18:26:52.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.989+0000 7ff9a49f7740 1 Processor -- start 2024-04-23T18:26:52.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.989+0000 7ff9a49f7740 1 -- start start 2024-04-23T18:26:52.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.989+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c603490 0x55e84c60da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.989+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c60df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c626380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1afd10 con 0x55e84c60e480 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1b1850 con 0x55e84c603db0 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1b03e0 con 0x55e84c603490 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c626380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c60df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c60df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:35890/0 (socket says 172.21.15.19:35890) 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c626380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59170/0 (socket says 172.21.15.19:59170) 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99affd640 1 -- 172.21.15.19:0/4094612543 learned_addr learned my addr 172.21.15.19:0/4094612543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/4094612543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c603490 0x55e84c60da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99affd640 1 -- 172.21.15.19:0/4094612543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c603490 msgr2=0x55e84c60da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:52.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99affd640 1 --2- 172.21.15.19:0/4094612543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c603490 0x55e84c60da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99affd640 1 -- 172.21.15.19:0/4094612543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 msgr2=0x55e84c626380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:52.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99affd640 1 --2- 172.21.15.19:0/4094612543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c626380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99affd640 1 -- 172.21.15.19:0/4094612543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e84c603be0 con 0x55e84c603db0 2024-04-23T18:26:52.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99bfff640 1 --2- 172.21.15.19:0/4094612543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c626380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:52.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.990+0000 7ff99affd640 1 --2- 172.21.15.19:0/4094612543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c60df40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff99001e9f0 tx=0x7ff99000b4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517a5fb52c5aa37e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:52.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9817fa640 1 -- 172.21.15.19:0/4094612543 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99003f070 con 0x55e84c603db0 2024-04-23T18:26:52.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9817fa640 1 -- 172.21.15.19:0/4094612543 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff990023070 con 0x55e84c603db0 2024-04-23T18:26:52.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9817fa640 1 -- 172.21.15.19:0/4094612543 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990028470 con 0x55e84c603db0 2024-04-23T18:26:52.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4094612543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 msgr2=0x55e84c60df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/4094612543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c60df40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff99001e9f0 tx=0x7ff99000b4b0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4094612543 shutdown_connections 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/4094612543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c60df40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/4094612543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c60e480 0x55e84c626380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/4094612543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c603490 0x55e84c60da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4094612543 >> 172.21.15.19:0/4094612543 conn(0x55e84c5ff7b0 msgr2=0x55e84c5fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4094612543 shutdown_connections 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.991+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4094612543 wait complete. 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.992+0000 7ff9a49f7740 1 -- start start 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.992+0000 7ff9a49f7740 1 Processor -- start 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.992+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603490 0x55e84c616c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.992+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c6116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.992+0000 7ff9a49f7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c60e480 0x55e84c611e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603490 0x55e84c616c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1b1180 con 0x55e84c603490 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff9a49f7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1b0ab0 con 0x55e84c603db0 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603490 0x55e84c616c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59180/0 (socket says 172.21.15.19:59180) 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4180908736 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1adcf0 con 0x55e84c60e480 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99b7fe640 1 -- 172.21.15.19:0/4180908736 learned_addr learned my addr 172.21.15.19:0/4180908736 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99affd640 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c6116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff9a49f7740 10 client.0 _ll_register_callbacks cb 0x7ff990003b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99bfff640 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c60e480 0x55e84c611e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99affd640 1 -- 172.21.15.19:0/4180908736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c60e480 msgr2=0x55e84c611e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99affd640 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c60e480 0x55e84c611e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99affd640 1 -- 172.21.15.19:0/4180908736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603490 msgr2=0x55e84c616c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99affd640 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603490 0x55e84c616c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99affd640 1 -- 172.21.15.19:0/4180908736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e84c684bf0 con 0x55e84c603db0 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603490 0x55e84c616c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff99affd640 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c6116d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55e84c603bb0 tx=0x7ff99001e660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34cb6a2b5a21ac87 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff97affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.993+0000 7ff97affd640 1 -- 172.21.15.19:0/4180908736 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99001ea20 con 0x55e84c603db0 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.994+0000 7ff97affd640 1 -- 172.21.15.19:0/4180908736 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff99001ebc0 con 0x55e84c603db0 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.994+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4180908736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff990023070 con 0x55e84c603db0 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.994+0000 7ff97affd640 1 -- 172.21.15.19:0/4180908736 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff99001ede0 con 0x55e84c603db0 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.994+0000 7ff9a49f7740 10 client.25352 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:52.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.994+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4180908736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e84c685d70 con 0x55e84c603db0 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.994+0000 7ff97affd640 1 -- 172.21.15.19:0/4180908736 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff990023070 con 0x55e84c603db0 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.994+0000 7ff97affd640 1 -- 172.21.15.19:0/4180908736 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff990004a10 con 0x55e84c603db0 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.995+0000 7ff9a49f7740 10 client.25352 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.995+0000 7ff9a49f7740 10 client.25352 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.995+0000 7ff9a49f7740 20 client.25352 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.995+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e84c61e680 0x55e84c6884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.995+0000 7ff9a49f7740 4 client.25352 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "92bdbb00", "release": true, "await": true}] 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.995+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4180908736 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "92bdbb00", "release": true, "await": true}) -- 0x55e84c5760f0 con 0x55e84c61e680 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.995+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e84c61e680 0x55e84c6884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.995+0000 7ff97affd640 10 client.25352 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.995+0000 7ff99b7fe640 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e84c61e680 0x55e84c6884b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e309dc5dacbb6eb server_cookie=d12056cb7fc6a053 in_seq=0 out_seq=0 2024-04-23T18:26:52.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff97affd640 1 -- 172.21.15.19:0/4180908736 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55e84c5760f0 con 0x55e84c61e680 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff97affd640 10 client.25352 handle_command_reply: tid=0 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 client.25352 shutdown 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4180908736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 msgr2=0x55e84c6116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c6116d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55e84c603bb0 tx=0x7ff99001e660 comp rx=0 tx=0).stop 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4180908736 shutdown_connections 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e84c61e680 0x55e84c6884b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff97affd640 0 client.25352 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e84c603db0 0x55e84c6116d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e84c60e480 0x55e84c611e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 --2- 172.21.15.19:0/4180908736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e84c603490 0x55e84c616c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4180908736 >> 172.21.15.19:0/4180908736 conn(0x55e84c5ff7b0 msgr2=0x55e84c604c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4180908736 shutdown_connections 2024-04-23T18:26:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 1 -- 172.21.15.19:0/4180908736 wait complete. 2024-04-23T18:26:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.998+0000 7ff9a49f7740 20 client.25352 trim_cache size 0 max 16384 2024-04-23T18:26:52.995 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:26:52.995 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:26:52.995 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:26:52.995 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 4, 2024-04-23T18:26:52.995 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:26:52.995 INFO:teuthology.orchestra.run.smithi019.stdout: "92bdbb00": { 2024-04-23T18:26:52.995 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 4, 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 78.5, 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:26:52.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:26:52.997 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:26:52.997 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:52.997 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:52.997 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:52.997 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:52.997 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:52.997 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:26:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 -- 172.21.15.19:0/1471383996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff9840a3f40 msgr2=0x7ff9840a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff9840a3f40 0x7ff9840a6400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff99c102d30 tx=0x7ff99000b390 comp rx=0 tx=0).stop 2024-04-23T18:26:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 -- 172.21.15.19:0/1471383996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff99c1031d0 msgr2=0x7ff99c1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff99c1031d0 0x7ff99c1a0630 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff98c005940 tx=0x7ff98c004270 comp rx=0 tx=0).stop 2024-04-23T18:26:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 -- 172.21.15.19:0/1471383996 shutdown_connections 2024-04-23T18:26:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff9840a3f40 0x7ff9840a6400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c103af0 0x7ff99c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff99c10c510 0x7ff99c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 --2- 172.21.15.19:0/1471383996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff99c1031d0 0x7ff99c1a0630 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 -- 172.21.15.19:0/1471383996 >> 172.21.15.19:0/1471383996 conn(0x7ff99c077f50 msgr2=0x7ff99c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 -- 172.21.15.19:0/1471383996 shutdown_connections 2024-04-23T18:26:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:52.999+0000 7ff9a1c05640 1 -- 172.21.15.19:0/1471383996 wait complete. 2024-04-23T18:26:53.003 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '92bdbb00', seconds elapsed: 19.9 2024-04-23T18:26:53.003 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.5 2024-04-23T18:26:53.003 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:53.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.115+0000 7f78d0508640 1 -- 172.21.15.19:0/187403609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c8107270 msgr2=0x7f78c810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.115+0000 7f78d0508640 1 --2- 172.21.15.19:0/187403609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c8107270 0x7f78c810ee40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009ca0 tx=0x7f78c002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.115+0000 7f78d0508640 1 -- 172.21.15.19:0/187403609 shutdown_connections 2024-04-23T18:26:53.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.116+0000 7f78d0508640 1 --2- 172.21.15.19:0/187403609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c8107270 0x7f78c810ee40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.116+0000 7f78d0508640 1 --2- 172.21.15.19:0/187403609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c8102b50 0x7f78c8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.116+0000 7f78d0508640 1 --2- 172.21.15.19:0/187403609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c81021a0 0x7f78c8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.116+0000 7f78d0508640 1 -- 172.21.15.19:0/187403609 >> 172.21.15.19:0/187403609 conn(0x7f78c8079350 msgr2=0x7f78c8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:53.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.116+0000 7f78d0508640 1 -- 172.21.15.19:0/187403609 shutdown_connections 2024-04-23T18:26:53.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.116+0000 7f78d0508640 1 -- 172.21.15.19:0/187403609 wait complete. 2024-04-23T18:26:53.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.116+0000 7f78d0508640 1 Processor -- start 2024-04-23T18:26:53.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.116+0000 7f78d0508640 1 -- start start 2024-04-23T18:26:53.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.116+0000 7f78d0508640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c81021a0 0x7f78c81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78d0508640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c8107270 0x7f78c8198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78d0508640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c819d010 0x7f78c819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78d0508640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78c81121e0 con 0x7f78c8107270 2024-04-23T18:26:53.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78d0508640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8112060 con 0x7f78c81021a0 2024-04-23T18:26:53.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78d0508640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8112360 con 0x7f78c819d010 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78ce2a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c81021a0 0x7f78c81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78cdaa3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c8107270 0x7f78c8198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78ce2a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c81021a0 0x7f78c81987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:35926/0 (socket says 172.21.15.19:35926) 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78ce2a4640 1 -- 172.21.15.19:0/1295687929 learned_addr learned my addr 172.21.15.19:0/1295687929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78ce2a4640 1 -- 172.21.15.19:0/1295687929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c819d010 msgr2=0x7f78c819d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78ceaa5640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c819d010 0x7f78c819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78ce2a4640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c819d010 0x7f78c819d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78ce2a4640 1 -- 172.21.15.19:0/1295687929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c8107270 msgr2=0x7f78c8198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78ce2a4640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c8107270 0x7f78c8198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78ce2a4640 1 -- 172.21.15.19:0/1295687929 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c819db00 con 0x7f78c81021a0 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78cdaa3640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c8107270 0x7f78c8198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.117+0000 7f78ce2a4640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c81021a0 0x7f78c81987a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f78c400e9f0 tx=0x7f78c400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.118+0000 7f78bf7fe640 1 -- 172.21.15.19:0/1295687929 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c400cd80 con 0x7f78c81021a0 2024-04-23T18:26:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.118+0000 7f78bf7fe640 1 -- 172.21.15.19:0/1295687929 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c40054d0 con 0x7f78c81021a0 2024-04-23T18:26:53.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.118+0000 7f78d0508640 1 -- 172.21.15.19:0/1295687929 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c819ddf0 con 0x7f78c81021a0 2024-04-23T18:26:53.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.119+0000 7f78bf7fe640 1 -- 172.21.15.19:0/1295687929 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4010690 con 0x7f78c81021a0 2024-04-23T18:26:53.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.119+0000 7f78d0508640 1 -- 172.21.15.19:0/1295687929 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c80744a0 con 0x7f78c81021a0 2024-04-23T18:26:53.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.120+0000 7f78d0508640 1 -- 172.21.15.19:0/1295687929 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c8068d70 con 0x7f78c81021a0 2024-04-23T18:26:53.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.121+0000 7f78bf7fe640 1 -- 172.21.15.19:0/1295687929 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f78c4004e90 con 0x7f78c81021a0 2024-04-23T18:26:53.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.122+0000 7f78bf7fe640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f78b40a3ec0 0x7f78b40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.122+0000 7f78bf7fe640 1 -- 172.21.15.19:0/1295687929 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f78c40d0f30 con 0x7f78c81021a0 2024-04-23T18:26:53.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.122+0000 7f78cdaa3640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f78b40a3ec0 0x7f78b40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.122+0000 7f78cdaa3640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f78b40a3ec0 0x7f78b40a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78c8101a80 tx=0x7f78b8005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:53.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.127+0000 7f78bf7fe640 1 -- 172.21.15.19:0/1295687929 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78c409d350 con 0x7f78c81021a0 2024-04-23T18:26:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.357+0000 7f78d0508640 1 -- 172.21.15.19:0/1295687929 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78c8199aa0 con 0x7f78c81021a0 2024-04-23T18:26:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.358+0000 7f78bf7fe640 1 -- 172.21.15.19:0/1295687929 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f78c409cc50 con 0x7f78c81021a0 2024-04-23T18:26:53.351 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:26:53.351 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:26:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:26:53.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 -- 172.21.15.19:0/1295687929 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f78b40a3ec0 msgr2=0x7f78b40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f78b40a3ec0 0x7f78b40a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78c8101a80 tx=0x7f78b8005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:53.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 -- 172.21.15.19:0/1295687929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c81021a0 msgr2=0x7f78c81987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c81021a0 0x7f78c81987a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f78c400e9f0 tx=0x7f78c400edf0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 -- 172.21.15.19:0/1295687929 shutdown_connections 2024-04-23T18:26:53.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f78b40a3ec0 0x7f78b40a6380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c819d010 0x7f78c819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c8107270 0x7f78c8198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 --2- 172.21.15.19:0/1295687929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c81021a0 0x7f78c81987a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 -- 172.21.15.19:0/1295687929 >> 172.21.15.19:0/1295687929 conn(0x7f78c8079350 msgr2=0x7f78c8100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:53.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 -- 172.21.15.19:0/1295687929 shutdown_connections 2024-04-23T18:26:53.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.361+0000 7f78d0508640 1 -- 172.21.15.19:0/1295687929 wait complete. 2024-04-23T18:26:53.370 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 78.5 --await-for 42.0' 2024-04-23T18:26:53.370 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 78.5 --await-for 42.0 2024-04-23T18:26:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.494+0000 7fd4913ad640 1 -- 172.21.15.19:0/913807287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48c06a5d0 msgr2=0x7fd48c06a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.494+0000 7fd4913ad640 1 --2- 172.21.15.19:0/913807287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48c06a5d0 0x7fd48c06a9b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009900 tx=0x7fd47c02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.494+0000 7fd4913ad640 1 -- 172.21.15.19:0/913807287 shutdown_connections 2024-04-23T18:26:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.494+0000 7fd4913ad640 1 --2- 172.21.15.19:0/913807287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd48c06aef0 0x7fd48c10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.494+0000 7fd4913ad640 1 --2- 172.21.15.19:0/913807287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd48c10e960 0x7fd48c110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.494+0000 7fd4913ad640 1 --2- 172.21.15.19:0/913807287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48c06a5d0 0x7fd48c06a9b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.494+0000 7fd4913ad640 1 -- 172.21.15.19:0/913807287 >> 172.21.15.19:0/913807287 conn(0x7fd48c077f20 msgr2=0x7fd48c078330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.494+0000 7fd4913ad640 1 -- 172.21.15.19:0/913807287 shutdown_connections 2024-04-23T18:26:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.494+0000 7fd4913ad640 1 -- 172.21.15.19:0/913807287 wait complete. 2024-04-23T18:26:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.495+0000 7fd4913ad640 1 Processor -- start 2024-04-23T18:26:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.495+0000 7fd4913ad640 1 -- start start 2024-04-23T18:26:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.495+0000 7fd4913ad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd48c06a5d0 0x7fd48c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.495+0000 7fd4913ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48c06aef0 0x7fd48c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.495+0000 7fd4913ad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd48c10e960 0x7fd48c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.495+0000 7fd483fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48c06aef0 0x7fd48c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.495+0000 7fd483fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48c06aef0 0x7fd48c1a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59226/0 (socket says 172.21.15.19:59226) 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.495+0000 7fd483fff640 1 -- 172.21.15.19:0/4260977508 learned_addr learned my addr 172.21.15.19:0/4260977508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.495+0000 7fd48affd640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd48c06a5d0 0x7fd48c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.495+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1143e0 con 0x7fd48c06aef0 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c114260 con 0x7fd48c10e960 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c114560 con 0x7fd48c06a5d0 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd48c10e960 0x7fd48c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd483fff640 1 -- 172.21.15.19:0/4260977508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd48c06a5d0 msgr2=0x7fd48c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd483fff640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd48c06a5d0 0x7fd48c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd483fff640 1 -- 172.21.15.19:0/4260977508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd48c10e960 msgr2=0x7fd48c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd483fff640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd48c10e960 0x7fd48c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd483fff640 1 -- 172.21.15.19:0/4260977508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c19af50 con 0x7fd48c06aef0 2024-04-23T18:26:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd48c10e960 0x7fd48c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd483fff640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48c06aef0 0x7fd48c1a0b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd47400e9f0 tx=0x7fd47400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd488ff9640 1 -- 172.21.15.19:0/4260977508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47400cd80 con 0x7fd48c06aef0 2024-04-23T18:26:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c19b240 con 0x7fd48c06aef0 2024-04-23T18:26:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.496+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a7460 con 0x7fd48c06aef0 2024-04-23T18:26:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.497+0000 7fd488ff9640 1 -- 172.21.15.19:0/4260977508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4740108e0 con 0x7fd48c06aef0 2024-04-23T18:26:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.497+0000 7fd488ff9640 1 -- 172.21.15.19:0/4260977508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474018980 con 0x7fd48c06aef0 2024-04-23T18:26:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.499+0000 7fd488ff9640 1 -- 172.21.15.19:0/4260977508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd474010430 con 0x7fd48c06aef0 2024-04-23T18:26:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.499+0000 7fd4941a3740 1 Processor -- start 2024-04-23T18:26:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.499+0000 7fd4941a3740 1 -- start start 2024-04-23T18:26:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f40a0 0x55aa092f4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd488ff9640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4780a3ec0 0x7fd4780a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f49c0 0x55aa092feb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092ff090 0x55aa09301480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd4941a3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55aa09259370 con 0x55aa092f40a0 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd4941a3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55aa091f62e0 con 0x55aa092f49c0 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd4941a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aa09255330 con 0x55aa092ff090 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f40a0 0x55aa092f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f40a0 0x55aa092f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59238/0 (socket says 172.21.15.19:59238) 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd48b7fe640 1 -- 172.21.15.19:0/1498144190 learned_addr learned my addr 172.21.15.19:0/1498144190 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd488ff9640 1 -- 172.21.15.19:0/4260977508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd4740d0d40 con 0x7fd48c06aef0 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd483fff640 1 --2- 172.21.15.19:0/1498144190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f49c0 0x55aa092feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd48b7fe640 1 -- 172.21.15.19:0/1498144190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092ff090 msgr2=0x55aa09301480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd48affd640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4780a3ec0 0x7fd4780a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/1498144190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092ff090 0x55aa09301480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd48b7fe640 1 -- 172.21.15.19:0/1498144190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f49c0 msgr2=0x55aa092feb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/1498144190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f49c0 0x55aa092feb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.500+0000 7fd48b7fe640 1 -- 172.21.15.19:0/1498144190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa09037d80 con 0x55aa092f40a0 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.501+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/1498144190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f40a0 0x55aa092f4480 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd484012470 tx=0x7fd484012870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561d3df2db28a02c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.501+0000 7fd4817fa640 1 -- 172.21.15.19:0/1498144190 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48400f480 con 0x55aa092f40a0 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.501+0000 7fd48affd640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4780a3ec0 0x7fd4780a6380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd47c031440 tx=0x7fd47c005b50 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.501+0000 7fd4817fa640 1 -- 172.21.15.19:0/1498144190 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484016030 con 0x55aa092f40a0 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.501+0000 7fd4817fa640 1 -- 172.21.15.19:0/1498144190 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484010960 con 0x55aa092f40a0 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.501+0000 7fd4941a3740 1 -- 172.21.15.19:0/1498144190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f40a0 msgr2=0x55aa092f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 --2- 172.21.15.19:0/1498144190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f40a0 0x55aa092f4480 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd484012470 tx=0x7fd484012870 comp rx=0 tx=0).stop 2024-04-23T18:26:53.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 -- 172.21.15.19:0/1498144190 shutdown_connections 2024-04-23T18:26:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 --2- 172.21.15.19:0/1498144190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f49c0 0x55aa092feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 --2- 172.21.15.19:0/1498144190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092ff090 0x55aa09301480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 --2- 172.21.15.19:0/1498144190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f40a0 0x55aa092f4480 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 -- 172.21.15.19:0/1498144190 >> 172.21.15.19:0/1498144190 conn(0x55aa092f03c0 msgr2=0x55aa092f07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 -- 172.21.15.19:0/1498144190 shutdown_connections 2024-04-23T18:26:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 -- 172.21.15.19:0/1498144190 wait complete. 2024-04-23T18:26:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 -- start start 2024-04-23T18:26:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 Processor -- start 2024-04-23T18:26:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f40a0 0x55aa092fc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f49c0 0x55aa092fc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa092fca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55aa09301e80 con 0x55aa092ff090 2024-04-23T18:26:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55aa091f3fb0 con 0x55aa092f49c0 2024-04-23T18:26:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.502+0000 7fd4941a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aa09302000 con 0x55aa092f40a0 2024-04-23T18:26:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd4941a3740 10 client.0 _ll_register_callbacks cb 0x55aa091f39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa092fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa092fca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59246/0 (socket says 172.21.15.19:59246) 2024-04-23T18:26:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd48b7fe640 1 -- 172.21.15.19:0/2341465463 learned_addr learned my addr 172.21.15.19:0/2341465463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd483fff640 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f49c0 0x55aa092fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd48affd640 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f40a0 0x55aa092fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd48b7fe640 1 -- 172.21.15.19:0/2341465463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f40a0 msgr2=0x55aa092fc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f40a0 0x55aa092fc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd48b7fe640 1 -- 172.21.15.19:0/2341465463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f49c0 msgr2=0x55aa092fc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f49c0 0x55aa092fc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd48b7fe640 1 -- 172.21.15.19:0/2341465463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa09098510 con 0x55aa092ff090 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa092fca90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd484012440 tx=0x7fd484010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccfd2bd4ceb9b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd4727fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd4727fc640 1 -- 172.21.15.19:0/2341465463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484004c40 con 0x55aa092ff090 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd4727fc640 1 -- 172.21.15.19:0/2341465463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd484004de0 con 0x55aa092ff090 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.503+0000 7fd4941a3740 1 -- 172.21.15.19:0/2341465463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd484016030 con 0x55aa092ff090 2024-04-23T18:26:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4727fc640 1 -- 172.21.15.19:0/2341465463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd48402a050 con 0x55aa092ff090 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4941a3740 10 client.15741 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4941a3740 1 -- 172.21.15.19:0/2341465463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa08f2b670 con 0x55aa092ff090 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4727fc640 1 -- 172.21.15.19:0/2341465463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484016030 con 0x55aa092ff090 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4727fc640 1 -- 172.21.15.19:0/2341465463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd48400f480 con 0x55aa092ff090 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4941a3740 10 client.15741 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4941a3740 10 client.15741 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4941a3740 20 client.15741 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55aa0930f3f0 0x55aa093117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4941a3740 4 client.15741 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.504+0000 7fd4941a3740 1 -- 172.21.15.19:0/2341465463 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa09266d90 con 0x55aa0930f3f0 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.505+0000 7fd48affd640 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55aa0930f3f0 0x55aa093117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.505+0000 7fd4727fc640 10 client.15741 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.505+0000 7fd48affd640 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55aa0930f3f0 0x55aa093117e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eeb072ab49b74cc server_cookie=dbcd84753433dc6b in_seq=0 out_seq=0 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4727fc640 1 -- 172.21.15.19:0/2341465463 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa09266d90 con 0x55aa0930f3f0 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4727fc640 10 client.15741 handle_command_reply: tid=0 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 client.15741 shutdown 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 -- 172.21.15.19:0/2341465463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 msgr2=0x55aa092fca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa092fca90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd484012440 tx=0x7fd484010d00 comp rx=0 tx=0).stop 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 -- 172.21.15.19:0/2341465463 shutdown_connections 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55aa0930f3f0 0x55aa093117e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f49c0 0x55aa092fc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4727fc640 0 client.15741 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa092fca90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2341465463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f40a0 0x55aa092fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 -- 172.21.15.19:0/2341465463 >> 172.21.15.19:0/2341465463 conn(0x55aa092f03c0 msgr2=0x55aa092f60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:53.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 -- 172.21.15.19:0/2341465463 shutdown_connections 2024-04-23T18:26:53.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 1 -- 172.21.15.19:0/2341465463 wait complete. 2024-04-23T18:26:53.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.506+0000 7fd4941a3740 20 client.15741 trim_cache size 0 max 16384 2024-04-23T18:26:53.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:53 smithi019 ceph-mon[29215]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T18:26:53.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:53 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3765189991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.524+0000 7fd4941a3740 1 Processor -- start 2024-04-23T18:26:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.524+0000 7fd4941a3740 1 -- start start 2024-04-23T18:26:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.524+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f40a0 0x55aa092fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.524+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f49c0 0x55aa092ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.524+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa09317000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.524+0000 7fd4941a3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c00e680 con 0x55aa092ff090 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.524+0000 7fd4941a3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c0101c0 con 0x55aa092f40a0 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.524+0000 7fd4941a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c00ed50 con 0x55aa092f49c0 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa09317000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa09317000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59258/0 (socket says 172.21.15.19:59258) 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd48b7fe640 1 -- 172.21.15.19:0/669674929 learned_addr learned my addr 172.21.15.19:0/669674929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd48affd640 1 --2- 172.21.15.19:0/669674929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f40a0 0x55aa092fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd483fff640 1 --2- 172.21.15.19:0/669674929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f49c0 0x55aa092ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd48b7fe640 1 -- 172.21.15.19:0/669674929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f49c0 msgr2=0x55aa092ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/669674929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f49c0 0x55aa092ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd48b7fe640 1 -- 172.21.15.19:0/669674929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f40a0 msgr2=0x55aa092fc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/669674929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f40a0 0x55aa092fc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd48b7fe640 1 -- 172.21.15.19:0/669674929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa092ebca0 con 0x55aa092ff090 2024-04-23T18:26:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd483fff640 1 --2- 172.21.15.19:0/669674929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f49c0 0x55aa092ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/669674929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa09317000 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd4840128b0 tx=0x7fd484009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84276fcfe19fd394 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd480ff9640 1 -- 172.21.15.19:0/669674929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484009ce0 con 0x55aa092ff090 2024-04-23T18:26:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd480ff9640 1 -- 172.21.15.19:0/669674929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484016070 con 0x55aa092ff090 2024-04-23T18:26:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.525+0000 7fd480ff9640 1 -- 172.21.15.19:0/669674929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48400f480 con 0x55aa092ff090 2024-04-23T18:26:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.526+0000 7fd4941a3740 1 -- 172.21.15.19:0/669674929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 msgr2=0x55aa09317000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.526+0000 7fd4941a3740 1 --2- 172.21.15.19:0/669674929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa09317000 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd4840128b0 tx=0x7fd484009a40 comp rx=0 tx=0).stop 2024-04-23T18:26:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.526+0000 7fd4941a3740 1 -- 172.21.15.19:0/669674929 shutdown_connections 2024-04-23T18:26:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.526+0000 7fd4941a3740 1 --2- 172.21.15.19:0/669674929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092ff090 0x55aa09317000 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.526+0000 7fd4941a3740 1 --2- 172.21.15.19:0/669674929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f49c0 0x55aa092ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.526+0000 7fd4941a3740 1 --2- 172.21.15.19:0/669674929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092f40a0 0x55aa092fc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.526+0000 7fd4941a3740 1 -- 172.21.15.19:0/669674929 >> 172.21.15.19:0/669674929 conn(0x55aa092f03c0 msgr2=0x55aa092eed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.526+0000 7fd4941a3740 1 -- 172.21.15.19:0/669674929 shutdown_connections 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.526+0000 7fd4941a3740 1 -- 172.21.15.19:0/669674929 wait complete. 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd4941a3740 1 -- start start 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd4941a3740 1 Processor -- start 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f40a0 0x55aa093072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f49c0 0x55aa093077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd4941a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092ff090 0x55aa09304280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd4941a3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c00faf0 con 0x55aa092f49c0 2024-04-23T18:26:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd4941a3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c00f420 con 0x55aa092ff090 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd4941a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c00c660 con 0x55aa092f40a0 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd483fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f49c0 0x55aa093077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd483fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f49c0 0x55aa093077f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59268/0 (socket says 172.21.15.19:59268) 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd483fff640 1 -- 172.21.15.19:0/2012037098 learned_addr learned my addr 172.21.15.19:0/2012037098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd4941a3740 10 client.0 _ll_register_callbacks cb 0x7fd484010ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.527+0000 7fd48affd640 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f40a0 0x55aa093072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd48b7fe640 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092ff090 0x55aa09304280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd483fff640 1 -- 172.21.15.19:0/2012037098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f40a0 msgr2=0x55aa093072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd483fff640 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f40a0 0x55aa093072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd483fff640 1 -- 172.21.15.19:0/2012037098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092ff090 msgr2=0x55aa09304280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd483fff640 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092ff090 0x55aa09304280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd483fff640 1 -- 172.21.15.19:0/2012037098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa093049c0 con 0x55aa092f49c0 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd483fff640 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f49c0 0x55aa093077f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd47409ca50 tx=0x7fd47409c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775beb1209ddebe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd4727fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd4727fc640 1 -- 172.21.15.19:0/2012037098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740a0e70 con 0x55aa092f49c0 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd4727fc640 1 -- 172.21.15.19:0/2012037098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd4740ce620 con 0x55aa092f49c0 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd4941a3740 1 -- 172.21.15.19:0/2012037098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd484016070 con 0x55aa092f49c0 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd4727fc640 1 -- 172.21.15.19:0/2012037098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4740a5730 con 0x55aa092f49c0 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd4941a3740 10 client.15747 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.528+0000 7fd4941a3740 1 -- 172.21.15.19:0/2012037098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa093762a0 con 0x55aa092f49c0 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.529+0000 7fd4727fc640 1 -- 172.21.15.19:0/2012037098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484016070 con 0x55aa092f49c0 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.529+0000 7fd4727fc640 1 -- 172.21.15.19:0/2012037098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd474018b20 con 0x55aa092f49c0 2024-04-23T18:26:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.529+0000 7fd4941a3740 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.529+0000 7fd4941a3740 10 client.15747 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:26:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.529+0000 7fd4941a3740 20 client.15747 populate_metadata read hostname 'smithi019' 2024-04-23T18:26:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.529+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55aa0930f3f0 0x55aa09378aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.529+0000 7fd4941a3740 4 client.15747 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.5, "await_for": 42.0}] 2024-04-23T18:26:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.529+0000 7fd4941a3740 1 -- 172.21.15.19:0/2012037098 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.5, "await_for": 42.0}) -- 0x55aa09260c00 con 0x55aa0930f3f0 2024-04-23T18:26:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.529+0000 7fd48affd640 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55aa0930f3f0 0x55aa09378aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.530+0000 7fd4727fc640 10 client.15747 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.530+0000 7fd48affd640 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55aa0930f3f0 0x55aa09378aa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e683cea0177b87 server_cookie=229de5e1ce477e7d in_seq=0 out_seq=0 2024-04-23T18:26:53.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4727fc640 1 -- 172.21.15.19:0/2012037098 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55aa09260c00 con 0x55aa0930f3f0 2024-04-23T18:26:53.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4727fc640 10 client.15747 handle_command_reply: tid=0 2024-04-23T18:26:53.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 client.15747 shutdown 2024-04-23T18:26:53.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 -- 172.21.15.19:0/2012037098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f49c0 msgr2=0x55aa093077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f49c0 0x55aa093077f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd47409ca50 tx=0x7fd47409c9b0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 -- 172.21.15.19:0/2012037098 shutdown_connections 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55aa0930f3f0 0x55aa09378aa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4727fc640 0 client.15747 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa092ff090 0x55aa09304280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa092f49c0 0x55aa093077f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 --2- 172.21.15.19:0/2012037098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa092f40a0 0x55aa093072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 -- 172.21.15.19:0/2012037098 >> 172.21.15.19:0/2012037098 conn(0x55aa092f03c0 msgr2=0x55aa092ef600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 -- 172.21.15.19:0/2012037098 shutdown_connections 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 1 -- 172.21.15.19:0/2012037098 wait complete. 2024-04-23T18:26:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.538+0000 7fd4941a3740 20 client.15747 trim_cache size 0 max 16384 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 6, 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "1d0c8c92": { 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 6, 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:26:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 78.5, 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:53.534 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:53.535 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:53.535 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:26:53.535 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:26:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4780a3ec0 msgr2=0x7fd4780a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4780a3ec0 0x7fd4780a6380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd47c031440 tx=0x7fd47c005b50 comp rx=0 tx=0).stop 2024-04-23T18:26:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48c06aef0 msgr2=0x7fd48c1a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48c06aef0 0x7fd48c1a0b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd47400e9f0 tx=0x7fd47400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 shutdown_connections 2024-04-23T18:26:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4780a3ec0 0x7fd4780a6380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd48c10e960 0x7fd48c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48c06aef0 0x7fd48c1a0b50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 --2- 172.21.15.19:0/4260977508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd48c06a5d0 0x7fd48c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 >> 172.21.15.19:0/4260977508 conn(0x7fd48c077f20 msgr2=0x7fd48c10fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 shutdown_connections 2024-04-23T18:26:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:26:53.539+0000 7fd4913ad640 1 -- 172.21.15.19:0/4260977508 wait complete. 2024-04-23T18:26:53.543 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1d0c8c92', quiesce took 0.5 seconds. Will release after: 18.5 2024-04-23T18:26:53.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:53 smithi044 ceph-mon[32458]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T18:26:53.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:53 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3765189991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:53.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:53 smithi155 ceph-mon[33468]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T18:26:53.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:53 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3765189991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:54.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:54 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1295687929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:54.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:54 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1295687929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:54.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:54 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1295687929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:55.463 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:55 smithi044 ceph-mon[32458]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:26:55.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:55 smithi019 ceph-mon[29215]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:26:55.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:55 smithi155 ceph-mon[33468]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:26:57.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:57 smithi019 ceph-mon[29215]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:26:57.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:57 smithi044 ceph-mon[32458]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:26:57.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:57 smithi155 ceph-mon[33468]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:26:59.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:26:59 smithi019 ceph-mon[29215]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:26:59.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:26:59 smithi044 ceph-mon[32458]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:26:59.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:26:59 smithi155 ceph-mon[33468]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:27:00.510 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:27:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:27:00.044+0000 7f6ae6331700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:27:00.510 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:27:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:27:00.044+0000 7f6ae6331700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:27:00.510 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:27:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:27:00.044+0000 7f6ae6331700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:27:00.510 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:27:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:27:00.044+0000 7f6ae6331700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:27:00.510 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:27:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:27:00.044+0000 7f6ae6331700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:27:01.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:01 smithi019 ceph-mon[29215]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:27:01.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:01 smithi019 ceph-mon[29215]: mgrmap e25: x(active, since 52s), standbys: y, z 2024-04-23T18:27:01.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:01 smithi044 ceph-mon[32458]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:27:01.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:01 smithi044 ceph-mon[32458]: mgrmap e25: x(active, since 52s), standbys: y, z 2024-04-23T18:27:01.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:01 smithi155 ceph-mon[33468]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:27:01.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:01 smithi155 ceph-mon[33468]: mgrmap e25: x(active, since 52s), standbys: y, z 2024-04-23T18:27:02.625 INFO:tasks.workunit.client.0.smithi019.stderr:Note: switching to '57e0caf76fdb7ab8b1358588f292d08519163844'. 2024-04-23T18:27:02.625 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-23T18:27:02.625 INFO:tasks.workunit.client.0.smithi019.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-23T18:27:02.625 INFO:tasks.workunit.client.0.smithi019.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-23T18:27:02.625 INFO:tasks.workunit.client.0.smithi019.stderr:state without impacting any branches by switching back to a branch. 2024-04-23T18:27:02.625 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-23T18:27:02.625 INFO:tasks.workunit.client.0.smithi019.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-23T18:27:02.625 INFO:tasks.workunit.client.0.smithi019.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-23T18:27:02.626 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-23T18:27:02.626 INFO:tasks.workunit.client.0.smithi019.stderr: git switch -c 2024-04-23T18:27:02.626 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-23T18:27:02.626 INFO:tasks.workunit.client.0.smithi019.stderr:Or undo this operation with: 2024-04-23T18:27:02.626 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-23T18:27:02.626 INFO:tasks.workunit.client.0.smithi019.stderr: git switch - 2024-04-23T18:27:02.626 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-23T18:27:02.626 INFO:tasks.workunit.client.0.smithi019.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-23T18:27:02.626 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-04-23T18:27:02.626 INFO:tasks.workunit.client.0.smithi019.stderr:HEAD is now at 57e0caf76fd qa/suites/fs: add quiescer to the fs suite 2024-04-23T18:27:02.627 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-23T18:27:02.672 INFO:tasks.workunit.client.0.smithi019.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-23T18:27:02.675 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T18:27:02.675 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-23T18:27:03.320 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-23T18:27:03.376 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-23T18:27:03.417 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T18:27:03.418 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T18:27:03.418 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-23T18:27:03.459 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T18:27:03.463 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:27:03.463 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-23T18:27:03.478 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-23T18:27:03.479 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-23T18:27:03.479 DEBUG:teuthology.orchestra.run.smithi019:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-23T18:27:03.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:03 smithi019 ceph-mon[29215]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:27:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.546+0000 7ff68976f640 1 -- 172.21.15.19:0/2642984093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff684103b20 msgr2=0x7ff68410c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.546+0000 7ff68976f640 1 --2- 172.21.15.19:0/2642984093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff684103b20 0x7ff68410c000 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff678009900 tx=0x7ff67802a480 comp rx=0 tx=0).stop 2024-04-23T18:27:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.546+0000 7ff68976f640 1 -- 172.21.15.19:0/2642984093 shutdown_connections 2024-04-23T18:27:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.546+0000 7ff68976f640 1 --2- 172.21.15.19:0/2642984093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff68410c5d0 0x7ff68410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.546+0000 7ff68976f640 1 --2- 172.21.15.19:0/2642984093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff684103b20 0x7ff68410c000 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.546+0000 7ff68976f640 1 --2- 172.21.15.19:0/2642984093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684103200 0x7ff6841035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.546+0000 7ff68976f640 1 -- 172.21.15.19:0/2642984093 >> 172.21.15.19:0/2642984093 conn(0x7ff6840771d0 msgr2=0x7ff6840775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.546+0000 7ff68976f640 1 -- 172.21.15.19:0/2642984093 shutdown_connections 2024-04-23T18:27:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.546+0000 7ff68976f640 1 -- 172.21.15.19:0/2642984093 wait complete. 2024-04-23T18:27:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff68976f640 1 Processor -- start 2024-04-23T18:27:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff68976f640 1 -- start start 2024-04-23T18:27:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff68976f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff684103200 0x7ff6841986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff68976f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684103b20 0x7ff684198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff68976f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff68410c5d0 0x7ff68419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff68976f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6841121f0 con 0x7ff68410c5d0 2024-04-23T18:27:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff68976f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff684112070 con 0x7ff684103b20 2024-04-23T18:27:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff68976f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff684112370 con 0x7ff684103200 2024-04-23T18:27:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff6837fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff68410c5d0 0x7ff68419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff6837fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff68410c5d0 0x7ff68419cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35126/0 (socket says 172.21.15.19:35126) 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff6837fe640 1 -- 172.21.15.19:0/1318513583 learned_addr learned my addr 172.21.15.19:0/1318513583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.547+0000 7ff682ffd640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff684103200 0x7ff6841986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff6827fc640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684103b20 0x7ff684198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff6837fe640 1 -- 172.21.15.19:0/1318513583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff684103200 msgr2=0x7ff6841986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff6837fe640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff684103200 0x7ff6841986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff6837fe640 1 -- 172.21.15.19:0/1318513583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684103b20 msgr2=0x7ff684198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff6837fe640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684103b20 0x7ff684198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff6837fe640 1 -- 172.21.15.19:0/1318513583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68419d6c0 con 0x7ff68410c5d0 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff6827fc640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684103b20 0x7ff684198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff6837fe640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff68410c5d0 0x7ff68419cf90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00cab0 tx=0x7ff67c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff66bfff640 1 -- 172.21.15.19:0/1318513583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00ae20 con 0x7ff68410c5d0 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff68976f640 1 -- 172.21.15.19:0/1318513583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68419d9b0 con 0x7ff68410c5d0 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff66bfff640 1 -- 172.21.15.19:0/1318513583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff67c00f900 con 0x7ff68410c5d0 2024-04-23T18:27:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff66bfff640 1 -- 172.21.15.19:0/1318513583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c018a50 con 0x7ff68410c5d0 2024-04-23T18:27:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.548+0000 7ff68976f640 1 -- 172.21.15.19:0/1318513583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6841a5240 con 0x7ff68410c5d0 2024-04-23T18:27:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.550+0000 7ff66bfff640 1 -- 172.21.15.19:0/1318513583 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7ff67c00f450 con 0x7ff68410c5d0 2024-04-23T18:27:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff68c567740 1 Processor -- start 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff68c567740 1 -- start start 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff66bfff640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff66c0a3f70 0x7ff66c0a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff68c567740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 0x56308db8fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff68c567740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8ff40 0x56308db9a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff68c567740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db9a800 0x56308db9cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff68c567740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56308daf4970 con 0x56308db8ff40 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff68c567740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56308da914a0 con 0x56308db9a800 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff68c567740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56308daf0930 con 0x56308db8f620 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6827fc640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff66c0a3f70 0x7ff66c0a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6827fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8ff40 0x56308db9a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff66bfff640 1 -- 172.21.15.19:0/1318513583 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff67c0d12a0 con 0x7ff68410c5d0 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6827fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8ff40 0x56308db9a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35142/0 (socket says 172.21.15.19:35142) 2024-04-23T18:27:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6827fc640 1 -- 172.21.15.19:0/637856783 learned_addr learned my addr 172.21.15.19:0/637856783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff682ffd640 1 --2- 172.21.15.19:0/637856783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 0x56308db8fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6837fe640 1 --2- 172.21.15.19:0/637856783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db9a800 0x56308db9cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6827fc640 1 -- 172.21.15.19:0/637856783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 msgr2=0x56308db8fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6827fc640 1 --2- 172.21.15.19:0/637856783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 0x56308db8fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6827fc640 1 -- 172.21.15.19:0/637856783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db9a800 msgr2=0x56308db9cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6827fc640 1 --2- 172.21.15.19:0/637856783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db9a800 0x56308db9cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6827fc640 1 -- 172.21.15.19:0/637856783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308d807210 con 0x56308db8ff40 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.551+0000 7ff6837fe640 1 --2- 172.21.15.19:0/637856783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db9a800 0x56308db9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff6827fc640 1 --2- 172.21.15.19:0/637856783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8ff40 0x56308db9a2c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff678031a90 tx=0x7ff678004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc101fe43cfe45b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff6827fc640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff66c0a3f70 0x7ff66c0a6430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff6780040b0 tx=0x7ff678037000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff6697fa640 1 -- 172.21.15.19:0/637856783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678035040 con 0x56308db8ff40 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff6697fa640 1 -- 172.21.15.19:0/637856783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff678004440 con 0x56308db8ff40 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff6697fa640 1 -- 172.21.15.19:0/637856783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678005020 con 0x56308db8ff40 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff68c567740 1 -- 172.21.15.19:0/637856783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8ff40 msgr2=0x56308db9a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff68c567740 1 --2- 172.21.15.19:0/637856783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8ff40 0x56308db9a2c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff678031a90 tx=0x7ff678004040 comp rx=0 tx=0).stop 2024-04-23T18:27:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff68c567740 1 -- 172.21.15.19:0/637856783 shutdown_connections 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff68c567740 1 --2- 172.21.15.19:0/637856783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db9a800 0x56308db9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff68c567740 1 --2- 172.21.15.19:0/637856783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8ff40 0x56308db9a2c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff68c567740 1 --2- 172.21.15.19:0/637856783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 0x56308db8fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff68c567740 1 -- 172.21.15.19:0/637856783 >> 172.21.15.19:0/637856783 conn(0x56308db8bb30 msgr2=0x56308db8bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff68c567740 1 -- 172.21.15.19:0/637856783 shutdown_connections 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.552+0000 7ff68c567740 1 -- 172.21.15.19:0/637856783 wait complete. 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff68c567740 1 -- start start 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff68c567740 1 Processor -- start 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff68c567740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308db9a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff68c567740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308db97490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff68c567740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 0x56308db97a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff68c567740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56308db9d3d0 con 0x56308db8f620 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff68c567740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56308da8f0b0 con 0x56308db8ff40 2024-04-23T18:27:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff68c567740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56308db9d550 con 0x56308db9a800 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff68c567740 10 client.0 _ll_register_callbacks cb 0x56308da7cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff682ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308db9a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff682ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308db9a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35150/0 (socket says 172.21.15.19:35150) 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff682ffd640 1 -- 172.21.15.19:0/595705675 learned_addr learned my addr 172.21.15.19:0/595705675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff6837fe640 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 0x56308db97a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff682ffd640 1 -- 172.21.15.19:0/595705675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 msgr2=0x56308db97a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff682ffd640 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 0x56308db97a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff682ffd640 1 -- 172.21.15.19:0/595705675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 msgr2=0x56308db97490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff682ffd640 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308db97490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff682ffd640 1 -- 172.21.15.19:0/595705675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308d8d3620 con 0x56308db8f620 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.553+0000 7ff6837fe640 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 0x56308db97a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff682ffd640 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308db9a1a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff674012750 tx=0x7ff674012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9a5e116ec2e4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff666ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff666ff5640 1 -- 172.21.15.19:0/595705675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67401f060 con 0x56308db8f620 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff666ff5640 1 -- 172.21.15.19:0/595705675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff674010dc0 con 0x56308db8f620 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff68c567740 1 -- 172.21.15.19:0/595705675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff678004440 con 0x56308db8f620 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff666ff5640 1 -- 172.21.15.19:0/595705675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff674012e60 con 0x56308db8f620 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff68c567740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff68c567740 1 -- 172.21.15.19:0/595705675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56308dba9310 con 0x56308db8f620 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff666ff5640 1 -- 172.21.15.19:0/595705675 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff678004440 con 0x56308db8f620 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff666ff5640 1 -- 172.21.15.19:0/595705675 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff6740224a0 con 0x56308db8f620 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff68c567740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff68c567740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff68c567740 20 client.15765 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff68c567740 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56308dbaa7d0 0x56308dbacc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff68c567740 4 client.15765 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.554+0000 7ff68c567740 1 -- 172.21.15.19:0/595705675 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56308db02390 con 0x56308dbaa7d0 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.555+0000 7ff6827fc640 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56308dbaa7d0 0x56308dbacc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.555+0000 7ff6827fc640 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56308dbaa7d0 0x56308dbacc40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98db830983842595 server_cookie=49daa2793a583454 in_seq=0 out_seq=0 2024-04-23T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.555+0000 7ff666ff5640 10 client.15765 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff666ff5640 1 -- 172.21.15.19:0/595705675 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56308db02390 con 0x56308dbaa7d0 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff666ff5640 10 client.15765 handle_command_reply: tid=0 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 client.15765 shutdown 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 -- 172.21.15.19:0/595705675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 msgr2=0x56308db9a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308db9a1a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff674012750 tx=0x7ff674012c10 comp rx=0 tx=0).stop 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 -- 172.21.15.19:0/595705675 shutdown_connections 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56308dbaa7d0 0x56308dbacc40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308db97490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 0x56308db97a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff666ff5640 0 client.15765 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 --2- 172.21.15.19:0/595705675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308db9a1a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 -- 172.21.15.19:0/595705675 >> 172.21.15.19:0/595705675 conn(0x56308db8bb30 msgr2=0x56308db8acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 -- 172.21.15.19:0/595705675 shutdown_connections 2024-04-23T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 1 -- 172.21.15.19:0/595705675 wait complete. 2024-04-23T18:27:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.556+0000 7ff68c567740 20 client.15765 trim_cache size 0 max 16384 2024-04-23T18:27:03.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:03 smithi044 ceph-mon[32458]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:27:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff68c567740 1 Processor -- start 2024-04-23T18:27:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff68c567740 1 -- start start 2024-04-23T18:27:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff68c567740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 0x56308db97430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff68c567740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308dba2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff68c567740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db9a800 0x56308db99cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff68c567740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6841adb20 con 0x56308db9a800 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff68c567740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff6841af660 con 0x56308db8ff40 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff68c567740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff6841ae1f0 con 0x56308db8f620 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff6837fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db9a800 0x56308db99cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff6837fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db9a800 0x56308db99cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35156/0 (socket says 172.21.15.19:35156) 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.571+0000 7ff6837fe640 1 -- 172.21.15.19:0/4253250516 learned_addr learned my addr 172.21.15.19:0/4253250516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff682ffd640 1 --2- 172.21.15.19:0/4253250516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 0x56308db97430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff6827fc640 1 --2- 172.21.15.19:0/4253250516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308dba2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff6837fe640 1 -- 172.21.15.19:0/4253250516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 msgr2=0x56308db97430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff6837fe640 1 --2- 172.21.15.19:0/4253250516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 0x56308db97430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff6837fe640 1 -- 172.21.15.19:0/4253250516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 msgr2=0x56308dba2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff6837fe640 1 --2- 172.21.15.19:0/4253250516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308dba2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff6837fe640 1 -- 172.21.15.19:0/4253250516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308db959e0 con 0x56308db9a800 2024-04-23T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff682ffd640 1 --2- 172.21.15.19:0/4253250516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 0x56308db97430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff6837fe640 1 --2- 172.21.15.19:0/4253250516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db9a800 0x56308db99cd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0184b0 tx=0x7ff67c09cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4f6270b854c4fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff668ff9640 1 -- 172.21.15.19:0/4253250516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c09d030 con 0x56308db9a800 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff668ff9640 1 -- 172.21.15.19:0/4253250516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff67c0a13e0 con 0x56308db9a800 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff668ff9640 1 -- 172.21.15.19:0/4253250516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00fab0 con 0x56308db9a800 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff68c567740 1 -- 172.21.15.19:0/4253250516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db9a800 msgr2=0x56308db99cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.572+0000 7ff68c567740 1 --2- 172.21.15.19:0/4253250516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db9a800 0x56308db99cd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0184b0 tx=0x7ff67c09cf30 comp rx=0 tx=0).stop 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 -- 172.21.15.19:0/4253250516 shutdown_connections 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 --2- 172.21.15.19:0/4253250516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308dba2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 --2- 172.21.15.19:0/4253250516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db9a800 0x56308db99cd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 --2- 172.21.15.19:0/4253250516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db8f620 0x56308db97430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 -- 172.21.15.19:0/4253250516 >> 172.21.15.19:0/4253250516 conn(0x56308db8bb30 msgr2=0x56308db96d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 -- 172.21.15.19:0/4253250516 shutdown_connections 2024-04-23T18:27:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 -- 172.21.15.19:0/4253250516 wait complete. 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 -- start start 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 Processor -- start 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308dc08d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308dc092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 0x56308dc09800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6841aef90 con 0x56308db8f620 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff6841ae8c0 con 0x56308db8ff40 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff6841abb00 con 0x56308db9a800 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff68c567740 10 client.0 _ll_register_callbacks cb 0x7ff67400af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff682ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308dc08d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff682ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308dc08d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35166/0 (socket says 172.21.15.19:35166) 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.573+0000 7ff682ffd640 1 -- 172.21.15.19:0/968745896 learned_addr learned my addr 172.21.15.19:0/968745896 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff6827fc640 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308dc092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff682ffd640 1 -- 172.21.15.19:0/968745896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 msgr2=0x56308dc09800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff6837fe640 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 0x56308dc09800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff682ffd640 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 0x56308dc09800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff682ffd640 1 -- 172.21.15.19:0/968745896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 msgr2=0x56308dc092c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff682ffd640 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308dc092c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff682ffd640 1 -- 172.21.15.19:0/968745896 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308dc13f80 con 0x56308db8f620 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff6827fc640 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308dc092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff682ffd640 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308dc08d80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff67400ac40 tx=0x7ff674010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cd10ca50e0e14a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff666ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff666ff5640 1 -- 172.21.15.19:0/968745896 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674005040 con 0x56308db8f620 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff666ff5640 1 -- 172.21.15.19:0/968745896 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff6740051e0 con 0x56308db8f620 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff68c567740 1 -- 172.21.15.19:0/968745896 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff67c0a13e0 con 0x56308db8f620 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff666ff5640 1 -- 172.21.15.19:0/968745896 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff67401f030 con 0x56308db8f620 2024-04-23T18:27:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff68c567740 10 client.15771 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff68c567740 1 -- 172.21.15.19:0/968745896 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56308dc15200 con 0x56308db8f620 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff666ff5640 1 -- 172.21.15.19:0/968745896 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff67c0a13e0 con 0x56308db8f620 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff666ff5640 1 -- 172.21.15.19:0/968745896 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff674024480 con 0x56308db8f620 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff68c567740 10 client.15771 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff68c567740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff68c567740 20 client.15771 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff68c567740 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56308dbaa7d0 0x56308dc179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff68c567740 4 client.15771 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.574+0000 7ff68c567740 1 -- 172.21.15.19:0/968745896 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56308dafc200 con 0x56308dbaa7d0 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.575+0000 7ff6827fc640 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56308dbaa7d0 0x56308dc179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.575+0000 7ff6827fc640 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56308dbaa7d0 0x56308dc179e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe9cc8cbd425eae server_cookie=bd39526987c09e48 in_seq=0 out_seq=0 2024-04-23T18:27:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.575+0000 7ff666ff5640 10 client.15771 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:03.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:03 smithi155 ceph-mon[33468]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:27:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff666ff5640 1 -- 172.21.15.19:0/968745896 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56308dafc200 con 0x56308dbaa7d0 2024-04-23T18:27:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff666ff5640 10 client.15771 handle_command_reply: tid=0 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff68c567740 1 client.15771 shutdown 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff68c567740 1 -- 172.21.15.19:0/968745896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 msgr2=0x56308dc08d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff68c567740 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308dc08d80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff67400ac40 tx=0x7ff674010dd0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff68c567740 1 -- 172.21.15.19:0/968745896 shutdown_connections 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff68c567740 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56308dbaa7d0 0x56308dc179e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff68c567740 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56308db8ff40 0x56308dc092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff68c567740 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56308db9a800 0x56308dc09800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff68c567740 1 --2- 172.21.15.19:0/968745896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56308db8f620 0x56308dc08d80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff68c567740 1 -- 172.21.15.19:0/968745896 >> 172.21.15.19:0/968745896 conn(0x56308db8bb30 msgr2=0x56308db8a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff666ff5640 0 client.15771 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.582+0000 7ff68c567740 1 -- 172.21.15.19:0/968745896 shutdown_connections 2024-04-23T18:27:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68c567740 1 -- 172.21.15.19:0/968745896 wait complete. 2024-04-23T18:27:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68c567740 20 client.15771 trim_cache size 0 max 16384 2024-04-23T18:27:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:27:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:27:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:27:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:27:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 -- 172.21.15.19:0/1318513583 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff66c0a3f70 msgr2=0x7ff66c0a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff66c0a3f70 0x7ff66c0a6430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff6780040b0 tx=0x7ff678037000 comp rx=0 tx=0).stop 2024-04-23T18:27:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 -- 172.21.15.19:0/1318513583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff68410c5d0 msgr2=0x7ff68419cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff68410c5d0 0x7ff68419cf90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00cab0 tx=0x7ff67c00cf70 comp rx=0 tx=0).stop 2024-04-23T18:27:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 -- 172.21.15.19:0/1318513583 shutdown_connections 2024-04-23T18:27:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff66c0a3f70 0x7ff66c0a6430 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684103b20 0x7ff684198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff68410c5d0 0x7ff68419cf90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 --2- 172.21.15.19:0/1318513583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff684103200 0x7ff6841986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 -- 172.21.15.19:0/1318513583 >> 172.21.15.19:0/1318513583 conn(0x7ff6840771d0 msgr2=0x7ff6840fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 -- 172.21.15.19:0/1318513583 shutdown_connections 2024-04-23T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:03.583+0000 7ff68976f640 1 -- 172.21.15.19:0/1318513583 wait complete. 2024-04-23T18:27:05.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:05 smithi019 ceph-mon[29215]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:27:05.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:05 smithi044 ceph-mon[32458]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:27:05.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:05 smithi155 ceph-mon[33468]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:27:07.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:07 smithi019 ceph-mon[29215]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-23T18:27:07.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:07 smithi044 ceph-mon[32458]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-23T18:27:07.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:07 smithi155 ceph-mon[33468]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-23T18:27:09.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:09 smithi019 ceph-mon[29215]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:27:09.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:09 smithi044 ceph-mon[32458]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:27:09.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:09 smithi155 ceph-mon[33468]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:27:11.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:11 smithi019 ceph-mon[29215]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-04-23T18:27:11.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:11 smithi044 ceph-mon[32458]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-04-23T18:27:11.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:11 smithi155 ceph-mon[33468]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-04-23T18:27:12.044 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:27:12.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.168+0000 7f32714d0640 1 -- 172.21.15.19:0/705875181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f326c101ac0 msgr2=0x7f326c105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.168+0000 7f32714d0640 1 --2- 172.21.15.19:0/705875181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f326c101ac0 0x7f326c105ab0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3260009900 tx=0x7f326002a410 comp rx=0 tx=0).stop 2024-04-23T18:27:12.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.168+0000 7f32714d0640 1 -- 172.21.15.19:0/705875181 shutdown_connections 2024-04-23T18:27:12.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.168+0000 7f32714d0640 1 --2- 172.21.15.19:0/705875181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c106190 0x7f326c10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.168+0000 7f32714d0640 1 --2- 172.21.15.19:0/705875181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f326c101ac0 0x7f326c105ab0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.168+0000 7f32714d0640 1 --2- 172.21.15.19:0/705875181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f326c101110 0x7f326c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.168+0000 7f32714d0640 1 -- 172.21.15.19:0/705875181 >> 172.21.15.19:0/705875181 conn(0x7f326c077fb0 msgr2=0x7f326c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:12.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.168+0000 7f32714d0640 1 -- 172.21.15.19:0/705875181 shutdown_connections 2024-04-23T18:27:12.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.168+0000 7f32714d0640 1 -- 172.21.15.19:0/705875181 wait complete. 2024-04-23T18:27:12.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.168+0000 7f32714d0640 1 Processor -- start 2024-04-23T18:27:12.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f32714d0640 1 -- start start 2024-04-23T18:27:12.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f32714d0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c101110 0x7f326c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f32714d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f326c106190 0x7f326c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f32714d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f326c198640 0x7f326c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f32714d0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f326c111150 con 0x7f326c198640 2024-04-23T18:27:12.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f32714d0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f326c110fd0 con 0x7f326c106190 2024-04-23T18:27:12.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f32714d0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f326c1112d0 con 0x7f326c101110 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f326c198640 0x7f326c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f326c198640 0x7f326c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35766/0 (socket says 172.21.15.19:35766) 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f326b7fe640 1 -- 172.21.15.19:0/1650415278 learned_addr learned my addr 172.21.15.19:0/1650415278 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f326a7fc640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f326c106190 0x7f326c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.169+0000 7f326affd640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c101110 0x7f326c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f326b7fe640 1 -- 172.21.15.19:0/1650415278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c101110 msgr2=0x7f326c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f326b7fe640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c101110 0x7f326c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f326b7fe640 1 -- 172.21.15.19:0/1650415278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f326c106190 msgr2=0x7f326c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f326b7fe640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f326c106190 0x7f326c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f326b7fe640 1 -- 172.21.15.19:0/1650415278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c199240 con 0x7f326c198640 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f326affd640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c101110 0x7f326c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f326b7fe640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f326c198640 0x7f326c198ac0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f326400b9e0 tx=0x7f326400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f3253fff640 1 -- 172.21.15.19:0/1650415278 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264019070 con 0x7f326c198640 2024-04-23T18:27:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f3253fff640 1 -- 172.21.15.19:0/1650415278 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3264014440 con 0x7f326c198640 2024-04-23T18:27:12.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f32714d0640 1 -- 172.21.15.19:0/1650415278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326c1a5320 con 0x7f326c198640 2024-04-23T18:27:12.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f3253fff640 1 -- 172.21.15.19:0/1650415278 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32640075d0 con 0x7f326c198640 2024-04-23T18:27:12.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.170+0000 7f32714d0640 1 -- 172.21.15.19:0/1650415278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326c1a56d0 con 0x7f326c198640 2024-04-23T18:27:12.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.171+0000 7f32714d0640 1 -- 172.21.15.19:0/1650415278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f326c102590 con 0x7f326c198640 2024-04-23T18:27:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.172+0000 7f3253fff640 1 -- 172.21.15.19:0/1650415278 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f326401c070 con 0x7f326c198640 2024-04-23T18:27:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.173+0000 7f3253fff640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32540a3ed0 0x7f32540a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.173+0000 7f3253fff640 1 -- 172.21.15.19:0/1650415278 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f32640cfe40 con 0x7f326c198640 2024-04-23T18:27:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.173+0000 7f326affd640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32540a3ed0 0x7f32540a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.173+0000 7f326affd640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32540a3ed0 0x7f32540a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f325c009b50 tx=0x7f325c005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:12.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.178+0000 7f3253fff640 1 -- 172.21.15.19:0/1650415278 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f326409bd90 con 0x7f326c198640 2024-04-23T18:27:12.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.405+0000 7f32714d0640 1 -- 172.21.15.19:0/1650415278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f326c101ac0 con 0x7f326c198640 2024-04-23T18:27:12.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.405+0000 7f3253fff640 1 -- 172.21.15.19:0/1650415278 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f32640a0090 con 0x7f326c198640 2024-04-23T18:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:27:12.399 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:27:12.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.407+0000 7f32714d0640 1 -- 172.21.15.19:0/1650415278 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32540a3ed0 msgr2=0x7f32540a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.407+0000 7f32714d0640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32540a3ed0 0x7f32540a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f325c009b50 tx=0x7f325c005f90 comp rx=0 tx=0).stop 2024-04-23T18:27:12.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.407+0000 7f32714d0640 1 -- 172.21.15.19:0/1650415278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f326c198640 msgr2=0x7f326c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.407+0000 7f32714d0640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f326c198640 0x7f326c198ac0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f326400b9e0 tx=0x7f326400bea0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.408+0000 7f32714d0640 1 -- 172.21.15.19:0/1650415278 shutdown_connections 2024-04-23T18:27:12.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.408+0000 7f32714d0640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32540a3ed0 0x7f32540a6390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.408+0000 7f32714d0640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f326c106190 0x7f326c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.408+0000 7f32714d0640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f326c198640 0x7f326c198ac0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.408+0000 7f32714d0640 1 --2- 172.21.15.19:0/1650415278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c101110 0x7f326c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.408+0000 7f32714d0640 1 -- 172.21.15.19:0/1650415278 >> 172.21.15.19:0/1650415278 conn(0x7f326c077fb0 msgr2=0x7f326c06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:12.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.408+0000 7f32714d0640 1 -- 172.21.15.19:0/1650415278 shutdown_connections 2024-04-23T18:27:12.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.408+0000 7f32714d0640 1 -- 172.21.15.19:0/1650415278 wait complete. 2024-04-23T18:27:12.411 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id 1d0c8c92 --release --await' 2024-04-23T18:27:12.412 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id 1d0c8c92 --release --await 2024-04-23T18:27:12.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.527+0000 7f959a4fa640 1 -- 172.21.15.19:0/3848588730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9594105e60 msgr2=0x7f9594106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.527+0000 7f959a4fa640 1 --2- 172.21.15.19:0/3848588730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9594105e60 0x7f9594106240 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9584009900 tx=0x7f958402a410 comp rx=0 tx=0).stop 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.527+0000 7f959a4fa640 1 -- 172.21.15.19:0/3848588730 shutdown_connections 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.527+0000 7f959a4fa640 1 --2- 172.21.15.19:0/3848588730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9594101580 0x7f9594101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.527+0000 7f959a4fa640 1 --2- 172.21.15.19:0/3848588730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9594101f40 0x7f959410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.527+0000 7f959a4fa640 1 --2- 172.21.15.19:0/3848588730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9594105e60 0x7f9594106240 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.527+0000 7f959a4fa640 1 -- 172.21.15.19:0/3848588730 >> 172.21.15.19:0/3848588730 conn(0x7f95940773e0 msgr2=0x7f95940fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.527+0000 7f959a4fa640 1 -- 172.21.15.19:0/3848588730 shutdown_connections 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.528+0000 7f959a4fa640 1 -- 172.21.15.19:0/3848588730 wait complete. 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.528+0000 7f959a4fa640 1 Processor -- start 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.528+0000 7f959a4fa640 1 -- start start 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.528+0000 7f959a4fa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9594101580 0x7f95941975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.528+0000 7f959a4fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9594101f40 0x7f9594197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.528+0000 7f959a4fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9594105e60 0x7f959419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.528+0000 7f959a4fa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95941110c0 con 0x7f9594101f40 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.528+0000 7f959a4fa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9594110f40 con 0x7f9594105e60 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.528+0000 7f959a4fa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9594111240 con 0x7f9594101580 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95937fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9594101f40 0x7f9594197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95937fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9594101f40 0x7f9594197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35778/0 (socket says 172.21.15.19:35778) 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95937fe640 1 -- 172.21.15.19:0/511398718 learned_addr learned my addr 172.21.15.19:0/511398718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f9593fff640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9594101580 0x7f95941975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f9598a97640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9594105e60 0x7f959419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95937fe640 1 -- 172.21.15.19:0/511398718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9594101580 msgr2=0x7f95941975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95937fe640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9594101580 0x7f95941975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95937fe640 1 -- 172.21.15.19:0/511398718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9594105e60 msgr2=0x7f959419be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95937fe640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9594105e60 0x7f959419be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95937fe640 1 -- 172.21.15.19:0/511398718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959419c590 con 0x7f9594101f40 2024-04-23T18:27:12.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95937fe640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9594101f40 0x7f9594197ae0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f957c00c890 tx=0x7f957c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:12.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95917fa640 1 -- 172.21.15.19:0/511398718 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c004220 con 0x7f9594101f40 2024-04-23T18:27:12.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f959a4fa640 1 -- 172.21.15.19:0/511398718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959419c880 con 0x7f9594101f40 2024-04-23T18:27:12.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95917fa640 1 -- 172.21.15.19:0/511398718 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f957c0043c0 con 0x7f9594101f40 2024-04-23T18:27:12.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f95917fa640 1 -- 172.21.15.19:0/511398718 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c004a00 con 0x7f9594101f40 2024-04-23T18:27:12.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.529+0000 7f959a4fa640 1 -- 172.21.15.19:0/511398718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95941a4110 con 0x7f9594101f40 2024-04-23T18:27:12.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f95917fa640 1 -- 172.21.15.19:0/511398718 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f957c0079b0 con 0x7f9594101f40 2024-04-23T18:27:12.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f959d211740 1 Processor -- start 2024-04-23T18:27:12.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f959d211740 1 -- start start 2024-04-23T18:27:12.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f959d211740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29bbe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f959d211740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bcb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f959d211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29bcda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f959d211740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cf29b236d0 con 0x55cf29bbeaf0 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f959d211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cf29ac01a0 con 0x55cf29bcb690 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f959d211740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf29b1f690 con 0x55cf29bbe140 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f95917fa640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f95800a3ed0 0x7f95800a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f9593fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bcb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f9593fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bcb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35792/0 (socket says 172.21.15.19:35792) 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f9593fff640 1 -- 172.21.15.19:0/2069208381 learned_addr learned my addr 172.21.15.19:0/2069208381 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f9598a97640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f95800a3ed0 0x7f95800a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f9598a97640 1 --2- 172.21.15.19:0/2069208381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29bbe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f95917fa640 1 -- 172.21.15.19:0/511398718 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f957c0d0750 con 0x7f9594101f40 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.532+0000 7f95937fe640 1 --2- 172.21.15.19:0/2069208381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29bcda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f9593fff640 1 -- 172.21.15.19:0/2069208381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 msgr2=0x55cf29bbe520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f9593fff640 1 --2- 172.21.15.19:0/2069208381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29bbe520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f9593fff640 1 -- 172.21.15.19:0/2069208381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 msgr2=0x55cf29bcda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f9593fff640 1 --2- 172.21.15.19:0/2069208381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29bcda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f9593fff640 1 -- 172.21.15.19:0/2069208381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf29aca5b0 con 0x55cf29bbeaf0 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f9598a97640 1 --2- 172.21.15.19:0/2069208381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29bbe520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f9598a97640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f95800a3ed0 0x7f95800a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f958c009fd0 tx=0x7f958c004ab0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f9593fff640 1 --2- 172.21.15.19:0/2069208381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bcb150 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9584009900 tx=0x7f9584004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fba3ea619e6c8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f958a7fc640 1 -- 172.21.15.19:0/2069208381 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584035040 con 0x55cf29bbeaf0 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f958a7fc640 1 -- 172.21.15.19:0/2069208381 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9584004740 con 0x55cf29bbeaf0 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.533+0000 7f958a7fc640 1 -- 172.21.15.19:0/2069208381 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584003850 con 0x55cf29bbeaf0 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 -- 172.21.15.19:0/2069208381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 msgr2=0x55cf29bcb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 --2- 172.21.15.19:0/2069208381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bcb150 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9584009900 tx=0x7f9584004310 comp rx=0 tx=0).stop 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 -- 172.21.15.19:0/2069208381 shutdown_connections 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 --2- 172.21.15.19:0/2069208381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29bcda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 --2- 172.21.15.19:0/2069208381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bcb150 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 --2- 172.21.15.19:0/2069208381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29bbe520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 -- 172.21.15.19:0/2069208381 >> 172.21.15.19:0/2069208381 conn(0x55cf29bba7b0 msgr2=0x55cf29bbab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 -- 172.21.15.19:0/2069208381 shutdown_connections 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 -- 172.21.15.19:0/2069208381 wait complete. 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 -- start start 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.534+0000 7f959d211740 1 Processor -- start 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f959d211740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29bc68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f959d211740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bc6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f959d211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29bd9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f959d211740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cf29bc1030 con 0x55cf29bbeaf0 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f959d211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cf29abdb20 con 0x55cf29bcb690 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f959d211740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf29bc11b0 con 0x55cf29bbe140 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f959d211740 10 client.0 _ll_register_callbacks cb 0x55cf29abd530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95937fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bc6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95937fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bc6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35800/0 (socket says 172.21.15.19:35800) 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95937fe640 1 -- 172.21.15.19:0/961653770 learned_addr learned my addr 172.21.15.19:0/961653770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95937fe640 1 -- 172.21.15.19:0/961653770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 msgr2=0x55cf29bc68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f9593fff640 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29bc68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95937fe640 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29bc68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95937fe640 1 -- 172.21.15.19:0/961653770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 msgr2=0x55cf29bd9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f9598a97640 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29bd9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95937fe640 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29bd9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95937fe640 1 -- 172.21.15.19:0/961653770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf29835920 con 0x55cf29bbeaf0 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95937fe640 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bc6de0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f957c098f60 tx=0x7f957c09c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1330892a6238bb1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95737fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95737fe640 1 -- 172.21.15.19:0/961653770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c01d040 con 0x55cf29bbeaf0 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f95737fe640 1 -- 172.21.15.19:0/961653770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f957c0982e0 con 0x55cf29bbeaf0 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.535+0000 7f959d211740 1 -- 172.21.15.19:0/961653770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9584004740 con 0x55cf29bbeaf0 2024-04-23T18:27:12.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f95737fe640 1 -- 172.21.15.19:0/961653770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f957c005290 con 0x55cf29bbeaf0 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f959d211740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f959d211740 1 -- 172.21.15.19:0/961653770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf29901d30 con 0x55cf29bbeaf0 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f95737fe640 1 -- 172.21.15.19:0/961653770 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9584004740 con 0x55cf29bbeaf0 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f95737fe640 1 -- 172.21.15.19:0/961653770 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f957c0984e0 con 0x55cf29bbeaf0 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f959d211740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f959d211740 10 client.15789 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f959d211740 20 client.15789 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f959d211740 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cf29bdd930 0x55cf29bdfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f959d211740 4 client.15789 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f959d211740 1 -- 172.21.15.19:0/961653770 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf29ae1500 con 0x55cf29bdd930 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.536+0000 7f9593fff640 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cf29bdd930 0x55cf29bdfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.537+0000 7f95737fe640 10 client.15789 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.537+0000 7f9593fff640 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cf29bdd930 0x55cf29bdfdb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d7d4f5e25ee02c server_cookie=d2cd951922bc4cf2 in_seq=0 out_seq=0 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.537+0000 7f95737fe640 1 -- 172.21.15.19:0/961653770 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf29ae1500 con 0x55cf29bdd930 2024-04-23T18:27:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.537+0000 7f95737fe640 10 client.15789 handle_command_reply: tid=0 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.537+0000 7f959d211740 1 client.15789 shutdown 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 1 -- 172.21.15.19:0/961653770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 msgr2=0x55cf29bc6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bc6de0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f957c098f60 tx=0x7f957c09c200 comp rx=0 tx=0).stop 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 1 -- 172.21.15.19:0/961653770 shutdown_connections 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cf29bdd930 0x55cf29bdfdb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29bd9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bc6de0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 1 --2- 172.21.15.19:0/961653770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29bc68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f95737fe640 0 client.15789 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 1 -- 172.21.15.19:0/961653770 >> 172.21.15.19:0/961653770 conn(0x55cf29bba7b0 msgr2=0x55cf29bc0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 1 -- 172.21.15.19:0/961653770 shutdown_connections 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 1 -- 172.21.15.19:0/961653770 wait complete. 2024-04-23T18:27:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.538+0000 7f959d211740 20 client.15789 trim_cache size 0 max 16384 2024-04-23T18:27:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f959d211740 1 Processor -- start 2024-04-23T18:27:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f959d211740 1 -- start start 2024-04-23T18:27:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f959d211740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29be5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f959d211740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f959d211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29be5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f959d211740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95941ab8a0 con 0x55cf29bbeaf0 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f959d211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95941ad3e0 con 0x55cf29bcb690 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f959d211740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f95941abf70 con 0x55cf29bbe140 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f95937fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f95937fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35804/0 (socket says 172.21.15.19:35804) 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f95937fe640 1 -- 172.21.15.19:0/1976695303 learned_addr learned my addr 172.21.15.19:0/1976695303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f9593fff640 1 --2- 172.21.15.19:0/1976695303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29be5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.555+0000 7f95937fe640 1 -- 172.21.15.19:0/1976695303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 msgr2=0x55cf29be5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f95937fe640 1 --2- 172.21.15.19:0/1976695303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29be5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f95937fe640 1 -- 172.21.15.19:0/1976695303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 msgr2=0x55cf29be5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f9598a97640 1 --2- 172.21.15.19:0/1976695303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29be5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f95937fe640 1 --2- 172.21.15.19:0/1976695303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29be5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f95937fe640 1 -- 172.21.15.19:0/1976695303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf29bbf8f0 con 0x55cf29bbeaf0 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f9598a97640 1 --2- 172.21.15.19:0/1976695303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29be5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f95937fe640 1 --2- 172.21.15.19:0/1976695303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd9630 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f957c09c520 tx=0x7f957c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ef6ca613d1b167 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f9589ffb640 1 -- 172.21.15.19:0/1976695303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c01d040 con 0x55cf29bbeaf0 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f9589ffb640 1 -- 172.21.15.19:0/1976695303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f957c0bb470 con 0x55cf29bbeaf0 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f9589ffb640 1 -- 172.21.15.19:0/1976695303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c0a0890 con 0x55cf29bbeaf0 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f959d211740 1 -- 172.21.15.19:0/1976695303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 msgr2=0x55cf29bd9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f959d211740 1 --2- 172.21.15.19:0/1976695303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd9630 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f957c09c520 tx=0x7f957c000bc0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f959d211740 1 -- 172.21.15.19:0/1976695303 shutdown_connections 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f959d211740 1 --2- 172.21.15.19:0/1976695303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bcb690 0x55cf29be5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f959d211740 1 --2- 172.21.15.19:0/1976695303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd9630 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f959d211740 1 --2- 172.21.15.19:0/1976695303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bbe140 0x55cf29be5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f959d211740 1 -- 172.21.15.19:0/1976695303 >> 172.21.15.19:0/1976695303 conn(0x55cf29bba7b0 msgr2=0x55cf29bc03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.556+0000 7f959d211740 1 -- 172.21.15.19:0/1976695303 shutdown_connections 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f959d211740 1 -- 172.21.15.19:0/1976695303 wait complete. 2024-04-23T18:27:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f959d211740 1 -- start start 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f959d211740 1 Processor -- start 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f959d211740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbe140 0x55cf29bd2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f959d211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f959d211740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bcb690 0x55cf29bd7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f959d211740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95941acd10 con 0x55cf29bbe140 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f959d211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95941ac640 con 0x55cf29bbeaf0 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f959d211740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f95941a9880 con 0x55cf29bcb690 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f959d211740 10 client.0 _ll_register_callbacks cb 0x7f957c09c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f9593fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbe140 0x55cf29bd2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f9593fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbe140 0x55cf29bd2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35814/0 (socket says 172.21.15.19:35814) 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f9593fff640 1 -- 172.21.15.19:0/2548245044 learned_addr learned my addr 172.21.15.19:0/2548245044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f9598a97640 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bcb690 0x55cf29bd7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.557+0000 7f95937fe640 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f9593fff640 1 -- 172.21.15.19:0/2548245044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bcb690 msgr2=0x55cf29bd7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f9593fff640 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bcb690 0x55cf29bd7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f9593fff640 1 -- 172.21.15.19:0/2548245044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bbeaf0 msgr2=0x55cf29bd33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f9593fff640 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f9593fff640 1 -- 172.21.15.19:0/2548245044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf29bd7ba0 con 0x55cf29bbe140 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f95937fe640 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f9593fff640 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbe140 0x55cf29bd2ea0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f958402aa00 tx=0x7f958402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af84d46bb52bd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f95737fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f95737fe640 1 -- 172.21.15.19:0/2548245044 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584055070 con 0x55cf29bbe140 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f95737fe640 1 -- 172.21.15.19:0/2548245044 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f9584005000 con 0x55cf29bbe140 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f959d211740 1 -- 172.21.15.19:0/2548245044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f957c0bb470 con 0x55cf29bbe140 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f95737fe640 1 -- 172.21.15.19:0/2548245044 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f958405e050 con 0x55cf29bbe140 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f959d211740 10 client.15795 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f959d211740 1 -- 172.21.15.19:0/2548245044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf29c44580 con 0x55cf29bbe140 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f95737fe640 1 -- 172.21.15.19:0/2548245044 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f957c0bb470 con 0x55cf29bbe140 2024-04-23T18:27:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.558+0000 7f95737fe640 1 -- 172.21.15.19:0/2548245044 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9584058020 con 0x55cf29bbe140 2024-04-23T18:27:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.559+0000 7f959d211740 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.559+0000 7f959d211740 10 client.15795 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:27:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.559+0000 7f959d211740 20 client.15795 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.559+0000 7f959d211740 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cf29c448c0 0x55cf29c46cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.559+0000 7f959d211740 4 client.15795 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1d0c8c92", "release": true, "await": true}] 2024-04-23T18:27:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.559+0000 7f959d211740 1 -- 172.21.15.19:0/2548245044 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1d0c8c92", "release": true, "await": true}) -- 0x55cf29b310f0 con 0x55cf29c448c0 2024-04-23T18:27:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.559+0000 7f95937fe640 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cf29c448c0 0x55cf29c46cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.559+0000 7f95937fe640 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cf29c448c0 0x55cf29c46cb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1bcdc9d657fd611 server_cookie=c82e3cd406816f1b in_seq=0 out_seq=0 2024-04-23T18:27:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.559+0000 7f95737fe640 10 client.15795 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:12.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f95737fe640 1 -- 172.21.15.19:0/2548245044 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55cf29b310f0 con 0x55cf29c448c0 2024-04-23T18:27:12.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f95737fe640 10 client.15795 handle_command_reply: tid=0 2024-04-23T18:27:12.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 client.15795 shutdown 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 -- 172.21.15.19:0/2548245044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbe140 msgr2=0x55cf29bd2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbe140 0x55cf29bd2ea0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f958402aa00 tx=0x7f958402a410 comp rx=0 tx=0).stop 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 -- 172.21.15.19:0/2548245044 shutdown_connections 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cf29c448c0 0x55cf29c46cb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf29bbeaf0 0x55cf29bd33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf29bcb690 0x55cf29bd7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f95737fe640 0 client.15795 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 --2- 172.21.15.19:0/2548245044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf29bbe140 0x55cf29bd2ea0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 -- 172.21.15.19:0/2548245044 >> 172.21.15.19:0/2548245044 conn(0x55cf29bba7b0 msgr2=0x55cf29bca970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 -- 172.21.15.19:0/2548245044 shutdown_connections 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 1 -- 172.21.15.19:0/2548245044 wait complete. 2024-04-23T18:27:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.564+0000 7f959d211740 20 client.15795 trim_cache size 0 max 16384 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 8, 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "1d0c8c92": { 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 8, 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 78.5, 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:27:12.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 -- 172.21.15.19:0/511398718 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f95800a3ed0 msgr2=0x7f95800a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f95800a3ed0 0x7f95800a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f958c009fd0 tx=0x7f958c004ab0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 -- 172.21.15.19:0/511398718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9594101f40 msgr2=0x7f9594197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9594101f40 0x7f9594197ae0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f957c00c890 tx=0x7f957c00cd50 comp rx=0 tx=0).stop 2024-04-23T18:27:12.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 -- 172.21.15.19:0/511398718 shutdown_connections 2024-04-23T18:27:12.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f95800a3ed0 0x7f95800a6390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9594105e60 0x7f959419be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9594101f40 0x7f9594197ae0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 --2- 172.21.15.19:0/511398718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9594101580 0x7f95941975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 -- 172.21.15.19:0/511398718 >> 172.21.15.19:0/511398718 conn(0x7f95940773e0 msgr2=0x7f9594103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:12.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 -- 172.21.15.19:0/511398718 shutdown_connections 2024-04-23T18:27:12.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:27:12.565+0000 7f959a4fa640 1 -- 172.21.15.19:0/511398718 wait complete. 2024-04-23T18:27:12.563 INFO:tasks.workunit.client.0.smithi019.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-23T18:27:12.564 INFO:tasks.workunit.client.0.smithi019.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-23T18:27:12.564 INFO:tasks.workunit.client.0.smithi019.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-23T18:27:12.569 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1d0c8c92', seconds elapsed: 19.6 2024-04-23T18:27:13.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:13 smithi019 ceph-mon[29215]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:27:13.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:13 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1650415278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:13.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:13 smithi044 ceph-mon[32458]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:27:13.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:13 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1650415278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:13.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:13 smithi155 ceph-mon[33468]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:27:13.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:13 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1650415278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:14.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:14 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:14.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:14 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:14 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:14 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:14 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:14 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:14 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:14 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:14.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:14 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:14 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:14 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:14 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:14 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:14 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:14 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:14.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:14 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:14 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:14 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:14 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:14 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:14.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:14 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:15.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:15 smithi019 ceph-mon[29215]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-23T18:27:15.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:15 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:15.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:15 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:15.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:15 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:15 smithi044 ceph-mon[32458]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-23T18:27:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:15 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:15 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:15 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:15 smithi155 ceph-mon[33468]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-23T18:27:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:15 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:15 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:15 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:27:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.683+0000 7f589fccf640 1 -- 172.21.15.19:0/1985890314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5898101aa0 msgr2=0x7f5898105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.683+0000 7f589fccf640 1 --2- 172.21.15.19:0/1985890314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5898101aa0 0x7f5898105a90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5888009900 tx=0x7f588802a410 comp rx=0 tx=0).stop 2024-04-23T18:27:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.683+0000 7f589fccf640 1 -- 172.21.15.19:0/1985890314 shutdown_connections 2024-04-23T18:27:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.683+0000 7f589fccf640 1 --2- 172.21.15.19:0/1985890314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898106170 0x7f589810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.683+0000 7f589fccf640 1 --2- 172.21.15.19:0/1985890314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5898101aa0 0x7f5898105a90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.683+0000 7f589fccf640 1 --2- 172.21.15.19:0/1985890314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58981010f0 0x7f58981014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.683+0000 7f589fccf640 1 -- 172.21.15.19:0/1985890314 >> 172.21.15.19:0/1985890314 conn(0x7f5898077fd0 msgr2=0x7f58980783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.683+0000 7f589fccf640 1 -- 172.21.15.19:0/1985890314 shutdown_connections 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.683+0000 7f589fccf640 1 -- 172.21.15.19:0/1985890314 wait complete. 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589fccf640 1 Processor -- start 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589fccf640 1 -- start start 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589fccf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58981010f0 0x7f5898198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589fccf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898106170 0x7f5898198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589fccf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f589819cfa0 0x7f589819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589fccf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58981121f0 con 0x7f58981010f0 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589fccf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5898112070 con 0x7f5898106170 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589fccf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5898112370 con 0x7f589819cfa0 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589da6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58981010f0 0x7f5898198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589da6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58981010f0 0x7f5898198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35840/0 (socket says 172.21.15.19:35840) 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589da6b640 1 -- 172.21.15.19:0/2715983673 learned_addr learned my addr 172.21.15.19:0/2715983673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589e26c640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f589819cfa0 0x7f589819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.684+0000 7f589d26a640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898106170 0x7f5898198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f589da6b640 1 -- 172.21.15.19:0/2715983673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f589819cfa0 msgr2=0x7f589819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f589da6b640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f589819cfa0 0x7f589819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f589da6b640 1 -- 172.21.15.19:0/2715983673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898106170 msgr2=0x7f5898198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f589da6b640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898106170 0x7f5898198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f589da6b640 1 -- 172.21.15.19:0/2715983673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589819da90 con 0x7f58981010f0 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f589da6b640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58981010f0 0x7f5898198730 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f589400d770 tx=0x7f589400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f588effd640 1 -- 172.21.15.19:0/2715983673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894017070 con 0x7f58981010f0 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f589fccf640 1 -- 172.21.15.19:0/2715983673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589819dd80 con 0x7f58981010f0 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f588effd640 1 -- 172.21.15.19:0/2715983673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58940054d0 con 0x7f58981010f0 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f588effd640 1 -- 172.21.15.19:0/2715983673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894004c30 con 0x7f58981010f0 2024-04-23T18:27:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.685+0000 7f589fccf640 1 -- 172.21.15.19:0/2715983673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58981a54f0 con 0x7f58981010f0 2024-04-23T18:27:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.688+0000 7f588effd640 1 -- 172.21.15.19:0/2715983673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5894004df0 con 0x7f58981010f0 2024-04-23T18:27:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.688+0000 7f58a29ee740 1 Processor -- start 2024-04-23T18:27:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.688+0000 7f58a29ee740 1 -- start start 2024-04-23T18:27:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.688+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 0x561c3615b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.688+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615be40 0x561c3616a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.688+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3616ac10 0x561c3616d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.688+0000 7f58a29ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561c360c0970 con 0x561c3616ac10 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.688+0000 7f58a29ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561c3605d4a0 con 0x561c3615be40 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.688+0000 7f58a29ee740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561c360bc930 con 0x561c3615b520 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589d26a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3616ac10 0x561c3616d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589e26c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 0x561c3615b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589d26a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3616ac10 0x561c3616d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35854/0 (socket says 172.21.15.19:35854) 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589d26a640 1 -- 172.21.15.19:0/4227298095 learned_addr learned my addr 172.21.15.19:0/4227298095 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f588effd640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58840a3f70 0x7f58840a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589da6b640 1 --2- 172.21.15.19:0/4227298095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615be40 0x561c3616a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f588effd640 1 -- 172.21.15.19:0/2715983673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f58940d04f0 con 0x7f58981010f0 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589e26c640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58840a3f70 0x7f58840a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589d26a640 1 -- 172.21.15.19:0/4227298095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 msgr2=0x561c3615b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589d26a640 1 --2- 172.21.15.19:0/4227298095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 0x561c3615b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589d26a640 1 -- 172.21.15.19:0/4227298095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615be40 msgr2=0x561c3616a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589d26a640 1 --2- 172.21.15.19:0/4227298095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615be40 0x561c3616a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589d26a640 1 -- 172.21.15.19:0/4227298095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c35dd3210 con 0x561c3616ac10 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589e26c640 1 --2- 172.21.15.19:0/4227298095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 0x561c3615b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589d26a640 1 --2- 172.21.15.19:0/4227298095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3616ac10 0x561c3616d000 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5888009900 tx=0x7f5888005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b6cc7c317bddf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.689+0000 7f589e26c640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58840a3f70 0x7f58840a6430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5890005fd0 tx=0x7f5890004ab0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f5883fff640 1 -- 172.21.15.19:0/4227298095 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888040070 con 0x561c3616ac10 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f5883fff640 1 -- 172.21.15.19:0/4227298095 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5888004030 con 0x561c3616ac10 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f5883fff640 1 -- 172.21.15.19:0/4227298095 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888035040 con 0x561c3616ac10 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f58a29ee740 1 -- 172.21.15.19:0/4227298095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3616ac10 msgr2=0x561c3616d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f58a29ee740 1 --2- 172.21.15.19:0/4227298095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3616ac10 0x561c3616d000 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5888009900 tx=0x7f5888005c00 comp rx=0 tx=0).stop 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f58a29ee740 1 -- 172.21.15.19:0/4227298095 shutdown_connections 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f58a29ee740 1 --2- 172.21.15.19:0/4227298095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615be40 0x561c3616a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f58a29ee740 1 --2- 172.21.15.19:0/4227298095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3616ac10 0x561c3616d000 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f58a29ee740 1 --2- 172.21.15.19:0/4227298095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 0x561c3615b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f58a29ee740 1 -- 172.21.15.19:0/4227298095 >> 172.21.15.19:0/4227298095 conn(0x561c36157b30 msgr2=0x561c36157f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.690+0000 7f58a29ee740 1 -- 172.21.15.19:0/4227298095 shutdown_connections 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.691+0000 7f58a29ee740 1 -- 172.21.15.19:0/4227298095 wait complete. 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.691+0000 7f58a29ee740 1 -- start start 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.691+0000 7f58a29ee740 1 Processor -- start 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.691+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 0x561c3616a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.691+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36167850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.691+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3616ac10 0x561c36167db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f58a29ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561c3616d7e0 con 0x561c3615be40 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f58a29ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561c3605b0b0 con 0x561c3616ac10 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f58a29ee740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561c3616d960 con 0x561c3615b520 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f58a29ee740 10 client.0 _ll_register_callbacks cb 0x561c36048d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589d26a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589d26a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36167850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35866/0 (socket says 172.21.15.19:35866) 2024-04-23T18:27:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589d26a640 1 -- 172.21.15.19:0/3812356084 learned_addr learned my addr 172.21.15.19:0/3812356084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589e26c640 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3616ac10 0x561c36167db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589da6b640 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 0x561c3616a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589d26a640 1 -- 172.21.15.19:0/3812356084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 msgr2=0x561c3616a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589d26a640 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 0x561c3616a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589d26a640 1 -- 172.21.15.19:0/3812356084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3616ac10 msgr2=0x561c36167db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589d26a640 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3616ac10 0x561c36167db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589d26a640 1 -- 172.21.15.19:0/3812356084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c35e9f620 con 0x561c3615be40 2024-04-23T18:27:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f589d26a640 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36167850 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f58880095a0 tx=0x7f588802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432ac6e34e24f21c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f58817fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f58817fa640 1 -- 172.21.15.19:0/3812356084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888040070 con 0x561c3615be40 2024-04-23T18:27:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f58817fa640 1 -- 172.21.15.19:0/3812356084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5888005460 con 0x561c3615be40 2024-04-23T18:27:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.692+0000 7f58a29ee740 1 -- 172.21.15.19:0/3812356084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5888004030 con 0x561c3615be40 2024-04-23T18:27:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58817fa640 1 -- 172.21.15.19:0/3812356084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5888056050 con 0x561c3615be40 2024-04-23T18:27:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58a29ee740 10 client.15807 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58a29ee740 1 -- 172.21.15.19:0/3812356084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c36179790 con 0x561c3615be40 2024-04-23T18:27:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58817fa640 1 -- 172.21.15.19:0/3812356084 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5888004030 con 0x561c3615be40 2024-04-23T18:27:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58817fa640 1 -- 172.21.15.19:0/3812356084 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5888004b40 con 0x561c3615be40 2024-04-23T18:27:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58a29ee740 10 client.15807 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58a29ee740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58a29ee740 20 client.15807 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58a29ee740 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561c3617aae0 0x561c3617ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58a29ee740 4 client.15807 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f58a29ee740 1 -- 172.21.15.19:0/3812356084 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c360ce390 con 0x561c3617aae0 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.693+0000 7f589da6b640 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561c3617aae0 0x561c3617ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.694+0000 7f589da6b640 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561c3617aae0 0x561c3617ced0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc7e25fba9eff56 server_cookie=692956338d10f980 in_seq=0 out_seq=0 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.694+0000 7f58817fa640 10 client.15807 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58817fa640 1 -- 172.21.15.19:0/3812356084 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c360ce390 con 0x561c3617aae0 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58817fa640 10 client.15807 handle_command_reply: tid=0 2024-04-23T18:27:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 client.15807 shutdown 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 -- 172.21.15.19:0/3812356084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 msgr2=0x561c36167850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36167850 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f58880095a0 tx=0x7f588802a410 comp rx=0 tx=0).stop 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 -- 172.21.15.19:0/3812356084 shutdown_connections 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561c3617aae0 0x561c3617ced0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3616ac10 0x561c36167db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58817fa640 0 client.15807 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36167850 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 --2- 172.21.15.19:0/3812356084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615b520 0x561c3616a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 -- 172.21.15.19:0/3812356084 >> 172.21.15.19:0/3812356084 conn(0x561c36157b30 msgr2=0x561c3615d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 -- 172.21.15.19:0/3812356084 shutdown_connections 2024-04-23T18:27:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 1 -- 172.21.15.19:0/3812356084 wait complete. 2024-04-23T18:27:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.695+0000 7f58a29ee740 20 client.15807 trim_cache size 0 max 16384 2024-04-23T18:27:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f58a29ee740 1 Processor -- start 2024-04-23T18:27:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f58a29ee740 1 -- start start 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615b520 0x561c361766a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36157f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3616ac10 0x561c3616a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f58a29ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58981ab790 con 0x561c3615be40 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f58a29ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f58981ad2d0 con 0x561c3615b520 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f58a29ee740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58981abe60 con 0x561c3616ac10 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f589d26a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f589d26a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35874/0 (socket says 172.21.15.19:35874) 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f589d26a640 1 -- 172.21.15.19:0/1959894902 learned_addr learned my addr 172.21.15.19:0/1959894902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f589da6b640 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615b520 0x561c361766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.711+0000 7f589e26c640 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3616ac10 0x561c3616a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f589d26a640 1 -- 172.21.15.19:0/1959894902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3616ac10 msgr2=0x561c3616a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f589d26a640 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3616ac10 0x561c3616a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f589d26a640 1 -- 172.21.15.19:0/1959894902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615b520 msgr2=0x561c361766a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f589d26a640 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615b520 0x561c361766a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f589d26a640 1 -- 172.21.15.19:0/1959894902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c36165da0 con 0x561c3615be40 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f589da6b640 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615b520 0x561c361766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f589e26c640 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3616ac10 0x561c3616a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f589d26a640 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36157f10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f58880050b0 tx=0x7f5888031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe15196fbe59ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f58837fe640 1 -- 172.21.15.19:0/1959894902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588805f070 con 0x561c3615be40 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f58837fe640 1 -- 172.21.15.19:0/1959894902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f588804d070 con 0x561c3615be40 2024-04-23T18:27:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.712+0000 7f58837fe640 1 -- 172.21.15.19:0/1959894902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888040070 con 0x561c3615be40 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 -- 172.21.15.19:0/1959894902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 msgr2=0x561c36157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36157f10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f58880050b0 tx=0x7f5888031400 comp rx=0 tx=0).stop 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 -- 172.21.15.19:0/1959894902 shutdown_connections 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3616ac10 0x561c3616a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615be40 0x561c36157f10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 --2- 172.21.15.19:0/1959894902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3615b520 0x561c361766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 -- 172.21.15.19:0/1959894902 >> 172.21.15.19:0/1959894902 conn(0x561c36157b30 msgr2=0x561c36162f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 -- 172.21.15.19:0/1959894902 shutdown_connections 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 -- 172.21.15.19:0/1959894902 wait complete. 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 -- start start 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.713+0000 7f58a29ee740 1 Processor -- start 2024-04-23T18:27:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615b520 0x561c36172e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615be40 0x561c36164080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58a29ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3616ac10 0x561c361645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58a29ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58981acc00 con 0x561c3615b520 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58a29ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f58981ac530 con 0x561c3616ac10 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58a29ee740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58981a9790 con 0x561c3615be40 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58a29ee740 10 client.0 _ll_register_callbacks cb 0x7f5888032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589da6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615b520 0x561c36172e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589da6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615b520 0x561c36172e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35886/0 (socket says 172.21.15.19:35886) 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589da6b640 1 -- 172.21.15.19:0/822276703 learned_addr learned my addr 172.21.15.19:0/822276703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589e26c640 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3616ac10 0x561c361645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589d26a640 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615be40 0x561c36164080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589da6b640 1 -- 172.21.15.19:0/822276703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615be40 msgr2=0x561c36164080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589da6b640 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615be40 0x561c36164080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589da6b640 1 -- 172.21.15.19:0/822276703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3616ac10 msgr2=0x561c361645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589da6b640 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3616ac10 0x561c361645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589da6b640 1 -- 172.21.15.19:0/822276703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c36164c30 con 0x561c3615b520 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f589da6b640 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615b520 0x561c36172e10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f58940cda90 tx=0x7f589406ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d5cf147ac7070d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58817fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58817fa640 1 -- 172.21.15.19:0/822276703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894020040 con 0x561c3615b520 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58817fa640 1 -- 172.21.15.19:0/822276703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f58940989a0 con 0x561c3615b520 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.714+0000 7f58a29ee740 1 -- 172.21.15.19:0/822276703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f588804d070 con 0x561c3615b520 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58817fa640 1 -- 172.21.15.19:0/822276703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58940a4ef0 con 0x561c3615b520 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58a29ee740 10 client.15813 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58a29ee740 1 -- 172.21.15.19:0/822276703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c361e0b30 con 0x561c3615b520 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58817fa640 1 -- 172.21.15.19:0/822276703 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f588804d070 con 0x561c3615b520 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58817fa640 1 -- 172.21.15.19:0/822276703 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f589409b780 con 0x561c3615b520 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58a29ee740 10 client.15813 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58a29ee740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58a29ee740 20 client.15813 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58a29ee740 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561c361e1200 0x561c361e3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58a29ee740 4 client.15813 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f58a29ee740 1 -- 172.21.15.19:0/822276703 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c360c8200 con 0x561c361e1200 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.715+0000 7f589d26a640 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561c361e1200 0x561c361e3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.716+0000 7f589d26a640 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561c361e1200 0x561c361e3620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=486222bfa4275ab3 server_cookie=9a0643853a5f3ea0 in_seq=0 out_seq=0 2024-04-23T18:27:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.716+0000 7f58817fa640 10 client.15813 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58817fa640 1 -- 172.21.15.19:0/822276703 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c360c8200 con 0x561c361e1200 2024-04-23T18:27:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58817fa640 10 client.15813 handle_command_reply: tid=0 2024-04-23T18:27:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 client.15813 shutdown 2024-04-23T18:27:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 -- 172.21.15.19:0/822276703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615b520 msgr2=0x561c36172e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615b520 0x561c36172e10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f58940cda90 tx=0x7f589406ee20 comp rx=0 tx=0).stop 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 -- 172.21.15.19:0/822276703 shutdown_connections 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561c361e1200 0x561c361e3620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561c3616ac10 0x561c361645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c3615be40 0x561c36164080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58817fa640 0 client.15813 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 --2- 172.21.15.19:0/822276703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c3615b520 0x561c36172e10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 -- 172.21.15.19:0/822276703 >> 172.21.15.19:0/822276703 conn(0x561c36157b30 msgr2=0x561c36162f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 -- 172.21.15.19:0/822276703 shutdown_connections 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 1 -- 172.21.15.19:0/822276703 wait complete. 2024-04-23T18:27:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.726+0000 7f58a29ee740 20 client.15813 trim_cache size 0 max 16384 2024-04-23T18:27:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:27:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:27:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:27:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:27:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 -- 172.21.15.19:0/2715983673 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58840a3f70 msgr2=0x7f58840a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58840a3f70 0x7f58840a6430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5890005fd0 tx=0x7f5890004ab0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 -- 172.21.15.19:0/2715983673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58981010f0 msgr2=0x7f5898198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58981010f0 0x7f5898198730 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f589400d770 tx=0x7f589400dc30 comp rx=0 tx=0).stop 2024-04-23T18:27:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 -- 172.21.15.19:0/2715983673 shutdown_connections 2024-04-23T18:27:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58840a3f70 0x7f58840a6430 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898106170 0x7f5898198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f589819cfa0 0x7f589819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 --2- 172.21.15.19:0/2715983673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58981010f0 0x7f5898198730 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 -- 172.21.15.19:0/2715983673 >> 172.21.15.19:0/2715983673 conn(0x7f5898077fd0 msgr2=0x7f58980fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 -- 172.21.15.19:0/2715983673 shutdown_connections 2024-04-23T18:27:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:16.727+0000 7f589fccf640 1 -- 172.21.15.19:0/2715983673 wait complete. 2024-04-23T18:27:17.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:17 smithi019 ceph-mon[29215]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-04-23T18:27:17.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:17 smithi044 ceph-mon[32458]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-04-23T18:27:17.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:17 smithi155 ceph-mon[33468]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-04-23T18:27:19.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:19 smithi019 ceph-mon[29215]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-23T18:27:19.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:19 smithi044 ceph-mon[32458]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-23T18:27:19.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:19 smithi155 ceph-mon[33468]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-23T18:27:21.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:21 smithi019 ceph-mon[29215]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-23T18:27:21.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:21 smithi044 ceph-mon[32458]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-23T18:27:21.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:21 smithi155 ceph-mon[33468]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-23T18:27:23.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:23 smithi019 ceph-mon[29215]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-23T18:27:23.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:23 smithi044 ceph-mon[32458]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-23T18:27:23.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:23 smithi155 ceph-mon[33468]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-23T18:27:25.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:25 smithi019 ceph-mon[29215]: pgmap v41: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-23T18:27:25.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:25 smithi044 ceph-mon[32458]: pgmap v41: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-23T18:27:25.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:25 smithi155 ceph-mon[33468]: pgmap v41: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-04-23T18:27:27.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:27 smithi019 ceph-mon[29215]: pgmap v42: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:27.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:27 smithi044 ceph-mon[32458]: pgmap v42: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:27.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:27 smithi155 ceph-mon[33468]: pgmap v42: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:29.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:29 smithi044 ceph-mon[32458]: pgmap v43: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-23T18:27:29.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:29 smithi155 ceph-mon[33468]: pgmap v43: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-23T18:27:29.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:29 smithi019 ceph-mon[29215]: pgmap v43: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-23T18:27:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.828+0000 7f72c5d6f640 1 -- 172.21.15.19:0/4158410273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c0106170 msgr2=0x7f72c010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.828+0000 7f72c5d6f640 1 --2- 172.21.15.19:0/4158410273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c0106170 0x7f72c010dd40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009900 tx=0x7f72b802a410 comp rx=0 tx=0).stop 2024-04-23T18:27:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.828+0000 7f72c5d6f640 1 -- 172.21.15.19:0/4158410273 shutdown_connections 2024-04-23T18:27:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.828+0000 7f72c5d6f640 1 --2- 172.21.15.19:0/4158410273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c0106170 0x7f72c010dd40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.828+0000 7f72c5d6f640 1 --2- 172.21.15.19:0/4158410273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c0101aa0 0x7f72c0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.828+0000 7f72c5d6f640 1 --2- 172.21.15.19:0/4158410273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72c01010f0 0x7f72c01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.828+0000 7f72c5d6f640 1 -- 172.21.15.19:0/4158410273 >> 172.21.15.19:0/4158410273 conn(0x7f72c0077fd0 msgr2=0x7f72c00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.828+0000 7f72c5d6f640 1 -- 172.21.15.19:0/4158410273 shutdown_connections 2024-04-23T18:27:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.828+0000 7f72c5d6f640 1 -- 172.21.15.19:0/4158410273 wait complete. 2024-04-23T18:27:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.828+0000 7f72c5d6f640 1 Processor -- start 2024-04-23T18:27:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72c5d6f640 1 -- start start 2024-04-23T18:27:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72c5d6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72c01010f0 0x7f72c0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72c5d6f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c0106170 0x7f72c0198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72c5d6f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c019cef0 0x7f72c019d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72c5d6f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01121f0 con 0x7f72c0106170 2024-04-23T18:27:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72c5d6f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0112070 con 0x7f72c01010f0 2024-04-23T18:27:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72c5d6f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0112370 con 0x7f72c019cef0 2024-04-23T18:27:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72beffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c0106170 0x7f72c0198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72beffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c0106170 0x7f72c0198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44998/0 (socket says 172.21.15.19:44998) 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72beffd640 1 -- 172.21.15.19:0/3994456653 learned_addr learned my addr 172.21.15.19:0/3994456653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72c01010f0 0x7f72c0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72bffff640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c019cef0 0x7f72c019d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72beffd640 1 -- 172.21.15.19:0/3994456653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c019cef0 msgr2=0x7f72c019d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72beffd640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c019cef0 0x7f72c019d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72beffd640 1 -- 172.21.15.19:0/3994456653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72c01010f0 msgr2=0x7f72c0198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72beffd640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72c01010f0 0x7f72c0198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.829+0000 7f72beffd640 1 -- 172.21.15.19:0/3994456653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c019d9e0 con 0x7f72c0106170 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.830+0000 7f72beffd640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c0106170 0x7f72c0198b70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f72b4005fd0 tx=0x7f72b400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.830+0000 7f72bcff9640 1 -- 172.21.15.19:0/3994456653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4017070 con 0x7f72c0106170 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.830+0000 7f72c5d6f640 1 -- 172.21.15.19:0/3994456653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c019dcd0 con 0x7f72c0106170 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.830+0000 7f72bcff9640 1 -- 172.21.15.19:0/3994456653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b4009c30 con 0x7f72c0106170 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.830+0000 7f72bcff9640 1 -- 172.21.15.19:0/3994456653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4003d30 con 0x7f72c0106170 2024-04-23T18:27:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.830+0000 7f72c5d6f640 1 -- 172.21.15.19:0/3994456653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c01a5510 con 0x7f72c0106170 2024-04-23T18:27:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.832+0000 7f72bcff9640 1 -- 172.21.15.19:0/3994456653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f72b4003ed0 con 0x7f72c0106170 2024-04-23T18:27:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72c8b64740 1 Processor -- start 2024-04-23T18:27:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72c8b64740 1 -- start start 2024-04-23T18:27:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483363520 0x55a483363900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a4833726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483372c10 0x55a483375000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72c8b64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a4832c8970 con 0x55a483363e40 2024-04-23T18:27:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72c8b64740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a4832654a0 con 0x55a483363520 2024-04-23T18:27:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72c8b64740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a4832c4930 con 0x55a483372c10 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a4833726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a4833726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45010/0 (socket says 172.21.15.19:45010) 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bf7fe640 1 -- 172.21.15.19:0/724364003 learned_addr learned my addr 172.21.15.19:0/724364003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bcff9640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f72a80a3f70 0x7f72a80a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72beffd640 1 --2- 172.21.15.19:0/724364003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483372c10 0x55a483375000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bffff640 1 --2- 172.21.15.19:0/724364003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483363520 0x55a483363900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bffff640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f72a80a3f70 0x7f72a80a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bf7fe640 1 -- 172.21.15.19:0/724364003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483372c10 msgr2=0x55a483375000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/724364003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483372c10 0x55a483375000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bf7fe640 1 -- 172.21.15.19:0/724364003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483363520 msgr2=0x55a483363900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/724364003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483363520 0x55a483363900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bcff9640 1 -- 172.21.15.19:0/3994456653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f72b40d0690 con 0x7f72c0106170 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.833+0000 7f72bf7fe640 1 -- 172.21.15.19:0/724364003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a482fdb210 con 0x55a483363e40 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.834+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/724364003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a4833726d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f72b00114f0 tx=0x7f72b00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f94b42779370ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.834+0000 7f72a5ffb640 1 -- 172.21.15.19:0/724364003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b001e070 con 0x55a483363e40 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.834+0000 7f72bffff640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f72a80a3f70 0x7f72a80a6430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f72b802a930 tx=0x7f72b8005fd0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.834+0000 7f72a5ffb640 1 -- 172.21.15.19:0/724364003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b000fad0 con 0x55a483363e40 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.834+0000 7f72a5ffb640 1 -- 172.21.15.19:0/724364003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0019b10 con 0x55a483363e40 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 -- 172.21.15.19:0/724364003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 msgr2=0x55a4833726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 --2- 172.21.15.19:0/724364003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a4833726d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f72b00114f0 tx=0x7f72b00119b0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 -- 172.21.15.19:0/724364003 shutdown_connections 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 --2- 172.21.15.19:0/724364003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483372c10 0x55a483375000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 --2- 172.21.15.19:0/724364003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a4833726d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 --2- 172.21.15.19:0/724364003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483363520 0x55a483363900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 -- 172.21.15.19:0/724364003 >> 172.21.15.19:0/724364003 conn(0x55a48335fb30 msgr2=0x55a48335ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 -- 172.21.15.19:0/724364003 shutdown_connections 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 -- 172.21.15.19:0/724364003 wait complete. 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 -- start start 2024-04-23T18:27:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.835+0000 7f72c8b64740 1 Processor -- start 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363520 0x55a483372510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a48336f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483372c10 0x55a48336fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72c8b64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a483375840 con 0x55a483363e40 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72c8b64740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a4832630b0 con 0x55a483372c10 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72c8b64740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a4833759c0 con 0x55a483363520 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72c8b64740 10 client.0 _ll_register_callbacks cb 0x55a483250d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72beffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a48336f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72beffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a48336f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45024/0 (socket says 172.21.15.19:45024) 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72beffd640 1 -- 172.21.15.19:0/2876913158 learned_addr learned my addr 172.21.15.19:0/2876913158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72bffff640 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483372c10 0x55a48336fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363520 0x55a483372510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72beffd640 1 -- 172.21.15.19:0/2876913158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363520 msgr2=0x55a483372510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72beffd640 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363520 0x55a483372510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72beffd640 1 -- 172.21.15.19:0/2876913158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483372c10 msgr2=0x55a48336fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72beffd640 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483372c10 0x55a48336fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72beffd640 1 -- 172.21.15.19:0/2876913158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4830a7620 con 0x55a483363e40 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72beffd640 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a48336f800 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f72b40cdcc0 tx=0x7f72b406de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e80a5cc063067a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f729effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f729effd640 1 -- 172.21.15.19:0/2876913158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b40a07d0 con 0x55a483363e40 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f729effd640 1 -- 172.21.15.19:0/2876913158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f72b40041c0 con 0x55a483363e40 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.836+0000 7f72c8b64740 1 -- 172.21.15.19:0/2876913158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b000fad0 con 0x55a483363e40 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f729effd640 1 -- 172.21.15.19:0/2876913158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f72b40a5090 con 0x55a483363e40 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f72c8b64740 10 client.15825 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f72c8b64740 1 -- 172.21.15.19:0/2876913158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a483381790 con 0x55a483363e40 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f729effd640 1 -- 172.21.15.19:0/2876913158 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b000fad0 con 0x55a483363e40 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f729effd640 1 -- 172.21.15.19:0/2876913158 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f72b406a030 con 0x55a483363e40 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f72c8b64740 10 client.15825 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f72c8b64740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f72c8b64740 20 client.15825 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f72c8b64740 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a483382c00 0x55a483384ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f72c8b64740 4 client.15825 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f72c8b64740 1 -- 172.21.15.19:0/2876913158 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4832d6390 con 0x55a483382c00 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.837+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a483382c00 0x55a483384ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.838+0000 7f729effd640 10 client.15825 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.838+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a483382c00 0x55a483384ff0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdcb69d393f73538 server_cookie=200002b9c1665a3e in_seq=0 out_seq=0 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.838+0000 7f729effd640 1 -- 172.21.15.19:0/2876913158 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4832d6390 con 0x55a483382c00 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.838+0000 7f729effd640 10 client.15825 handle_command_reply: tid=0 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.838+0000 7f72c8b64740 1 client.15825 shutdown 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 1 -- 172.21.15.19:0/2876913158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 msgr2=0x55a48336f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a48336f800 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f72b40cdcc0 tx=0x7f72b406de60 comp rx=0 tx=0).stop 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 1 -- 172.21.15.19:0/2876913158 shutdown_connections 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a483382c00 0x55a483384ff0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483372c10 0x55a48336fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363e40 0x55a48336f800 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 1 --2- 172.21.15.19:0/2876913158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363520 0x55a483372510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f729effd640 0 client.15825 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 1 -- 172.21.15.19:0/2876913158 >> 172.21.15.19:0/2876913158 conn(0x55a48335fb30 msgr2=0x55a4833659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 1 -- 172.21.15.19:0/2876913158 shutdown_connections 2024-04-23T18:27:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 1 -- 172.21.15.19:0/2876913158 wait complete. 2024-04-23T18:27:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.839+0000 7f72c8b64740 20 client.15825 trim_cache size 0 max 16384 2024-04-23T18:27:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72c8b64740 1 Processor -- start 2024-04-23T18:27:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72c8b64740 1 -- start start 2024-04-23T18:27:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363520 0x55a48336af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483363e40 0x55a48335eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483372c10 0x55a483372040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72c8b64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01ab8a0 con 0x55a483372c10 2024-04-23T18:27:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72c8b64740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01ad3e0 con 0x55a483363e40 2024-04-23T18:27:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72c8b64740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01abf70 con 0x55a483363520 2024-04-23T18:27:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483372c10 0x55a483372040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483372c10 0x55a483372040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45032/0 (socket says 172.21.15.19:45032) 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.854+0000 7f72bffff640 1 -- 172.21.15.19:0/2981745137 learned_addr learned my addr 172.21.15.19:0/2981745137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72beffd640 1 --2- 172.21.15.19:0/2981745137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483363e40 0x55a48335eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/2981745137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363520 0x55a48336af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72bffff640 1 -- 172.21.15.19:0/2981745137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363520 msgr2=0x55a48336af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72bffff640 1 --2- 172.21.15.19:0/2981745137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363520 0x55a48336af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72bffff640 1 -- 172.21.15.19:0/2981745137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483363e40 msgr2=0x55a48335eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72bffff640 1 --2- 172.21.15.19:0/2981745137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483363e40 0x55a48335eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72bffff640 1 -- 172.21.15.19:0/2981745137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a48335f960 con 0x55a483372c10 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72bffff640 1 --2- 172.21.15.19:0/2981745137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483372c10 0x55a483372040 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009900 tx=0x7f72b8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf881359ff50e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72a57fa640 1 -- 172.21.15.19:0/2981745137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b804e070 con 0x55a483372c10 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72a57fa640 1 -- 172.21.15.19:0/2981745137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b8031e50 con 0x55a483372c10 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.855+0000 7f72a57fa640 1 -- 172.21.15.19:0/2981745137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8042070 con 0x55a483372c10 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 -- 172.21.15.19:0/2981745137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483372c10 msgr2=0x55a483372040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 --2- 172.21.15.19:0/2981745137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483372c10 0x55a483372040 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009900 tx=0x7f72b8031400 comp rx=0 tx=0).stop 2024-04-23T18:27:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 -- 172.21.15.19:0/2981745137 shutdown_connections 2024-04-23T18:27:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 --2- 172.21.15.19:0/2981745137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483363e40 0x55a48335eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 --2- 172.21.15.19:0/2981745137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483372c10 0x55a483372040 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 --2- 172.21.15.19:0/2981745137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363520 0x55a48336af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 -- 172.21.15.19:0/2981745137 >> 172.21.15.19:0/2981745137 conn(0x55a48335fb30 msgr2=0x55a48335e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 -- 172.21.15.19:0/2981745137 shutdown_connections 2024-04-23T18:27:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 -- 172.21.15.19:0/2981745137 wait complete. 2024-04-23T18:27:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 -- start start 2024-04-23T18:27:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.856+0000 7f72c8b64740 1 Processor -- start 2024-04-23T18:27:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363520 0x55a483376810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363e40 0x55a483376d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72c8b64740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483372c10 0x55a483377290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72c8b64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01acd10 con 0x55a483363520 2024-04-23T18:27:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72c8b64740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01ac640 con 0x55a483372c10 2024-04-23T18:27:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72c8b64740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01a98a0 con 0x55a483363e40 2024-04-23T18:27:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72c8b64740 10 client.0 _ll_register_callbacks cb 0x7f72b409bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363520 0x55a483376810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363520 0x55a483376810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45036/0 (socket says 172.21.15.19:45036) 2024-04-23T18:27:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72bf7fe640 1 -- 172.21.15.19:0/1459306464 learned_addr learned my addr 172.21.15.19:0/1459306464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72bffff640 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483372c10 0x55a483377290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72beffd640 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363e40 0x55a483376d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72bf7fe640 1 -- 172.21.15.19:0/1459306464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363e40 msgr2=0x55a483376d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363e40 0x55a483376d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72bf7fe640 1 -- 172.21.15.19:0/1459306464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483372c10 msgr2=0x55a483377290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483372c10 0x55a483377290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72bf7fe640 1 -- 172.21.15.19:0/1459306464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4833e8330 con 0x55a483363520 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72beffd640 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363e40 0x55a483376d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72bf7fe640 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363520 0x55a483376810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009a00 tx=0x7f72b0009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952f928ff7c27232 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f729effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f729effd640 1 -- 172.21.15.19:0/1459306464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b002d060 con 0x55a483363520 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f729effd640 1 -- 172.21.15.19:0/1459306464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f72b000fe20 con 0x55a483363520 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.857+0000 7f72c8b64740 1 -- 172.21.15.19:0/1459306464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b8031e50 con 0x55a483363520 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f729effd640 1 -- 172.21.15.19:0/1459306464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f72b001e040 con 0x55a483363520 2024-04-23T18:27:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f72c8b64740 10 client.15831 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f72c8b64740 1 -- 172.21.15.19:0/1459306464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4833e9620 con 0x55a483363520 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f729effd640 1 -- 172.21.15.19:0/1459306464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b8031e50 con 0x55a483363520 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f729effd640 1 -- 172.21.15.19:0/1459306464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f72b00196e0 con 0x55a483363520 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f72c8b64740 10 client.15831 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f72c8b64740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f72c8b64740 20 client.15831 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f72c8b64740 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a4833e9b50 0x55a4833ebf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f72c8b64740 4 client.15831 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f72c8b64740 1 -- 172.21.15.19:0/1459306464 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4832d0200 con 0x55a4833e9b50 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.858+0000 7f72beffd640 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a4833e9b50 0x55a4833ebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.859+0000 7f72beffd640 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a4833e9b50 0x55a4833ebf40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d9ca88ef98aa8d5 server_cookie=5d570b5a05f9bc5b in_seq=0 out_seq=0 2024-04-23T18:27:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.859+0000 7f729effd640 10 client.15831 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f729effd640 1 -- 172.21.15.19:0/1459306464 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4832d0200 con 0x55a4833e9b50 2024-04-23T18:27:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f729effd640 10 client.15831 handle_command_reply: tid=0 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 client.15831 shutdown 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 -- 172.21.15.19:0/1459306464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363520 msgr2=0x55a483376810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363520 0x55a483376810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009a00 tx=0x7f72b0009590 comp rx=0 tx=0).stop 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 -- 172.21.15.19:0/1459306464 shutdown_connections 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a4833e9b50 0x55a4833ebf40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a483372c10 0x55a483377290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f729effd640 0 client.15831 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a483363e40 0x55a483376d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 --2- 172.21.15.19:0/1459306464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a483363520 0x55a483376810 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 -- 172.21.15.19:0/1459306464 >> 172.21.15.19:0/1459306464 conn(0x55a48335fb30 msgr2=0x55a483365900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 -- 172.21.15.19:0/1459306464 shutdown_connections 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 1 -- 172.21.15.19:0/1459306464 wait complete. 2024-04-23T18:27:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.872+0000 7f72c8b64740 20 client.15831 trim_cache size 0 max 16384 2024-04-23T18:27:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:27:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:27:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:27:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 -- 172.21.15.19:0/3994456653 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f72a80a3f70 msgr2=0x7f72a80a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f72a80a3f70 0x7f72a80a6430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f72b802a930 tx=0x7f72b8005fd0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 -- 172.21.15.19:0/3994456653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c0106170 msgr2=0x7f72c0198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c0106170 0x7f72c0198b70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f72b4005fd0 tx=0x7f72b400dde0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 -- 172.21.15.19:0/3994456653 shutdown_connections 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f72a80a3f70 0x7f72a80a6430 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c019cef0 0x7f72c019d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72c0106170 0x7f72c0198b70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 --2- 172.21.15.19:0/3994456653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72c01010f0 0x7f72c0198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 -- 172.21.15.19:0/3994456653 >> 172.21.15.19:0/3994456653 conn(0x7f72c0077fd0 msgr2=0x7f72c00fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 -- 172.21.15.19:0/3994456653 shutdown_connections 2024-04-23T18:27:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:29.873+0000 7f72c5d6f640 1 -- 172.21.15.19:0/3994456653 wait complete. 2024-04-23T18:27:31.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:31 smithi044 ceph-mon[32458]: pgmap v44: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T18:27:31.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:31 smithi155 ceph-mon[33468]: pgmap v44: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T18:27:31.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:31 smithi019 ceph-mon[29215]: pgmap v44: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T18:27:34.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:33 smithi019 ceph-mon[29215]: pgmap v45: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T18:27:34.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:33 smithi044 ceph-mon[32458]: pgmap v45: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T18:27:34.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:33 smithi155 ceph-mon[33468]: pgmap v45: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T18:27:36.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:35 smithi019 ceph-mon[29215]: pgmap v46: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-23T18:27:36.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:35 smithi044 ceph-mon[32458]: pgmap v46: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-23T18:27:36.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:35 smithi155 ceph-mon[33468]: pgmap v46: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-23T18:27:38.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:37 smithi019 ceph-mon[29215]: pgmap v47: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:27:38.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:37 smithi044 ceph-mon[32458]: pgmap v47: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:27:38.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:37 smithi155 ceph-mon[33468]: pgmap v47: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:39 smithi019 ceph-mon[29215]: pgmap v48: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:27:40.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:39 smithi044 ceph-mon[32458]: pgmap v48: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:27:40.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:39 smithi155 ceph-mon[33468]: pgmap v48: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:27:42.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:41 smithi019 ceph-mon[29215]: pgmap v49: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-23T18:27:42.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:41 smithi019 ceph-mon[29215]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-23T18:27:42.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:41 smithi044 ceph-mon[32458]: pgmap v49: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-23T18:27:42.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:41 smithi044 ceph-mon[32458]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-23T18:27:42.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:41 smithi155 ceph-mon[33468]: pgmap v49: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-23T18:27:42.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:41 smithi155 ceph-mon[33468]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-23T18:27:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.972+0000 7f3fe7cae640 1 -- 172.21.15.19:0/159564593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe0106660 msgr2=0x7f3fe0106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.972+0000 7f3fe7cae640 1 --2- 172.21.15.19:0/159564593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe0106660 0x7f3fe0106a40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009930 tx=0x7f3fdc02a420 comp rx=0 tx=0).stop 2024-04-23T18:27:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.972+0000 7f3fe7cae640 1 -- 172.21.15.19:0/159564593 shutdown_connections 2024-04-23T18:27:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.972+0000 7f3fe7cae640 1 --2- 172.21.15.19:0/159564593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe01005c0 0x7f3fe0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.972+0000 7f3fe7cae640 1 --2- 172.21.15.19:0/159564593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fe0100f80 0x7f3fe010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.972+0000 7f3fe7cae640 1 --2- 172.21.15.19:0/159564593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe0106660 0x7f3fe0106a40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.972+0000 7f3fe7cae640 1 -- 172.21.15.19:0/159564593 >> 172.21.15.19:0/159564593 conn(0x7f3fe0077390 msgr2=0x7f3fe00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.973+0000 7f3fe7cae640 1 -- 172.21.15.19:0/159564593 shutdown_connections 2024-04-23T18:27:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.973+0000 7f3fe7cae640 1 -- 172.21.15.19:0/159564593 wait complete. 2024-04-23T18:27:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.973+0000 7f3fe7cae640 1 Processor -- start 2024-04-23T18:27:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.973+0000 7f3fe7cae640 1 -- start start 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.973+0000 7f3fe7cae640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fe01005c0 0x7f3fe0102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.973+0000 7f3fe7cae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe0100f80 0x7f3fe01027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe7cae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe0106660 0x7f3fe0102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe7cae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01110c0 con 0x7f3fe0106660 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe7cae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0110f40 con 0x7f3fe0100f80 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe7cae640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0111240 con 0x7f3fe01005c0 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe624b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe0106660 0x7f3fe0102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe624b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe0106660 0x7f3fe0102ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40124/0 (socket says 172.21.15.19:40124) 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe624b640 1 -- 172.21.15.19:0/1989982334 learned_addr learned my addr 172.21.15.19:0/1989982334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe0100f80 0x7f3fe01027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe5a4a640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fe01005c0 0x7f3fe0102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe624b640 1 -- 172.21.15.19:0/1989982334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fe01005c0 msgr2=0x7f3fe0102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe624b640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fe01005c0 0x7f3fe0102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe624b640 1 -- 172.21.15.19:0/1989982334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe0100f80 msgr2=0x7f3fe01027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe624b640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe0100f80 0x7f3fe01027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe624b640 1 -- 172.21.15.19:0/1989982334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe01a6bd0 con 0x7f3fe0106660 2024-04-23T18:27:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe624b640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe0106660 0x7f3fe0102ce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800eb10 tx=0x7f3fd800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fd6ffd640 1 -- 172.21.15.19:0/1989982334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8004b20 con 0x7f3fe0106660 2024-04-23T18:27:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fd6ffd640 1 -- 172.21.15.19:0/1989982334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd8004cc0 con 0x7f3fe0106660 2024-04-23T18:27:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fe7cae640 1 -- 172.21.15.19:0/1989982334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe01a6ec0 con 0x7f3fe0106660 2024-04-23T18:27:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.974+0000 7f3fd6ffd640 1 -- 172.21.15.19:0/1989982334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8010670 con 0x7f3fe0106660 2024-04-23T18:27:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.975+0000 7f3fe7cae640 1 -- 172.21.15.19:0/1989982334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe01a7430 con 0x7f3fe0106660 2024-04-23T18:27:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.977+0000 7f3fd6ffd640 1 -- 172.21.15.19:0/1989982334 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3fd8005020 con 0x7f3fe0106660 2024-04-23T18:27:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.977+0000 7f3fea9c7740 1 Processor -- start 2024-04-23T18:27:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fea9c7740 1 -- start start 2024-04-23T18:27:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fd6ffd640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3fcc0a3e40 0x7f3fcc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 0x55cad61aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cad610f970 con 0x55cad61aaf40 2024-04-23T18:27:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cad60ac4a0 con 0x55cad61aa620 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cad610b930 con 0x55cad61b5800 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3fcc0a3e40 0x7f3fcc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fd6ffd640 1 -- 172.21.15.19:0/1989982334 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3fd80d0c80 con 0x7f3fe0106660 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fe5249640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fe624b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fe624b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:34188/0 (socket says 172.21.15.19:34188) 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fe624b640 1 -- 172.21.15.19:0/25909621 learned_addr learned my addr 172.21.15.19:0/25909621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fe5249640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40132/0 (socket says 172.21.15.19:40132) 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.978+0000 7f3fe5a4a640 1 --2- 172.21.15.19:0/25909621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 0x55cad61aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.979+0000 7f3fe624b640 1 -- 172.21.15.19:0/25909621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 msgr2=0x55cad61aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.979+0000 7f3fe624b640 1 --2- 172.21.15.19:0/25909621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 0x55cad61aaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.979+0000 7f3fe624b640 1 -- 172.21.15.19:0/25909621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 msgr2=0x55cad61b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.979+0000 7f3fe624b640 1 --2- 172.21.15.19:0/25909621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61b52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.979+0000 7f3fe624b640 1 -- 172.21.15.19:0/25909621 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad5e22210 con 0x55cad61b5800 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.979+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3fcc0a3e40 0x7f3fcc0a6300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000bfd0 tx=0x7f3fd000bb40 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.979+0000 7f3fe624b640 1 --2- 172.21.15.19:0/25909621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61b7bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80a5390 tx=0x7f3fd800cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4975fd689cc90ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.979+0000 7f3fcbfff640 1 -- 172.21.15.19:0/25909621 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80b63d0 con 0x55cad61b5800 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.979+0000 7f3fcbfff640 1 -- 172.21.15.19:0/25909621 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd809d350 con 0x55cad61b5800 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.979+0000 7f3fcbfff640 1 -- 172.21.15.19:0/25909621 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80ce6a0 con 0x55cad61b5800 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 -- 172.21.15.19:0/25909621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 msgr2=0x55cad61b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/25909621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61b7bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80a5390 tx=0x7f3fd800cee0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 -- 172.21.15.19:0/25909621 shutdown_connections 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/25909621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61b7bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/25909621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/25909621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 0x55cad61aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 -- 172.21.15.19:0/25909621 >> 172.21.15.19:0/25909621 conn(0x55cad61a6b30 msgr2=0x55cad61a6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 -- 172.21.15.19:0/25909621 shutdown_connections 2024-04-23T18:27:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 -- 172.21.15.19:0/25909621 wait complete. 2024-04-23T18:27:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 -- start start 2024-04-23T18:27:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.980+0000 7f3fea9c7740 1 Processor -- start 2024-04-23T18:27:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61aaf40 0x55cad61b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61b5800 0x55cad61b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cad61b8430 con 0x55cad61aa620 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cad60aa0b0 con 0x55cad61b5800 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cad61b85b0 con 0x55cad61aaf40 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fea9c7740 10 client.0 _ll_register_callbacks cb 0x55cad6097d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe5a4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe5a4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40142/0 (socket says 172.21.15.19:40142) 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe5a4a640 1 -- 172.21.15.19:0/1504928647 learned_addr learned my addr 172.21.15.19:0/1504928647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61aaf40 0x55cad61b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe624b640 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61b5800 0x55cad61b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe5a4a640 1 -- 172.21.15.19:0/1504928647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61aaf40 msgr2=0x55cad61b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe5a4a640 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61aaf40 0x55cad61b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe5a4a640 1 -- 172.21.15.19:0/1504928647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61b5800 msgr2=0x55cad61b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe5a4a640 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61b5800 0x55cad61b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe5a4a640 1 -- 172.21.15.19:0/1504928647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad5eee620 con 0x55cad61aa620 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.981+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61aaf40 0x55cad61b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fe5a4a640 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61b5070 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc005c40 tx=0x7f3fdc0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3ce7a8a8283b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1504928647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc035040 con 0x55cad61aa620 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1504928647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fd809d350 con 0x55cad61aa620 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1504928647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3fdc004240 con 0x55cad61aa620 2024-04-23T18:27:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1504928647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3fdc031900 con 0x55cad61aa620 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fea9c7740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1504928647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cad61c42a0 con 0x55cad61aa620 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1504928647 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd809d350 con 0x55cad61aa620 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1504928647 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3fdc03c560 con 0x55cad61aa620 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fea9c7740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fea9c7740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.982+0000 7f3fea9c7740 20 client.15843 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.983+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cad61c5760 0x55cad61c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.983+0000 7f3fea9c7740 4 client.15843 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.983+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1504928647 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cad611d390 con 0x55cad61c5760 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.983+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cad61c5760 0x55cad61c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.983+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cad61c5760 0x55cad61c7bb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851b0668ce19a7c8 server_cookie=dede801747e08264 in_seq=0 out_seq=0 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.983+0000 7f3fc97fa640 10 client.15843 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1504928647 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cad611d390 con 0x55cad61c5760 2024-04-23T18:27:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fc97fa640 10 client.15843 handle_command_reply: tid=0 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 client.15843 shutdown 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1504928647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 msgr2=0x55cad61b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61b5070 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc005c40 tx=0x7f3fdc0316d0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1504928647 shutdown_connections 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cad61c5760 0x55cad61c7bb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61b5800 0x55cad61b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61aaf40 0x55cad61b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1504928647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61b5070 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fc97fa640 0 client.15843 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1504928647 >> 172.21.15.19:0/1504928647 conn(0x55cad61a6b30 msgr2=0x55cad61a5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1504928647 shutdown_connections 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.984+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1504928647 wait complete. 2024-04-23T18:27:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:42.985+0000 7f3fea9c7740 20 client.15843 trim_cache size 0 max 16384 2024-04-23T18:27:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fea9c7740 1 Processor -- start 2024-04-23T18:27:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fea9c7740 1 -- start start 2024-04-23T18:27:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 0x55cad61c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01afde0 con 0x55cad61aaf40 2024-04-23T18:27:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01b1920 con 0x55cad61aa620 2024-04-23T18:27:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01b04b0 con 0x55cad61b5800 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5249640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5249640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40152/0 (socket says 172.21.15.19:40152) 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5249640 1 -- 172.21.15.19:0/1472657916 learned_addr learned my addr 172.21.15.19:0/1472657916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe624b640 1 --2- 172.21.15.19:0/1472657916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5249640 1 -- 172.21.15.19:0/1472657916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 msgr2=0x55cad61a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1472657916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5a4a640 1 --2- 172.21.15.19:0/1472657916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 0x55cad61c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5249640 1 -- 172.21.15.19:0/1472657916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 msgr2=0x55cad61c12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1472657916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 0x55cad61c12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5249640 1 -- 172.21.15.19:0/1472657916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad61aad70 con 0x55cad61aaf40 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5a4a640 1 --2- 172.21.15.19:0/1472657916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 0x55cad61c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.000+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1472657916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61adaf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0103cf0 tx=0x7f3fd001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94420689a470a3dd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fcb7fe640 1 -- 172.21.15.19:0/1472657916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0010b00 con 0x55cad61aaf40 2024-04-23T18:27:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fcb7fe640 1 -- 172.21.15.19:0/1472657916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd0031070 con 0x55cad61aaf40 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fcb7fe640 1 -- 172.21.15.19:0/1472657916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0021770 con 0x55cad61aaf40 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1472657916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 msgr2=0x55cad61adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1472657916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61adaf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0103cf0 tx=0x7f3fd001f000 comp rx=0 tx=0).stop 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1472657916 shutdown_connections 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1472657916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1472657916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aaf40 0x55cad61adaf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1472657916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aa620 0x55cad61c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1472657916 >> 172.21.15.19:0/1472657916 conn(0x55cad61a6b30 msgr2=0x55cad61a5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1472657916 shutdown_connections 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.001+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1472657916 wait complete. 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fea9c7740 1 -- start start 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fea9c7740 1 Processor -- start 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aaf40 0x55cad61af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fea9c7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01b1250 con 0x55cad61aa620 2024-04-23T18:27:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01b0b80 con 0x55cad61aaf40 2024-04-23T18:27:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fea9c7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01addc0 con 0x55cad61b5800 2024-04-23T18:27:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fea9c7740 10 client.0 _ll_register_callbacks cb 0x7f3fdc032ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fe5a4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fe5a4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40166/0 (socket says 172.21.15.19:40166) 2024-04-23T18:27:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fe5a4a640 1 -- 172.21.15.19:0/1269239148 learned_addr learned my addr 172.21.15.19:0/1269239148 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fe624b640 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.002+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aaf40 0x55cad61af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fe5a4a640 1 -- 172.21.15.19:0/1269239148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 msgr2=0x55cad61af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fe5a4a640 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fe5a4a640 1 -- 172.21.15.19:0/1269239148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aaf40 msgr2=0x55cad61af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fe5a4a640 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aaf40 0x55cad61af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fe5a4a640 1 -- 172.21.15.19:0/1269239148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad622ad30 con 0x55cad61aa620 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fe624b640 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fe5a4a640 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61aee70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc03caf0 tx=0x7f3fdc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48861e4718446683 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1269239148 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc051070 con 0x55cad61aa620 2024-04-23T18:27:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1269239148 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3fdc004030 con 0x55cad61aa620 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1269239148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fd0031070 con 0x55cad61aa620 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1269239148 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3fdc035040 con 0x55cad61aa620 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fea9c7740 10 client.15849 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1269239148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cad622bdf0 con 0x55cad61aa620 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.003+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1269239148 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd0031070 con 0x55cad61aa620 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.004+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1269239148 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3fdc044420 con 0x55cad61aa620 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.004+0000 7f3fea9c7740 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.004+0000 7f3fea9c7740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.004+0000 7f3fea9c7740 20 client.15849 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.004+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cad622c5f0 0x55cad622ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.004+0000 7f3fea9c7740 4 client.15849 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.004+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1269239148 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cad6117200 con 0x55cad622c5f0 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.004+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cad622c5f0 0x55cad622ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.004+0000 7f3fe5249640 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cad622c5f0 0x55cad622ea10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=872f8d0468ea6703 server_cookie=c8c2b2b429bd2273 in_seq=0 out_seq=0 2024-04-23T18:27:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.004+0000 7f3fc97fa640 10 client.15849 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.017+0000 7f3fc97fa640 1 -- 172.21.15.19:0/1269239148 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cad6117200 con 0x55cad622c5f0 2024-04-23T18:27:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.017+0000 7f3fc97fa640 10 client.15849 handle_command_reply: tid=0 2024-04-23T18:27:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.017+0000 7f3fea9c7740 1 client.15849 shutdown 2024-04-23T18:27:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1269239148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 msgr2=0x55cad61aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61aee70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc03caf0 tx=0x7f3fdc0056d0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1269239148 shutdown_connections 2024-04-23T18:27:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55cad622c5f0 0x55cad622ea10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cad61aaf40 0x55cad61af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cad61b5800 0x55cad61af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fc97fa640 0 client.15849 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 1 --2- 172.21.15.19:0/1269239148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cad61aa620 0x55cad61aee70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1269239148 >> 172.21.15.19:0/1269239148 conn(0x55cad61a6b30 msgr2=0x55cad61ad100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1269239148 shutdown_connections 2024-04-23T18:27:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 1 -- 172.21.15.19:0/1269239148 wait complete. 2024-04-23T18:27:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fea9c7740 20 client.15849 trim_cache size 0 max 16384 2024-04-23T18:27:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:27:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:27:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:27:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:27:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fe7cae640 1 -- 172.21.15.19:0/1989982334 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3fcc0a3e40 msgr2=0x7f3fcc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fe7cae640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3fcc0a3e40 0x7f3fcc0a6300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000bfd0 tx=0x7f3fd000bb40 comp rx=0 tx=0).stop 2024-04-23T18:27:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fe7cae640 1 -- 172.21.15.19:0/1989982334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe0106660 msgr2=0x7f3fe0102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fe7cae640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe0106660 0x7f3fe0102ce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800eb10 tx=0x7f3fd800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fe7cae640 1 -- 172.21.15.19:0/1989982334 shutdown_connections 2024-04-23T18:27:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fe7cae640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3fcc0a3e40 0x7f3fcc0a6300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fe7cae640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe0100f80 0x7f3fe01027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fe7cae640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe0106660 0x7f3fe0102ce0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fe7cae640 1 --2- 172.21.15.19:0/1989982334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fe01005c0 0x7f3fe0102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.018+0000 7f3fe7cae640 1 -- 172.21.15.19:0/1989982334 >> 172.21.15.19:0/1989982334 conn(0x7f3fe0077390 msgr2=0x7f3fe00ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.019+0000 7f3fe7cae640 1 -- 172.21.15.19:0/1989982334 shutdown_connections 2024-04-23T18:27:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:43.019+0000 7f3fe7cae640 1 -- 172.21.15.19:0/1989982334 wait complete. 2024-04-23T18:27:44.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:43 smithi019 ceph-mon[29215]: pgmap v50: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:44.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:43 smithi044 ceph-mon[32458]: pgmap v50: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:44.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:43 smithi155 ceph-mon[33468]: pgmap v50: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:46.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:45 smithi019 ceph-mon[29215]: pgmap v51: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-23T18:27:46.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:45 smithi044 ceph-mon[32458]: pgmap v51: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-23T18:27:46.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:45 smithi155 ceph-mon[33468]: pgmap v51: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-23T18:27:48.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:47 smithi019 ceph-mon[29215]: pgmap v52: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:48.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:47 smithi044 ceph-mon[32458]: pgmap v52: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:48.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:47 smithi155 ceph-mon[33468]: pgmap v52: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:50.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:49 smithi019 ceph-mon[29215]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:50.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:49 smithi044 ceph-mon[32458]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:50.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:49 smithi155 ceph-mon[33468]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:27:52.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:51 smithi019 ceph-mon[29215]: pgmap v54: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-23T18:27:52.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:51 smithi044 ceph-mon[32458]: pgmap v54: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-23T18:27:52.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:51 smithi155 ceph-mon[33468]: pgmap v54: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-23T18:27:54.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:53 smithi019 ceph-mon[29215]: pgmap v55: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:27:54.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:53 smithi044 ceph-mon[32458]: pgmap v55: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:27:54.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:53 smithi155 ceph-mon[33468]: pgmap v55: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:27:56.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:55 smithi019 ceph-mon[29215]: pgmap v56: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-23T18:27:56.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:55 smithi044 ceph-mon[32458]: pgmap v56: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-23T18:27:56.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:55 smithi155 ceph-mon[33468]: pgmap v56: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-23T18:27:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.120+0000 7ff4184ef640 1 -- 172.21.15.19:0/2958868536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff410102b80 msgr2=0x7ff410106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.120+0000 7ff4184ef640 1 --2- 172.21.15.19:0/2958868536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff410102b80 0x7ff410106bc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff400009900 tx=0x7ff40002a410 comp rx=0 tx=0).stop 2024-04-23T18:27:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 -- 172.21.15.19:0/2958868536 shutdown_connections 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 --2- 172.21.15.19:0/2958868536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4101072a0 0x7ff41010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 --2- 172.21.15.19:0/2958868536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff410102b80 0x7ff410106bc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 --2- 172.21.15.19:0/2958868536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4101021d0 0x7ff4101025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 -- 172.21.15.19:0/2958868536 >> 172.21.15.19:0/2958868536 conn(0x7ff4100793e0 msgr2=0x7ff4100797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 -- 172.21.15.19:0/2958868536 shutdown_connections 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 -- 172.21.15.19:0/2958868536 wait complete. 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 Processor -- start 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 -- start start 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4101021d0 0x7ff410198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4101072a0 0x7ff410198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41019cf00 0x7ff41019d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101121e0 con 0x7ff4101021d0 2024-04-23T18:27:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff410112060 con 0x7ff4101072a0 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.121+0000 7ff4184ef640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff410112360 con 0x7ff41019cf00 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff41628b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4101021d0 0x7ff410198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff41628b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4101021d0 0x7ff410198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34708/0 (socket says 172.21.15.19:34708) 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff41628b640 1 -- 172.21.15.19:0/1715841604 learned_addr learned my addr 172.21.15.19:0/1715841604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff415a8a640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4101072a0 0x7ff410198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff416a8c640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41019cf00 0x7ff41019d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff41628b640 1 -- 172.21.15.19:0/1715841604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41019cf00 msgr2=0x7ff41019d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff41628b640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41019cf00 0x7ff41019d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff41628b640 1 -- 172.21.15.19:0/1715841604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4101072a0 msgr2=0x7ff410198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff41628b640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4101072a0 0x7ff410198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff41628b640 1 -- 172.21.15.19:0/1715841604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41019d9f0 con 0x7ff4101021d0 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff41628b640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4101021d0 0x7ff410198640 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00d950 tx=0x7ff40c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff4077fe640 1 -- 172.21.15.19:0/1715841604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c017070 con 0x7ff4101021d0 2024-04-23T18:27:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff4077fe640 1 -- 172.21.15.19:0/1715841604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff40c009c30 con 0x7ff4101021d0 2024-04-23T18:27:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff4184ef640 1 -- 172.21.15.19:0/1715841604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41019dce0 con 0x7ff4101021d0 2024-04-23T18:27:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff4077fe640 1 -- 172.21.15.19:0/1715841604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c00f620 con 0x7ff4101021d0 2024-04-23T18:27:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.122+0000 7ff4184ef640 1 -- 172.21.15.19:0/1715841604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4101a54a0 con 0x7ff4101021d0 2024-04-23T18:27:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.125+0000 7ff4077fe640 1 -- 172.21.15.19:0/1715841604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff40c010460 con 0x7ff4101021d0 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.125+0000 7ff41b209740 1 Processor -- start 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff41b209740 1 -- start start 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff41b209740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a98f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff41b209740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a99042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff41b209740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a9906bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff41b209740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5566a985e970 con 0x5566a9904800 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff41b209740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5566a97fb4a0 con 0x5566a98f9f40 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff41b209740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5566a985a930 con 0x5566a98f9620 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff415a8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a9906bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff415a8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a9906bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34722/0 (socket says 172.21.15.19:34722) 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff415a8a640 1 -- 172.21.15.19:0/3027282380 learned_addr learned my addr 172.21.15.19:0/3027282380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff41628b640 1 --2- 172.21.15.19:0/3027282380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a99042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff416a8c640 1 --2- 172.21.15.19:0/3027282380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a98f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.126+0000 7ff4077fe640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff3fc0a3dc0 0x7ff3fc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff416a8c640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff3fc0a3dc0 0x7ff3fc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff415a8a640 1 -- 172.21.15.19:0/3027282380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 msgr2=0x5566a98f9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff415a8a640 1 --2- 172.21.15.19:0/3027282380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a98f9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff415a8a640 1 -- 172.21.15.19:0/3027282380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 msgr2=0x5566a99042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff415a8a640 1 --2- 172.21.15.19:0/3027282380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a99042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff415a8a640 1 -- 172.21.15.19:0/3027282380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a9571210 con 0x5566a9904800 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff4077fe640 1 -- 172.21.15.19:0/1715841604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff40c0d0520 con 0x7ff4101021d0 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff415a8a640 1 --2- 172.21.15.19:0/3027282380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a9906bf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff400031440 tx=0x7ff400032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ed5cf2412e0454 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff416a8c640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff3fc0a3dc0 0x7ff3fc0a6280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff408005fd0 tx=0x7ff408004ab0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff404bfe640 1 -- 172.21.15.19:0/3027282380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40003f070 con 0x5566a9904800 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff404bfe640 1 -- 172.21.15.19:0/3027282380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff400032e40 con 0x5566a9904800 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.127+0000 7ff404bfe640 1 -- 172.21.15.19:0/3027282380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400035070 con 0x5566a9904800 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.128+0000 7ff41b209740 1 -- 172.21.15.19:0/3027282380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 msgr2=0x5566a9906bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.128+0000 7ff41b209740 1 --2- 172.21.15.19:0/3027282380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a9906bf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff400031440 tx=0x7ff400032c50 comp rx=0 tx=0).stop 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.128+0000 7ff41b209740 1 -- 172.21.15.19:0/3027282380 shutdown_connections 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.128+0000 7ff41b209740 1 --2- 172.21.15.19:0/3027282380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a99042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.128+0000 7ff41b209740 1 --2- 172.21.15.19:0/3027282380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a9906bf0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.128+0000 7ff41b209740 1 --2- 172.21.15.19:0/3027282380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a98f9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.128+0000 7ff41b209740 1 -- 172.21.15.19:0/3027282380 >> 172.21.15.19:0/3027282380 conn(0x5566a98f5b30 msgr2=0x5566a98f5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.128+0000 7ff41b209740 1 -- 172.21.15.19:0/3027282380 shutdown_connections 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.128+0000 7ff41b209740 1 -- 172.21.15.19:0/3027282380 wait complete. 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.128+0000 7ff41b209740 1 -- start start 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41b209740 1 Processor -- start 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41b209740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9620 0x5566a9904120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41b209740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a9901410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41b209740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a9904800 0x5566a9901950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41b209740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5566a99073d0 con 0x5566a98f9620 2024-04-23T18:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41b209740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5566a97f90b0 con 0x5566a98f9f40 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41b209740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5566a9907550 con 0x5566a9904800 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41b209740 10 client.0 _ll_register_callbacks cb 0x5566a97e6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41628b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9620 0x5566a9904120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41628b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9620 0x5566a9904120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34732/0 (socket says 172.21.15.19:34732) 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41628b640 1 -- 172.21.15.19:0/3396492534 learned_addr learned my addr 172.21.15.19:0/3396492534 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff416a8c640 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a9904800 0x5566a9901950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff415a8a640 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a9901410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41628b640 1 -- 172.21.15.19:0/3396492534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a9904800 msgr2=0x5566a9901950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41628b640 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a9904800 0x5566a9901950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41628b640 1 -- 172.21.15.19:0/3396492534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 msgr2=0x5566a9901410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41628b640 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a9901410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41628b640 1 -- 172.21.15.19:0/3396492534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a963d620 con 0x5566a98f9620 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff41628b640 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9620 0x5566a9904120 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff40c098170 tx=0x7ff40c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375d0ea61f65ea12 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.129+0000 7ff3f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/3396492534 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c020070 con 0x5566a98f9620 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/3396492534 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff40c098b00 con 0x5566a98f9620 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff41b209740 1 -- 172.21.15.19:0/3396492534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566a986c390 con 0x5566a98f9620 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/3396492534 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff40c0d3e20 con 0x5566a98f9620 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff41b209740 10 client.15861 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff41b209740 1 -- 172.21.15.19:0/3396492534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566a9913300 con 0x5566a98f9620 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/3396492534 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5566a986c390 con 0x5566a98f9620 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/3396492534 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff40c0cdec0 con 0x5566a98f9620 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff41b209740 10 client.15861 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff41b209740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff41b209740 20 client.15861 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff41b209740 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5566a99147c0 0x5566a9916bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff41b209740 4 client.15861 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.130+0000 7ff41b209740 1 -- 172.21.15.19:0/3396492534 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566a9866200 con 0x5566a99147c0 2024-04-23T18:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.131+0000 7ff415a8a640 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5566a99147c0 0x5566a9916bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.131+0000 7ff415a8a640 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5566a99147c0 0x5566a9916bb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84defccd83d48fbd server_cookie=f70ee64b0a2ac23c in_seq=0 out_seq=0 2024-04-23T18:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.131+0000 7ff3f9ffb640 10 client.15861 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/3396492534 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff40004eae0 con 0x5566a99147c0 2024-04-23T18:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff3f9ffb640 10 client.15861 handle_command_reply: tid=0 2024-04-23T18:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 client.15861 shutdown 2024-04-23T18:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 -- 172.21.15.19:0/3396492534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9620 msgr2=0x5566a9904120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9620 0x5566a9904120 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff40c098170 tx=0x7ff40c010cb0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 -- 172.21.15.19:0/3396492534 shutdown_connections 2024-04-23T18:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5566a99147c0 0x5566a9916bb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a9901410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a9904800 0x5566a9901950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff3f9ffb640 0 client.15861 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 --2- 172.21.15.19:0/3396492534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9620 0x5566a9904120 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 -- 172.21.15.19:0/3396492534 >> 172.21.15.19:0/3396492534 conn(0x5566a98f5b30 msgr2=0x5566a98fb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 -- 172.21.15.19:0/3396492534 shutdown_connections 2024-04-23T18:27:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 1 -- 172.21.15.19:0/3396492534 wait complete. 2024-04-23T18:27:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.132+0000 7ff41b209740 20 client.15861 trim_cache size 0 max 16384 2024-04-23T18:27:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.147+0000 7ff41b209740 1 Processor -- start 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.147+0000 7ff41b209740 1 -- start start 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff41b209740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a9910290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff41b209740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9f40 0x5566a9903c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff41b209740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a9904800 0x5566a991c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff41b209740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101ad980 con 0x5566a98f9f40 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff41b209740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101af4c0 con 0x5566a9904800 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff41b209740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101ae050 con 0x5566a98f9620 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff415a8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9f40 0x5566a9903c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff415a8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9f40 0x5566a9903c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34742/0 (socket says 172.21.15.19:34742) 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff415a8a640 1 -- 172.21.15.19:0/851283347 learned_addr learned my addr 172.21.15.19:0/851283347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff41628b640 1 --2- 172.21.15.19:0/851283347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a9910290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff416a8c640 1 --2- 172.21.15.19:0/851283347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a9904800 0x5566a991c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff415a8a640 1 -- 172.21.15.19:0/851283347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 msgr2=0x5566a9910290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff415a8a640 1 --2- 172.21.15.19:0/851283347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a9910290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff415a8a640 1 -- 172.21.15.19:0/851283347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a9904800 msgr2=0x5566a991c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff415a8a640 1 --2- 172.21.15.19:0/851283347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a9904800 0x5566a991c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff415a8a640 1 -- 172.21.15.19:0/851283347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a9917190 con 0x5566a98f9f40 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff416a8c640 1 --2- 172.21.15.19:0/851283347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a9904800 0x5566a991c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff415a8a640 1 --2- 172.21.15.19:0/851283347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9f40 0x5566a9903c50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff400033000 tx=0x7ff400005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577d6222737765d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.148+0000 7ff3fbfff640 1 -- 172.21.15.19:0/851283347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400051020 con 0x5566a98f9f40 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff3fbfff640 1 -- 172.21.15.19:0/851283347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff400049040 con 0x5566a98f9f40 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff3fbfff640 1 -- 172.21.15.19:0/851283347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40003f040 con 0x5566a98f9f40 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff41b209740 1 -- 172.21.15.19:0/851283347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9f40 msgr2=0x5566a9903c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff41b209740 1 --2- 172.21.15.19:0/851283347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9f40 0x5566a9903c50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff400033000 tx=0x7ff400005be0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff41b209740 1 -- 172.21.15.19:0/851283347 shutdown_connections 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff41b209740 1 --2- 172.21.15.19:0/851283347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a9904800 0x5566a991c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff41b209740 1 --2- 172.21.15.19:0/851283347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a98f9f40 0x5566a9903c50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff41b209740 1 --2- 172.21.15.19:0/851283347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a9910290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff41b209740 1 -- 172.21.15.19:0/851283347 >> 172.21.15.19:0/851283347 conn(0x5566a98f5b30 msgr2=0x5566a98f4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff41b209740 1 -- 172.21.15.19:0/851283347 shutdown_connections 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.149+0000 7ff41b209740 1 -- 172.21.15.19:0/851283347 wait complete. 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff41b209740 1 -- start start 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff41b209740 1 Processor -- start 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff41b209740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a98fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff41b209740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a98fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff41b209740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a98fec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff41b209740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101aedf0 con 0x5566a9904800 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff41b209740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101ae720 con 0x5566a98f9f40 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff41b209740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101ab980 con 0x5566a98f9620 2024-04-23T18:27:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff41b209740 10 client.0 _ll_register_callbacks cb 0x7ff40c09c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff416a8c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a98fec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff416a8c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a98fec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34746/0 (socket says 172.21.15.19:34746) 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff416a8c640 1 -- 172.21.15.19:0/237783923 learned_addr learned my addr 172.21.15.19:0/237783923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff415a8a640 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a98fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff41628b640 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a98fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff416a8c640 1 -- 172.21.15.19:0/237783923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 msgr2=0x5566a98fe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff416a8c640 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a98fe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff416a8c640 1 -- 172.21.15.19:0/237783923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 msgr2=0x5566a98fe730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff416a8c640 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a98fe730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.150+0000 7ff416a8c640 1 -- 172.21.15.19:0/237783923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a9979d30 con 0x5566a9904800 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff416a8c640 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a98fec70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff408011820 tx=0x7ff408004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c7a2c32c001f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff3f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:27:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/237783923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408019040 con 0x5566a9904800 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/237783923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff408010bd0 con 0x5566a9904800 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff41b209740 1 -- 172.21.15.19:0/237783923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff400049040 con 0x5566a9904800 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/237783923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff408034050 con 0x5566a9904800 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff41b209740 10 client.15867 fetch_fsmap learned FSMap version 27 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff41b209740 1 -- 172.21.15.19:0/237783923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566a997aff0 con 0x5566a9904800 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/237783923 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff400049040 con 0x5566a9904800 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/237783923 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff408004250 con 0x5566a9904800 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff41b209740 10 client.15867 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff41b209740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff41b209740 20 client.15867 populate_metadata read hostname 'smithi019' 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff41b209740 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5566a997b7d0 0x5566a997dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff41b209740 4 client.15867 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:27:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.151+0000 7ff41b209740 1 -- 172.21.15.19:0/237783923 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566a97e25f0 con 0x5566a997b7d0 2024-04-23T18:27:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.152+0000 7ff41628b640 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5566a997b7d0 0x5566a997dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.152+0000 7ff41628b640 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5566a997b7d0 0x5566a997dbf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e736faa2f8299b6c server_cookie=bc9881e08e8e7caf in_seq=0 out_seq=0 2024-04-23T18:27:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.152+0000 7ff3f9ffb640 10 client.15867 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff3f9ffb640 1 -- 172.21.15.19:0/237783923 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566a97e25f0 con 0x5566a997b7d0 2024-04-23T18:27:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff3f9ffb640 10 client.15867 handle_command_reply: tid=0 2024-04-23T18:27:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 client.15867 shutdown 2024-04-23T18:27:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 -- 172.21.15.19:0/237783923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 msgr2=0x5566a98fec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a98fec70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff408011820 tx=0x7ff408004a40 comp rx=0 tx=0).stop 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 -- 172.21.15.19:0/237783923 shutdown_connections 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5566a997b7d0 0x5566a997dbf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566a98f9f40 0x5566a98fe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff3f9ffb640 0 client.15867 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566a9904800 0x5566a98fec70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 --2- 172.21.15.19:0/237783923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566a98f9620 0x5566a98fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 -- 172.21.15.19:0/237783923 >> 172.21.15.19:0/237783923 conn(0x5566a98f5b30 msgr2=0x5566a9900c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 -- 172.21.15.19:0/237783923 shutdown_connections 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 1 -- 172.21.15.19:0/237783923 wait complete. 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.167+0000 7ff41b209740 20 client.15867 trim_cache size 0 max 16384 2024-04-23T18:27:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:27:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:27:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:27:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 -- 172.21.15.19:0/1715841604 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff3fc0a3dc0 msgr2=0x7ff3fc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff3fc0a3dc0 0x7ff3fc0a6280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff408005fd0 tx=0x7ff408004ab0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 -- 172.21.15.19:0/1715841604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4101021d0 msgr2=0x7ff410198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4101021d0 0x7ff410198640 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00d950 tx=0x7ff40c00de10 comp rx=0 tx=0).stop 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 -- 172.21.15.19:0/1715841604 shutdown_connections 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff3fc0a3dc0 0x7ff3fc0a6280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4101072a0 0x7ff410198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41019cf00 0x7ff41019d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 --2- 172.21.15.19:0/1715841604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4101021d0 0x7ff410198640 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 -- 172.21.15.19:0/1715841604 >> 172.21.15.19:0/1715841604 conn(0x7ff4100793e0 msgr2=0x7ff4100ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 -- 172.21.15.19:0/1715841604 shutdown_connections 2024-04-23T18:27:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:27:56.168+0000 7ff4184ef640 1 -- 172.21.15.19:0/1715841604 wait complete. 2024-04-23T18:27:58.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:57 smithi019 ceph-mon[29215]: pgmap v57: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T18:27:58.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:57 smithi044 ceph-mon[32458]: pgmap v57: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T18:27:58.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:57 smithi155 ceph-mon[33468]: pgmap v57: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T18:28:00.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:27:59 smithi019 ceph-mon[29215]: pgmap v58: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T18:28:00.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:27:59 smithi044 ceph-mon[32458]: pgmap v58: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T18:28:00.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:27:59 smithi155 ceph-mon[33468]: pgmap v58: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T18:28:02.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:01 smithi019 ceph-mon[29215]: pgmap v59: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-23T18:28:02.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:01 smithi044 ceph-mon[32458]: pgmap v59: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-23T18:28:02.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:01 smithi155 ceph-mon[33468]: pgmap v59: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-23T18:28:04.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:03 smithi019 ceph-mon[29215]: pgmap v60: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:28:04.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:03 smithi044 ceph-mon[32458]: pgmap v60: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:28:04.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:03 smithi155 ceph-mon[33468]: pgmap v60: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T18:28:06.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:05 smithi019 ceph-mon[29215]: pgmap v61: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 37 op/s 2024-04-23T18:28:06.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:05 smithi044 ceph-mon[32458]: pgmap v61: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 37 op/s 2024-04-23T18:28:06.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:05 smithi155 ceph-mon[33468]: pgmap v61: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 37 op/s 2024-04-23T18:28:08.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:07 smithi019 ceph-mon[29215]: pgmap v62: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 30 op/s 2024-04-23T18:28:08.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:07 smithi044 ceph-mon[32458]: pgmap v62: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 30 op/s 2024-04-23T18:28:08.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:07 smithi155 ceph-mon[33468]: pgmap v62: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 30 op/s 2024-04-23T18:28:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.268+0000 7fbbcaecb640 1 -- 172.21.15.19:0/1038220363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbc41010e0 msgr2=0x7fbbc41014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.268+0000 7fbbcaecb640 1 --2- 172.21.15.19:0/1038220363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbc41010e0 0x7fbbc41014c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009900 tx=0x7fbbc002a410 comp rx=0 tx=0).stop 2024-04-23T18:28:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 -- 172.21.15.19:0/1038220363 shutdown_connections 2024-04-23T18:28:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 --2- 172.21.15.19:0/1038220363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbc4101a90 0x7fbbc4105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 --2- 172.21.15.19:0/1038220363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc41061b0 0x7fbbc410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 --2- 172.21.15.19:0/1038220363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbc41010e0 0x7fbbc41014c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 -- 172.21.15.19:0/1038220363 >> 172.21.15.19:0/1038220363 conn(0x7fbbc40771d0 msgr2=0x7fbbc40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 -- 172.21.15.19:0/1038220363 shutdown_connections 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 -- 172.21.15.19:0/1038220363 wait complete. 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 Processor -- start 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 -- start start 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc41010e0 0x7fbbc41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.269+0000 7fbbcaecb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbc41061b0 0x7fbbc4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbcaecb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbc419cf70 0x7fbbc419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbcaecb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4111120 con 0x7fbbc41061b0 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbcaecb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4110fa0 con 0x7fbbc419cf70 2024-04-23T18:28:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbcaecb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41112a0 con 0x7fbbc41010e0 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbc41061b0 0x7fbbc4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbc41061b0 0x7fbbc4198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40874/0 (socket says 172.21.15.19:40874) 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbbbfff640 1 -- 172.21.15.19:0/2495007043 learned_addr learned my addr 172.21.15.19:0/2495007043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbc8c67640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc41010e0 0x7fbbc41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbc9468640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbc419cf70 0x7fbbc419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbbbfff640 1 -- 172.21.15.19:0/2495007043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc41010e0 msgr2=0x7fbbc41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc41010e0 0x7fbbc41986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbbbfff640 1 -- 172.21.15.19:0/2495007043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbc419cf70 msgr2=0x7fbbc419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbc419cf70 0x7fbbc419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbbbfff640 1 -- 172.21.15.19:0/2495007043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc419da60 con 0x7fbbc41061b0 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbc41061b0 0x7fbbc4198bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4005fd0 tx=0x7fbbb400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbb9ffb640 1 -- 172.21.15.19:0/2495007043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4017070 con 0x7fbbc41061b0 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbb9ffb640 1 -- 172.21.15.19:0/2495007043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb4009c30 con 0x7fbbc41061b0 2024-04-23T18:28:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbcaecb640 1 -- 172.21.15.19:0/2495007043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc419dd50 con 0x7fbbc41061b0 2024-04-23T18:28:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.270+0000 7fbbb9ffb640 1 -- 172.21.15.19:0/2495007043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4003d30 con 0x7fbbc41061b0 2024-04-23T18:28:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.271+0000 7fbbcaecb640 1 -- 172.21.15.19:0/2495007043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc41a5540 con 0x7fbbc41061b0 2024-04-23T18:28:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.273+0000 7fbbb9ffb640 1 -- 172.21.15.19:0/2495007043 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbbb4003ed0 con 0x7fbbc41061b0 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.273+0000 7fbbcdbdb740 1 Processor -- start 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.273+0000 7fbbcdbdb740 1 -- start start 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.273+0000 7fbbb9ffb640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbbac0a3dc0 0x7fbbac0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255620 0x563797255a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255f40 0x5637972602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x563797262bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5637971ba970 con 0x563797260800 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5637971574a0 con 0x563797255620 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5637971b6930 con 0x563797255f40 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbb9ffb640 1 -- 172.21.15.19:0/2495007043 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbbb40d04d0 con 0x7fbbc41061b0 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbbac0a3dc0 0x7fbbac0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x563797262bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x563797262bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40884/0 (socket says 172.21.15.19:40884) 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 -- 172.21.15.19:0/3502336948 learned_addr learned my addr 172.21.15.19:0/3502336948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc8c67640 1 --2- 172.21.15.19:0/3502336948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255620 0x563797255a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/3502336948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255f40 0x5637972602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 -- 172.21.15.19:0/3502336948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255f40 msgr2=0x5637972602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 --2- 172.21.15.19:0/3502336948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255f40 0x5637972602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 -- 172.21.15.19:0/3502336948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255620 msgr2=0x563797255a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 --2- 172.21.15.19:0/3502336948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255620 0x563797255a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 -- 172.21.15.19:0/3502336948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563796ecd210 con 0x563797260800 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbbac0a3dc0 0x7fbbac0a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009fd0 tx=0x7fbbbc009dc0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.274+0000 7fbbc9468640 1 --2- 172.21.15.19:0/3502336948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x563797262bf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc010c30 tx=0x7fbbbc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bc8ba559f926fc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbaaffd640 1 -- 172.21.15.19:0/3502336948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc025070 con 0x563797260800 2024-04-23T18:28:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbaaffd640 1 -- 172.21.15.19:0/3502336948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc01fc00 con 0x563797260800 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbaaffd640 1 -- 172.21.15.19:0/3502336948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc004050 con 0x563797260800 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3502336948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 msgr2=0x563797262bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3502336948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x563797262bf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc010c30 tx=0x7fbbbc01c000 comp rx=0 tx=0).stop 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3502336948 shutdown_connections 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3502336948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x563797262bf0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3502336948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255f40 0x5637972602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3502336948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255620 0x563797255a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3502336948 >> 172.21.15.19:0/3502336948 conn(0x563797251b30 msgr2=0x563797251f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3502336948 shutdown_connections 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.275+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3502336948 wait complete. 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.276+0000 7fbbcdbdb740 1 -- start start 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.276+0000 7fbbcdbdb740 1 Processor -- start 2024-04-23T18:28:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.276+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797255620 0x5637972600e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.276+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379725d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.276+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797260800 0x56379725d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.276+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5637972677e0 con 0x563797255620 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.276+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5637971550b0 con 0x563797255f40 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.276+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563797267960 con 0x563797260800 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.276+0000 7fbbc8c67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797255620 0x5637972600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.276+0000 7fbbcdbdb740 10 client.0 _ll_register_callbacks cb 0x563797142d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbc8c67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797255620 0x5637972600e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40896/0 (socket says 172.21.15.19:40896) 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbc8c67640 1 -- 172.21.15.19:0/2065547415 learned_addr learned my addr 172.21.15.19:0/2065547415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbc9468640 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797260800 0x56379725d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379725d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbc8c67640 1 -- 172.21.15.19:0/2065547415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797260800 msgr2=0x56379725d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbc8c67640 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797260800 0x56379725d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbc8c67640 1 -- 172.21.15.19:0/2065547415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 msgr2=0x56379725d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbc8c67640 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379725d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbc8c67640 1 -- 172.21.15.19:0/2065547415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563796f99620 con 0x563797255620 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbc9468640 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797260800 0x56379725d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379725d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbc8c67640 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797255620 0x5637972600e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0005c40 tx=0x7fbbc0031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de839756a87eb8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbba3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbba3fff640 1 -- 172.21.15.19:0/2065547415 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0035040 con 0x563797255620 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbba3fff640 1 -- 172.21.15.19:0/2065547415 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbbc0004240 con 0x563797255620 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/2065547415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbbc01fc00 con 0x563797255620 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbba3fff640 1 -- 172.21.15.19:0/2065547415 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbc00318d0 con 0x563797255620 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbcdbdb740 10 client.15879 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.277+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/2065547415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563797273710 con 0x563797255620 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.278+0000 7fbba3fff640 1 -- 172.21.15.19:0/2065547415 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc01fc00 con 0x563797255620 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.278+0000 7fbba3fff640 1 -- 172.21.15.19:0/2065547415 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbbc0004a20 con 0x563797255620 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.278+0000 7fbbcdbdb740 10 client.15879 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.278+0000 7fbbcdbdb740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.278+0000 7fbbcdbdb740 20 client.15879 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.278+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563797274b80 0x563797276f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.278+0000 7fbbcdbdb740 4 client.15879 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.278+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/2065547415 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637971c8390 con 0x563797274b80 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.278+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563797274b80 0x563797276f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.279+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563797274b80 0x563797276f70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f9aa3b2448b888 server_cookie=5dbff93d5609eb0a in_seq=0 out_seq=0 2024-04-23T18:28:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.279+0000 7fbba3fff640 10 client.15879 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbba3fff640 1 -- 172.21.15.19:0/2065547415 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637971c8390 con 0x563797274b80 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbba3fff640 10 client.15879 handle_command_reply: tid=0 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 client.15879 shutdown 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/2065547415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797255620 msgr2=0x5637972600e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797255620 0x5637972600e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0005c40 tx=0x7fbbc0031720 comp rx=0 tx=0).stop 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/2065547415 shutdown_connections 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563797274b80 0x563797276f70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379725d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797260800 0x56379725d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbba3fff640 0 client.15879 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/2065547415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797255620 0x5637972600e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/2065547415 >> 172.21.15.19:0/2065547415 conn(0x563797251b30 msgr2=0x5637972578b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/2065547415 shutdown_connections 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/2065547415 wait complete. 2024-04-23T18:28:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.280+0000 7fbbcdbdb740 20 client.15879 trim_cache size 0 max 16384 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbcdbdb740 1 Processor -- start 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbcdbdb740 1 -- start start 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255620 0x5637972706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379727c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379727cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41ab890 con 0x563797260800 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41ad3d0 con 0x563797255f40 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41abf60 con 0x563797255620 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbc9468640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379727cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbc9468640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379727cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40904/0 (socket says 172.21.15.19:40904) 2024-04-23T18:28:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.295+0000 7fbbc9468640 1 -- 172.21.15.19:0/3765348536 learned_addr learned my addr 172.21.15.19:0/3765348536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbc8c67640 1 --2- 172.21.15.19:0/3765348536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255620 0x5637972706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/3765348536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379727c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbc9468640 1 -- 172.21.15.19:0/3765348536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255620 msgr2=0x5637972706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbc9468640 1 --2- 172.21.15.19:0/3765348536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255620 0x5637972706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbc9468640 1 -- 172.21.15.19:0/3765348536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 msgr2=0x56379727c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbc9468640 1 --2- 172.21.15.19:0/3765348536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379727c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbc9468640 1 -- 172.21.15.19:0/3765348536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56379725c380 con 0x563797260800 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/3765348536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379727c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbc9468640 1 --2- 172.21.15.19:0/3765348536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379727cda0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc006010 tx=0x7fbbbc01fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649c432d8b8d6319 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbaa7fc640 1 -- 172.21.15.19:0/3765348536 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc025040 con 0x563797260800 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbaa7fc640 1 -- 172.21.15.19:0/3765348536 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc003e20 con 0x563797260800 2024-04-23T18:28:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.296+0000 7fbbaa7fc640 1 -- 172.21.15.19:0/3765348536 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc005600 con 0x563797260800 2024-04-23T18:28:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3765348536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 msgr2=0x56379727cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3765348536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379727cda0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc006010 tx=0x7fbbbc01fde0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3765348536 shutdown_connections 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3765348536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379727c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3765348536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379727cda0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3765348536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255620 0x5637972706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3765348536 >> 172.21.15.19:0/3765348536 conn(0x563797251b30 msgr2=0x563797251f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3765348536 shutdown_connections 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3765348536 wait complete. 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 -- start start 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.297+0000 7fbbcdbdb740 1 Processor -- start 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255620 0x56379726a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379726ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbcdbdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379726b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41acd00 con 0x563797260800 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41ac630 con 0x563797255f40 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbcdbdb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41a9890 con 0x563797255620 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbcdbdb740 10 client.0 _ll_register_callbacks cb 0x7fbbc0005e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbc9468640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379726b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbc9468640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379726b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40908/0 (socket says 172.21.15.19:40908) 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbc9468640 1 -- 172.21.15.19:0/3551106459 learned_addr learned my addr 172.21.15.19:0/3551106459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbbbfff640 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379726ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbc8c67640 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255620 0x56379726a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbc9468640 1 -- 172.21.15.19:0/3551106459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255620 msgr2=0x56379726a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbc9468640 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255620 0x56379726a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbc9468640 1 -- 172.21.15.19:0/3551106459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 msgr2=0x56379726ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbc9468640 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379726ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbc9468640 1 -- 172.21.15.19:0/3551106459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637972772e0 con 0x563797260800 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbc9468640 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379726b220 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc023040 tx=0x7fbbbc0127f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9472ef7839c64626 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbba3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbba3fff640 1 -- 172.21.15.19:0/3551106459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc036070 con 0x563797260800 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbba3fff640 1 -- 172.21.15.19:0/3551106459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbbbc012240 con 0x563797260800 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.298+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3551106459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbbc003e20 con 0x563797260800 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbba3fff640 1 -- 172.21.15.19:0/3551106459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbbc025040 con 0x563797260800 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbbcdbdb740 10 client.15885 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3551106459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563797255d70 con 0x563797260800 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbba3fff640 1 -- 172.21.15.19:0/3551106459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc003e20 con 0x563797260800 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbba3fff640 1 -- 172.21.15.19:0/3551106459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbbbc028070 con 0x563797260800 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbbcdbdb740 10 client.15885 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbbcdbdb740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbbcdbdb740 20 client.15885 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5637972db710 0x5637972ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbbcdbdb740 4 client.15885 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3551106459 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637971c2200 con 0x5637972db710 2024-04-23T18:28:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.299+0000 7fbbc8c67640 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5637972db710 0x5637972ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.300+0000 7fbba3fff640 10 client.15885 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.300+0000 7fbbc8c67640 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5637972db710 0x5637972ddb00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2934b73fe33f67d server_cookie=b66bf2cd78762036 in_seq=0 out_seq=0 2024-04-23T18:28:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbba3fff640 1 -- 172.21.15.19:0/3551106459 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637971c2200 con 0x5637972db710 2024-04-23T18:28:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbba3fff640 10 client.15885 handle_command_reply: tid=0 2024-04-23T18:28:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 client.15885 shutdown 2024-04-23T18:28:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3551106459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 msgr2=0x56379726b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379726b220 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc023040 tx=0x7fbbbc0127f0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3551106459 shutdown_connections 2024-04-23T18:28:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5637972db710 0x5637972ddb00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563797255f40 0x56379726ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563797260800 0x56379726b220 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbba3fff640 0 client.15885 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 --2- 172.21.15.19:0/3551106459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563797255620 0x56379726a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3551106459 >> 172.21.15.19:0/3551106459 conn(0x563797251b30 msgr2=0x563797250cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3551106459 shutdown_connections 2024-04-23T18:28:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 1 -- 172.21.15.19:0/3551106459 wait complete. 2024-04-23T18:28:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.316+0000 7fbbcdbdb740 20 client.15885 trim_cache size 0 max 16384 2024-04-23T18:28:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:28:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:28:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:28:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:28:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 -- 172.21.15.19:0/2495007043 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbbac0a3dc0 msgr2=0x7fbbac0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbbac0a3dc0 0x7fbbac0a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009fd0 tx=0x7fbbbc009dc0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 -- 172.21.15.19:0/2495007043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbc41061b0 msgr2=0x7fbbc4198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbc41061b0 0x7fbbc4198bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4005fd0 tx=0x7fbbb400dde0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 -- 172.21.15.19:0/2495007043 shutdown_connections 2024-04-23T18:28:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbbac0a3dc0 0x7fbbac0a6280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbc419cf70 0x7fbbc419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbc41061b0 0x7fbbc4198bf0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 --2- 172.21.15.19:0/2495007043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc41010e0 0x7fbbc41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 -- 172.21.15.19:0/2495007043 >> 172.21.15.19:0/2495007043 conn(0x7fbbc40771d0 msgr2=0x7fbbc406aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 -- 172.21.15.19:0/2495007043 shutdown_connections 2024-04-23T18:28:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:09.317+0000 7fbbcaecb640 1 -- 172.21.15.19:0/2495007043 wait complete. 2024-04-23T18:28:10.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:09 smithi019 ceph-mon[29215]: pgmap v63: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 30 op/s 2024-04-23T18:28:10.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:09 smithi044 ceph-mon[32458]: pgmap v63: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 30 op/s 2024-04-23T18:28:10.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:09 smithi155 ceph-mon[33468]: pgmap v63: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 30 op/s 2024-04-23T18:28:12.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:11 smithi019 ceph-mon[29215]: pgmap v64: 97 pgs: 97 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 37 op/s 2024-04-23T18:28:12.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:11 smithi044 ceph-mon[32458]: pgmap v64: 97 pgs: 97 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 37 op/s 2024-04-23T18:28:12.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:11 smithi155 ceph-mon[33468]: pgmap v64: 97 pgs: 97 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 37 op/s 2024-04-23T18:28:14.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:13 smithi019 ceph-mon[29215]: pgmap v65: 97 pgs: 97 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-23T18:28:14.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:13 smithi044 ceph-mon[32458]: pgmap v65: 97 pgs: 97 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-23T18:28:14.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:13 smithi155 ceph-mon[33468]: pgmap v65: 97 pgs: 97 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-23T18:28:15.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:14 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:15.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:14 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:15.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:14 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:16.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:15 smithi019 ceph-mon[29215]: pgmap v66: 97 pgs: 97 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-23T18:28:16.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:15 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:16.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:15 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:16.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:15 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:28:16.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:15 smithi044 ceph-mon[32458]: pgmap v66: 97 pgs: 97 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-23T18:28:16.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:15 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:16.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:15 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:16.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:15 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:28:16.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:15 smithi155 ceph-mon[33468]: pgmap v66: 97 pgs: 97 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-23T18:28:16.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:15 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:16.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:15 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:16.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:15 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:28:16.072 INFO:tasks.workunit.client.0.smithi019.stdout:1/600: open fd = 3 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:2/600: open fd = 3 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:3/600: open fd = 3 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:4/600: open fd = 3 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.073 INFO:tasks.workunit.client.0.smithi019.stdout:5/600: open fd = 3 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:6/600: open fd = 3 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:7/600: open fd = 3 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:8/600: open fd = 3 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.074 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:9/600: open fd = 3 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:10/600: open fd = 3 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:11/600: open fd = 3 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:12/600: open fd = 3 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.075 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:13/600: open fd = 3 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:14/600: open fd = 3 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:15/600: open fd = 3 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:16/600: open fd = 3 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.076 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:17/600: open fd = 3 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:18/600: open fd = 3 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:19/600: open fd = 3 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:20/600: open fd = 3 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.077 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:21/600: open fd = 3 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:22/600: open fd = 3 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:23/600: open fd = 3 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:24/600: open fd = 3 2024-04-23T18:28:16.078 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:25/600: open fd = 3 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:26/600: open fd = 3 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:27/600: open fd = 3 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:28/600: open fd = 3 2024-04-23T18:28:16.079 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:29/600: open fd = 3 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:30/600: open fd = 3 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:31/600: open fd = 3 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:32/600: open fd = 3 2024-04-23T18:28:16.080 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:33/600: open fd = 3 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:34/600: open fd = 3 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:35/600: open fd = 3 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.081 INFO:tasks.workunit.client.0.smithi019.stdout:36/600: open fd = 3 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:37/600: open fd = 3 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:38/600: open fd = 3 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:39/600: open fd = 3 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.082 INFO:tasks.workunit.client.0.smithi019.stdout:40/600: open fd = 3 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:41/600: open fd = 3 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:42/600: open fd = 3 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:43/600: open fd = 3 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.083 INFO:tasks.workunit.client.0.smithi019.stdout:44/600: open fd = 3 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:45/600: open fd = 3 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:46/600: open fd = 3 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:47/600: open fd = 3 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.084 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:48/600: open fd = 3 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:49/600: open fd = 3 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:50/600: open fd = 3 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:51/600: open fd = 3 2024-04-23T18:28:16.085 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:52/600: open fd = 3 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:53/600: open fd = 3 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:54/600: open fd = 3 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.086 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:55/600: open fd = 3 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:56/600: open fd = 3 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:57/600: open fd = 3 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:58/600: open fd = 3 2024-04-23T18:28:16.087 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:59/600: open fd = 3 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:60/600: open fd = 3 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:61/600: open fd = 3 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.088 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.089 INFO:tasks.workunit.client.0.smithi019.stdout:62/600: open fd = 3 2024-04-23T18:28:16.089 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.089 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.089 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:16.089 INFO:tasks.workunit.client.0.smithi019.stdout:63/600: open fd = 3 2024-04-23T18:28:16.089 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.089 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:28:16.089 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:28:18.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:17 smithi019 ceph-mon[29215]: pgmap v67: 97 pgs: 97 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T18:28:18.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:17 smithi044 ceph-mon[32458]: pgmap v67: 97 pgs: 97 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T18:28:18.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:17 smithi155 ceph-mon[33468]: pgmap v67: 97 pgs: 97 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T18:28:20.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:19 smithi019 ceph-mon[29215]: pgmap v68: 97 pgs: 97 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T18:28:20.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:19 smithi044 ceph-mon[32458]: pgmap v68: 97 pgs: 97 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T18:28:20.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:19 smithi155 ceph-mon[33468]: pgmap v68: 97 pgs: 97 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T18:28:21.284 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.6 2024-04-23T18:28:21.285 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:28:21.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.408+0000 7f8b4facf640 1 -- 172.21.15.19:0/2519030013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b481016d0 msgr2=0x7f8b4810e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.408+0000 7f8b4facf640 1 --2- 172.21.15.19:0/2519030013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b481016d0 0x7f8b4810e420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009900 tx=0x7f8b3802a410 comp rx=0 tx=0).stop 2024-04-23T18:28:21.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.408+0000 7f8b4facf640 1 -- 172.21.15.19:0/2519030013 shutdown_connections 2024-04-23T18:28:21.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.408+0000 7f8b4facf640 1 --2- 172.21.15.19:0/2519030013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b4810e960 0x7f8b48110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.408+0000 7f8b4facf640 1 --2- 172.21.15.19:0/2519030013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b481016d0 0x7f8b4810e420 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.408+0000 7f8b4facf640 1 --2- 172.21.15.19:0/2519030013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b48100db0 0x7f8b48101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.408+0000 7f8b4facf640 1 -- 172.21.15.19:0/2519030013 >> 172.21.15.19:0/2519030013 conn(0x7f8b48077fb0 msgr2=0x7f8b480783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.408+0000 7f8b4facf640 1 -- 172.21.15.19:0/2519030013 shutdown_connections 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.408+0000 7f8b4facf640 1 -- 172.21.15.19:0/2519030013 wait complete. 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4facf640 1 Processor -- start 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4facf640 1 -- start start 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4facf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b48100db0 0x7f8b481a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4facf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b481016d0 0x7f8b481a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4facf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b4810e960 0x7f8b4819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4facf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48114410 con 0x7f8b48100db0 2024-04-23T18:28:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4facf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48114290 con 0x7f8b481016d0 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4facf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48114590 con 0x7f8b4810e960 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4d86b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b48100db0 0x7f8b481a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4d86b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b48100db0 0x7f8b481a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43944/0 (socket says 172.21.15.19:43944) 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.409+0000 7f8b4d86b640 1 -- 172.21.15.19:0/2320598959 learned_addr learned my addr 172.21.15.19:0/2320598959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b4e06c640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b4810e960 0x7f8b4819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b4d06a640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b481016d0 0x7f8b481a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b4d86b640 1 -- 172.21.15.19:0/2320598959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b4810e960 msgr2=0x7f8b4819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b4d86b640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b4810e960 0x7f8b4819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b4d86b640 1 -- 172.21.15.19:0/2320598959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b481016d0 msgr2=0x7f8b481a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b4d86b640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b481016d0 0x7f8b481a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b4d86b640 1 -- 172.21.15.19:0/2320598959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4819af80 con 0x7f8b48100db0 2024-04-23T18:28:21.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b4d86b640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b48100db0 0x7f8b481a0640 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400d950 tx=0x7f8b4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:21.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b3effd640 1 -- 172.21.15.19:0/2320598959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44017070 con 0x7f8b48100db0 2024-04-23T18:28:21.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b3effd640 1 -- 172.21.15.19:0/2320598959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b44009c30 con 0x7f8b48100db0 2024-04-23T18:28:21.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b3effd640 1 -- 172.21.15.19:0/2320598959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400f620 con 0x7f8b48100db0 2024-04-23T18:28:21.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b4facf640 1 -- 172.21.15.19:0/2320598959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4819b270 con 0x7f8b48100db0 2024-04-23T18:28:21.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.410+0000 7f8b4facf640 1 -- 172.21.15.19:0/2320598959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b481a7490 con 0x7f8b48100db0 2024-04-23T18:28:21.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.411+0000 7f8b4facf640 1 -- 172.21.15.19:0/2320598959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4806ad80 con 0x7f8b48100db0 2024-04-23T18:28:21.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.413+0000 7f8b3effd640 1 -- 172.21.15.19:0/2320598959 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b44010460 con 0x7f8b48100db0 2024-04-23T18:28:21.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.413+0000 7f8b3effd640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b340a3d20 0x7f8b340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.413+0000 7f8b3effd640 1 -- 172.21.15.19:0/2320598959 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8b440d0340 con 0x7f8b48100db0 2024-04-23T18:28:21.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.413+0000 7f8b4d06a640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b340a3d20 0x7f8b340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.414+0000 7f8b4d06a640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b340a3d20 0x7f8b340a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b38031440 tx=0x7f8b38005b50 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:21.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.417+0000 7f8b3effd640 1 -- 172.21.15.19:0/2320598959 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b44009de0 con 0x7f8b48100db0 2024-04-23T18:28:21.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.644+0000 7f8b4facf640 1 -- 172.21.15.19:0/2320598959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b48109b20 con 0x7f8b48100db0 2024-04-23T18:28:21.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.645+0000 7f8b3effd640 1 -- 172.21.15.19:0/2320598959 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f8b4409c3c0 con 0x7f8b48100db0 2024-04-23T18:28:21.638 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:28:21.638 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:28:21.638 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:28:21.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 -- 172.21.15.19:0/2320598959 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b340a3d20 msgr2=0x7f8b340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b340a3d20 0x7f8b340a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b38031440 tx=0x7f8b38005b50 comp rx=0 tx=0).stop 2024-04-23T18:28:21.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 -- 172.21.15.19:0/2320598959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b48100db0 msgr2=0x7f8b481a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b48100db0 0x7f8b481a0640 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400d950 tx=0x7f8b4400de10 comp rx=0 tx=0).stop 2024-04-23T18:28:21.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 -- 172.21.15.19:0/2320598959 shutdown_connections 2024-04-23T18:28:21.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b340a3d20 0x7f8b340a61e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b481016d0 0x7f8b481a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b4810e960 0x7f8b4819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 --2- 172.21.15.19:0/2320598959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b48100db0 0x7f8b481a0640 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 -- 172.21.15.19:0/2320598959 >> 172.21.15.19:0/2320598959 conn(0x7f8b48077fb0 msgr2=0x7f8b4810ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:21.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 -- 172.21.15.19:0/2320598959 shutdown_connections 2024-04-23T18:28:21.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.647+0000 7f8b4facf640 1 -- 172.21.15.19:0/2320598959 wait complete. 2024-04-23T18:28:21.651 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 73.6 --await-for 42.0' 2024-04-23T18:28:21.651 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 73.6 --await-for 42.0 2024-04-23T18:28:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.770+0000 7f98e78ef640 1 -- 172.21.15.19:0/124583995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0107260 msgr2=0x7f98e010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.770+0000 7f98e78ef640 1 --2- 172.21.15.19:0/124583995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0107260 0x7f98e010ee30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009ca0 tx=0x7f98d802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.770+0000 7f98e78ef640 1 -- 172.21.15.19:0/124583995 shutdown_connections 2024-04-23T18:28:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.770+0000 7f98e78ef640 1 --2- 172.21.15.19:0/124583995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0107260 0x7f98e010ee30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.770+0000 7f98e78ef640 1 --2- 172.21.15.19:0/124583995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e0102b40 0x7f98e0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.770+0000 7f98e78ef640 1 --2- 172.21.15.19:0/124583995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98e0102190 0x7f98e0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.770+0000 7f98e78ef640 1 -- 172.21.15.19:0/124583995 >> 172.21.15.19:0/124583995 conn(0x7f98e0079310 msgr2=0x7f98e0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.770+0000 7f98e78ef640 1 -- 172.21.15.19:0/124583995 shutdown_connections 2024-04-23T18:28:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.770+0000 7f98e78ef640 1 -- 172.21.15.19:0/124583995 wait complete. 2024-04-23T18:28:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e78ef640 1 Processor -- start 2024-04-23T18:28:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e78ef640 1 -- start start 2024-04-23T18:28:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e78ef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0102190 0x7f98e019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e78ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98e0107260 0x7f98e019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e78ef640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e01985a0 0x7f98e0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e78ef640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0112240 con 0x7f98e0102190 2024-04-23T18:28:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e78ef640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01120c0 con 0x7f98e0107260 2024-04-23T18:28:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e78ef640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01123c0 con 0x7f98e01985a0 2024-04-23T18:28:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e568b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0102190 0x7f98e019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e568b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0102190 0x7f98e019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43968/0 (socket says 172.21.15.19:43968) 2024-04-23T18:28:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e568b640 1 -- 172.21.15.19:0/319303936 learned_addr learned my addr 172.21.15.19:0/319303936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98e0107260 0x7f98e019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.771+0000 7f98e5e8c640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e01985a0 0x7f98e0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98e568b640 1 -- 172.21.15.19:0/319303936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e01985a0 msgr2=0x7f98e0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98e568b640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e01985a0 0x7f98e0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98e568b640 1 -- 172.21.15.19:0/319303936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98e0107260 msgr2=0x7f98e019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98e568b640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98e0107260 0x7f98e019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98e568b640 1 -- 172.21.15.19:0/319303936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e01990c0 con 0x7f98e0102190 2024-04-23T18:28:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98e5e8c640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e01985a0 0x7f98e0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98e568b640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0102190 0x7f98e019e3f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00b4a0 tx=0x7f98dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98d67fc640 1 -- 172.21.15.19:0/319303936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc014070 con 0x7f98e0102190 2024-04-23T18:28:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98e78ef640 1 -- 172.21.15.19:0/319303936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e01993b0 con 0x7f98e0102190 2024-04-23T18:28:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98d67fc640 1 -- 172.21.15.19:0/319303936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc010ac0 con 0x7f98e0102190 2024-04-23T18:28:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98d67fc640 1 -- 172.21.15.19:0/319303936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc017c30 con 0x7f98e0102190 2024-04-23T18:28:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.772+0000 7f98e78ef640 1 -- 172.21.15.19:0/319303936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e01a5600 con 0x7f98e0102190 2024-04-23T18:28:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.774+0000 7f98d67fc640 1 -- 172.21.15.19:0/319303936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f98dc017070 con 0x7f98e0102190 2024-04-23T18:28:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.774+0000 7f98ea606740 1 Processor -- start 2024-04-23T18:28:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98ea606740 1 -- start start 2024-04-23T18:28:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98ea606740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98ea606740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6a19c0 0x55797e6abb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98ea606740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6ac090 0x55797e6ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98ea606740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55797e606370 con 0x55797e6ac090 2024-04-23T18:28:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98ea606740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55797e5a32e0 con 0x55797e6a19c0 2024-04-23T18:28:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98ea606740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55797e602330 con 0x55797e6a10a0 2024-04-23T18:28:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e4e8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6ac090 0x55797e6ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e4e8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6ac090 0x55797e6ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43984/0 (socket says 172.21.15.19:43984) 2024-04-23T18:28:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e4e8a640 1 -- 172.21.15.19:0/3961451563 learned_addr learned my addr 172.21.15.19:0/3961451563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98d67fc640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f98cc0a3d20 0x7f98cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e568b640 1 --2- 172.21.15.19:0/3961451563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6a19c0 0x55797e6abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e5e8c640 1 --2- 172.21.15.19:0/3961451563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e5e8c640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f98cc0a3d20 0x7f98cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e4e8a640 1 -- 172.21.15.19:0/3961451563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 msgr2=0x55797e6a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/3961451563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e4e8a640 1 -- 172.21.15.19:0/3961451563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6a19c0 msgr2=0x55797e6abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/3961451563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6a19c0 0x55797e6abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e4e8a640 1 -- 172.21.15.19:0/3961451563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55797e3e4d80 con 0x55797e6ac090 2024-04-23T18:28:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e568b640 1 --2- 172.21.15.19:0/3961451563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6a19c0 0x55797e6abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98d67fc640 1 -- 172.21.15.19:0/319303936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f98dc0cfdc0 con 0x7f98e0102190 2024-04-23T18:28:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.775+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/3961451563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6ac090 0x55797e6ae480 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f98d000ab70 tx=0x7f98d000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbd6bd5b2f27ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98e5e8c640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f98cc0a3d20 0x7f98cc0a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009430 tx=0x7f98d80095c0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98cb7fe640 1 -- 172.21.15.19:0/3961451563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0004220 con 0x55797e6ac090 2024-04-23T18:28:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98cb7fe640 1 -- 172.21.15.19:0/3961451563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d00043c0 con 0x55797e6ac090 2024-04-23T18:28:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98cb7fe640 1 -- 172.21.15.19:0/3961451563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0018790 con 0x55797e6ac090 2024-04-23T18:28:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98ea606740 1 -- 172.21.15.19:0/3961451563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6ac090 msgr2=0x55797e6ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98ea606740 1 --2- 172.21.15.19:0/3961451563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6ac090 0x55797e6ae480 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f98d000ab70 tx=0x7f98d000f790 comp rx=0 tx=0).stop 2024-04-23T18:28:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98ea606740 1 -- 172.21.15.19:0/3961451563 shutdown_connections 2024-04-23T18:28:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98ea606740 1 --2- 172.21.15.19:0/3961451563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6a19c0 0x55797e6abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98ea606740 1 --2- 172.21.15.19:0/3961451563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6ac090 0x55797e6ae480 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98ea606740 1 --2- 172.21.15.19:0/3961451563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98ea606740 1 -- 172.21.15.19:0/3961451563 >> 172.21.15.19:0/3961451563 conn(0x55797e69d3c0 msgr2=0x55797e69d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98ea606740 1 -- 172.21.15.19:0/3961451563 shutdown_connections 2024-04-23T18:28:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.776+0000 7f98ea606740 1 -- 172.21.15.19:0/3961451563 wait complete. 2024-04-23T18:28:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98ea606740 1 -- start start 2024-04-23T18:28:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98ea606740 1 Processor -- start 2024-04-23T18:28:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98ea606740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98ea606740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98ea606740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98ea606740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55797e6aee80 con 0x55797e6a19c0 2024-04-23T18:28:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98ea606740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55797e5a0fb0 con 0x55797e6ac090 2024-04-23T18:28:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98ea606740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55797e6af000 con 0x55797e6a10a0 2024-04-23T18:28:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98ea606740 10 client.0 _ll_register_callbacks cb 0x55797e5a09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98e4e8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98e4e8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43996/0 (socket says 172.21.15.19:43996) 2024-04-23T18:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98e4e8a640 1 -- 172.21.15.19:0/3287685397 learned_addr learned my addr 172.21.15.19:0/3287685397 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.777+0000 7f98e568b640 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98e5e8c640 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98e4e8a640 1 -- 172.21.15.19:0/3287685397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 msgr2=0x55797e6a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98e4e8a640 1 -- 172.21.15.19:0/3287685397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 msgr2=0x55797e6a9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6a9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98e4e8a640 1 -- 172.21.15.19:0/3287685397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55797e445510 con 0x55797e6a19c0 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a9510 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f98d0003e50 tx=0x7f98d000afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20b63a6eb4c5928 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98c8ff9640 1 -- 172.21.15.19:0/3287685397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0027020 con 0x55797e6a19c0 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98c8ff9640 1 -- 172.21.15.19:0/3287685397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f98d0003980 con 0x55797e6a19c0 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98ea606740 1 -- 172.21.15.19:0/3287685397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d00043c0 con 0x55797e6a19c0 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98c8ff9640 1 -- 172.21.15.19:0/3287685397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98d0032050 con 0x55797e6a19c0 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98ea606740 10 client.15903 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98ea606740 1 -- 172.21.15.19:0/3287685397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55797e2d8670 con 0x55797e6a19c0 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98c8ff9640 1 -- 172.21.15.19:0/3287685397 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d00043c0 con 0x55797e6a19c0 2024-04-23T18:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.778+0000 7f98c8ff9640 1 -- 172.21.15.19:0/3287685397 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f98d0004bf0 con 0x55797e6a19c0 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.779+0000 7f98ea606740 10 client.15903 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.779+0000 7f98ea606740 10 client.15903 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.779+0000 7f98ea606740 20 client.15903 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.779+0000 7f98ea606740 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55797e6bc280 0x55797e6be670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.779+0000 7f98ea606740 4 client.15903 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.779+0000 7f98ea606740 1 -- 172.21.15.19:0/3287685397 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55797e613d90 con 0x55797e6bc280 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.779+0000 7f98e568b640 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55797e6bc280 0x55797e6be670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.779+0000 7f98e568b640 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55797e6bc280 0x55797e6be670 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00c11c91099c0df server_cookie=963ed05a92c8ef51 in_seq=0 out_seq=0 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.779+0000 7f98c8ff9640 10 client.15903 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98c8ff9640 1 -- 172.21.15.19:0/3287685397 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55797e613d90 con 0x55797e6bc280 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98c8ff9640 10 client.15903 handle_command_reply: tid=0 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 client.15903 shutdown 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 -- 172.21.15.19:0/3287685397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 msgr2=0x55797e6a9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a9510 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f98d0003e50 tx=0x7f98d000afb0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 -- 172.21.15.19:0/3287685397 shutdown_connections 2024-04-23T18:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55797e6bc280 0x55797e6be670 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a9510 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98c8ff9640 0 client.15903 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 --2- 172.21.15.19:0/3287685397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 -- 172.21.15.19:0/3287685397 >> 172.21.15.19:0/3287685397 conn(0x55797e69d3c0 msgr2=0x55797e6a31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 -- 172.21.15.19:0/3287685397 shutdown_connections 2024-04-23T18:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 1 -- 172.21.15.19:0/3287685397 wait complete. 2024-04-23T18:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.780+0000 7f98ea606740 20 client.15903 trim_cache size 0 max 16384 2024-04-23T18:28:21.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98ea606740 1 Processor -- start 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98ea606740 1 -- start start 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98ea606740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98ea606740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98ea606740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98ea606740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01adbd0 con 0x55797e6a19c0 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98ea606740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01af710 con 0x55797e6ac090 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98ea606740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01ae2a0 con 0x55797e6a10a0 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e4e8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e4e8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44008/0 (socket says 172.21.15.19:44008) 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e4e8a640 1 -- 172.21.15.19:0/878910866 learned_addr learned my addr 172.21.15.19:0/878910866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e568b640 1 --2- 172.21.15.19:0/878910866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e4e8a640 1 -- 172.21.15.19:0/878910866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 msgr2=0x55797e6b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/878910866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e4e8a640 1 -- 172.21.15.19:0/878910866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 msgr2=0x55797e6c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/878910866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e4e8a640 1 -- 172.21.15.19:0/878910866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55797e6a06b0 con 0x55797e6a19c0 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e568b640 1 --2- 172.21.15.19:0/878910866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.798+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/878910866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a3910 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f98d0018670 tx=0x7f98d0018440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f24a25ddfb9ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98caffd640 1 -- 172.21.15.19:0/878910866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0027070 con 0x55797e6a19c0 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98caffd640 1 -- 172.21.15.19:0/878910866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d003b070 con 0x55797e6a19c0 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98caffd640 1 -- 172.21.15.19:0/878910866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d00048c0 con 0x55797e6a19c0 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98ea606740 1 -- 172.21.15.19:0/878910866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 msgr2=0x55797e6a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98ea606740 1 --2- 172.21.15.19:0/878910866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a3910 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f98d0018670 tx=0x7f98d0018440 comp rx=0 tx=0).stop 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98ea606740 1 -- 172.21.15.19:0/878910866 shutdown_connections 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98ea606740 1 --2- 172.21.15.19:0/878910866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98ea606740 1 --2- 172.21.15.19:0/878910866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a3910 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98ea606740 1 --2- 172.21.15.19:0/878910866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98ea606740 1 -- 172.21.15.19:0/878910866 >> 172.21.15.19:0/878910866 conn(0x55797e69d3c0 msgr2=0x55797e69c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98ea606740 1 -- 172.21.15.19:0/878910866 shutdown_connections 2024-04-23T18:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.799+0000 7f98ea606740 1 -- 172.21.15.19:0/878910866 wait complete. 2024-04-23T18:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98ea606740 1 -- start start 2024-04-23T18:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98ea606740 1 Processor -- start 2024-04-23T18:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98ea606740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98ea606740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98ea606740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98ea606740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01af040 con 0x55797e6a19c0 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98ea606740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01ae970 con 0x55797e6ac090 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98ea606740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01abbb0 con 0x55797e6a10a0 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98ea606740 10 client.0 _ll_register_callbacks cb 0x7f98d000ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98e4e8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98e4e8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44022/0 (socket says 172.21.15.19:44022) 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98e4e8a640 1 -- 172.21.15.19:0/1659977236 learned_addr learned my addr 172.21.15.19:0/1659977236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98e568b640 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.800+0000 7f98e5e8c640 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98e4e8a640 1 -- 172.21.15.19:0/1659977236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 msgr2=0x55797e6a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98e4e8a640 1 -- 172.21.15.19:0/1659977236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 msgr2=0x55797e6a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98e4e8a640 1 -- 172.21.15.19:0/1659977236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55797e725c10 con 0x55797e6a19c0 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98e5e8c640 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98e4e8a640 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a63e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f98d003a000 tx=0x7f98d000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6115f60c4d8a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98c8ff9640 1 -- 172.21.15.19:0/1659977236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0027040 con 0x55797e6a19c0 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98ea606740 1 -- 172.21.15.19:0/1659977236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d003b070 con 0x55797e6a19c0 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98c8ff9640 1 -- 172.21.15.19:0/1659977236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f98d0005190 con 0x55797e6a19c0 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98c8ff9640 1 -- 172.21.15.19:0/1659977236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98d004d050 con 0x55797e6a19c0 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98ea606740 10 client.15909 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98ea606740 1 -- 172.21.15.19:0/1659977236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55797e726e40 con 0x55797e6a19c0 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98c8ff9640 1 -- 172.21.15.19:0/1659977236 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d003b070 con 0x55797e6a19c0 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98c8ff9640 1 -- 172.21.15.19:0/1659977236 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f98d0018920 con 0x55797e6a19c0 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98ea606740 10 client.15909 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.801+0000 7f98ea606740 10 client.15909 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.802+0000 7f98ea606740 20 client.15909 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.802+0000 7f98ea606740 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55797e727510 0x55797e729930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.802+0000 7f98ea606740 4 client.15909 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 73.6, "await_for": 42.0}] 2024-04-23T18:28:21.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.802+0000 7f98ea606740 1 -- 172.21.15.19:0/1659977236 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 73.6, "await_for": 42.0}) -- 0x55797e60dc00 con 0x55797e727510 2024-04-23T18:28:21.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.802+0000 7f98e568b640 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55797e727510 0x55797e729930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:21.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.802+0000 7f98e568b640 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55797e727510 0x55797e729930 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da7dfdaba13dc854 server_cookie=73f67ca327092f4f in_seq=0 out_seq=0 2024-04-23T18:28:21.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.802+0000 7f98c8ff9640 10 client.15909 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.824+0000 7f98c8ff9640 1 -- 172.21.15.19:0/1659977236 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55797e60dc00 con 0x55797e727510 2024-04-23T18:28:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.824+0000 7f98c8ff9640 10 client.15909 handle_command_reply: tid=0 2024-04-23T18:28:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.824+0000 7f98ea606740 1 client.15909 shutdown 2024-04-23T18:28:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 1 -- 172.21.15.19:0/1659977236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 msgr2=0x55797e6a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a63e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f98d003a000 tx=0x7f98d000fc90 comp rx=0 tx=0).stop 2024-04-23T18:28:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 1 -- 172.21.15.19:0/1659977236 shutdown_connections 2024-04-23T18:28:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55797e727510 0x55797e729930 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55797e6ac090 0x55797e6a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98c8ff9640 0 client.15909 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797e6a19c0 0x55797e6a63e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 1 --2- 172.21.15.19:0/1659977236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55797e6a10a0 0x55797e6a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 1 -- 172.21.15.19:0/1659977236 >> 172.21.15.19:0/1659977236 conn(0x55797e69d3c0 msgr2=0x55797e6a8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 1 -- 172.21.15.19:0/1659977236 shutdown_connections 2024-04-23T18:28:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 1 -- 172.21.15.19:0/1659977236 wait complete. 2024-04-23T18:28:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98ea606740 20 client.15909 trim_cache size 0 max 16384 2024-04-23T18:28:21.821 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:28:21.822 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:28:21.822 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:28:21.822 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 10, 2024-04-23T18:28:21.822 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:28:21.822 INFO:teuthology.orchestra.run.smithi019.stdout: "d3b5be3e": { 2024-04-23T18:28:21.822 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 10, 2024-04-23T18:28:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:28:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:28:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:28:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:28:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:28:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:28:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 73.6, 2024-04-23T18:28:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:28:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:28:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:28:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:28:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:28:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:28:21.825 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:28:21.825 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:28:21.825 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:28:21.825 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:28:21.825 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:28:21.825 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:28:21.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 -- 172.21.15.19:0/319303936 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f98cc0a3d20 msgr2=0x7f98cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f98cc0a3d20 0x7f98cc0a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009430 tx=0x7f98d80095c0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 -- 172.21.15.19:0/319303936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0102190 msgr2=0x7f98e019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:21.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0102190 0x7f98e019e3f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00b4a0 tx=0x7f98dc00b960 comp rx=0 tx=0).stop 2024-04-23T18:28:21.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 -- 172.21.15.19:0/319303936 shutdown_connections 2024-04-23T18:28:21.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f98cc0a3d20 0x7f98cc0a61e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98e0107260 0x7f98e019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e01985a0 0x7f98e0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 --2- 172.21.15.19:0/319303936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0102190 0x7f98e019e3f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:21.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 -- 172.21.15.19:0/319303936 >> 172.21.15.19:0/319303936 conn(0x7f98e0079310 msgr2=0x7f98e01001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:21.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.825+0000 7f98e78ef640 1 -- 172.21.15.19:0/319303936 shutdown_connections 2024-04-23T18:28:21.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:21.826+0000 7f98e78ef640 1 -- 172.21.15.19:0/319303936 wait complete. 2024-04-23T18:28:21.831 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd3b5be3e', quiesce took 0.5 seconds. Will release after: 13.6 2024-04-23T18:28:22.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:21 smithi019 ceph-mon[29215]: pgmap v69: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T18:28:22.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:21 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2320598959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:28:22.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:21 smithi044 ceph-mon[32458]: pgmap v69: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T18:28:22.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:21 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2320598959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:28:22.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:21 smithi155 ceph-mon[33468]: pgmap v69: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T18:28:22.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:21 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2320598959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:28:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.413+0000 7fa4c74cf640 1 -- 172.21.15.19:0/3268528365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4c01062f0 msgr2=0x7fa4c0106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.413+0000 7fa4c74cf640 1 --2- 172.21.15.19:0/3268528365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4c01062f0 0x7fa4c0106780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009900 tx=0x7fa4b802a510 comp rx=0 tx=0).stop 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.413+0000 7fa4c74cf640 1 -- 172.21.15.19:0/3268528365 shutdown_connections 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.413+0000 7fa4c74cf640 1 --2- 172.21.15.19:0/3268528365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4c01062f0 0x7fa4c0106780 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.413+0000 7fa4c74cf640 1 --2- 172.21.15.19:0/3268528365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c01018f0 0x7fa4c0105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.413+0000 7fa4c74cf640 1 --2- 172.21.15.19:0/3268528365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4c0100fd0 0x7fa4c01013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.413+0000 7fa4c74cf640 1 -- 172.21.15.19:0/3268528365 >> 172.21.15.19:0/3268528365 conn(0x7fa4c0077fd0 msgr2=0x7fa4c00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.413+0000 7fa4c74cf640 1 -- 172.21.15.19:0/3268528365 shutdown_connections 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.413+0000 7fa4c74cf640 1 -- 172.21.15.19:0/3268528365 wait complete. 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c74cf640 1 Processor -- start 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c74cf640 1 -- start start 2024-04-23T18:28:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c74cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c0100fd0 0x7fa4c01a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c74cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4c01018f0 0x7fa4c01a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c74cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4c01062f0 0x7fa4c019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c74cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0118790 con 0x7fa4c01062f0 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c74cf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0118610 con 0x7fa4c01018f0 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c74cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0118910 con 0x7fa4c0100fd0 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c5a6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4c01062f0 0x7fa4c019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c5a6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4c01062f0 0x7fa4c019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44038/0 (socket says 172.21.15.19:44038) 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c5a6c640 1 -- 172.21.15.19:0/11489761 learned_addr learned my addr 172.21.15.19:0/11489761 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4c01018f0 0x7fa4c01a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c526b640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c0100fd0 0x7fa4c01a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c5a6c640 1 -- 172.21.15.19:0/11489761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c0100fd0 msgr2=0x7fa4c01a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c5a6c640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c0100fd0 0x7fa4c01a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c5a6c640 1 -- 172.21.15.19:0/11489761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4c01018f0 msgr2=0x7fa4c01a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c5a6c640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4c01018f0 0x7fa4c01a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c5a6c640 1 -- 172.21.15.19:0/11489761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c019f230 con 0x7fa4c01062f0 2024-04-23T18:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c526b640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c0100fd0 0x7fa4c01a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.414+0000 7fa4c5a6c640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4c01062f0 0x7fa4c019eab0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8004770 tx=0x7fa4b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.415+0000 7fa4be7fc640 1 -- 172.21.15.19:0/11489761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b803f030 con 0x7fa4c01062f0 2024-04-23T18:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.415+0000 7fa4be7fc640 1 -- 172.21.15.19:0/11489761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4b8003aa0 con 0x7fa4c01062f0 2024-04-23T18:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.415+0000 7fa4c74cf640 1 -- 172.21.15.19:0/11489761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c019f4c0 con 0x7fa4c01062f0 2024-04-23T18:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.415+0000 7fa4be7fc640 1 -- 172.21.15.19:0/11489761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8035070 con 0x7fa4c01062f0 2024-04-23T18:28:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.415+0000 7fa4c74cf640 1 -- 172.21.15.19:0/11489761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c01ab770 con 0x7fa4c01062f0 2024-04-23T18:28:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.417+0000 7fa4be7fc640 1 -- 172.21.15.19:0/11489761 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa4b8042030 con 0x7fa4c01062f0 2024-04-23T18:28:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.417+0000 7fa4ca1e4740 1 Processor -- start 2024-04-23T18:28:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.417+0000 7fa4ca1e4740 1 -- start start 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206fe9fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feaa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206feaa800 0x56206feacbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56206fe04970 con 0x56206fe9ff40 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56206fda14a0 con 0x56206fe9f620 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56206fe00930 con 0x56206feaa800 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c526b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feaa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c526b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feaa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44046/0 (socket says 172.21.15.19:44046) 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c526b640 1 -- 172.21.15.19:0/3442100077 learned_addr learned my addr 172.21.15.19:0/3442100077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/3442100077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206fe9fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4be7fc640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa4a80a3e40 0x7fa4a80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c526b640 1 -- 172.21.15.19:0/3442100077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206feaa800 msgr2=0x56206feacbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/3442100077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206feaa800 0x56206feacbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c5a6c640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa4a80a3e40 0x7fa4a80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c526b640 1 --2- 172.21.15.19:0/3442100077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206feaa800 0x56206feacbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c526b640 1 -- 172.21.15.19:0/3442100077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 msgr2=0x56206fe9fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c526b640 1 --2- 172.21.15.19:0/3442100077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206fe9fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c526b640 1 -- 172.21.15.19:0/3442100077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56206fb17210 con 0x56206fe9ff40 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/3442100077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206feaa800 0x56206feacbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4be7fc640 1 -- 172.21.15.19:0/11489761 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa4b80f0e90 con 0x7fa4c01062f0 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.418+0000 7fa4c526b640 1 --2- 172.21.15.19:0/3442100077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feaa2c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000f3e0 tx=0x7fa4b000f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec113b7a0e6e638c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4c5a6c640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa4a80a3e40 0x7fa4a80a6300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80b9830 tx=0x7fa4b808c740 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4a77fe640 1 -- 172.21.15.19:0/3442100077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0004d20 con 0x56206fe9ff40 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4a77fe640 1 -- 172.21.15.19:0/3442100077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4b0003e20 con 0x56206fe9ff40 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4a77fe640 1 -- 172.21.15.19:0/3442100077 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0011500 con 0x56206fe9ff40 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/3442100077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 msgr2=0x56206feaa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/3442100077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feaa2c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000f3e0 tx=0x7fa4b000f8a0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/3442100077 shutdown_connections 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/3442100077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206feaa800 0x56206feacbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/3442100077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feaa2c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/3442100077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206fe9fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.419+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/3442100077 >> 172.21.15.19:0/3442100077 conn(0x56206fe9bb30 msgr2=0x56206fe9bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.420+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/3442100077 shutdown_connections 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.420+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/3442100077 wait complete. 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.420+0000 7fa4ca1e4740 1 -- start start 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.420+0000 7fa4ca1e4740 1 Processor -- start 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.420+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9f620 0x56206feaa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.420+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9ff40 0x56206fea73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.420+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206feaa800 0x56206fea7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.420+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56206fead430 con 0x56206feaa800 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.420+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56206fd9f0b0 con 0x56206fe9ff40 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.420+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56206fead5b0 con 0x56206fe9f620 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4ca1e4740 10 client.0 _ll_register_callbacks cb 0x56206fd8cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c4a6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206feaa800 0x56206fea7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c4a6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206feaa800 0x56206fea7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44050/0 (socket says 172.21.15.19:44050) 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/901720719 learned_addr learned my addr 172.21.15.19:0/901720719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c526b640 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9ff40 0x56206fea73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9f620 0x56206feaa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/901720719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9f620 msgr2=0x56206feaa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9f620 0x56206feaa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/901720719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9ff40 msgr2=0x56206fea73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9ff40 0x56206fea73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/901720719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206fbe3620 con 0x56206feaa800 2024-04-23T18:28:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206feaa800 0x56206fea7920 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa4c01a0250 tx=0x7fa4b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de74ee2e23c4d20 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/901720719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4010e40 con 0x56206feaa800 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/901720719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa4b401b040 con 0x56206feaa800 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/901720719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4b0003e20 con 0x56206feaa800 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/901720719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4b4004020 con 0x56206feaa800 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4ca1e4740 10 client.15921 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.421+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/901720719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56206feb9310 con 0x56206feaa800 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.422+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/901720719 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4b0003e20 con 0x56206feaa800 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.422+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/901720719 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa4b4004810 con 0x56206feaa800 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.422+0000 7fa4ca1e4740 10 client.15921 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.422+0000 7fa4ca1e4740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.422+0000 7fa4ca1e4740 20 client.15921 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.422+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56206feba7d0 0x56206febcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.422+0000 7fa4ca1e4740 4 client.15921 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.422+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/901720719 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56206fe12390 con 0x56206feba7d0 2024-04-23T18:28:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.422+0000 7fa4c526b640 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56206feba7d0 0x56206febcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.423+0000 7fa4a4ff9640 10 client.15921 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.423+0000 7fa4c526b640 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56206feba7d0 0x56206febcbc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2800137b9a3188a server_cookie=a99c9ebf2c845b61 in_seq=0 out_seq=0 2024-04-23T18:28:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.423+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/901720719 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56206fe12390 con 0x56206feba7d0 2024-04-23T18:28:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.423+0000 7fa4a4ff9640 10 client.15921 handle_command_reply: tid=0 2024-04-23T18:28:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.423+0000 7fa4ca1e4740 1 client.15921 shutdown 2024-04-23T18:28:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.423+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/901720719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206feaa800 msgr2=0x56206fea7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.423+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206feaa800 0x56206fea7920 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa4c01a0250 tx=0x7fa4b4004730 comp rx=0 tx=0).stop 2024-04-23T18:28:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.424+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/901720719 shutdown_connections 2024-04-23T18:28:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.424+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56206feba7d0 0x56206febcbc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.424+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9ff40 0x56206fea73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.424+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206feaa800 0x56206fea7920 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.424+0000 7fa4a4ff9640 0 client.15921 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.424+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/901720719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9f620 0x56206feaa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.424+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/901720719 >> 172.21.15.19:0/901720719 conn(0x56206fe9bb30 msgr2=0x56206fea18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.424+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/901720719 shutdown_connections 2024-04-23T18:28:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.424+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/901720719 wait complete. 2024-04-23T18:28:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.424+0000 7fa4ca1e4740 20 client.15921 trim_cache size 0 max 16384 2024-04-23T18:28:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4ca1e4740 1 Processor -- start 2024-04-23T18:28:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4ca1e4740 1 -- start start 2024-04-23T18:28:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9bb30 0x56206feb62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206fea2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9ff40 0x56206fea9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01b4080 con 0x56206fe9bb30 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01b5bc0 con 0x56206fe9f620 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01b4750 con 0x56206fe9ff40 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c4a6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9bb30 0x56206feb62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c4a6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9bb30 0x56206feb62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44056/0 (socket says 172.21.15.19:44056) 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/3408191484 learned_addr learned my addr 172.21.15.19:0/3408191484 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c526b640 1 --2- 172.21.15.19:0/3408191484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206fea2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/3408191484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9ff40 0x56206fea9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/3408191484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9ff40 msgr2=0x56206fea9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/3408191484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9ff40 0x56206fea9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/3408191484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 msgr2=0x56206fea2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/3408191484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206fea2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/3408191484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56206fe9cd60 con 0x56206fe9bb30 2024-04-23T18:28:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.439+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/3408191484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9bb30 0x56206feb62f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4012840 tx=0x7fa4b4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c516a2938dbf923 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4a6ffd640 1 -- 172.21.15.19:0/3408191484 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b402d070 con 0x56206fe9bb30 2024-04-23T18:28:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4a6ffd640 1 -- 172.21.15.19:0/3408191484 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4b4005850 con 0x56206fe9bb30 2024-04-23T18:28:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4a6ffd640 1 -- 172.21.15.19:0/3408191484 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b40041c0 con 0x56206fe9bb30 2024-04-23T18:28:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/3408191484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9bb30 msgr2=0x56206feb62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/3408191484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9bb30 0x56206feb62f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4012840 tx=0x7fa4b4005fb0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/3408191484 shutdown_connections 2024-04-23T18:28:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/3408191484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206fea2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/3408191484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9ff40 0x56206fea9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/3408191484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9bb30 0x56206feb62f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/3408191484 >> 172.21.15.19:0/3408191484 conn(0x56206fea7a80 msgr2=0x56206fea2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/3408191484 shutdown_connections 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.440+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/3408191484 wait complete. 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4ca1e4740 1 -- start start 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4ca1e4740 1 Processor -- start 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9bb30 0x56206feb29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206feaddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4ca1e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01b54f0 con 0x56206fe9ff40 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01b4e20 con 0x56206fe9f620 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4ca1e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01b2060 con 0x56206fe9bb30 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4ca1e4740 10 client.0 _ll_register_callbacks cb 0x7fa4b4012900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4c4a6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4c4a6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44068/0 (socket says 172.21.15.19:44068) 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/173841338 learned_addr learned my addr 172.21.15.19:0/173841338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4c526b640 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206feaddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.441+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9bb30 0x56206feb29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/173841338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9bb30 msgr2=0x56206feb29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9bb30 0x56206feb29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/173841338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 msgr2=0x56206feaddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206feaddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4c4a6a640 1 -- 172.21.15.19:0/173841338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206fea08e0 con 0x56206fe9ff40 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4c4a6a640 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feae300 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x56206fe9c1c0 tx=0x7fa4b40048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8f01cafb4214a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/173841338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b402d040 con 0x56206fe9ff40 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/173841338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa4b4018070 con 0x56206fe9ff40 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/173841338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4b4005850 con 0x56206fe9ff40 2024-04-23T18:28:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/173841338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4b400bc90 con 0x56206fe9ff40 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4ca1e4740 10 client.15927 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/173841338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56206ff20950 con 0x56206fe9ff40 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/173841338 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4b4005850 con 0x56206fe9ff40 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/173841338 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa4b4004ea0 con 0x56206fe9ff40 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4ca1e4740 10 client.15927 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4ca1e4740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4ca1e4740 20 client.15927 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56206feaa800 0x56206ff22fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4ca1e4740 4 client.15927 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.442+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/173841338 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56206fe0c200 con 0x56206feaa800 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.443+0000 7fa4c526b640 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56206feaa800 0x56206ff22fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.443+0000 7fa4c526b640 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56206feaa800 0x56206ff22fe0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b706fe3f7c22c5c9 server_cookie=9dfb714e55ac59c9 in_seq=0 out_seq=0 2024-04-23T18:28:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.443+0000 7fa4a4ff9640 10 client.15927 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4a4ff9640 1 -- 172.21.15.19:0/173841338 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56206fe0c200 con 0x56206feaa800 2024-04-23T18:28:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4a4ff9640 10 client.15927 handle_command_reply: tid=0 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4ca1e4740 1 client.15927 shutdown 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/173841338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 msgr2=0x56206feae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feae300 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x56206fe9c1c0 tx=0x7fa4b40048d0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/173841338 shutdown_connections 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56206feaa800 0x56206ff22fe0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56206fe9f620 0x56206feaddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56206fe9ff40 0x56206feae300 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4ca1e4740 1 --2- 172.21.15.19:0/173841338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56206fe9bb30 0x56206feb29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4a4ff9640 0 client.15927 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.458+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/173841338 >> 172.21.15.19:0/173841338 conn(0x56206fea7a80 msgr2=0x56206fea5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/173841338 shutdown_connections 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4ca1e4740 1 -- 172.21.15.19:0/173841338 wait complete. 2024-04-23T18:28:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4ca1e4740 20 client.15927 trim_cache size 0 max 16384 2024-04-23T18:28:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:28:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:28:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:28:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 -- 172.21.15.19:0/11489761 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa4a80a3e40 msgr2=0x7fa4a80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa4a80a3e40 0x7fa4a80a6300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80b9830 tx=0x7fa4b808c740 comp rx=0 tx=0).stop 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 -- 172.21.15.19:0/11489761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4c01062f0 msgr2=0x7fa4c019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4c01062f0 0x7fa4c019eab0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8004770 tx=0x7fa4b8004270 comp rx=0 tx=0).stop 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 -- 172.21.15.19:0/11489761 shutdown_connections 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa4a80a3e40 0x7fa4a80a6300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4c01018f0 0x7fa4c01a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4c01062f0 0x7fa4c019eab0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 --2- 172.21.15.19:0/11489761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c0100fd0 0x7fa4c01a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 -- 172.21.15.19:0/11489761 >> 172.21.15.19:0/11489761 conn(0x7fa4c0077fd0 msgr2=0x7fa4c0103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 -- 172.21.15.19:0/11489761 shutdown_connections 2024-04-23T18:28:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:22.459+0000 7fa4c74cf640 1 -- 172.21.15.19:0/11489761 wait complete. 2024-04-23T18:28:24.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:23 smithi019 ceph-mon[29215]: pgmap v70: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T18:28:24.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:23 smithi044 ceph-mon[32458]: pgmap v70: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T18:28:24.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:23 smithi155 ceph-mon[33468]: pgmap v70: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T18:28:26.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:25 smithi019 ceph-mon[29215]: pgmap v71: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 23 op/s 2024-04-23T18:28:26.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:25 smithi044 ceph-mon[32458]: pgmap v71: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 23 op/s 2024-04-23T18:28:26.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:25 smithi155 ceph-mon[33468]: pgmap v71: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 23 op/s 2024-04-23T18:28:28.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:27 smithi019 ceph-mon[29215]: pgmap v72: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T18:28:28.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:27 smithi044 ceph-mon[32458]: pgmap v72: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T18:28:28.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:27 smithi155 ceph-mon[33468]: pgmap v72: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T18:28:30.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:29 smithi044 ceph-mon[32458]: pgmap v73: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T18:28:30.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:29 smithi155 ceph-mon[33468]: pgmap v73: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T18:28:30.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:29 smithi019 ceph-mon[29215]: pgmap v73: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T18:28:32.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:31 smithi044 ceph-mon[32458]: pgmap v74: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-23T18:28:32.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:31 smithi155 ceph-mon[33468]: pgmap v74: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-23T18:28:32.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:31 smithi019 ceph-mon[29215]: pgmap v74: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-23T18:28:34.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:33 smithi044 ceph-mon[32458]: pgmap v75: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-04-23T18:28:34.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:33 smithi155 ceph-mon[33468]: pgmap v75: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-04-23T18:28:34.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:33 smithi019 ceph-mon[29215]: pgmap v75: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-04-23T18:28:35.432 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:28:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.555+0000 7f76308fa640 1 -- 172.21.15.19:0/814786362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f762c1010a0 msgr2=0x7f762c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.555+0000 7f76308fa640 1 --2- 172.21.15.19:0/814786362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f762c1010a0 0x7f762c101480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7624009900 tx=0x7f762402a410 comp rx=0 tx=0).stop 2024-04-23T18:28:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.555+0000 7f76308fa640 1 -- 172.21.15.19:0/814786362 shutdown_connections 2024-04-23T18:28:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.555+0000 7f76308fa640 1 --2- 172.21.15.19:0/814786362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762c101a50 0x7f762c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.555+0000 7f76308fa640 1 --2- 172.21.15.19:0/814786362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c106170 0x7f762c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.555+0000 7f76308fa640 1 --2- 172.21.15.19:0/814786362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f762c1010a0 0x7f762c101480 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.555+0000 7f76308fa640 1 -- 172.21.15.19:0/814786362 >> 172.21.15.19:0/814786362 conn(0x7f762c077f50 msgr2=0x7f762c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.555+0000 7f76308fa640 1 -- 172.21.15.19:0/814786362 shutdown_connections 2024-04-23T18:28:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.555+0000 7f76308fa640 1 -- 172.21.15.19:0/814786362 wait complete. 2024-04-23T18:28:35.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f76308fa640 1 Processor -- start 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f76308fa640 1 -- start start 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f76308fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f762c1010a0 0x7f762c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f76308fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762c106170 0x7f762c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f76308fa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c19a060 0x7f762c1a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f76308fa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f762c111130 con 0x7f762c1010a0 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f76308fa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f762c110fb0 con 0x7f762c106170 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f76308fa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f762c1112b0 con 0x7f762c19a060 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f762a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f762c1010a0 0x7f762c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f762a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f762c1010a0 0x7f762c199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33930/0 (socket says 172.21.15.19:33930) 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f762a59c640 1 -- 172.21.15.19:0/1768503594 learned_addr learned my addr 172.21.15.19:0/1768503594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f7629d9b640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762c106170 0x7f762c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f762ad9d640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c19a060 0x7f762c1a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.556+0000 7fdebaa91640 1 -- 172.21.15.19:0/854450901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb41031e0 msgr2=0x7fdeb41035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.556+0000 7fdebaa91640 1 --2- 172.21.15.19:0/854450901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb41031e0 0x7fdeb41035c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009900 tx=0x7fdea802a510 comp rx=0 tx=0).stop 2024-04-23T18:28:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f762a59c640 1 -- 172.21.15.19:0/1768503594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c19a060 msgr2=0x7f762c1a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f762a59c640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c19a060 0x7f762c1a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f762a59c640 1 -- 172.21.15.19:0/1768503594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762c106170 msgr2=0x7f762c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f762a59c640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762c106170 0x7f762c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.556+0000 7f762a59c640 1 -- 172.21.15.19:0/1768503594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c1a4de0 con 0x7f762c1010a0 2024-04-23T18:28:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.556+0000 7fdebaa91640 1 -- 172.21.15.19:0/854450901 shutdown_connections 2024-04-23T18:28:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.556+0000 7fdebaa91640 1 --2- 172.21.15.19:0/854450901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb410c520 0x7fdeb410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.556+0000 7fdebaa91640 1 --2- 172.21.15.19:0/854450901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdeb4103b00 0x7fdeb410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.556+0000 7fdebaa91640 1 --2- 172.21.15.19:0/854450901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb41031e0 0x7fdeb41035c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.556+0000 7fdebaa91640 1 -- 172.21.15.19:0/854450901 >> 172.21.15.19:0/854450901 conn(0x7fdeb4077fd0 msgr2=0x7fdeb40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.556+0000 7fdebaa91640 1 -- 172.21.15.19:0/854450901 shutdown_connections 2024-04-23T18:28:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.556+0000 7fdebaa91640 1 -- 172.21.15.19:0/854450901 wait complete. 2024-04-23T18:28:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.557+0000 7f762a59c640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f762c1010a0 0x7f762c199590 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7624005ab0 tx=0x7f7624003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.557+0000 7f761b7fe640 1 -- 172.21.15.19:0/1768503594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762403f070 con 0x7f762c1010a0 2024-04-23T18:28:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.557+0000 7f76308fa640 1 -- 172.21.15.19:0/1768503594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762c1a5070 con 0x7f762c1010a0 2024-04-23T18:28:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.557+0000 7f761b7fe640 1 -- 172.21.15.19:0/1768503594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7624031d60 con 0x7f762c1010a0 2024-04-23T18:28:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.557+0000 7f761b7fe640 1 -- 172.21.15.19:0/1768503594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624035030 con 0x7f762c1010a0 2024-04-23T18:28:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.557+0000 7f76308fa640 1 -- 172.21.15.19:0/1768503594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762c1a55b0 con 0x7f762c1010a0 2024-04-23T18:28:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.557+0000 7fdebaa91640 1 Processor -- start 2024-04-23T18:28:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.557+0000 7fdebaa91640 1 -- start start 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.557+0000 7fdebaa91640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdeb41031e0 0x7fdeb41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.557+0000 7fdebaa91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4103b00 0x7fdeb41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.557+0000 7fdebaa91640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb410c520 0x7fdeb419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.557+0000 7fdebaa91640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4112190 con 0x7fdeb4103b00 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.557+0000 7fdebaa91640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4112010 con 0x7fdeb410c520 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.557+0000 7fdebaa91640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4112310 con 0x7fdeb41031e0 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4103b00 0x7fdeb41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4103b00 0x7fdeb41a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33934/0 (socket says 172.21.15.19:33934) 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb3fff640 1 -- 172.21.15.19:0/1180028404 learned_addr learned my addr 172.21.15.19:0/1180028404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb902e640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb410c520 0x7fdeb419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb882d640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdeb41031e0 0x7fdeb41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb3fff640 1 -- 172.21.15.19:0/1180028404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdeb41031e0 msgr2=0x7fdeb41a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb3fff640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdeb41031e0 0x7fdeb41a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb3fff640 1 -- 172.21.15.19:0/1180028404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb410c520 msgr2=0x7fdeb419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb3fff640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb410c520 0x7fdeb419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb3fff640 1 -- 172.21.15.19:0/1180028404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb419afb0 con 0x7fdeb4103b00 2024-04-23T18:28:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb902e640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb410c520 0x7fdeb419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb882d640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdeb41031e0 0x7fdeb41a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb3fff640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4103b00 0x7fdeb41a0b70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdea000d950 tx=0x7fdea000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.558+0000 7f76308fa640 1 -- 172.21.15.19:0/1768503594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f762c068d70 con 0x7f762c1010a0 2024-04-23T18:28:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.558+0000 7fdeb1ffb640 1 -- 172.21.15.19:0/1180028404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0017030 con 0x7fdeb4103b00 2024-04-23T18:28:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.559+0000 7fdeb1ffb640 1 -- 172.21.15.19:0/1180028404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea0004020 con 0x7fdeb4103b00 2024-04-23T18:28:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.559+0000 7fdeb1ffb640 1 -- 172.21.15.19:0/1180028404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea00047a0 con 0x7fdeb4103b00 2024-04-23T18:28:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.559+0000 7fdebaa91640 1 -- 172.21.15.19:0/1180028404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb419b2a0 con 0x7fdeb4103b00 2024-04-23T18:28:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.559+0000 7fdebaa91640 1 -- 172.21.15.19:0/1180028404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb41a75b0 con 0x7fdeb4103b00 2024-04-23T18:28:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.560+0000 7f761b7fe640 1 -- 172.21.15.19:0/1768503594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7624042070 con 0x7f762c1010a0 2024-04-23T18:28:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.561+0000 7f761b7fe640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76140a3d20 0x7f76140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.561+0000 7f7629d9b640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76140a3d20 0x7f76140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.561+0000 7f761b7fe640 1 -- 172.21.15.19:0/1768503594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f76240f0fe0 con 0x7f762c1010a0 2024-04-23T18:28:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.561+0000 7f7629d9b640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76140a3d20 0x7f76140a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f762c069c30 tx=0x7f762000a2d0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdeb1ffb640 1 -- 172.21.15.19:0/1180028404 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdea0009a30 con 0x7fdeb4103b00 2024-04-23T18:28:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdebd7ad740 1 Processor -- start 2024-04-23T18:28:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdebd7ad740 1 -- start start 2024-04-23T18:28:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30416a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff304212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 0x55ff30423bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ff3037b970 con 0x55ff30416f40 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ff303184a0 con 0x55ff30421800 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ff30377930 con 0x55ff30416620 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdeb882d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff304212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdeb882d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff304212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33940/0 (socket says 172.21.15.19:33940) 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdeb882d640 1 -- 172.21.15.19:0/4143316113 learned_addr learned my addr 172.21.15.19:0/4143316113 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.562+0000 7fdeb1ffb640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdea40a3dc0 0x7fdea40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb3fff640 1 --2- 172.21.15.19:0/4143316113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 0x55ff30423bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb902e640 1 --2- 172.21.15.19:0/4143316113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30416a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb902e640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdea40a3dc0 0x7fdea40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb882d640 1 -- 172.21.15.19:0/4143316113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 msgr2=0x55ff30416a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb882d640 1 --2- 172.21.15.19:0/4143316113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30416a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb882d640 1 -- 172.21.15.19:0/4143316113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 msgr2=0x55ff30423bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb882d640 1 --2- 172.21.15.19:0/4143316113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 0x55ff30423bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb882d640 1 -- 172.21.15.19:0/4143316113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3008e210 con 0x55ff30416f40 2024-04-23T18:28:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb1ffb640 1 -- 172.21.15.19:0/1180028404 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fdea00d0250 con 0x7fdeb4103b00 2024-04-23T18:28:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb882d640 1 --2- 172.21.15.19:0/4143316113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff304212c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdea802aa30 tx=0x7fdea8032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fd174752135ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.563+0000 7fdeb902e640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdea40a3dc0 0x7fdea40a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00a780 tx=0x7fdeac00a4d0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.566+0000 7fde9effd640 1 -- 172.21.15.19:0/4143316113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea803f070 con 0x55ff30416f40 2024-04-23T18:28:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.566+0000 7fde9effd640 1 -- 172.21.15.19:0/4143316113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea8031760 con 0x55ff30416f40 2024-04-23T18:28:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.566+0000 7fde9effd640 1 -- 172.21.15.19:0/4143316113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8035070 con 0x55ff30416f40 2024-04-23T18:28:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.566+0000 7f761b7fe640 1 -- 172.21.15.19:0/1768503594 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76240bd060 con 0x7f762c1010a0 2024-04-23T18:28:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 -- 172.21.15.19:0/4143316113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 msgr2=0x55ff304212c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/4143316113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff304212c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdea802aa30 tx=0x7fdea8032e70 comp rx=0 tx=0).stop 2024-04-23T18:28:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 -- 172.21.15.19:0/4143316113 shutdown_connections 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/4143316113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 0x55ff30423bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/4143316113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff304212c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/4143316113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30416a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 -- 172.21.15.19:0/4143316113 >> 172.21.15.19:0/4143316113 conn(0x55ff30412b30 msgr2=0x55ff30412f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 -- 172.21.15.19:0/4143316113 shutdown_connections 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 -- 172.21.15.19:0/4143316113 wait complete. 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 -- start start 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.567+0000 7fdebd7ad740 1 Processor -- start 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30421110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416f40 0x55ff3041e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff3041e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ff304243d0 con 0x55ff30421800 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ff303160b0 con 0x55ff30416f40 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ff30424550 con 0x55ff30416620 2024-04-23T18:28:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdebd7ad740 10 client.0 _ll_register_callbacks cb 0x55ff30303d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdeb902e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff3041e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdeb902e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff3041e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33946/0 (socket says 172.21.15.19:33946) 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdeb902e640 1 -- 172.21.15.19:0/2689555200 learned_addr learned my addr 172.21.15.19:0/2689555200 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdeb3fff640 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416f40 0x55ff3041e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdeb882d640 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30421110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdeb902e640 1 -- 172.21.15.19:0/2689555200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 msgr2=0x55ff30421110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdeb902e640 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30421110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.568+0000 7fdeb902e640 1 -- 172.21.15.19:0/2689555200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416f40 msgr2=0x55ff3041e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fdeb902e640 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416f40 0x55ff3041e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fdeb902e640 1 -- 172.21.15.19:0/2689555200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3015a620 con 0x55ff30421800 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fdeb882d640 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30421110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fdeb902e640 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff3041e940 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00fa00 tx=0x7fdeac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8a355eb7442100 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fde93fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fde93fff640 1 -- 172.21.15.19:0/2689555200 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac024070 con 0x55ff30421800 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fde93fff640 1 -- 172.21.15.19:0/2689555200 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fdeac00ed40 con 0x55ff30421800 2024-04-23T18:28:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2689555200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea8031760 con 0x55ff30421800 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fde93fff640 1 -- 172.21.15.19:0/2689555200 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdeac02f050 con 0x55ff30421800 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fdebd7ad740 10 client.15945 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2689555200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ff30430310 con 0x55ff30421800 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.569+0000 7fde93fff640 1 -- 172.21.15.19:0/2689555200 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea8031760 con 0x55ff30421800 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.570+0000 7fde93fff640 1 -- 172.21.15.19:0/2689555200 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdeac01f070 con 0x55ff30421800 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.570+0000 7fdebd7ad740 10 client.15945 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.570+0000 7fdebd7ad740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.570+0000 7fdebd7ad740 20 client.15945 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.570+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ff304317d0 0x55ff30433be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.570+0000 7fdebd7ad740 4 client.15945 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.570+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2689555200 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff30389390 con 0x55ff304317d0 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.570+0000 7fdeb882d640 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ff304317d0 0x55ff30433be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.571+0000 7fde93fff640 10 client.15945 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.571+0000 7fdeb882d640 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ff304317d0 0x55ff30433be0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc1e75ee975e0fb server_cookie=93436e4d92f1ed80 in_seq=0 out_seq=0 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.571+0000 7fde93fff640 1 -- 172.21.15.19:0/2689555200 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff30389390 con 0x55ff304317d0 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.571+0000 7fde93fff640 10 client.15945 handle_command_reply: tid=0 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.571+0000 7fdebd7ad740 1 client.15945 shutdown 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2689555200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 msgr2=0x55ff3041e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff3041e940 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00fa00 tx=0x7fdeac004a40 comp rx=0 tx=0).stop 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2689555200 shutdown_connections 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ff304317d0 0x55ff30433be0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416f40 0x55ff3041e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff3041e940 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/2689555200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30421110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2689555200 >> 172.21.15.19:0/2689555200 conn(0x55ff30412b30 msgr2=0x55ff30418860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fde93fff640 0 client.15945 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2689555200 shutdown_connections 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2689555200 wait complete. 2024-04-23T18:28:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.572+0000 7fdebd7ad740 20 client.15945 trim_cache size 0 max 16384 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdebd7ad740 1 Processor -- start 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdebd7ad740 1 -- start start 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416620 0x55ff30420c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416f40 0x55ff304394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff30439a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41afcb0 con 0x55ff30421800 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41b17f0 con 0x55ff30416620 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41b0380 con 0x55ff30416f40 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdeb902e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff30439a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdeb902e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff30439a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33954/0 (socket says 172.21.15.19:33954) 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdeb902e640 1 -- 172.21.15.19:0/2460304345 learned_addr learned my addr 172.21.15.19:0/2460304345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdeb882d640 1 --2- 172.21.15.19:0/2460304345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416620 0x55ff30420c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.587+0000 7fdeb3fff640 1 --2- 172.21.15.19:0/2460304345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416f40 0x55ff304394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.588+0000 7fdeb902e640 1 -- 172.21.15.19:0/2460304345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416f40 msgr2=0x55ff304394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.588+0000 7fdeb902e640 1 --2- 172.21.15.19:0/2460304345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416f40 0x55ff304394f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.588+0000 7fdeb902e640 1 -- 172.21.15.19:0/2460304345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416620 msgr2=0x55ff30420c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.588+0000 7fdeb902e640 1 --2- 172.21.15.19:0/2460304345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416620 0x55ff30420c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.588+0000 7fdeb902e640 1 -- 172.21.15.19:0/2460304345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3040e2e0 con 0x55ff30421800 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.588+0000 7fdeb882d640 1 --2- 172.21.15.19:0/2460304345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416620 0x55ff30420c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.588+0000 7fdeb902e640 1 --2- 172.21.15.19:0/2460304345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff30439a30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdeac016040 tx=0x7fdeac014770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675309925be82c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.588+0000 7fde9e7fc640 1 -- 172.21.15.19:0/2460304345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac024040 con 0x55ff30421800 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.588+0000 7fde9e7fc640 1 -- 172.21.15.19:0/2460304345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac01a070 con 0x55ff30421800 2024-04-23T18:28:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.588+0000 7fde9e7fc640 1 -- 172.21.15.19:0/2460304345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0203f0 con 0x55ff30421800 2024-04-23T18:28:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2460304345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 msgr2=0x55ff30439a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/2460304345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff30439a30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdeac016040 tx=0x7fdeac014770 comp rx=0 tx=0).stop 2024-04-23T18:28:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2460304345 shutdown_connections 2024-04-23T18:28:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/2460304345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30421800 0x55ff30439a30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/2460304345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416f40 0x55ff304394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/2460304345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30416620 0x55ff30420c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2460304345 >> 172.21.15.19:0/2460304345 conn(0x55ff30412b30 msgr2=0x55ff3041e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2460304345 shutdown_connections 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 -- 172.21.15.19:0/2460304345 wait complete. 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 -- start start 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.589+0000 7fdebd7ad740 1 Processor -- start 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30429b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff30426ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdebd7ad740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 0x55ff30427400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41b1120 con 0x55ff30416f40 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41b0a50 con 0x55ff30421800 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdebd7ad740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41adc90 con 0x55ff30416620 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdebd7ad740 10 client.0 _ll_register_callbacks cb 0x7fdeac011dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff30426ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff30426ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33964/0 (socket says 172.21.15.19:33964) 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb3fff640 1 -- 172.21.15.19:0/3796607765 learned_addr learned my addr 172.21.15.19:0/3796607765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb882d640 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30429b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb902e640 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 0x55ff30427400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb3fff640 1 -- 172.21.15.19:0/3796607765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 msgr2=0x55ff30429b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb3fff640 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30429b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb3fff640 1 -- 172.21.15.19:0/3796607765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 msgr2=0x55ff30427400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb3fff640 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 0x55ff30427400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb3fff640 1 -- 172.21.15.19:0/3796607765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff30427c10 con 0x55ff30416f40 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb902e640 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 0x55ff30427400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb882d640 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30429b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdeb3fff640 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff30426ec0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdea0098680 tx=0x7fdea006bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927d1e6d5ebf35f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fde93fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fde93fff640 1 -- 172.21.15.19:0/3796607765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea007e150 con 0x55ff30416f40 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fde93fff640 1 -- 172.21.15.19:0/3796607765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fdea00cdc50 con 0x55ff30416f40 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.590+0000 7fdebd7ad740 1 -- 172.21.15.19:0/3796607765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeac01a070 con 0x55ff30416f40 2024-04-23T18:28:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fde93fff640 1 -- 172.21.15.19:0/3796607765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdea009c220 con 0x55ff30416f40 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fdebd7ad740 10 client.15951 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fdebd7ad740 1 -- 172.21.15.19:0/3796607765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ff30497db0 con 0x55ff30416f40 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fde93fff640 1 -- 172.21.15.19:0/3796607765 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac01a070 con 0x55ff30416f40 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fde93fff640 1 -- 172.21.15.19:0/3796607765 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdea00a0170 con 0x55ff30416f40 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fdebd7ad740 10 client.15951 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fdebd7ad740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fdebd7ad740 20 client.15951 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ff30498500 0x55ff3049a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fdebd7ad740 4 client.15951 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fdebd7ad740 1 -- 172.21.15.19:0/3796607765 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff30383200 con 0x55ff30498500 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.591+0000 7fdeb882d640 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ff30498500 0x55ff3049a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.592+0000 7fde93fff640 10 client.15951 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.592+0000 7fdeb882d640 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ff30498500 0x55ff3049a920 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3567fbde25f22240 server_cookie=d73d2aa28483297 in_seq=0 out_seq=0 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.605+0000 7fde93fff640 1 -- 172.21.15.19:0/3796607765 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff30383200 con 0x55ff30498500 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.605+0000 7fde93fff640 10 client.15951 handle_command_reply: tid=0 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 client.15951 shutdown 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 -- 172.21.15.19:0/3796607765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 msgr2=0x55ff30426ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff30426ec0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdea0098680 tx=0x7fdea006bb00 comp rx=0 tx=0).stop 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 -- 172.21.15.19:0/3796607765 shutdown_connections 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ff30498500 0x55ff3049a920 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff30421800 0x55ff30427400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff30416f40 0x55ff30426ec0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fde93fff640 0 client.15951 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 --2- 172.21.15.19:0/3796607765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff30416620 0x55ff30429b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 -- 172.21.15.19:0/3796607765 >> 172.21.15.19:0/3796607765 conn(0x55ff30412b30 msgr2=0x55ff30411420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 -- 172.21.15.19:0/3796607765 shutdown_connections 2024-04-23T18:28:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 1 -- 172.21.15.19:0/3796607765 wait complete. 2024-04-23T18:28:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.606+0000 7fdebd7ad740 20 client.15951 trim_cache size 0 max 16384 2024-04-23T18:28:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:28:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:28:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:28:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:28:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 -- 172.21.15.19:0/1180028404 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdea40a3dc0 msgr2=0x7fdea40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdea40a3dc0 0x7fdea40a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00a780 tx=0x7fdeac00a4d0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 -- 172.21.15.19:0/1180028404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4103b00 msgr2=0x7fdeb41a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4103b00 0x7fdeb41a0b70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdea000d950 tx=0x7fdea000de10 comp rx=0 tx=0).stop 2024-04-23T18:28:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 -- 172.21.15.19:0/1180028404 shutdown_connections 2024-04-23T18:28:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdea40a3dc0 0x7fdea40a6280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb410c520 0x7fdeb419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4103b00 0x7fdeb41a0b70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 --2- 172.21.15.19:0/1180028404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdeb41031e0 0x7fdeb41a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 -- 172.21.15.19:0/1180028404 >> 172.21.15.19:0/1180028404 conn(0x7fdeb4077fd0 msgr2=0x7fdeb41083f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 -- 172.21.15.19:0/1180028404 shutdown_connections 2024-04-23T18:28:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:35.607+0000 7fdebaa91640 1 -- 172.21.15.19:0/1180028404 wait complete. 2024-04-23T18:28:35.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.796+0000 7f76308fa640 1 -- 172.21.15.19:0/1768503594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f762c1093a0 con 0x7f762c1010a0 2024-04-23T18:28:35.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.796+0000 7f761b7fe640 1 -- 172.21.15.19:0/1768503594 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f76240c1360 con 0x7f762c1010a0 2024-04-23T18:28:35.789 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:28:35.789 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:28:35.789 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:28:35.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 -- 172.21.15.19:0/1768503594 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76140a3d20 msgr2=0x7f76140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76140a3d20 0x7f76140a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f762c069c30 tx=0x7f762000a2d0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 -- 172.21.15.19:0/1768503594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f762c1010a0 msgr2=0x7f762c199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f762c1010a0 0x7f762c199590 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7624005ab0 tx=0x7f7624003cd0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 -- 172.21.15.19:0/1768503594 shutdown_connections 2024-04-23T18:28:35.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76140a3d20 0x7f76140a61e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f762c106170 0x7f762c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c19a060 0x7f762c1a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 --2- 172.21.15.19:0/1768503594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f762c1010a0 0x7f762c199590 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 -- 172.21.15.19:0/1768503594 >> 172.21.15.19:0/1768503594 conn(0x7f762c077f50 msgr2=0x7f762c0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.798+0000 7f76308fa640 1 -- 172.21.15.19:0/1768503594 shutdown_connections 2024-04-23T18:28:35.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.799+0000 7f76308fa640 1 -- 172.21.15.19:0/1768503594 wait complete. 2024-04-23T18:28:35.802 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id d3b5be3e --release --await' 2024-04-23T18:28:35.802 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id d3b5be3e --release --await 2024-04-23T18:28:35.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.918+0000 7f2dc9376640 1 -- 172.21.15.19:0/1481547521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc4101a40 msgr2=0x7f2dc4105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.918+0000 7f2dc9376640 1 --2- 172.21.15.19:0/1481547521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc4101a40 0x7f2dc4105a80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009900 tx=0x7f2dac02a410 comp rx=0 tx=0).stop 2024-04-23T18:28:35.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.918+0000 7f2dc9376640 1 -- 172.21.15.19:0/1481547521 shutdown_connections 2024-04-23T18:28:35.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.918+0000 7f2dc9376640 1 --2- 172.21.15.19:0/1481547521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc4106160 0x7f2dc410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.918+0000 7f2dc9376640 1 --2- 172.21.15.19:0/1481547521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc4101a40 0x7f2dc4105a80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.918+0000 7f2dc9376640 1 --2- 172.21.15.19:0/1481547521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4101090 0x7f2dc4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.918+0000 7f2dc9376640 1 -- 172.21.15.19:0/1481547521 >> 172.21.15.19:0/1481547521 conn(0x7f2dc4077f50 msgr2=0x7f2dc4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.918+0000 7f2dc9376640 1 -- 172.21.15.19:0/1481547521 shutdown_connections 2024-04-23T18:28:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.918+0000 7f2dc9376640 1 -- 172.21.15.19:0/1481547521 wait complete. 2024-04-23T18:28:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.919+0000 7f2dc9376640 1 Processor -- start 2024-04-23T18:28:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.919+0000 7f2dc9376640 1 -- start start 2024-04-23T18:28:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.919+0000 7f2dc9376640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc4101090 0x7f2dc4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.919+0000 7f2dc9376640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4106160 0x7f2dc4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.919+0000 7f2dc9376640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc419d000 0x7f2dc419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.919+0000 7f2dc9376640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41121e0 con 0x7f2dc419d000 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.919+0000 7f2dc9376640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc4112060 con 0x7f2dc4106160 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.919+0000 7f2dc9376640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc4112360 con 0x7f2dc4101090 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc419d000 0x7f2dc419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4106160 0x7f2dc4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc419d000 0x7f2dc419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33988/0 (socket says 172.21.15.19:33988) 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc37fe640 1 -- 172.21.15.19:0/2998755908 learned_addr learned my addr 172.21.15.19:0/2998755908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc2ffd640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc4101090 0x7f2dc4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc37fe640 1 -- 172.21.15.19:0/2998755908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc4101090 msgr2=0x7f2dc4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc4101090 0x7f2dc4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc37fe640 1 -- 172.21.15.19:0/2998755908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4106160 msgr2=0x7f2dc4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4106160 0x7f2dc4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc37fe640 1 -- 172.21.15.19:0/2998755908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc419daf0 con 0x7f2dc419d000 2024-04-23T18:28:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc27fc640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4106160 0x7f2dc4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc419d000 0x7f2dc419d4b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00d750 tx=0x7f2dbc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dbbfff640 1 -- 172.21.15.19:0/2998755908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc017070 con 0x7f2dc419d000 2024-04-23T18:28:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc9376640 1 -- 172.21.15.19:0/2998755908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc419dde0 con 0x7f2dc419d000 2024-04-23T18:28:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dbbfff640 1 -- 172.21.15.19:0/2998755908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc0054d0 con 0x7f2dc419d000 2024-04-23T18:28:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dbbfff640 1 -- 172.21.15.19:0/2998755908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc004c60 con 0x7f2dc419d000 2024-04-23T18:28:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.920+0000 7f2dc9376640 1 -- 172.21.15.19:0/2998755908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc41a55a0 con 0x7f2dc419d000 2024-04-23T18:28:35.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.923+0000 7f2dbbfff640 1 -- 172.21.15.19:0/2998755908 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2dbc009bd0 con 0x7f2dc419d000 2024-04-23T18:28:35.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.923+0000 7f2dcc16e740 1 Processor -- start 2024-04-23T18:28:35.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.923+0000 7f2dcc16e740 1 -- start start 2024-04-23T18:28:35.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f321870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dbbfff640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2db00a3d20 0x7f2db00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f32bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 0x55dd3f32e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dd3f2866d0 con 0x55dd3f321db0 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dd3f2231a0 con 0x55dd3f32c480 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dd3f282690 con 0x55dd3f321490 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc2ffd640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2db00a3d20 0x7f2db00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f32bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f32bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34002/0 (socket says 172.21.15.19:34002) 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc27fc640 1 -- 172.21.15.19:0/952539909 learned_addr learned my addr 172.21.15.19:0/952539909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dbbfff640 1 -- 172.21.15.19:0/2998755908 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2dbc0d0050 con 0x7f2dc419d000 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/952539909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 0x55dd3f32e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc27fc640 1 --2- 172.21.15.19:0/952539909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f321870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc37fe640 1 -- 172.21.15.19:0/952539909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 msgr2=0x55dd3f321870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/952539909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f321870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc37fe640 1 -- 172.21.15.19:0/952539909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 msgr2=0x55dd3f32bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/952539909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f32bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc37fe640 1 -- 172.21.15.19:0/952539909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd3f22d5b0 con 0x55dd3f32c480 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc27fc640 1 --2- 172.21.15.19:0/952539909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f321870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.924+0000 7f2dc2ffd640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2db00a3d20 0x7f2db00a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2db4005fd0 tx=0x7f2db4009500 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.925+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/952539909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 0x55dd3f32e870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0cd7d0 tx=0x7f2dbc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=316a9a0455a188ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.925+0000 7f2db97fa640 1 -- 172.21.15.19:0/952539909 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc020040 con 0x55dd3f32c480 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.925+0000 7f2db97fa640 1 -- 172.21.15.19:0/952539909 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc005680 con 0x55dd3f32c480 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.925+0000 7f2db97fa640 1 -- 172.21.15.19:0/952539909 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0cda50 con 0x55dd3f32c480 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.925+0000 7f2dcc16e740 1 -- 172.21.15.19:0/952539909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 msgr2=0x55dd3f32e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.925+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/952539909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 0x55dd3f32e870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0cd7d0 tx=0x7f2dbc000bc0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.925+0000 7f2dcc16e740 1 -- 172.21.15.19:0/952539909 shutdown_connections 2024-04-23T18:28:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.925+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/952539909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 0x55dd3f32e870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/952539909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f32bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/952539909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f321870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 -- 172.21.15.19:0/952539909 >> 172.21.15.19:0/952539909 conn(0x55dd3f31d7b0 msgr2=0x55dd3f31db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 -- 172.21.15.19:0/952539909 shutdown_connections 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 -- 172.21.15.19:0/952539909 wait complete. 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 -- start start 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 Processor -- start 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f32be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f329180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 0x55dd3f3297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dd3f32f250 con 0x55dd3f321db0 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dd3f220b20 con 0x55dd3f32c480 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dd3f32f3d0 con 0x55dd3f321490 2024-04-23T18:28:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dcc16e740 10 client.0 _ll_register_callbacks cb 0x55dd3f220530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dc27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f329180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dc27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f329180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34006/0 (socket says 172.21.15.19:34006) 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dc27fc640 1 -- 172.21.15.19:0/44460304 learned_addr learned my addr 172.21.15.19:0/44460304 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 0x55dd3f3297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.926+0000 7f2dc2ffd640 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f32be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2dc27fc640 1 -- 172.21.15.19:0/44460304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 msgr2=0x55dd3f32be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2dc27fc640 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f32be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2dc27fc640 1 -- 172.21.15.19:0/44460304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 msgr2=0x55dd3f3297b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2dc27fc640 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 0x55dd3f3297b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2dc27fc640 1 -- 172.21.15.19:0/44460304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd3ef98920 con 0x55dd3f321db0 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2dc27fc640 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f329180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2dac02a930 tx=0x7f2dac005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c32d26a08780a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2daa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2daa7fc640 1 -- 172.21.15.19:0/44460304 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac035070 con 0x55dd3f321db0 2024-04-23T18:28:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2daa7fc640 1 -- 172.21.15.19:0/44460304 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2dac004300 con 0x55dd3f321db0 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2dcc16e740 1 -- 172.21.15.19:0/44460304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dbc005680 con 0x55dd3f321db0 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2daa7fc640 1 -- 172.21.15.19:0/44460304 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2dac005020 con 0x55dd3f321db0 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2dcc16e740 10 client.15963 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2dcc16e740 1 -- 172.21.15.19:0/44460304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd3f064d30 con 0x55dd3f321db0 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2daa7fc640 1 -- 172.21.15.19:0/44460304 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc005680 con 0x55dd3f321db0 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.927+0000 7f2daa7fc640 1 -- 172.21.15.19:0/44460304 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2dac004ce0 con 0x55dd3f321db0 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.928+0000 7f2dcc16e740 10 client.15963 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.928+0000 7f2dcc16e740 10 client.15963 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.928+0000 7f2dcc16e740 20 client.15963 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.928+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55dd3f340900 0x55dd3f342d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.928+0000 7f2dcc16e740 4 client.15963 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.928+0000 7f2dcc16e740 1 -- 172.21.15.19:0/44460304 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd3f244500 con 0x55dd3f340900 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.928+0000 7f2dc2ffd640 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55dd3f340900 0x55dd3f342d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.928+0000 7f2dc2ffd640 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55dd3f340900 0x55dd3f342d20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2451d7b06108116d server_cookie=cac3cc41ce21efe8 in_seq=0 out_seq=0 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.928+0000 7f2daa7fc640 10 client.15963 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2daa7fc640 1 -- 172.21.15.19:0/44460304 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd3f244500 con 0x55dd3f340900 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2daa7fc640 10 client.15963 handle_command_reply: tid=0 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 client.15963 shutdown 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 -- 172.21.15.19:0/44460304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 msgr2=0x55dd3f329180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f329180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2dac02a930 tx=0x7f2dac005bc0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 -- 172.21.15.19:0/44460304 shutdown_connections 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55dd3f340900 0x55dd3f342d20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f32c480 0x55dd3f3297b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2daa7fc640 0 client.15963 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f321db0 0x55dd3f329180 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/44460304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f32be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 -- 172.21.15.19:0/44460304 >> 172.21.15.19:0/44460304 conn(0x55dd3f31d7b0 msgr2=0x55dd3f3235b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 -- 172.21.15.19:0/44460304 shutdown_connections 2024-04-23T18:28:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 1 -- 172.21.15.19:0/44460304 wait complete. 2024-04-23T18:28:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.929+0000 7f2dcc16e740 20 client.15963 trim_cache size 0 max 16384 2024-04-23T18:28:35.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.946+0000 7f2dcc16e740 1 Processor -- start 2024-04-23T18:28:35.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.946+0000 7f2dcc16e740 1 -- start start 2024-04-23T18:28:35.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f328aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 0x55dd3f3485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f348b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41ab870 con 0x55dd3f32c480 2024-04-23T18:28:35.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41ad3b0 con 0x55dd3f321db0 2024-04-23T18:28:35.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41abf40 con 0x55dd3f321490 2024-04-23T18:28:35.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f348b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f348b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34016/0 (socket says 172.21.15.19:34016) 2024-04-23T18:28:35.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc37fe640 1 -- 172.21.15.19:0/1230945803 learned_addr learned my addr 172.21.15.19:0/1230945803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc2ffd640 1 --2- 172.21.15.19:0/1230945803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f328aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc37fe640 1 -- 172.21.15.19:0/1230945803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 msgr2=0x55dd3f328aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/1230945803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f328aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc37fe640 1 -- 172.21.15.19:0/1230945803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 msgr2=0x55dd3f3485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc27fc640 1 --2- 172.21.15.19:0/1230945803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 0x55dd3f3485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/1230945803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 0x55dd3f3485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc37fe640 1 -- 172.21.15.19:0/1230945803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd3f324db0 con 0x55dd3f32c480 2024-04-23T18:28:35.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc27fc640 1 --2- 172.21.15.19:0/1230945803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 0x55dd3f3485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:35.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.947+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/1230945803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f348b30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc09c060 tx=0x7f2dbc09bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef7540aa3805e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2db8ff9640 1 -- 172.21.15.19:0/1230945803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc020040 con 0x55dd3f32c480 2024-04-23T18:28:35.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2db8ff9640 1 -- 172.21.15.19:0/1230945803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc0a03d0 con 0x55dd3f32c480 2024-04-23T18:28:35.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2db8ff9640 1 -- 172.21.15.19:0/1230945803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0b5260 con 0x55dd3f32c480 2024-04-23T18:28:35.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2dcc16e740 1 -- 172.21.15.19:0/1230945803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 msgr2=0x55dd3f348b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/1230945803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f348b30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc09c060 tx=0x7f2dbc09bd40 comp rx=0 tx=0).stop 2024-04-23T18:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2dcc16e740 1 -- 172.21.15.19:0/1230945803 shutdown_connections 2024-04-23T18:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/1230945803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 0x55dd3f3485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/1230945803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f348b30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/1230945803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f328aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2dcc16e740 1 -- 172.21.15.19:0/1230945803 >> 172.21.15.19:0/1230945803 conn(0x55dd3f31d7b0 msgr2=0x55dd3f31db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2dcc16e740 1 -- 172.21.15.19:0/1230945803 shutdown_connections 2024-04-23T18:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.948+0000 7f2dcc16e740 1 -- 172.21.15.19:0/1230945803 wait complete. 2024-04-23T18:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dcc16e740 1 -- start start 2024-04-23T18:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dcc16e740 1 Processor -- start 2024-04-23T18:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f338f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 0x55dd3f326710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dcc16e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f326cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41acce0 con 0x55dd3f32c480 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41ac610 con 0x55dd3f321db0 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dcc16e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41a9870 con 0x55dd3f321490 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dcc16e740 10 client.0 _ll_register_callbacks cb 0x7f2dac031740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dc37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f326cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dc37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f326cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34024/0 (socket says 172.21.15.19:34024) 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dc37fe640 1 -- 172.21.15.19:0/3798426889 learned_addr learned my addr 172.21.15.19:0/3798426889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dc27fc640 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 0x55dd3f326710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dc2ffd640 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f338f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dc37fe640 1 -- 172.21.15.19:0/3798426889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 msgr2=0x55dd3f338f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f338f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dc37fe640 1 -- 172.21.15.19:0/3798426889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 msgr2=0x55dd3f326710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 0x55dd3f326710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.949+0000 7f2dc37fe640 1 -- 172.21.15.19:0/3798426889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd3f32d3c0 con 0x55dd3f32c480 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f326cb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc098530 tx=0x7f2dbc0a4d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb7b4a2cc8a545d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2daa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2daa7fc640 1 -- 172.21.15.19:0/3798426889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0b9070 con 0x55dd3f32c480 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2daa7fc640 1 -- 172.21.15.19:0/3798426889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2dbc09b520 con 0x55dd3f32c480 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2dcc16e740 1 -- 172.21.15.19:0/3798426889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dbc0a03d0 con 0x55dd3f32c480 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2daa7fc640 1 -- 172.21.15.19:0/3798426889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2dbc020050 con 0x55dd3f32c480 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2dcc16e740 10 client.15969 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2dcc16e740 1 -- 172.21.15.19:0/3798426889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd3f3a71b0 con 0x55dd3f32c480 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2daa7fc640 1 -- 172.21.15.19:0/3798426889 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc0a03d0 con 0x55dd3f32c480 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2daa7fc640 1 -- 172.21.15.19:0/3798426889 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2dbc0b0ac0 con 0x55dd3f32c480 2024-04-23T18:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2dcc16e740 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2dcc16e740 10 client.15969 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:28:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2dcc16e740 20 client.15969 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55dd3f3a7430 0x55dd3f3a9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2dcc16e740 4 client.15969 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d3b5be3e", "release": true, "await": true}] 2024-04-23T18:28:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.950+0000 7f2dcc16e740 1 -- 172.21.15.19:0/3798426889 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d3b5be3e", "release": true, "await": true}) -- 0x55dd3f2940f0 con 0x55dd3f3a7430 2024-04-23T18:28:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.951+0000 7f2dc2ffd640 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55dd3f3a7430 0x55dd3f3a9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.951+0000 7f2dc2ffd640 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55dd3f3a7430 0x55dd3f3a9820 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14340d6a7a14485 server_cookie=3cd2d4b3f7ca59b8 in_seq=0 out_seq=0 2024-04-23T18:28:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.951+0000 7f2daa7fc640 10 client.15969 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2daa7fc640 1 -- 172.21.15.19:0/3798426889 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55dd3f2940f0 con 0x55dd3f3a7430 2024-04-23T18:28:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2daa7fc640 10 client.15969 handle_command_reply: tid=0 2024-04-23T18:28:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 client.15969 shutdown 2024-04-23T18:28:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 -- 172.21.15.19:0/3798426889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 msgr2=0x55dd3f326cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f326cb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc098530 tx=0x7f2dbc0a4d10 comp rx=0 tx=0).stop 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 -- 172.21.15.19:0/3798426889 shutdown_connections 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55dd3f3a7430 0x55dd3f3a9820 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd3f321db0 0x55dd3f326710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd3f32c480 0x55dd3f326cb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2daa7fc640 0 client.15969 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 --2- 172.21.15.19:0/3798426889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd3f321490 0x55dd3f338f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 -- 172.21.15.19:0/3798426889 >> 172.21.15.19:0/3798426889 conn(0x55dd3f31d7b0 msgr2=0x55dd3f32a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 -- 172.21.15.19:0/3798426889 shutdown_connections 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 1 -- 172.21.15.19:0/3798426889 wait complete. 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.955+0000 7f2dcc16e740 20 client.15969 trim_cache size 0 max 16384 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:28:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 12, 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "d3b5be3e": { 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 12, 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 73.6, 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:28:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:28:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:28:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:28:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:28:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:28:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:28:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:28:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:28:35.953 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 -- 172.21.15.19:0/2998755908 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2db00a3d20 msgr2=0x7f2db00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2db00a3d20 0x7f2db00a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2db4005fd0 tx=0x7f2db4009500 comp rx=0 tx=0).stop 2024-04-23T18:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 -- 172.21.15.19:0/2998755908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc419d000 msgr2=0x7f2dc419d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc419d000 0x7f2dc419d4b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00d750 tx=0x7f2dbc00dc10 comp rx=0 tx=0).stop 2024-04-23T18:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 -- 172.21.15.19:0/2998755908 shutdown_connections 2024-04-23T18:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2db00a3d20 0x7f2db00a61e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dc4106160 0x7f2dc4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc419d000 0x7f2dc419d4b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 --2- 172.21.15.19:0/2998755908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc4101090 0x7f2dc4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 -- 172.21.15.19:0/2998755908 >> 172.21.15.19:0/2998755908 conn(0x7f2dc4077f50 msgr2=0x7f2dc40ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 -- 172.21.15.19:0/2998755908 shutdown_connections 2024-04-23T18:28:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:28:35.956+0000 7f2dc9376640 1 -- 172.21.15.19:0/2998755908 wait complete. 2024-04-23T18:28:35.959 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd3b5be3e', seconds elapsed: 14.7 2024-04-23T18:28:36.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:35 smithi044 ceph-mon[32458]: pgmap v76: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 16 op/s 2024-04-23T18:28:36.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:35 smithi155 ceph-mon[33468]: pgmap v76: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 16 op/s 2024-04-23T18:28:36.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:35 smithi019 ceph-mon[29215]: pgmap v76: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 16 op/s 2024-04-23T18:28:37.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:36 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1768503594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:28:37.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:36 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1768503594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:28:37.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:36 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1768503594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:28:38.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:37 smithi044 ceph-mon[32458]: pgmap v77: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:28:38.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:37 smithi155 ceph-mon[33468]: pgmap v77: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:28:38.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:37 smithi019 ceph-mon[29215]: pgmap v77: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:28:40.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:39 smithi044 ceph-mon[32458]: pgmap v78: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:28:40.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:39 smithi155 ceph-mon[33468]: pgmap v78: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:28:40.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:39 smithi019 ceph-mon[29215]: pgmap v78: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:28:42.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:41 smithi044 ceph-mon[32458]: pgmap v79: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-23T18:28:42.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:41 smithi155 ceph-mon[33468]: pgmap v79: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-23T18:28:42.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:41 smithi019 ceph-mon[29215]: pgmap v79: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-23T18:28:44.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:43 smithi044 ceph-mon[32458]: pgmap v80: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 12 op/s 2024-04-23T18:28:44.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:43 smithi155 ceph-mon[33468]: pgmap v80: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 12 op/s 2024-04-23T18:28:44.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:43 smithi019 ceph-mon[29215]: pgmap v80: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 12 op/s 2024-04-23T18:28:46.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:45 smithi044 ceph-mon[32458]: pgmap v81: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-23T18:28:46.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:45 smithi155 ceph-mon[33468]: pgmap v81: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-23T18:28:46.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:45 smithi019 ceph-mon[29215]: pgmap v81: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-23T18:28:48.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:47 smithi044 ceph-mon[32458]: pgmap v82: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:28:48.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:47 smithi155 ceph-mon[33468]: pgmap v82: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:28:48.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:47 smithi019 ceph-mon[29215]: pgmap v82: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:28:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.706+0000 7f80034bb640 1 -- 172.21.15.19:0/1932073199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ffc10e8b0 msgr2=0x7f7ffc110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.706+0000 7f80034bb640 1 --2- 172.21.15.19:0/1932073199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ffc10e8b0 0x7f7ffc110ca0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009ca0 tx=0x7f7ff402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.706+0000 7f80034bb640 1 -- 172.21.15.19:0/1932073199 shutdown_connections 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.706+0000 7f80034bb640 1 --2- 172.21.15.19:0/1932073199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ffc1016b0 0x7f7ffc10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.706+0000 7f80034bb640 1 --2- 172.21.15.19:0/1932073199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ffc10e8b0 0x7f7ffc110ca0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.706+0000 7f80034bb640 1 --2- 172.21.15.19:0/1932073199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ffc100d00 0x7f7ffc1010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.706+0000 7f80034bb640 1 -- 172.21.15.19:0/1932073199 >> 172.21.15.19:0/1932073199 conn(0x7f7ffc077fd0 msgr2=0x7f7ffc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.707+0000 7f80034bb640 1 -- 172.21.15.19:0/1932073199 shutdown_connections 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.707+0000 7f80034bb640 1 -- 172.21.15.19:0/1932073199 wait complete. 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.707+0000 7f80034bb640 1 Processor -- start 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.707+0000 7f80034bb640 1 -- start start 2024-04-23T18:28:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.707+0000 7f80034bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ffc100d00 0x7f7ffc1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.707+0000 7f80034bb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ffc1016b0 0x7f7ffc1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.707+0000 7f80034bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ffc10e8b0 0x7f7ffc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.707+0000 7f80034bb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1143d0 con 0x7f7ffc10e8b0 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.707+0000 7f80034bb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc114250 con 0x7f7ffc100d00 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.707+0000 7f80034bb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc114550 con 0x7f7ffc1016b0 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8001a58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ffc10e8b0 0x7f7ffc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8001a58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ffc10e8b0 0x7f7ffc19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33286/0 (socket says 172.21.15.19:33286) 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8001a58640 1 -- 172.21.15.19:0/1601048476 learned_addr learned my addr 172.21.15.19:0/1601048476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8001257640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ffc100d00 0x7f7ffc1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8000a56640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ffc1016b0 0x7f7ffc1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8001a58640 1 -- 172.21.15.19:0/1601048476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ffc1016b0 msgr2=0x7f7ffc1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8001a58640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ffc1016b0 0x7f7ffc1a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8001a58640 1 -- 172.21.15.19:0/1601048476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ffc100d00 msgr2=0x7f7ffc1a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8001a58640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ffc100d00 0x7f7ffc1a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8001a58640 1 -- 172.21.15.19:0/1601048476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc19af90 con 0x7f7ffc10e8b0 2024-04-23T18:28:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f8001a58640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ffc10e8b0 0x7f7ffc19a6d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40319f0 tx=0x7f7ff4031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f7ff27fc640 1 -- 172.21.15.19:0/1601048476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4003bf0 con 0x7f7ffc10e8b0 2024-04-23T18:28:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f80034bb640 1 -- 172.21.15.19:0/1601048476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc19b220 con 0x7f7ffc10e8b0 2024-04-23T18:28:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f7ff27fc640 1 -- 172.21.15.19:0/1601048476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff4003d90 con 0x7f7ffc10e8b0 2024-04-23T18:28:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.708+0000 7f7ff27fc640 1 -- 172.21.15.19:0/1601048476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff402fc00 con 0x7f7ffc10e8b0 2024-04-23T18:28:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.709+0000 7f80034bb640 1 -- 172.21.15.19:0/1601048476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc1a7540 con 0x7f7ffc10e8b0 2024-04-23T18:28:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.711+0000 7f7ff27fc640 1 -- 172.21.15.19:0/1601048476 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7ff4004610 con 0x7f7ffc10e8b0 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.711+0000 7f80061d0740 1 Processor -- start 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.711+0000 7f80061d0740 1 -- start start 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.711+0000 7f80061d0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.711+0000 7f80061d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9dc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.711+0000 7f80061d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 0x55e7ac9debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.711+0000 7f80061d0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e7ac936970 con 0x55e7ac9d1620 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.711+0000 7f80061d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e7ac8d34a0 con 0x55e7ac9dc800 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.711+0000 7f80061d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e7ac932930 con 0x55e7ac9d1f40 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8000a56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8000a56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33294/0 (socket says 172.21.15.19:33294) 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8000a56640 1 -- 172.21.15.19:0/5603009 learned_addr learned my addr 172.21.15.19:0/5603009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8001257640 1 --2- 172.21.15.19:0/5603009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9dc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8000a56640 1 --2- 172.21.15.19:0/5603009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 0x55e7ac9debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8000a56640 1 -- 172.21.15.19:0/5603009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 msgr2=0x55e7ac9dc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8000a56640 1 --2- 172.21.15.19:0/5603009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9dc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8000a56640 1 -- 172.21.15.19:0/5603009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 msgr2=0x55e7ac9debf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8000a56640 1 --2- 172.21.15.19:0/5603009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 0x55e7ac9debf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8000a56640 1 -- 172.21.15.19:0/5603009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ac649210 con 0x55e7ac9d1620 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f7ff27fc640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7fe80a3dc0 0x7f7fe80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8001257640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7fe80a3dc0 0x7f7fe80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f8000a56640 1 --2- 172.21.15.19:0/5603009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9d1a00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0058a0 tx=0x7f7fec00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621dffc736907169 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f7ff27fc640 1 -- 172.21.15.19:0/1601048476 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7ff40f1be0 con 0x7f7ffc10e8b0 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f7fe77fe640 1 -- 172.21.15.19:0/5603009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec0120d0 con 0x55e7ac9d1620 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f7fe77fe640 1 -- 172.21.15.19:0/5603009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fec005020 con 0x55e7ac9d1620 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.712+0000 7f7fe77fe640 1 -- 172.21.15.19:0/5603009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec004a00 con 0x55e7ac9d1620 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.713+0000 7f80061d0740 1 -- 172.21.15.19:0/5603009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 msgr2=0x55e7ac9d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.713+0000 7f8001257640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7fe80a3dc0 0x7f7fe80a6280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55e7ac9d2bc0 tx=0x7f7ff800b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.713+0000 7f80061d0740 1 --2- 172.21.15.19:0/5603009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9d1a00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0058a0 tx=0x7f7fec00b7c0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.713+0000 7f80061d0740 1 -- 172.21.15.19:0/5603009 shutdown_connections 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.713+0000 7f80061d0740 1 --2- 172.21.15.19:0/5603009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 0x55e7ac9debf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.713+0000 7f80061d0740 1 --2- 172.21.15.19:0/5603009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9dc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.713+0000 7f80061d0740 1 --2- 172.21.15.19:0/5603009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9d1a00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.713+0000 7f80061d0740 1 -- 172.21.15.19:0/5603009 >> 172.21.15.19:0/5603009 conn(0x55e7ac9cdb30 msgr2=0x55e7ac9cdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.713+0000 7f80061d0740 1 -- 172.21.15.19:0/5603009 shutdown_connections 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.713+0000 7f80061d0740 1 -- 172.21.15.19:0/5603009 wait complete. 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f80061d0740 1 -- start start 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f80061d0740 1 Processor -- start 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f80061d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1620 0x55e7ac9dc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f80061d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f80061d0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9d99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f80061d0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e7ac9df430 con 0x55e7ac9dc800 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f80061d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e7ac8d10b0 con 0x55e7ac9d1f40 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f80061d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e7ac9df5b0 con 0x55e7ac9d1620 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f80061d0740 10 client.0 _ll_register_callbacks cb 0x55e7ac8bed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f8001a58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9d99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f8001257640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1620 0x55e7ac9dc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f8001a58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9d99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33304/0 (socket says 172.21.15.19:33304) 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f8001a58640 1 -- 172.21.15.19:0/4004312805 learned_addr learned my addr 172.21.15.19:0/4004312805 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.714+0000 7f8000a56640 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f8001a58640 1 -- 172.21.15.19:0/4004312805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1620 msgr2=0x55e7ac9dc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f8001a58640 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1620 0x55e7ac9dc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f8001a58640 1 -- 172.21.15.19:0/4004312805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1f40 msgr2=0x55e7ac9d9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f8001a58640 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9d9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f8001a58640 1 -- 172.21.15.19:0/4004312805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ac715620 con 0x55e7ac9dc800 2024-04-23T18:28:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f8001a58640 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9d99a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40ba120 tx=0x7f7ff40bdb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111ace762966634e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f7fe4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/4004312805 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40d7060 con 0x55e7ac9dc800 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/4004312805 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7ff40ba330 con 0x55e7ac9dc800 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f80061d0740 1 -- 172.21.15.19:0/4004312805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fec005020 con 0x55e7ac9dc800 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/4004312805 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ff40be3c0 con 0x55e7ac9dc800 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f80061d0740 10 client.15981 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f80061d0740 1 -- 172.21.15.19:0/4004312805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e7ac9df430 con 0x55e7ac9dc800 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.715+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/4004312805 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fec005020 con 0x55e7ac9dc800 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.716+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/4004312805 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7ff40de0c0 con 0x55e7ac9dc800 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.716+0000 7f80061d0740 10 client.15981 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.716+0000 7f80061d0740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.716+0000 7f80061d0740 20 client.15981 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.716+0000 7f80061d0740 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e7ac9ec7f0 0x55e7ac9eec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.716+0000 7f80061d0740 4 client.15981 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.716+0000 7f80061d0740 1 -- 172.21.15.19:0/4004312805 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ac944390 con 0x55e7ac9ec7f0 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.716+0000 7f8001257640 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e7ac9ec7f0 0x55e7ac9eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.716+0000 7f8001257640 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e7ac9ec7f0 0x55e7ac9eec10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee79ae7733771a26 server_cookie=d0f8507cf8faa1f9 in_seq=0 out_seq=0 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.716+0000 7f7fe4ff9640 10 client.15981 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/4004312805 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7ac944390 con 0x55e7ac9ec7f0 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f7fe4ff9640 10 client.15981 handle_command_reply: tid=0 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f80061d0740 1 client.15981 shutdown 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f80061d0740 1 -- 172.21.15.19:0/4004312805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 msgr2=0x55e7ac9d99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f80061d0740 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9d99a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40ba120 tx=0x7f7ff40bdb90 comp rx=0 tx=0).stop 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f80061d0740 1 -- 172.21.15.19:0/4004312805 shutdown_connections 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f80061d0740 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e7ac9ec7f0 0x55e7ac9eec10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f80061d0740 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f80061d0740 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9d99a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f7fe4ff9640 0 client.15981 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f80061d0740 1 --2- 172.21.15.19:0/4004312805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1620 0x55e7ac9dc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.717+0000 7f80061d0740 1 -- 172.21.15.19:0/4004312805 >> 172.21.15.19:0/4004312805 conn(0x55e7ac9cdb30 msgr2=0x55e7ac9d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.718+0000 7f80061d0740 1 -- 172.21.15.19:0/4004312805 shutdown_connections 2024-04-23T18:28:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.718+0000 7f80061d0740 1 -- 172.21.15.19:0/4004312805 wait complete. 2024-04-23T18:28:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.718+0000 7f80061d0740 20 client.15981 trim_cache size 0 max 16384 2024-04-23T18:28:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f80061d0740 1 Processor -- start 2024-04-23T18:28:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f80061d0740 1 -- start start 2024-04-23T18:28:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f80061d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f80061d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9dbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f80061d0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9f4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f80061d0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1afc20 con 0x55e7ac9dc800 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f80061d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1b1760 con 0x55e7ac9d1620 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f80061d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1b02f0 con 0x55e7ac9d1f40 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001a58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001a58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33320/0 (socket says 172.21.15.19:33320) 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001a58640 1 -- 172.21.15.19:0/219123017 learned_addr learned my addr 172.21.15.19:0/219123017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8000a56640 1 --2- 172.21.15.19:0/219123017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9dbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001257640 1 --2- 172.21.15.19:0/219123017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001a58640 1 -- 172.21.15.19:0/219123017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 msgr2=0x55e7ac9dbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001a58640 1 --2- 172.21.15.19:0/219123017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9dbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001a58640 1 -- 172.21.15.19:0/219123017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1620 msgr2=0x55e7ac9e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001a58640 1 --2- 172.21.15.19:0/219123017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001a58640 1 -- 172.21.15.19:0/219123017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ac9d1d70 con 0x55e7ac9dc800 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001257640 1 --2- 172.21.15.19:0/219123017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f8001a58640 1 --2- 172.21.15.19:0/219123017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9f4520 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40bdc50 tx=0x7f7ff402ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8147fd37b564e8b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.733+0000 7f7fe6ffd640 1 -- 172.21.15.19:0/219123017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40c6980 con 0x55e7ac9dc800 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f7fe6ffd640 1 -- 172.21.15.19:0/219123017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff4036070 con 0x55e7ac9dc800 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f7fe6ffd640 1 -- 172.21.15.19:0/219123017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40d7060 con 0x55e7ac9dc800 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f80061d0740 1 -- 172.21.15.19:0/219123017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 msgr2=0x55e7ac9f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f80061d0740 1 --2- 172.21.15.19:0/219123017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9f4520 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40bdc50 tx=0x7f7ff402ffb0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f80061d0740 1 -- 172.21.15.19:0/219123017 shutdown_connections 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f80061d0740 1 --2- 172.21.15.19:0/219123017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9dc800 0x55e7ac9f4520 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f80061d0740 1 --2- 172.21.15.19:0/219123017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9dbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f80061d0740 1 --2- 172.21.15.19:0/219123017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f80061d0740 1 -- 172.21.15.19:0/219123017 >> 172.21.15.19:0/219123017 conn(0x55e7ac9cdb30 msgr2=0x55e7ac9d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f80061d0740 1 -- 172.21.15.19:0/219123017 shutdown_connections 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.734+0000 7f80061d0740 1 -- 172.21.15.19:0/219123017 wait complete. 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f80061d0740 1 -- start start 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f80061d0740 1 Processor -- start 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f80061d0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f80061d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9e0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f80061d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 0x55e7ac9e0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f80061d0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1b1090 con 0x55e7ac9d1620 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f80061d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1b09c0 con 0x55e7ac9dc800 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f80061d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1adc00 con 0x55e7ac9d1f40 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f80061d0740 10 client.0 _ll_register_callbacks cb 0x7f7ff40b9860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f8001257640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f8001257640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33322/0 (socket says 172.21.15.19:33322) 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f8001257640 1 -- 172.21.15.19:0/177395324 learned_addr learned my addr 172.21.15.19:0/177395324 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:28:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f8001a58640 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 0x55e7ac9e0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.735+0000 7f8000a56640 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f8001257640 1 -- 172.21.15.19:0/177395324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 msgr2=0x55e7ac9e0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f8001257640 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9e0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f8001257640 1 -- 172.21.15.19:0/177395324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 msgr2=0x55e7ac9e0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f8001257640 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 0x55e7ac9e0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f8001257640 1 -- 172.21.15.19:0/177395324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7aca51f40 con 0x55e7ac9d1620 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f8001a58640 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 0x55e7ac9e0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f8001257640 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e0440 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800bfd0 tx=0x7f7ff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c8b9becd12c73c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f7fe4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/177395324 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff801ea70 con 0x55e7ac9d1620 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/177395324 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7ff801ec10 con 0x55e7ac9d1620 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f80061d0740 1 -- 172.21.15.19:0/177395324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff4036070 con 0x55e7ac9d1620 2024-04-23T18:28:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/177395324 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ff8038050 con 0x55e7ac9d1620 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f80061d0740 10 client.15987 fetch_fsmap learned FSMap version 27 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f80061d0740 1 -- 172.21.15.19:0/177395324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e7aca531e0 con 0x55e7ac9d1620 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/177395324 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff4036070 con 0x55e7ac9d1620 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.736+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/177395324 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7ff80117e0 con 0x55e7ac9d1620 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.737+0000 7f80061d0740 10 client.15987 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.737+0000 7f80061d0740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.737+0000 7f80061d0740 20 client.15987 populate_metadata read hostname 'smithi019' 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.737+0000 7f80061d0740 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e7aca53900 0x55e7aca55cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.737+0000 7f80061d0740 4 client.15987 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.737+0000 7f80061d0740 1 -- 172.21.15.19:0/177395324 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7ac93e200 con 0x55e7aca53900 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.737+0000 7f8000a56640 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e7aca53900 0x55e7aca55cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.737+0000 7f8000a56640 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e7aca53900 0x55e7aca55cf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea9d18963501a026 server_cookie=2367afc6caec0a7e in_seq=0 out_seq=0 2024-04-23T18:28:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.737+0000 7f7fe4ff9640 10 client.15987 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.751+0000 7f7fe4ff9640 1 -- 172.21.15.19:0/177395324 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7ac93e200 con 0x55e7aca53900 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.751+0000 7f7fe4ff9640 10 client.15987 handle_command_reply: tid=0 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.751+0000 7f80061d0740 1 client.15987 shutdown 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 1 -- 172.21.15.19:0/177395324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 msgr2=0x55e7ac9e0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e0440 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800bfd0 tx=0x7f7ff80046c0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 1 -- 172.21.15.19:0/177395324 shutdown_connections 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e7aca53900 0x55e7aca55cf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e7ac9dc800 0x55e7ac9e0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e7ac9d1f40 0x55e7ac9e0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 1 --2- 172.21.15.19:0/177395324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7ac9d1620 0x55e7ac9e0440 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f7fe4ff9640 0 client.15987 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 1 -- 172.21.15.19:0/177395324 >> 172.21.15.19:0/177395324 conn(0x55e7ac9cdb30 msgr2=0x55e7ac9e72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 1 -- 172.21.15.19:0/177395324 shutdown_connections 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 1 -- 172.21.15.19:0/177395324 wait complete. 2024-04-23T18:28:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80061d0740 20 client.15987 trim_cache size 0 max 16384 2024-04-23T18:28:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:28:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:28:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:28:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:28:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80034bb640 1 -- 172.21.15.19:0/1601048476 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7fe80a3dc0 msgr2=0x7f7fe80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80034bb640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7fe80a3dc0 0x7f7fe80a6280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55e7ac9d2bc0 tx=0x7f7ff800b390 comp rx=0 tx=0).stop 2024-04-23T18:28:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80034bb640 1 -- 172.21.15.19:0/1601048476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ffc10e8b0 msgr2=0x7f7ffc19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80034bb640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ffc10e8b0 0x7f7ffc19a6d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40319f0 tx=0x7f7ff4031a20 comp rx=0 tx=0).stop 2024-04-23T18:28:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80034bb640 1 -- 172.21.15.19:0/1601048476 shutdown_connections 2024-04-23T18:28:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80034bb640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7fe80a3dc0 0x7f7fe80a6280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80034bb640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ffc10e8b0 0x7f7ffc19a6d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80034bb640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ffc1016b0 0x7f7ffc1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80034bb640 1 --2- 172.21.15.19:0/1601048476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ffc100d00 0x7f7ffc1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.752+0000 7f80034bb640 1 -- 172.21.15.19:0/1601048476 >> 172.21.15.19:0/1601048476 conn(0x7f7ffc077fd0 msgr2=0x7f7ffc10fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.753+0000 7f80034bb640 1 -- 172.21.15.19:0/1601048476 shutdown_connections 2024-04-23T18:28:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:28:48.753+0000 7f80034bb640 1 -- 172.21.15.19:0/1601048476 wait complete. 2024-04-23T18:28:50.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:49 smithi044 ceph-mon[32458]: pgmap v83: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:28:50.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:49 smithi155 ceph-mon[33468]: pgmap v83: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:28:50.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:49 smithi019 ceph-mon[29215]: pgmap v83: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:28:52.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:51 smithi044 ceph-mon[32458]: pgmap v84: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 23 op/s 2024-04-23T18:28:52.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:51 smithi155 ceph-mon[33468]: pgmap v84: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 23 op/s 2024-04-23T18:28:52.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:51 smithi019 ceph-mon[29215]: pgmap v84: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 23 op/s 2024-04-23T18:28:54.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:53 smithi044 ceph-mon[32458]: pgmap v85: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:28:54.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:53 smithi155 ceph-mon[33468]: pgmap v85: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:28:54.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:53 smithi019 ceph-mon[29215]: pgmap v85: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:28:56.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:55 smithi044 ceph-mon[32458]: pgmap v86: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T18:28:56.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:55 smithi155 ceph-mon[33468]: pgmap v86: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T18:28:56.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:55 smithi019 ceph-mon[29215]: pgmap v86: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T18:28:58.062 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:57 smithi019 ceph-mon[29215]: pgmap v87: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:28:58.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:57 smithi044 ceph-mon[32458]: pgmap v87: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:28:58.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:57 smithi155 ceph-mon[33468]: pgmap v87: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:29:00.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:28:59 smithi155 ceph-mon[33468]: pgmap v88: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:29:00.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:28:59 smithi019 ceph-mon[29215]: pgmap v88: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:29:00.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:28:59 smithi044 ceph-mon[32458]: pgmap v88: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:29:01.217 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:29:01.217 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.65 2024-04-23T18:29:01.218 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:29:01.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.340+0000 7fdccdbfe640 1 -- 172.21.15.19:0/196961484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc8102240 msgr2=0x7fdcc8102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.340+0000 7fdccdbfe640 1 --2- 172.21.15.19:0/196961484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc8102240 0x7fdcc8102620 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009900 tx=0x7fdcb802a410 comp rx=0 tx=0).stop 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.340+0000 7fdccdbfe640 1 -- 172.21.15.19:0/196961484 shutdown_connections 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.340+0000 7fdccdbfe640 1 --2- 172.21.15.19:0/196961484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcc8102bf0 0x7fdcc8106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.340+0000 7fdccdbfe640 1 --2- 172.21.15.19:0/196961484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc8107280 0x7fdcc810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.340+0000 7fdccdbfe640 1 --2- 172.21.15.19:0/196961484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc8102240 0x7fdcc8102620 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.340+0000 7fdccdbfe640 1 -- 172.21.15.19:0/196961484 >> 172.21.15.19:0/196961484 conn(0x7fdcc80790e0 msgr2=0x7fdcc80794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.341+0000 7fdccdbfe640 1 -- 172.21.15.19:0/196961484 shutdown_connections 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.341+0000 7fdccdbfe640 1 -- 172.21.15.19:0/196961484 wait complete. 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.341+0000 7fdccdbfe640 1 Processor -- start 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.341+0000 7fdccdbfe640 1 -- start start 2024-04-23T18:29:01.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdccdbfe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc8102240 0x7fdcc819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdccdbfe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcc8107280 0x7fdcc819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdccdbfe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc81985e0 0x7fdcc8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdccdbfe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81121e0 con 0x7fdcc8102240 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdccdbfe640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8112060 con 0x7fdcc8107280 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdccdbfe640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8112360 con 0x7fdcc81985e0 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc8102240 0x7fdcc819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc8102240 0x7fdcc819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45976/0 (socket says 172.21.15.19:45976) 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc77fe640 1 -- 172.21.15.19:0/3913201741 learned_addr learned my addr 172.21.15.19:0/3913201741 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc6ffd640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcc8107280 0x7fdcc819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc7fff640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc81985e0 0x7fdcc8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc77fe640 1 -- 172.21.15.19:0/3913201741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc81985e0 msgr2=0x7fdcc8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc77fe640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc81985e0 0x7fdcc8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc77fe640 1 -- 172.21.15.19:0/3913201741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcc8107280 msgr2=0x7fdcc819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc77fe640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcc8107280 0x7fdcc819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc77fe640 1 -- 172.21.15.19:0/3913201741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc8199100 con 0x7fdcc8102240 2024-04-23T18:29:01.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc7fff640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc81985e0 0x7fdcc8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:01.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc77fe640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc8102240 0x7fdcc819e3e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8005940 tx=0x7fdcb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc4ff9640 1 -- 172.21.15.19:0/3913201741 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb803f030 con 0x7fdcc8102240 2024-04-23T18:29:01.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdcc4ff9640 1 -- 172.21.15.19:0/3913201741 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb8003cb0 con 0x7fdcc8102240 2024-04-23T18:29:01.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.342+0000 7fdccdbfe640 1 -- 172.21.15.19:0/3913201741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc8199390 con 0x7fdcc8102240 2024-04-23T18:29:01.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.343+0000 7fdcc4ff9640 1 -- 172.21.15.19:0/3913201741 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8035070 con 0x7fdcc8102240 2024-04-23T18:29:01.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.343+0000 7fdccdbfe640 1 -- 172.21.15.19:0/3913201741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc81a5530 con 0x7fdcc8102240 2024-04-23T18:29:01.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.343+0000 7fdccdbfe640 1 -- 172.21.15.19:0/3913201741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc80691c0 con 0x7fdcc8102240 2024-04-23T18:29:01.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.345+0000 7fdcc4ff9640 1 -- 172.21.15.19:0/3913201741 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdcb8042030 con 0x7fdcc8102240 2024-04-23T18:29:01.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.345+0000 7fdcc4ff9640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdcb00a3d20 0x7fdcb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.345+0000 7fdcc6ffd640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdcb00a3d20 0x7fdcb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.345+0000 7fdcc4ff9640 1 -- 172.21.15.19:0/3913201741 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fdcb80f1340 con 0x7fdcc8102240 2024-04-23T18:29:01.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.346+0000 7fdcc6ffd640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdcb00a3d20 0x7fdcb00a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8101cf0 tx=0x7fdcbc00b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.349+0000 7fdcc4ff9640 1 -- 172.21.15.19:0/3913201741 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdcb80bda10 con 0x7fdcc8102240 2024-04-23T18:29:01.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.575+0000 7fdccdbfe640 1 -- 172.21.15.19:0/3913201741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcc8102bf0 con 0x7fdcc8102240 2024-04-23T18:29:01.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.576+0000 7fdcc4ff9640 1 -- 172.21.15.19:0/3913201741 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7fdcb80bd310 con 0x7fdcc8102240 2024-04-23T18:29:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:29:01.569 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:29:01.569 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:29:01.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.578+0000 7fdccdbfe640 1 -- 172.21.15.19:0/3913201741 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdcb00a3d20 msgr2=0x7fdcb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.578+0000 7fdccdbfe640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdcb00a3d20 0x7fdcb00a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8101cf0 tx=0x7fdcbc00b390 comp rx=0 tx=0).stop 2024-04-23T18:29:01.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.578+0000 7fdccdbfe640 1 -- 172.21.15.19:0/3913201741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc8102240 msgr2=0x7fdcc819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.578+0000 7fdccdbfe640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc8102240 0x7fdcc819e3e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8005940 tx=0x7fdcb80042f0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.578+0000 7fdccdbfe640 1 -- 172.21.15.19:0/3913201741 shutdown_connections 2024-04-23T18:29:01.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.578+0000 7fdccdbfe640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdcb00a3d20 0x7fdcb00a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.578+0000 7fdccdbfe640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcc8107280 0x7fdcc819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.578+0000 7fdccdbfe640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc81985e0 0x7fdcc8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.578+0000 7fdccdbfe640 1 --2- 172.21.15.19:0/3913201741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc8102240 0x7fdcc819e3e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.578+0000 7fdccdbfe640 1 -- 172.21.15.19:0/3913201741 >> 172.21.15.19:0/3913201741 conn(0x7fdcc80790e0 msgr2=0x7fdcc81004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.579+0000 7fdccdbfe640 1 -- 172.21.15.19:0/3913201741 shutdown_connections 2024-04-23T18:29:01.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.579+0000 7fdccdbfe640 1 -- 172.21.15.19:0/3913201741 wait complete. 2024-04-23T18:29:01.582 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 77.65 --await-for 42.0' 2024-04-23T18:29:01.582 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 77.65 --await-for 42.0 2024-04-23T18:29:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.702+0000 7f9ab80c2640 1 -- 172.21.15.19:0/3414614873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab0107260 msgr2=0x7f9ab010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.702+0000 7f9ab80c2640 1 --2- 172.21.15.19:0/3414614873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab0107260 0x7f9ab010ee30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009ca0 tx=0x7f9aa802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.702+0000 7f9ab80c2640 1 -- 172.21.15.19:0/3414614873 shutdown_connections 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.702+0000 7f9ab80c2640 1 --2- 172.21.15.19:0/3414614873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab0107260 0x7f9ab010ee30 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.702+0000 7f9ab80c2640 1 --2- 172.21.15.19:0/3414614873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0102b40 0x7f9ab0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.702+0000 7f9ab80c2640 1 --2- 172.21.15.19:0/3414614873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab0102190 0x7f9ab0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.702+0000 7f9ab80c2640 1 -- 172.21.15.19:0/3414614873 >> 172.21.15.19:0/3414614873 conn(0x7f9ab0079310 msgr2=0x7f9ab0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.702+0000 7f9ab80c2640 1 -- 172.21.15.19:0/3414614873 shutdown_connections 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.702+0000 7f9ab80c2640 1 -- 172.21.15.19:0/3414614873 wait complete. 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab80c2640 1 Processor -- start 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab80c2640 1 -- start start 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab80c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab0102190 0x7f9ab019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab80c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab0107260 0x7f9ab019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab80c2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0198680 0x7f9ab0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab80c2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01122a0 con 0x7f9ab0102190 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab80c2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0112120 con 0x7f9ab0107260 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab80c2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0112420 con 0x7f9ab0198680 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab5e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab0102190 0x7f9ab019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab5e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab0102190 0x7f9ab019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45994/0 (socket says 172.21.15.19:45994) 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/1112636257 learned_addr learned my addr 172.21.15.19:0/1112636257 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab565d640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab0107260 0x7f9ab019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab665f640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0198680 0x7f9ab0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/1112636257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0198680 msgr2=0x7f9ab0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0198680 0x7f9ab0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/1112636257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab0107260 msgr2=0x7f9ab019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab0107260 0x7f9ab019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.703+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/1112636257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab0199280 con 0x7f9ab0102190 2024-04-23T18:29:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.704+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab0102190 0x7f9ab019e5c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00b4a0 tx=0x7f9aac00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.704+0000 7f9aa6ffd640 1 -- 172.21.15.19:0/1112636257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac015020 con 0x7f9ab0102190 2024-04-23T18:29:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.704+0000 7f9aa6ffd640 1 -- 172.21.15.19:0/1112636257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aac011ce0 con 0x7f9ab0102190 2024-04-23T18:29:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.704+0000 7f9ab80c2640 1 -- 172.21.15.19:0/1112636257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab01a5360 con 0x7f9ab0102190 2024-04-23T18:29:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.704+0000 7f9aa6ffd640 1 -- 172.21.15.19:0/1112636257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac010dc0 con 0x7f9ab0102190 2024-04-23T18:29:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.704+0000 7f9ab80c2640 1 -- 172.21.15.19:0/1112636257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab01a5760 con 0x7f9ab0102190 2024-04-23T18:29:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.706+0000 7f9aa6ffd640 1 -- 172.21.15.19:0/1112636257 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9aac018020 con 0x7f9ab0102190 2024-04-23T18:29:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.706+0000 7f9abadd1740 1 Processor -- start 2024-04-23T18:29:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.706+0000 7f9abadd1740 1 -- start start 2024-04-23T18:29:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89020a0 0x557bd8902480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89029c0 0x557bd890cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd890f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9abadd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bd8867370 con 0x557bd89029c0 2024-04-23T18:29:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9abadd1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557bd88042e0 con 0x557bd890d090 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9abadd1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557bd8863330 con 0x557bd89020a0 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab5e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89029c0 0x557bd890cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab5e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89029c0 0x557bd890cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46010/0 (socket says 172.21.15.19:46010) 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9aa6ffd640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a9c0a3d20 0x7f9a9c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/2093115484 learned_addr learned my addr 172.21.15.19:0/2093115484 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab665f640 1 --2- 172.21.15.19:0/2093115484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89020a0 0x557bd8902480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab565d640 1 --2- 172.21.15.19:0/2093115484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd890f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab665f640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a9c0a3d20 0x7f9a9c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9aa6ffd640 1 -- 172.21.15.19:0/1112636257 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f9aac0cffc0 con 0x7f9ab0102190 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/2093115484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89020a0 msgr2=0x557bd8902480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/2093115484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89020a0 0x557bd8902480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/2093115484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 msgr2=0x557bd890f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/2093115484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd890f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/2093115484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bd8645d80 con 0x557bd89029c0 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/2093115484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89029c0 0x557bd890cb50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9aac061200 tx=0x7f9aac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61be05207d8ec966 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.707+0000 7f9ab665f640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a9c0a3d20 0x7f9a9c0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009a60 tx=0x7f9aa80095c0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9a9bfff640 1 -- 172.21.15.19:0/2093115484 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac061020 con 0x557bd89029c0 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9a9bfff640 1 -- 172.21.15.19:0/2093115484 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aac09c0a0 con 0x557bd89029c0 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9a9bfff640 1 -- 172.21.15.19:0/2093115484 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0cd090 con 0x557bd89029c0 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9abadd1740 1 -- 172.21.15.19:0/2093115484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89029c0 msgr2=0x557bd890cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9abadd1740 1 --2- 172.21.15.19:0/2093115484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89029c0 0x557bd890cb50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9aac061200 tx=0x7f9aac000bc0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9abadd1740 1 -- 172.21.15.19:0/2093115484 shutdown_connections 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9abadd1740 1 --2- 172.21.15.19:0/2093115484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd890f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9abadd1740 1 --2- 172.21.15.19:0/2093115484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89029c0 0x557bd890cb50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9abadd1740 1 --2- 172.21.15.19:0/2093115484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89020a0 0x557bd8902480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9abadd1740 1 -- 172.21.15.19:0/2093115484 >> 172.21.15.19:0/2093115484 conn(0x557bd88fe3c0 msgr2=0x557bd88fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9abadd1740 1 -- 172.21.15.19:0/2093115484 shutdown_connections 2024-04-23T18:29:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.708+0000 7f9abadd1740 1 -- 172.21.15.19:0/2093115484 wait complete. 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9abadd1740 1 -- start start 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9abadd1740 1 Processor -- start 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd890a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 0x557bd890a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd890aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9abadd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bd890fdc0 con 0x557bd89020a0 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9abadd1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557bd8801fb0 con 0x557bd890d090 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9abadd1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557bd890ff40 con 0x557bd89029c0 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9abadd1740 10 client.0 _ll_register_callbacks cb 0x557bd88019a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9ab5e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd890a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9ab5e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd890a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46012/0 (socket says 172.21.15.19:46012) 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/312675738 learned_addr learned my addr 172.21.15.19:0/312675738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.709+0000 7f9ab565d640 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 0x557bd890a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9ab665f640 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd890aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/312675738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 msgr2=0x557bd890a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 0x557bd890a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/312675738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 msgr2=0x557bd890aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd890aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/312675738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bd86a6510 con 0x557bd89020a0 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9ab665f640 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd890aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd890a060 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9aac09c030 tx=0x7f9aac010380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4560e4c22312b3d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9a997fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9a997fa640 1 -- 172.21.15.19:0/312675738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac061020 con 0x557bd89020a0 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9a997fa640 1 -- 172.21.15.19:0/312675738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f9aac0a00d0 con 0x557bd89020a0 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9abadd1740 1 -- 172.21.15.19:0/312675738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aac09c0a0 con 0x557bd89020a0 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9a997fa640 1 -- 172.21.15.19:0/312675738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9aac098340 con 0x557bd89020a0 2024-04-23T18:29:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9abadd1740 10 client.16005 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9abadd1740 1 -- 172.21.15.19:0/312675738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bd8539670 con 0x557bd89020a0 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9a997fa640 1 -- 172.21.15.19:0/312675738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aac09c0a0 con 0x557bd89020a0 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9a997fa640 1 -- 172.21.15.19:0/312675738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9aac098710 con 0x557bd89020a0 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9abadd1740 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9abadd1740 10 client.16005 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9abadd1740 20 client.16005 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.710+0000 7f9abadd1740 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557bd891d230 0x557bd891f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.711+0000 7f9abadd1740 4 client.16005 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.711+0000 7f9abadd1740 1 -- 172.21.15.19:0/312675738 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bd8874d90 con 0x557bd891d230 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.711+0000 7f9ab565d640 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557bd891d230 0x557bd891f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.711+0000 7f9a997fa640 10 client.16005 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.711+0000 7f9ab565d640 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557bd891d230 0x557bd891f620 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e92ae4fc5c4b8b server_cookie=c08cdf1abee66328 in_seq=0 out_seq=0 2024-04-23T18:29:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9a997fa640 1 -- 172.21.15.19:0/312675738 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bd8874d90 con 0x557bd891d230 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9a997fa640 10 client.16005 handle_command_reply: tid=0 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 client.16005 shutdown 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 -- 172.21.15.19:0/312675738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 msgr2=0x557bd890a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd890a060 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9aac09c030 tx=0x7f9aac010380 comp rx=0 tx=0).stop 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 -- 172.21.15.19:0/312675738 shutdown_connections 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557bd891d230 0x557bd891f620 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd890aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 0x557bd890a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 --2- 172.21.15.19:0/312675738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd890a060 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9a997fa640 0 client.16005 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 -- 172.21.15.19:0/312675738 >> 172.21.15.19:0/312675738 conn(0x557bd88fe3c0 msgr2=0x557bd88fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 -- 172.21.15.19:0/312675738 shutdown_connections 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 1 -- 172.21.15.19:0/312675738 wait complete. 2024-04-23T18:29:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.712+0000 7f9abadd1740 20 client.16005 trim_cache size 0 max 16384 2024-04-23T18:29:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.729+0000 7f9abadd1740 1 Processor -- start 2024-04-23T18:29:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.729+0000 7f9abadd1740 1 -- start start 2024-04-23T18:29:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8922a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 0x557bd89072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd8907800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9abadd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01adc90 con 0x557bd89020a0 2024-04-23T18:29:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9abadd1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01af7d0 con 0x557bd890d090 2024-04-23T18:29:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9abadd1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01ae360 con 0x557bd89029c0 2024-04-23T18:29:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9ab5e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8922a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9ab5e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8922a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46026/0 (socket says 172.21.15.19:46026) 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/2047865960 learned_addr learned my addr 172.21.15.19:0/2047865960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/2047865960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 msgr2=0x557bd89072c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/2047865960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 0x557bd89072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/2047865960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 msgr2=0x557bd8907800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/2047865960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd8907800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/2047865960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bd89085b0 con 0x557bd89020a0 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/2047865960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8922a30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0cd900 tx=0x7f9aac0cd6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89b1cea9ca5156c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9a9b7fe640 1 -- 172.21.15.19:0/2047865960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac09c0a0 con 0x557bd89020a0 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9a9b7fe640 1 -- 172.21.15.19:0/2047865960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aac0cd730 con 0x557bd89020a0 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.730+0000 7f9a9b7fe640 1 -- 172.21.15.19:0/2047865960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0cd180 con 0x557bd89020a0 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 -- 172.21.15.19:0/2047865960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 msgr2=0x557bd8922a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 --2- 172.21.15.19:0/2047865960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8922a30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0cd900 tx=0x7f9aac0cd6c0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 -- 172.21.15.19:0/2047865960 shutdown_connections 2024-04-23T18:29:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 --2- 172.21.15.19:0/2047865960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd8907800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 --2- 172.21.15.19:0/2047865960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 0x557bd89072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 --2- 172.21.15.19:0/2047865960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8922a30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 -- 172.21.15.19:0/2047865960 >> 172.21.15.19:0/2047865960 conn(0x557bd88fe3c0 msgr2=0x557bd8918a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 -- 172.21.15.19:0/2047865960 shutdown_connections 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 -- 172.21.15.19:0/2047865960 wait complete. 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 -- start start 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.731+0000 7f9abadd1740 1 Processor -- start 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8980c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 0x557bd89811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9abadd1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd897b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9abadd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01af100 con 0x557bd89020a0 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9abadd1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01aea30 con 0x557bd890d090 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9abadd1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01abc70 con 0x557bd89029c0 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9abadd1740 10 client.0 _ll_register_callbacks cb 0x7f9aac010f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9ab5e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8980c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9ab5e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8980c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46030/0 (socket says 172.21.15.19:46030) 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/443203012 learned_addr learned my addr 172.21.15.19:0/443203012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/443203012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 msgr2=0x557bd89811d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 0x557bd89811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/443203012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 msgr2=0x557bd897b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd897b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9ab5e5e640 1 -- 172.21.15.19:0/443203012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bd897b790 con 0x557bd89020a0 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9ab5e5e640 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8980c90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0cdd40 tx=0x7f9aac0107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fdaf0595e72092 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9a997fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9a997fa640 1 -- 172.21.15.19:0/443203012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0b9030 con 0x557bd89020a0 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9a997fa640 1 -- 172.21.15.19:0/443203012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f9aac0cd930 con 0x557bd89020a0 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.732+0000 7f9abadd1740 1 -- 172.21.15.19:0/443203012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aac0cd730 con 0x557bd89020a0 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9a997fa640 1 -- 172.21.15.19:0/443203012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9aac098060 con 0x557bd89020a0 2024-04-23T18:29:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9abadd1740 10 client.16011 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9abadd1740 1 -- 172.21.15.19:0/443203012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bd897b790 con 0x557bd89020a0 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9a997fa640 1 -- 172.21.15.19:0/443203012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aac0cd730 con 0x557bd89020a0 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9a997fa640 1 -- 172.21.15.19:0/443203012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9aac0b0b40 con 0x557bd89020a0 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9abadd1740 10 client.16011 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9abadd1740 10 client.16011 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9abadd1740 20 client.16011 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9abadd1740 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557bd8988740 0x557bd898ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9abadd1740 4 client.16011 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.65, "await_for": 42.0}] 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9abadd1740 1 -- 172.21.15.19:0/443203012 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.65, "await_for": 42.0}) -- 0x557bd886ec00 con 0x557bd8988740 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.733+0000 7f9ab565d640 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557bd8988740 0x557bd898ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.734+0000 7f9ab565d640 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557bd8988740 0x557bd898ab70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ba46fb962b5ff31 server_cookie=3e54c07562fc85b3 in_seq=0 out_seq=0 2024-04-23T18:29:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.734+0000 7f9a997fa640 10 client.16011 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.754+0000 7f9a997fa640 1 -- 172.21.15.19:0/443203012 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557bd886ec00 con 0x557bd8988740 2024-04-23T18:29:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.754+0000 7f9a997fa640 10 client.16011 handle_command_reply: tid=0 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.754+0000 7f9abadd1740 1 client.16011 shutdown 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 1 -- 172.21.15.19:0/443203012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 msgr2=0x557bd8980c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8980c90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0cdd40 tx=0x7f9aac0107e0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 1 -- 172.21.15.19:0/443203012 shutdown_connections 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557bd8988740 0x557bd898ab70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557bd890d090 0x557bd897b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bd89029c0 0x557bd89811d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 1 --2- 172.21.15.19:0/443203012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd89020a0 0x557bd8980c90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9a997fa640 0 client.16011 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 1 -- 172.21.15.19:0/443203012 >> 172.21.15.19:0/443203012 conn(0x557bd88fe3c0 msgr2=0x557bd88fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 1 -- 172.21.15.19:0/443203012 shutdown_connections 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 1 -- 172.21.15.19:0/443203012 wait complete. 2024-04-23T18:29:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9abadd1740 20 client.16011 trim_cache size 0 max 16384 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 14, 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "dc70017e": { 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 14, 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:29:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 77.7, 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:01.750 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:29:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 -- 172.21.15.19:0/1112636257 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a9c0a3d20 msgr2=0x7f9a9c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a9c0a3d20 0x7f9a9c0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009a60 tx=0x7f9aa80095c0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 -- 172.21.15.19:0/1112636257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab0102190 msgr2=0x7f9ab019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab0102190 0x7f9ab019e5c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00b4a0 tx=0x7f9aac00b8a0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 -- 172.21.15.19:0/1112636257 shutdown_connections 2024-04-23T18:29:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a9c0a3d20 0x7f9a9c0a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab0107260 0x7f9ab019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0198680 0x7f9ab0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 --2- 172.21.15.19:0/1112636257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ab0102190 0x7f9ab019e5c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 -- 172.21.15.19:0/1112636257 >> 172.21.15.19:0/1112636257 conn(0x7f9ab0079310 msgr2=0x7f9ab0100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 -- 172.21.15.19:0/1112636257 shutdown_connections 2024-04-23T18:29:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:01.755+0000 7f9ab80c2640 1 -- 172.21.15.19:0/1112636257 wait complete. 2024-04-23T18:29:01.758 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dc70017e', quiesce took 0.5 seconds. Will release after: 17.65 2024-04-23T18:29:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.842+0000 7f148df19640 1 -- 172.21.15.19:0/1799785926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148810e930 msgr2=0x7f1488110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.843+0000 7f148df19640 1 --2- 172.21.15.19:0/1799785926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148810e930 0x7f1488110d20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1480009ca0 tx=0x7f148002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.843+0000 7f148df19640 1 -- 172.21.15.19:0/1799785926 shutdown_connections 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.843+0000 7f148df19640 1 --2- 172.21.15.19:0/1799785926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148810e930 0x7f1488110d20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.843+0000 7f148df19640 1 --2- 172.21.15.19:0/1799785926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14881016a0 0x7f148810e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.843+0000 7f148df19640 1 --2- 172.21.15.19:0/1799785926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1488100d80 0x7f1488101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.843+0000 7f148df19640 1 -- 172.21.15.19:0/1799785926 >> 172.21.15.19:0/1799785926 conn(0x7f1488077fd0 msgr2=0x7f14880783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.843+0000 7f148df19640 1 -- 172.21.15.19:0/1799785926 shutdown_connections 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.843+0000 7f148df19640 1 -- 172.21.15.19:0/1799785926 wait complete. 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.843+0000 7f148df19640 1 Processor -- start 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.843+0000 7f148df19640 1 -- start start 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f148df19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488100d80 0x7f14881a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f148df19640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14881016a0 0x7f14881a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f148df19640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f148810e930 0x7f148819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f148df19640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1488114390 con 0x7f1488100d80 2024-04-23T18:29:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f148df19640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1488114210 con 0x7f148810e930 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f148df19640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1488114510 con 0x7f14881016a0 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f14877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488100d80 0x7f14881a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f14877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488100d80 0x7f14881a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46044/0 (socket says 172.21.15.19:46044) 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f14877fe640 1 -- 172.21.15.19:0/1373128428 learned_addr learned my addr 172.21.15.19:0/1373128428 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f1486ffd640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14881016a0 0x7f14881a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f14877fe640 1 -- 172.21.15.19:0/1373128428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14881016a0 msgr2=0x7f14881a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f14877fe640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14881016a0 0x7f14881a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f1487fff640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f148810e930 0x7f148819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f14877fe640 1 -- 172.21.15.19:0/1373128428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f148810e930 msgr2=0x7f148819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f14877fe640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f148810e930 0x7f148819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f14877fe640 1 -- 172.21.15.19:0/1373128428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148819afb0 con 0x7f1488100d80 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f1486ffd640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14881016a0 0x7f14881a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f14877fe640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488100d80 0x7f14881a05a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f147800d950 tx=0x7f147800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f1484ff9640 1 -- 172.21.15.19:0/1373128428 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478017070 con 0x7f1488100d80 2024-04-23T18:29:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f148df19640 1 -- 172.21.15.19:0/1373128428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148819b2a0 con 0x7f1488100d80 2024-04-23T18:29:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.844+0000 7f1484ff9640 1 -- 172.21.15.19:0/1373128428 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1478009c30 con 0x7f1488100d80 2024-04-23T18:29:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.845+0000 7f148df19640 1 -- 172.21.15.19:0/1373128428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881a7590 con 0x7f1488100d80 2024-04-23T18:29:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.845+0000 7f1484ff9640 1 -- 172.21.15.19:0/1373128428 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478020060 con 0x7f1488100d80 2024-04-23T18:29:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1484ff9640 1 -- 172.21.15.19:0/1373128428 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1478010460 con 0x7f1488100d80 2024-04-23T18:29:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1490c2d740 1 Processor -- start 2024-04-23T18:29:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1490c2d740 1 -- start start 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1484ff9640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f14700a3dc0 0x7f14700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150620 0x55fffb150a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150f40 0x55fffb15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1490c2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fffb0b5970 con 0x55fffb150f40 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1490c2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fffb0524a0 con 0x55fffb150620 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1490c2d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fffb0b1930 con 0x55fffb15b800 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1487fff640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f14700a3dc0 0x7f14700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1486ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150f40 0x55fffb15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1484ff9640 1 -- 172.21.15.19:0/1373128428 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f14780d0420 con 0x7f1488100d80 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1486ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150f40 0x55fffb15b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46060/0 (socket says 172.21.15.19:46060) 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.847+0000 7f1486ffd640 1 -- 172.21.15.19:0/714684755 learned_addr learned my addr 172.21.15.19:0/714684755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f14877fe640 1 --2- 172.21.15.19:0/714684755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150620 0x55fffb150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f1487fff640 1 --2- 172.21.15.19:0/714684755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f1486ffd640 1 -- 172.21.15.19:0/714684755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 msgr2=0x55fffb15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f1486ffd640 1 --2- 172.21.15.19:0/714684755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb15dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f1486ffd640 1 -- 172.21.15.19:0/714684755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150620 msgr2=0x55fffb150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f1486ffd640 1 --2- 172.21.15.19:0/714684755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150620 0x55fffb150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f1486ffd640 1 -- 172.21.15.19:0/714684755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffadc8210 con 0x55fffb150f40 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f1487fff640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f14700a3dc0 0x7f14700a6280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1488102f60 tx=0x7f1480031af0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f1486ffd640 1 --2- 172.21.15.19:0/714684755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150f40 0x55fffb15b2c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f147c00ba70 tx=0x7f147c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3486e8d738264df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f146dffb640 1 -- 172.21.15.19:0/714684755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00ddf0 con 0x55fffb150f40 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f146dffb640 1 -- 172.21.15.19:0/714684755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147c0178d0 con 0x55fffb150f40 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.848+0000 7f146dffb640 1 -- 172.21.15.19:0/714684755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00f880 con 0x55fffb150f40 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 -- 172.21.15.19:0/714684755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150f40 msgr2=0x55fffb15b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 --2- 172.21.15.19:0/714684755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150f40 0x55fffb15b2c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f147c00ba70 tx=0x7f147c00be70 comp rx=0 tx=0).stop 2024-04-23T18:29:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 -- 172.21.15.19:0/714684755 shutdown_connections 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 --2- 172.21.15.19:0/714684755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 --2- 172.21.15.19:0/714684755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150f40 0x55fffb15b2c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 --2- 172.21.15.19:0/714684755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150620 0x55fffb150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 -- 172.21.15.19:0/714684755 >> 172.21.15.19:0/714684755 conn(0x55fffb14cb30 msgr2=0x55fffb14cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 -- 172.21.15.19:0/714684755 shutdown_connections 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 -- 172.21.15.19:0/714684755 wait complete. 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 -- start start 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.849+0000 7f1490c2d740 1 Processor -- start 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb15b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 0x55fffb158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb158940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1490c2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fffb15e430 con 0x55fffb150620 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1490c2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fffb0500b0 con 0x55fffb150f40 2024-04-23T18:29:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1490c2d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fffb15e5b0 con 0x55fffb15b800 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1490c2d740 10 client.0 _ll_register_callbacks cb 0x55fffb03dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f14877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb15b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f14877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb15b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46064/0 (socket says 172.21.15.19:46064) 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f14877fe640 1 -- 172.21.15.19:0/3847489200 learned_addr learned my addr 172.21.15.19:0/3847489200 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1486ffd640 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 0x55fffb158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1487fff640 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f14877fe640 1 -- 172.21.15.19:0/3847489200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 msgr2=0x55fffb158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f14877fe640 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f14877fe640 1 -- 172.21.15.19:0/3847489200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 msgr2=0x55fffb158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f14877fe640 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 0x55fffb158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f14877fe640 1 -- 172.21.15.19:0/3847489200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffae94620 con 0x55fffb150620 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1486ffd640 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 0x55fffb158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f14877fe640 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb15b110 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f147809c330 tx=0x7f1478009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da0c891d71fe27 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1466ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1466ffd640 1 -- 172.21.15.19:0/3847489200 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147809c3f0 con 0x55fffb150620 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1466ffd640 1 -- 172.21.15.19:0/3847489200 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f147806c0c0 con 0x55fffb150620 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.850+0000 7f1490c2d740 1 -- 172.21.15.19:0/3847489200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147c0178d0 con 0x55fffb150620 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1466ffd640 1 -- 172.21.15.19:0/3847489200 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14780a07a0 con 0x55fffb150620 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1490c2d740 10 client.16023 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1490c2d740 1 -- 172.21.15.19:0/3847489200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fffb16a350 con 0x55fffb150620 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1466ffd640 1 -- 172.21.15.19:0/3847489200 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147c0178d0 con 0x55fffb150620 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1466ffd640 1 -- 172.21.15.19:0/3847489200 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f14780cf040 con 0x55fffb150620 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1490c2d740 10 client.16023 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1490c2d740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1490c2d740 20 client.16023 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1490c2d740 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fffb16b7c0 0x55fffb16dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1490c2d740 4 client.16023 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1490c2d740 1 -- 172.21.15.19:0/3847489200 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fffb0c3390 con 0x55fffb16b7c0 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.851+0000 7f1486ffd640 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fffb16b7c0 0x55fffb16dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.852+0000 7f1486ffd640 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fffb16b7c0 0x55fffb16dc10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88afa874e5fce892 server_cookie=a6834cdb524b004b in_seq=0 out_seq=0 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.852+0000 7f1466ffd640 10 client.16023 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.852+0000 7f1466ffd640 1 -- 172.21.15.19:0/3847489200 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fffb0c3390 con 0x55fffb16b7c0 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.852+0000 7f1466ffd640 10 client.16023 handle_command_reply: tid=0 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.852+0000 7f1490c2d740 1 client.16023 shutdown 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 1 -- 172.21.15.19:0/3847489200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 msgr2=0x55fffb15b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb15b110 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f147809c330 tx=0x7f1478009ea0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 1 -- 172.21.15.19:0/3847489200 shutdown_connections 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fffb16b7c0 0x55fffb16dc10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 0x55fffb158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1466ffd640 0 client.16023 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 1 --2- 172.21.15.19:0/3847489200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb15b110 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 1 -- 172.21.15.19:0/3847489200 >> 172.21.15.19:0/3847489200 conn(0x55fffb14cb30 msgr2=0x55fffb1528b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 1 -- 172.21.15.19:0/3847489200 shutdown_connections 2024-04-23T18:29:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 1 -- 172.21.15.19:0/3847489200 wait complete. 2024-04-23T18:29:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.853+0000 7f1490c2d740 20 client.16023 trim_cache size 0 max 16384 2024-04-23T18:29:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.867+0000 7f1490c2d740 1 Processor -- start 2024-04-23T18:29:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.867+0000 7f1490c2d740 1 -- start start 2024-04-23T18:29:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb1672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 0x55fffb14bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb15ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f1490c2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14881afcc0 con 0x55fffb150620 2024-04-23T18:29:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f1490c2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14881b1800 con 0x55fffb150f40 2024-04-23T18:29:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f1490c2d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14881b0390 con 0x55fffb15b800 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f14877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb1672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f14877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb1672f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46074/0 (socket says 172.21.15.19:46074) 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f14877fe640 1 -- 172.21.15.19:0/2879978885 learned_addr learned my addr 172.21.15.19:0/2879978885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f1487fff640 1 --2- 172.21.15.19:0/2879978885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb15ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f14877fe640 1 -- 172.21.15.19:0/2879978885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 msgr2=0x55fffb15ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f14877fe640 1 --2- 172.21.15.19:0/2879978885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb15ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f14877fe640 1 -- 172.21.15.19:0/2879978885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 msgr2=0x55fffb14bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f1486ffd640 1 --2- 172.21.15.19:0/2879978885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 0x55fffb14bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f14877fe640 1 --2- 172.21.15.19:0/2879978885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 0x55fffb14bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f14877fe640 1 -- 172.21.15.19:0/2879978885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffb156990 con 0x55fffb150620 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f14877fe640 1 --2- 172.21.15.19:0/2879978885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb1672f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1478000c00 tx=0x7f14780cdc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab19026e417bba7d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f146d7fa640 1 -- 172.21.15.19:0/2879978885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780a5060 con 0x55fffb150620 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f146d7fa640 1 -- 172.21.15.19:0/2879978885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1478098a60 con 0x55fffb150620 2024-04-23T18:29:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.868+0000 7f146d7fa640 1 -- 172.21.15.19:0/2879978885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147806c2d0 con 0x55fffb150620 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.869+0000 7f1490c2d740 1 -- 172.21.15.19:0/2879978885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 msgr2=0x55fffb1672f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.869+0000 7f1490c2d740 1 --2- 172.21.15.19:0/2879978885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb1672f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1478000c00 tx=0x7f14780cdc50 comp rx=0 tx=0).stop 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.869+0000 7f1490c2d740 1 -- 172.21.15.19:0/2879978885 shutdown_connections 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.869+0000 7f1490c2d740 1 --2- 172.21.15.19:0/2879978885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb150f40 0x55fffb14bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.869+0000 7f1490c2d740 1 --2- 172.21.15.19:0/2879978885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb15b800 0x55fffb15ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.869+0000 7f1490c2d740 1 --2- 172.21.15.19:0/2879978885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb1672f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.869+0000 7f1490c2d740 1 -- 172.21.15.19:0/2879978885 >> 172.21.15.19:0/2879978885 conn(0x55fffb14cb30 msgr2=0x55fffb1539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.869+0000 7f1490c2d740 1 -- 172.21.15.19:0/2879978885 shutdown_connections 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.869+0000 7f1490c2d740 1 -- 172.21.15.19:0/2879978885 wait complete. 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.869+0000 7f1490c2d740 1 -- start start 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1490c2d740 1 Processor -- start 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb163bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb150f40 0x55fffb154db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1490c2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb15b800 0x55fffb1552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1490c2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14881b1130 con 0x55fffb150620 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1490c2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14881b0a60 con 0x55fffb15b800 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1490c2d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14881adca0 con 0x55fffb150f40 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1490c2d740 10 client.0 _ll_register_callbacks cb 0x7f147809be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f14877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb163bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f14877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb163bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46086/0 (socket says 172.21.15.19:46086) 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f14877fe640 1 -- 172.21.15.19:0/4254675594 learned_addr learned my addr 172.21.15.19:0/4254675594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1486ffd640 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb150f40 0x55fffb154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f14877fe640 1 -- 172.21.15.19:0/4254675594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb150f40 msgr2=0x55fffb154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f14877fe640 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb150f40 0x55fffb154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f14877fe640 1 -- 172.21.15.19:0/4254675594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb15b800 msgr2=0x55fffb1552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f14877fe640 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb15b800 0x55fffb1552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f14877fe640 1 -- 172.21.15.19:0/4254675594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffb155af0 con 0x55fffb150620 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f14877fe640 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb163bf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1478098dc0 tx=0x7f14780bb4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9840255cd5dcc68 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1466ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.870+0000 7f1466ffd640 1 -- 172.21.15.19:0/4254675594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780a5060 con 0x55fffb150620 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1466ffd640 1 -- 172.21.15.19:0/4254675594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f147809b840 con 0x55fffb150620 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1490c2d740 1 -- 172.21.15.19:0/4254675594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1478098a60 con 0x55fffb150620 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1466ffd640 1 -- 172.21.15.19:0/4254675594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f147809ba40 con 0x55fffb150620 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1490c2d740 10 client.16029 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1490c2d740 1 -- 172.21.15.19:0/4254675594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fffb1d1db0 con 0x55fffb150620 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1466ffd640 1 -- 172.21.15.19:0/4254675594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1478098a60 con 0x55fffb150620 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1466ffd640 1 -- 172.21.15.19:0/4254675594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f147809b440 con 0x55fffb150620 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1490c2d740 10 client.16029 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1490c2d740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1490c2d740 20 client.16029 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1490c2d740 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fffb1d2280 0x55fffb1d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1490c2d740 4 client.16029 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1490c2d740 1 -- 172.21.15.19:0/4254675594 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fffb0bd200 con 0x55fffb1d2280 2024-04-23T18:29:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.871+0000 7f1486ffd640 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fffb1d2280 0x55fffb1d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.872+0000 7f1486ffd640 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fffb1d2280 0x55fffb1d4670 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d72f9da59176885 server_cookie=2e2c674091ac8cce in_seq=0 out_seq=0 2024-04-23T18:29:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.872+0000 7f1466ffd640 10 client.16029 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.888+0000 7f1466ffd640 1 -- 172.21.15.19:0/4254675594 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fffb0bd200 con 0x55fffb1d2280 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.888+0000 7f1466ffd640 10 client.16029 handle_command_reply: tid=0 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.888+0000 7f1490c2d740 1 client.16029 shutdown 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 1 -- 172.21.15.19:0/4254675594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 msgr2=0x55fffb163bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb163bf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1478098dc0 tx=0x7f14780bb4c0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 1 -- 172.21.15.19:0/4254675594 shutdown_connections 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fffb1d2280 0x55fffb1d4670 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fffb15b800 0x55fffb1552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fffb150f40 0x55fffb154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 1 --2- 172.21.15.19:0/4254675594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffb150620 0x55fffb163bf0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1466ffd640 0 client.16029 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 1 -- 172.21.15.19:0/4254675594 >> 172.21.15.19:0/4254675594 conn(0x55fffb14cb30 msgr2=0x55fffb151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 1 -- 172.21.15.19:0/4254675594 shutdown_connections 2024-04-23T18:29:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 1 -- 172.21.15.19:0/4254675594 wait complete. 2024-04-23T18:29:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f1490c2d740 20 client.16029 trim_cache size 0 max 16384 2024-04-23T18:29:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:29:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:29:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:29:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:29:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f148df19640 1 -- 172.21.15.19:0/1373128428 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f14700a3dc0 msgr2=0x7f14700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f148df19640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f14700a3dc0 0x7f14700a6280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1488102f60 tx=0x7f1480031af0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f148df19640 1 -- 172.21.15.19:0/1373128428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488100d80 msgr2=0x7f14881a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f148df19640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488100d80 0x7f14881a05a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f147800d950 tx=0x7f147800de10 comp rx=0 tx=0).stop 2024-04-23T18:29:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f148df19640 1 -- 172.21.15.19:0/1373128428 shutdown_connections 2024-04-23T18:29:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f148df19640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f14700a3dc0 0x7f14700a6280 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f148df19640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f148810e930 0x7f148819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f148df19640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14881016a0 0x7f14881a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.889+0000 7f148df19640 1 --2- 172.21.15.19:0/1373128428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488100d80 0x7f14881a05a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.890+0000 7f148df19640 1 -- 172.21.15.19:0/1373128428 >> 172.21.15.19:0/1373128428 conn(0x7f1488077fd0 msgr2=0x7f148810fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.890+0000 7f148df19640 1 -- 172.21.15.19:0/1373128428 shutdown_connections 2024-04-23T18:29:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:01.890+0000 7f148df19640 1 -- 172.21.15.19:0/1373128428 wait complete. 2024-04-23T18:29:02.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:01 smithi019 ceph-mon[29215]: pgmap v89: 97 pgs: 97 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 21 op/s 2024-04-23T18:29:02.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:01 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3913201741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:02.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:01 smithi155 ceph-mon[33468]: pgmap v89: 97 pgs: 97 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 21 op/s 2024-04-23T18:29:02.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:01 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3913201741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:02.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:01 smithi044 ceph-mon[32458]: pgmap v89: 97 pgs: 97 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 21 op/s 2024-04-23T18:29:02.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:01 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3913201741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:04.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:03 smithi155 ceph-mon[33468]: pgmap v90: 97 pgs: 97 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:29:04.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:03 smithi019 ceph-mon[29215]: pgmap v90: 97 pgs: 97 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:29:04.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:03 smithi044 ceph-mon[32458]: pgmap v90: 97 pgs: 97 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:29:06.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:05 smithi019 ceph-mon[29215]: pgmap v91: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 36 op/s 2024-04-23T18:29:06.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:05 smithi044 ceph-mon[32458]: pgmap v91: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 36 op/s 2024-04-23T18:29:06.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:05 smithi155 ceph-mon[33468]: pgmap v91: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 36 op/s 2024-04-23T18:29:08.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:07 smithi019 ceph-mon[29215]: pgmap v92: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 29 op/s 2024-04-23T18:29:08.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:07 smithi044 ceph-mon[32458]: pgmap v92: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 29 op/s 2024-04-23T18:29:08.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:07 smithi155 ceph-mon[33468]: pgmap v92: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 29 op/s 2024-04-23T18:29:10.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:09 smithi019 ceph-mon[29215]: pgmap v93: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 29 op/s 2024-04-23T18:29:10.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:09 smithi044 ceph-mon[32458]: pgmap v93: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 29 op/s 2024-04-23T18:29:10.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:09 smithi155 ceph-mon[33468]: pgmap v93: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 29 op/s 2024-04-23T18:29:11.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:10 smithi019 ceph-mon[29215]: pgmap v94: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 KiB/s wr, 33 op/s 2024-04-23T18:29:11.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:10 smithi044 ceph-mon[32458]: pgmap v94: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 KiB/s wr, 33 op/s 2024-04-23T18:29:11.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:10 smithi155 ceph-mon[33468]: pgmap v94: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 KiB/s wr, 33 op/s 2024-04-23T18:29:13.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:13 smithi019 ceph-mon[29215]: pgmap v95: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 KiB/s wr, 27 op/s 2024-04-23T18:29:13.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:13 smithi044 ceph-mon[32458]: pgmap v95: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 KiB/s wr, 27 op/s 2024-04-23T18:29:13.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:13 smithi155 ceph-mon[33468]: pgmap v95: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 KiB/s wr, 27 op/s 2024-04-23T18:29:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.988+0000 7f62d8a72640 1 -- 172.21.15.19:0/4138871090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d41061b0 msgr2=0x7f62d410dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.988+0000 7f62d8a72640 1 --2- 172.21.15.19:0/4138871090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d41061b0 0x7f62d410dd80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009ca0 tx=0x7f62cc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.988+0000 7f62d8a72640 1 -- 172.21.15.19:0/4138871090 shutdown_connections 2024-04-23T18:29:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.988+0000 7f62d8a72640 1 --2- 172.21.15.19:0/4138871090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d41061b0 0x7f62d410dd80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.988+0000 7f62d8a72640 1 --2- 172.21.15.19:0/4138871090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d4101a90 0x7f62d4105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.988+0000 7f62d8a72640 1 --2- 172.21.15.19:0/4138871090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d41010e0 0x7f62d41014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.988+0000 7f62d8a72640 1 -- 172.21.15.19:0/4138871090 >> 172.21.15.19:0/4138871090 conn(0x7f62d40771d0 msgr2=0x7f62d40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.989+0000 7f62d8a72640 1 -- 172.21.15.19:0/4138871090 shutdown_connections 2024-04-23T18:29:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.989+0000 7f62d8a72640 1 -- 172.21.15.19:0/4138871090 wait complete. 2024-04-23T18:29:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.989+0000 7f62d8a72640 1 Processor -- start 2024-04-23T18:29:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.989+0000 7f62d8a72640 1 -- start start 2024-04-23T18:29:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d8a72640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d41010e0 0x7f62d4102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d8a72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d41061b0 0x7f62d41030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d8a72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d41035f0 0x7f62d41a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d8a72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4111150 con 0x7f62d41035f0 2024-04-23T18:29:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d8a72640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4110fd0 con 0x7f62d41061b0 2024-04-23T18:29:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d8a72640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41112d0 con 0x7f62d41010e0 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d41035f0 0x7f62d41a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d41035f0 0x7f62d41a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42408/0 (socket says 172.21.15.19:42408) 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d2d9d640 1 -- 172.21.15.19:0/2612075418 learned_addr learned my addr 172.21.15.19:0/2612075418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d259c640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d41010e0 0x7f62d4102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62cbfff640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d41061b0 0x7f62d41030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d2d9d640 1 -- 172.21.15.19:0/2612075418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d41010e0 msgr2=0x7f62d4102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d2d9d640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d41010e0 0x7f62d4102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d2d9d640 1 -- 172.21.15.19:0/2612075418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d41061b0 msgr2=0x7f62d41030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d2d9d640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d41061b0 0x7f62d41030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.990+0000 7f62d2d9d640 1 -- 172.21.15.19:0/2612075418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d41a6f20 con 0x7f62d41035f0 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.991+0000 7f62d2d9d640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d41035f0 0x7f62d41a6880 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009430 tx=0x7f62cc0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.991+0000 7f62cb7fe640 1 -- 172.21.15.19:0/2612075418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc003c30 con 0x7f62d41035f0 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.991+0000 7f62cb7fe640 1 -- 172.21.15.19:0/2612075418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62cc003dd0 con 0x7f62d41035f0 2024-04-23T18:29:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.991+0000 7f62d8a72640 1 -- 172.21.15.19:0/2612075418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d41a71b0 con 0x7f62d41035f0 2024-04-23T18:29:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.991+0000 7f62cb7fe640 1 -- 172.21.15.19:0/2612075418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc02faa0 con 0x7f62d41035f0 2024-04-23T18:29:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.991+0000 7f62d8a72640 1 -- 172.21.15.19:0/2612075418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d41a7700 con 0x7f62d41035f0 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.993+0000 7f62cb7fe640 1 -- 172.21.15.19:0/2612075418 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62cc02fc40 con 0x7f62d41035f0 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.993+0000 7f62db788740 1 Processor -- start 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62db788740 1 -- start start 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62db788740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23c620 0x56483a23ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62db788740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23cf40 0x56483a2472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62db788740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a249bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62db788740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56483a1a1970 con 0x56483a247800 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62db788740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56483a13e4a0 con 0x56483a23cf40 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62db788740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56483a19d930 con 0x56483a23c620 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a249bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a249bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42418/0 (socket says 172.21.15.19:42418) 2024-04-23T18:29:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62cbfff640 1 -- 172.21.15.19:0/4005711098 learned_addr learned my addr 172.21.15.19:0/4005711098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62cb7fe640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f62c00a3dc0 0x7f62c00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62d259c640 1 --2- 172.21.15.19:0/4005711098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23cf40 0x56483a2472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.994+0000 7f62cbfff640 1 --2- 172.21.15.19:0/4005711098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23c620 0x56483a23ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62d2d9d640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f62c00a3dc0 0x7f62c00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62cbfff640 1 -- 172.21.15.19:0/4005711098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23c620 msgr2=0x56483a23ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62cbfff640 1 --2- 172.21.15.19:0/4005711098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23c620 0x56483a23ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62cb7fe640 1 -- 172.21.15.19:0/2612075418 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f62cc0f1ba0 con 0x7f62d41035f0 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62cbfff640 1 -- 172.21.15.19:0/4005711098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23cf40 msgr2=0x56483a2472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62cbfff640 1 --2- 172.21.15.19:0/4005711098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23cf40 0x56483a2472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62cbfff640 1 -- 172.21.15.19:0/4005711098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564839eb4210 con 0x56483a247800 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62d259c640 1 --2- 172.21.15.19:0/4005711098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23cf40 0x56483a2472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62cbfff640 1 --2- 172.21.15.19:0/4005711098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a249bf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f62bc011690 tx=0x7f62bc00a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3862f097edb81113 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62d2d9d640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f62c00a3dc0 0x7f62c00a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0bdc10 tx=0x7f62cc0de040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62c8ff9640 1 -- 172.21.15.19:0/4005711098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc011260 con 0x56483a247800 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62c8ff9640 1 -- 172.21.15.19:0/4005711098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62bc0035f0 con 0x56483a247800 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.995+0000 7f62c8ff9640 1 -- 172.21.15.19:0/4005711098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00f6b0 con 0x56483a247800 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.996+0000 7f62db788740 1 -- 172.21.15.19:0/4005711098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 msgr2=0x56483a249bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.996+0000 7f62db788740 1 --2- 172.21.15.19:0/4005711098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a249bf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f62bc011690 tx=0x7f62bc00a6b0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.996+0000 7f62db788740 1 -- 172.21.15.19:0/4005711098 shutdown_connections 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.996+0000 7f62db788740 1 --2- 172.21.15.19:0/4005711098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23cf40 0x56483a2472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.996+0000 7f62db788740 1 --2- 172.21.15.19:0/4005711098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a249bf0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.996+0000 7f62db788740 1 --2- 172.21.15.19:0/4005711098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23c620 0x56483a23ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.996+0000 7f62db788740 1 -- 172.21.15.19:0/4005711098 >> 172.21.15.19:0/4005711098 conn(0x56483a238b30 msgr2=0x56483a238f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.996+0000 7f62db788740 1 -- 172.21.15.19:0/4005711098 shutdown_connections 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.996+0000 7f62db788740 1 -- 172.21.15.19:0/4005711098 wait complete. 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.996+0000 7f62db788740 1 -- start start 2024-04-23T18:29:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62db788740 1 Processor -- start 2024-04-23T18:29:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62db788740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23c620 0x56483a247140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62db788740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23cf40 0x56483a2444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62db788740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 0x56483a244a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62db788740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56483a24a3d0 con 0x56483a23cf40 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62db788740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56483a13c0b0 con 0x56483a247800 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62db788740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56483a24a550 con 0x56483a23c620 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62db788740 10 client.0 _ll_register_callbacks cb 0x56483a129d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23cf40 0x56483a2444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23cf40 0x56483a2444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42428/0 (socket says 172.21.15.19:42428) 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62d259c640 1 -- 172.21.15.19:0/1626174382 learned_addr learned my addr 172.21.15.19:0/1626174382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62cbfff640 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23c620 0x56483a247140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62cbfff640 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 0x56483a244a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62d259c640 1 -- 172.21.15.19:0/1626174382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23c620 msgr2=0x56483a247140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62d259c640 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23c620 0x56483a247140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62d259c640 1 -- 172.21.15.19:0/1626174382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 msgr2=0x56483a244a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62d259c640 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 0x56483a244a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62d259c640 1 -- 172.21.15.19:0/1626174382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564839f80620 con 0x56483a23cf40 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62cbfff640 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 0x56483a244a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62d259c640 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23cf40 0x56483a2444a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f62c400baa0 tx=0x7f62c400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96e0fc2ebe422c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.997+0000 7f62b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1626174382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4011440 con 0x56483a23cf40 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1626174382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f62c40115e0 con 0x56483a23cf40 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62db788740 1 -- 172.21.15.19:0/1626174382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62bc0035f0 con 0x56483a23cf40 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1626174382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62c40117e0 con 0x56483a23cf40 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62db788740 10 client.16041 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62db788740 1 -- 172.21.15.19:0/1626174382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56483a2563a0 con 0x56483a23cf40 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1626174382 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62bc0035f0 con 0x56483a23cf40 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1626174382 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f62c401b990 con 0x56483a23cf40 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62db788740 10 client.16041 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62db788740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62db788740 20 client.16041 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62db788740 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56483a257920 0x56483a259d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62db788740 4 client.16041 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.998+0000 7f62db788740 1 -- 172.21.15.19:0/1626174382 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56483a1af390 con 0x56483a257920 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.999+0000 7f62cbfff640 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56483a257920 0x56483a259d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.999+0000 7f62b9ffb640 10 client.16041 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:14.999+0000 7f62cbfff640 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56483a257920 0x56483a259d10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8f033945fdc974 server_cookie=727f47cc3a8a244e in_seq=0 out_seq=0 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1626174382 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56483a1af390 con 0x56483a257920 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62b9ffb640 10 client.16041 handle_command_reply: tid=0 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 client.16041 shutdown 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 -- 172.21.15.19:0/1626174382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23cf40 msgr2=0x56483a2444a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23cf40 0x56483a2444a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f62c400baa0 tx=0x7f62c400bf60 comp rx=0 tx=0).stop 2024-04-23T18:29:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 -- 172.21.15.19:0/1626174382 shutdown_connections 2024-04-23T18:29:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56483a257920 0x56483a259d10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 0x56483a244a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23cf40 0x56483a2444a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62b9ffb640 0 client.16041 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 --2- 172.21.15.19:0/1626174382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23c620 0x56483a247140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 -- 172.21.15.19:0/1626174382 >> 172.21.15.19:0/1626174382 conn(0x56483a238b30 msgr2=0x56483a237cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 -- 172.21.15.19:0/1626174382 shutdown_connections 2024-04-23T18:29:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 1 -- 172.21.15.19:0/1626174382 wait complete. 2024-04-23T18:29:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.000+0000 7f62db788740 20 client.16041 trim_cache size 0 max 16384 2024-04-23T18:29:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.015+0000 7f62db788740 1 Processor -- start 2024-04-23T18:29:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.015+0000 7f62db788740 1 -- start start 2024-04-23T18:29:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.015+0000 7f62db788740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23c620 0x56483a253290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.015+0000 7f62db788740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 0x56483a25f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.015+0000 7f62db788740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a25fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.015+0000 7f62db788740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41afc50 con 0x56483a247800 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.015+0000 7f62db788740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41b1790 con 0x56483a23c620 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.015+0000 7f62db788740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41b0320 con 0x56483a23cf40 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a25fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a25fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42432/0 (socket says 172.21.15.19:42432) 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62cbfff640 1 -- 172.21.15.19:0/120197126 learned_addr learned my addr 172.21.15.19:0/120197126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62cbfff640 1 --2- 172.21.15.19:0/120197126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23c620 0x56483a253290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62cbfff640 1 -- 172.21.15.19:0/120197126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 msgr2=0x56483a25f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62d259c640 1 --2- 172.21.15.19:0/120197126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 0x56483a25f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62cbfff640 1 --2- 172.21.15.19:0/120197126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 0x56483a25f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62cbfff640 1 -- 172.21.15.19:0/120197126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23c620 msgr2=0x56483a253290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62cbfff640 1 --2- 172.21.15.19:0/120197126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23c620 0x56483a253290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62cbfff640 1 -- 172.21.15.19:0/120197126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56483a2342e0 con 0x56483a247800 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62cbfff640 1 --2- 172.21.15.19:0/120197126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a25fb20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f62bc011690 tx=0x7f62bc00a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3884ca2cc9d83548 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62bbfff640 1 -- 172.21.15.19:0/120197126 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc004220 con 0x56483a247800 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62bbfff640 1 -- 172.21.15.19:0/120197126 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62bc0043c0 con 0x56483a247800 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.016+0000 7f62bbfff640 1 -- 172.21.15.19:0/120197126 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc005520 con 0x56483a247800 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 -- 172.21.15.19:0/120197126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 msgr2=0x56483a25fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 --2- 172.21.15.19:0/120197126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a25fb20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f62bc011690 tx=0x7f62bc00a6b0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 -- 172.21.15.19:0/120197126 shutdown_connections 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 --2- 172.21.15.19:0/120197126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a247800 0x56483a25fb20 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 --2- 172.21.15.19:0/120197126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 0x56483a25f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 --2- 172.21.15.19:0/120197126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a23c620 0x56483a253290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 -- 172.21.15.19:0/120197126 >> 172.21.15.19:0/120197126 conn(0x56483a238b30 msgr2=0x56483a237cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 -- 172.21.15.19:0/120197126 shutdown_connections 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 -- 172.21.15.19:0/120197126 wait complete. 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 -- start start 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 Processor -- start 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.017+0000 7f62db788740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23c620 0x56483a24fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62db788740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 0x56483a240f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62db788740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 0x56483a2414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62db788740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41b10c0 con 0x56483a23c620 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62db788740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41b09f0 con 0x56483a247800 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62db788740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41adc50 con 0x56483a23cf40 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62db788740 10 client.0 _ll_register_callbacks cb 0x7f62c40044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23c620 0x56483a24fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23c620 0x56483a24fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42448/0 (socket says 172.21.15.19:42448) 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62cbfff640 1 -- 172.21.15.19:0/1158937859 learned_addr learned my addr 172.21.15.19:0/1158937859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62d259c640 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 0x56483a240f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62cbfff640 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 0x56483a2414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62cbfff640 1 -- 172.21.15.19:0/1158937859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 msgr2=0x56483a240f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62cbfff640 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 0x56483a240f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62cbfff640 1 -- 172.21.15.19:0/1158937859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 msgr2=0x56483a2414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62cbfff640 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 0x56483a2414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62cbfff640 1 -- 172.21.15.19:0/1158937859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56483a241d20 con 0x56483a23c620 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62d259c640 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 0x56483a240f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62cbfff640 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23c620 0x56483a24fc80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00ade0 tx=0x7f62bc004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6567cf1772b688 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1158937859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0054a0 con 0x56483a23c620 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1158937859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f62bc005640 con 0x56483a23c620 2024-04-23T18:29:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.018+0000 7f62db788740 1 -- 172.21.15.19:0/1158937859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62bc0043c0 con 0x56483a23c620 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1158937859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62bc02a3b0 con 0x56483a23c620 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62db788740 10 client.16047 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62db788740 1 -- 172.21.15.19:0/1158937859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56483a2bdf00 con 0x56483a23c620 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1158937859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62bc0043c0 con 0x56483a23c620 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1158937859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f62bc003c40 con 0x56483a23c620 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62db788740 10 client.16047 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62db788740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62db788740 20 client.16047 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62db788740 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56483a2be380 0x56483a2c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62db788740 4 client.16047 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62db788740 1 -- 172.21.15.19:0/1158937859 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56483a1a9200 con 0x56483a2be380 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.019+0000 7f62d259c640 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56483a2be380 0x56483a2c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.020+0000 7f62b9ffb640 10 client.16047 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.020+0000 7f62d259c640 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56483a2be380 0x56483a2c0770 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0cc263592c8c71 server_cookie=9d9fb4e3c9bd6322 in_seq=0 out_seq=0 2024-04-23T18:29:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.033+0000 7f62b9ffb640 1 -- 172.21.15.19:0/1158937859 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56483a1a9200 con 0x56483a2be380 2024-04-23T18:29:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.033+0000 7f62b9ffb640 10 client.16047 handle_command_reply: tid=0 2024-04-23T18:29:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.033+0000 7f62db788740 1 client.16047 shutdown 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.033+0000 7f62db788740 1 -- 172.21.15.19:0/1158937859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23c620 msgr2=0x56483a24fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.033+0000 7f62db788740 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23c620 0x56483a24fc80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00ade0 tx=0x7f62bc004570 comp rx=0 tx=0).stop 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62db788740 1 -- 172.21.15.19:0/1158937859 shutdown_connections 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62db788740 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56483a2be380 0x56483a2c0770 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62db788740 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483a247800 0x56483a2414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62db788740 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56483a23cf40 0x56483a240f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62b9ffb640 0 client.16047 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62db788740 1 --2- 172.21.15.19:0/1158937859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483a23c620 0x56483a24fc80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62db788740 1 -- 172.21.15.19:0/1158937859 >> 172.21.15.19:0/1158937859 conn(0x56483a238b30 msgr2=0x56483a2439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62db788740 1 -- 172.21.15.19:0/1158937859 shutdown_connections 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62db788740 1 -- 172.21.15.19:0/1158937859 wait complete. 2024-04-23T18:29:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62db788740 20 client.16047 trim_cache size 0 max 16384 2024-04-23T18:29:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:29:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:29:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:29:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 -- 172.21.15.19:0/2612075418 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f62c00a3dc0 msgr2=0x7f62c00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f62c00a3dc0 0x7f62c00a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0bdc10 tx=0x7f62cc0de040 comp rx=0 tx=0).stop 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 -- 172.21.15.19:0/2612075418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d41035f0 msgr2=0x7f62d41a6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d41035f0 0x7f62d41a6880 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009430 tx=0x7f62cc0319f0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 -- 172.21.15.19:0/2612075418 shutdown_connections 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f62c00a3dc0 0x7f62c00a6280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d41061b0 0x7f62d41030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d41035f0 0x7f62d41a6880 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 --2- 172.21.15.19:0/2612075418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d41010e0 0x7f62d4102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 -- 172.21.15.19:0/2612075418 >> 172.21.15.19:0/2612075418 conn(0x7f62d40771d0 msgr2=0x7f62d406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 -- 172.21.15.19:0/2612075418 shutdown_connections 2024-04-23T18:29:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:15.034+0000 7f62d8a72640 1 -- 172.21.15.19:0/2612075418 wait complete. 2024-04-23T18:29:15.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:15 smithi019 ceph-mon[29215]: pgmap v96: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-04-23T18:29:15.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:15 smithi044 ceph-mon[32458]: pgmap v96: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-04-23T18:29:15.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:15 smithi155 ceph-mon[33468]: pgmap v96: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-04-23T18:29:16.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:16 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:16.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:16 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:16.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:16 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:16.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:16 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:29:16.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:16 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:16.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:16 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:16.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:16 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:16.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:16 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:29:16.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:16 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:16.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:16 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:16.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:16 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:16.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:16 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:29:17.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:17 smithi019 ceph-mon[29215]: pgmap v97: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:17.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:17 smithi044 ceph-mon[32458]: pgmap v97: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:17.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:17 smithi155 ceph-mon[33468]: pgmap v97: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:19.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:29:19.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:19 smithi019 ceph-mon[29215]: pgmap v98: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 -- 172.21.15.19:0/911628363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc10e960 msgr2=0x7f41cc110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 --2- 172.21.15.19:0/911628363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc10e960 0x7f41cc110d50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009ca0 tx=0x7f41c402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 -- 172.21.15.19:0/911628363 shutdown_connections 2024-04-23T18:29:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 --2- 172.21.15.19:0/911628363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc10e960 0x7f41cc110d50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 --2- 172.21.15.19:0/911628363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41cc1016d0 0x7f41cc10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 --2- 172.21.15.19:0/911628363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41cc100db0 0x7f41cc101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 -- 172.21.15.19:0/911628363 >> 172.21.15.19:0/911628363 conn(0x7f41cc077fb0 msgr2=0x7f41cc0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 -- 172.21.15.19:0/911628363 shutdown_connections 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 -- 172.21.15.19:0/911628363 wait complete. 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 Processor -- start 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.532+0000 7f41d3e91640 1 -- start start 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d3e91640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41cc100db0 0x7f41cc1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d3e91640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41cc1016d0 0x7f41cc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d3e91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc10e960 0x7f41cc19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d3e91640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc114440 con 0x7f41cc10e960 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d3e91640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1142c0 con 0x7f41cc1016d0 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d3e91640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1145c0 con 0x7f41cc100db0 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d242e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc10e960 0x7f41cc19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d242e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc10e960 0x7f41cc19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42468/0 (socket says 172.21.15.19:42468) 2024-04-23T18:29:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d242e640 1 -- 172.21.15.19:0/2509805150 learned_addr learned my addr 172.21.15.19:0/2509805150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d142c640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41cc1016d0 0x7f41cc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d1c2d640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41cc100db0 0x7f41cc1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d242e640 1 -- 172.21.15.19:0/2509805150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41cc100db0 msgr2=0x7f41cc1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d242e640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41cc100db0 0x7f41cc1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d242e640 1 -- 172.21.15.19:0/2509805150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41cc1016d0 msgr2=0x7f41cc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d242e640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41cc1016d0 0x7f41cc1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d242e640 1 -- 172.21.15.19:0/2509805150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc19afe0 con 0x7f41cc10e960 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d242e640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc10e960 0x7f41cc19a860 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f41c402ade0 tx=0x7f41c40316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41c2ffd640 1 -- 172.21.15.19:0/2509805150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4003b70 con 0x7f41cc10e960 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41c2ffd640 1 -- 172.21.15.19:0/2509805150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41c40318e0 con 0x7f41cc10e960 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d3e91640 1 -- 172.21.15.19:0/2509805150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41cc19b270 con 0x7f41cc10e960 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41c2ffd640 1 -- 172.21.15.19:0/2509805150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c402f990 con 0x7f41cc10e960 2024-04-23T18:29:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.533+0000 7f41d3e91640 1 -- 172.21.15.19:0/2509805150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41cc1a7570 con 0x7f41cc10e960 2024-04-23T18:29:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.535+0000 7f41d3e91640 1 -- 172.21.15.19:0/2509805150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41cc0691c0 con 0x7f41cc10e960 2024-04-23T18:29:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.536+0000 7f41c2ffd640 1 -- 172.21.15.19:0/2509805150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f41c402fb30 con 0x7f41cc10e960 2024-04-23T18:29:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.536+0000 7f41c2ffd640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f41b80a3d20 0x7f41b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.536+0000 7f41c2ffd640 1 -- 172.21.15.19:0/2509805150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f41c40f1ae0 con 0x7f41cc10e960 2024-04-23T18:29:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.536+0000 7f41d1c2d640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f41b80a3d20 0x7f41b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.537+0000 7f41d1c2d640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f41b80a3d20 0x7f41b80a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f41c8005fd0 tx=0x7f41c8005850 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.541+0000 7f41c2ffd640 1 -- 172.21.15.19:0/2509805150 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41c40be2c0 con 0x7f41cc10e960 2024-04-23T18:29:19.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:19 smithi044 ceph-mon[32458]: pgmap v98: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:19.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:19 smithi155 ceph-mon[33468]: pgmap v98: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.765+0000 7f41d3e91640 1 -- 172.21.15.19:0/2509805150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41cc109b20 con 0x7f41cc10e960 2024-04-23T18:29:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.765+0000 7f41c2ffd640 1 -- 172.21.15.19:0/2509805150 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f41c40bdbc0 con 0x7f41cc10e960 2024-04-23T18:29:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:29:19.758 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:29:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:29:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 -- 172.21.15.19:0/2509805150 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f41b80a3d20 msgr2=0x7f41b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f41b80a3d20 0x7f41b80a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f41c8005fd0 tx=0x7f41c8005850 comp rx=0 tx=0).stop 2024-04-23T18:29:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 -- 172.21.15.19:0/2509805150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc10e960 msgr2=0x7f41cc19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc10e960 0x7f41cc19a860 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f41c402ade0 tx=0x7f41c40316f0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 -- 172.21.15.19:0/2509805150 shutdown_connections 2024-04-23T18:29:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f41b80a3d20 0x7f41b80a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41cc1016d0 0x7f41cc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc10e960 0x7f41cc19a860 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 --2- 172.21.15.19:0/2509805150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41cc100db0 0x7f41cc1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 -- 172.21.15.19:0/2509805150 >> 172.21.15.19:0/2509805150 conn(0x7f41cc077fb0 msgr2=0x7f41cc10ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 -- 172.21.15.19:0/2509805150 shutdown_connections 2024-04-23T18:29:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.767+0000 7f41d3e91640 1 -- 172.21.15.19:0/2509805150 wait complete. 2024-04-23T18:29:19.771 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id dc70017e --release --await' 2024-04-23T18:29:19.771 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id dc70017e --release --await 2024-04-23T18:29:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 -- 172.21.15.19:0/2392423122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a4103af0 msgr2=0x7f07a410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 --2- 172.21.15.19:0/2392423122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a4103af0 0x7f07a410bfd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0794009900 tx=0x7f079402a530 comp rx=0 tx=0).stop 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 -- 172.21.15.19:0/2392423122 shutdown_connections 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 --2- 172.21.15.19:0/2392423122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a410c510 0x7f07a410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 --2- 172.21.15.19:0/2392423122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a4103af0 0x7f07a410bfd0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 --2- 172.21.15.19:0/2392423122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a41031d0 0x7f07a41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 -- 172.21.15.19:0/2392423122 >> 172.21.15.19:0/2392423122 conn(0x7f07a4077f50 msgr2=0x7f07a4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 -- 172.21.15.19:0/2392423122 shutdown_connections 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 -- 172.21.15.19:0/2392423122 wait complete. 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 Processor -- start 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.887+0000 7f07abfbb640 1 -- start start 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07abfbb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a41031d0 0x7f07a419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07abfbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a4103af0 0x7f07a419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07abfbb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a410c510 0x7f07a4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07abfbb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4112220 con 0x7f07a4103af0 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07abfbb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41120a0 con 0x7f07a41031d0 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07abfbb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41123a0 con 0x7f07a410c510 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9556640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a4103af0 0x7f07a419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9556640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a4103af0 0x7f07a419e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60796/0 (socket says 172.21.15.19:60796) 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9556640 1 -- 172.21.15.19:0/3931259535 learned_addr learned my addr 172.21.15.19:0/3931259535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9d57640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a41031d0 0x7f07a419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9556640 1 -- 172.21.15.19:0/3931259535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a410c510 msgr2=0x7f07a4198600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9556640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a410c510 0x7f07a4198600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9556640 1 -- 172.21.15.19:0/3931259535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a41031d0 msgr2=0x7f07a419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9556640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a41031d0 0x7f07a419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9556640 1 -- 172.21.15.19:0/3931259535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a4198d80 con 0x7f07a4103af0 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9d57640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a41031d0 0x7f07a419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07a9556640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a4103af0 0x7f07a419e990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0794005940 tx=0x7f0794004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f079affd640 1 -- 172.21.15.19:0/3931259535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079403f030 con 0x7f07a4103af0 2024-04-23T18:29:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f079affd640 1 -- 172.21.15.19:0/3931259535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0794003aa0 con 0x7f07a4103af0 2024-04-23T18:29:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f07abfbb640 1 -- 172.21.15.19:0/3931259535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a4199010 con 0x7f07a4103af0 2024-04-23T18:29:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.888+0000 7f079affd640 1 -- 172.21.15.19:0/3931259535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794035070 con 0x7f07a4103af0 2024-04-23T18:29:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.889+0000 7f07abfbb640 1 -- 172.21.15.19:0/3931259535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a41a5310 con 0x7f07a4103af0 2024-04-23T18:29:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.891+0000 7f079affd640 1 -- 172.21.15.19:0/3931259535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0794042030 con 0x7f07a4103af0 2024-04-23T18:29:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.891+0000 7f07aedab740 1 Processor -- start 2024-04-23T18:29:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.891+0000 7f07aedab740 1 -- start start 2024-04-23T18:29:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aedab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24ae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aedab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aedab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aedab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55baf24136d0 con 0x55baf24ae490 2024-04-23T18:29:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aedab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55baf23b01a0 con 0x55baf24b9480 2024-04-23T18:29:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aedab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55baf240f690 con 0x55baf24aedb0 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aa558640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aa558640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60812/0 (socket says 172.21.15.19:60812) 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aa558640 1 -- 172.21.15.19:0/2324177509 learned_addr learned my addr 172.21.15.19:0/2324177509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07a9d57640 1 --2- 172.21.15.19:0/2324177509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07a9556640 1 --2- 172.21.15.19:0/2324177509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f079affd640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f07900a3d20 0x7f07900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aa558640 1 -- 172.21.15.19:0/2324177509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 msgr2=0x55baf24b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aa558640 1 --2- 172.21.15.19:0/2324177509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24b8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aa558640 1 -- 172.21.15.19:0/2324177509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 msgr2=0x55baf24bb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aa558640 1 --2- 172.21.15.19:0/2324177509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24bb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aa558640 1 -- 172.21.15.19:0/2324177509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baf23ba5b0 con 0x55baf24ae490 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f079affd640 1 -- 172.21.15.19:0/3931259535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f07940f0f50 con 0x7f07a4103af0 2024-04-23T18:29:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aa558640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f07900a3d20 0x7f07900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.892+0000 7f07aa558640 1 --2- 172.21.15.19:0/2324177509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24ae870 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f079c005c10 tx=0x7f079c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1b8846e2b4859d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f078ffff640 1 -- 172.21.15.19:0/2324177509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c012e00 con 0x55baf24ae490 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f078ffff640 1 -- 172.21.15.19:0/2324177509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f079c01bd90 con 0x55baf24ae490 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f078ffff640 1 -- 172.21.15.19:0/2324177509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c01ab50 con 0x55baf24ae490 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f07aedab740 1 -- 172.21.15.19:0/2324177509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 msgr2=0x55baf24ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f07aa558640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f07900a3d20 0x7f07900a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f079c01f000 tx=0x7f079c020000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f07aedab740 1 --2- 172.21.15.19:0/2324177509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24ae870 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f079c005c10 tx=0x7f079c00e7e0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f07aedab740 1 -- 172.21.15.19:0/2324177509 shutdown_connections 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f07aedab740 1 --2- 172.21.15.19:0/2324177509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24bb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f07aedab740 1 --2- 172.21.15.19:0/2324177509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f07aedab740 1 --2- 172.21.15.19:0/2324177509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24ae870 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f07aedab740 1 -- 172.21.15.19:0/2324177509 >> 172.21.15.19:0/2324177509 conn(0x55baf24aa7b0 msgr2=0x55baf24aab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f07aedab740 1 -- 172.21.15.19:0/2324177509 shutdown_connections 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.893+0000 7f07aedab740 1 -- 172.21.15.19:0/2324177509 wait complete. 2024-04-23T18:29:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07aedab740 1 -- start start 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07aedab740 1 Processor -- start 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07aedab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07aedab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24b6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07aedab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07aedab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55baf24bc310 con 0x55baf24ae490 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07aedab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55baf23adb20 con 0x55baf24b9480 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07aedab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55baf24bc490 con 0x55baf24aedb0 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07aedab740 10 client.0 _ll_register_callbacks cb 0x55baf23ad530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07a9d57640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07a9d57640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60826/0 (socket says 172.21.15.19:60826) 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07a9d57640 1 -- 172.21.15.19:0/272999627 learned_addr learned my addr 172.21.15.19:0/272999627 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07aa558640 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.894+0000 7f07a9556640 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f07a9d57640 1 -- 172.21.15.19:0/272999627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 msgr2=0x55baf24b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f07a9d57640 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f07a9d57640 1 -- 172.21.15.19:0/272999627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 msgr2=0x55baf24b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f07a9d57640 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f07a9d57640 1 -- 172.21.15.19:0/272999627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baf2125920 con 0x55baf24ae490 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f07a9d57640 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b8e90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f07a000f4f0 tx=0x7f07a000f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b8a70452d9f4cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f078d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f078d7fa640 1 -- 172.21.15.19:0/272999627 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0013850 con 0x55baf24ae490 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f078d7fa640 1 -- 172.21.15.19:0/272999627 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f07a00139f0 con 0x55baf24ae490 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f07aedab740 1 -- 172.21.15.19:0/272999627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f079c01bd90 con 0x55baf24ae490 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f078d7fa640 1 -- 172.21.15.19:0/272999627 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07a0013bf0 con 0x55baf24ae490 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f07aedab740 10 client.16065 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f07aedab740 1 -- 172.21.15.19:0/272999627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55baf21f1d30 con 0x55baf24ae490 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.895+0000 7f078d7fa640 1 -- 172.21.15.19:0/272999627 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f079c01bd90 con 0x55baf24ae490 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.896+0000 7f078d7fa640 1 -- 172.21.15.19:0/272999627 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f07a00113e0 con 0x55baf24ae490 2024-04-23T18:29:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.896+0000 7f07aedab740 10 client.16065 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.896+0000 7f07aedab740 10 client.16065 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.896+0000 7f07aedab740 20 client.16065 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.896+0000 7f07aedab740 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55baf24c9690 0x55baf24cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.896+0000 7f07aedab740 4 client.16065 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.896+0000 7f07aedab740 1 -- 172.21.15.19:0/272999627 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baf23d1500 con 0x55baf24c9690 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.896+0000 7f07a9556640 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55baf24c9690 0x55baf24cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.896+0000 7f078d7fa640 10 client.16065 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.896+0000 7f07a9556640 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55baf24c9690 0x55baf24cba80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d0bad7fdf2a0d0a server_cookie=428259166d454d2d in_seq=0 out_seq=0 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f078d7fa640 1 -- 172.21.15.19:0/272999627 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55baf23d1500 con 0x55baf24c9690 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f078d7fa640 10 client.16065 handle_command_reply: tid=0 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 client.16065 shutdown 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 -- 172.21.15.19:0/272999627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 msgr2=0x55baf24b8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b8e90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f07a000f4f0 tx=0x7f07a000f9b0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 -- 172.21.15.19:0/272999627 shutdown_connections 2024-04-23T18:29:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55baf24c9690 0x55baf24cba80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f078d7fa640 0 client.16065 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 --2- 172.21.15.19:0/272999627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b8e90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 -- 172.21.15.19:0/272999627 >> 172.21.15.19:0/272999627 conn(0x55baf24aa7b0 msgr2=0x55baf24b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 -- 172.21.15.19:0/272999627 shutdown_connections 2024-04-23T18:29:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.897+0000 7f07aedab740 1 -- 172.21.15.19:0/272999627 wait complete. 2024-04-23T18:29:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.898+0000 7f07aedab740 20 client.16065 trim_cache size 0 max 16384 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07aedab740 1 Processor -- start 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07aedab740 1 -- start start 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07aedab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07aedab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07aedab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07aedab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41adb00 con 0x55baf24ae490 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07aedab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41af640 con 0x55baf24b9480 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07aedab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41ae1d0 con 0x55baf24aedb0 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07a9d57640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07a9d57640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60840/0 (socket says 172.21.15.19:60840) 2024-04-23T18:29:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07a9d57640 1 -- 172.21.15.19:0/1207756981 learned_addr learned my addr 172.21.15.19:0/1207756981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07aa558640 1 --2- 172.21.15.19:0/1207756981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07a9d57640 1 -- 172.21.15.19:0/1207756981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 msgr2=0x55baf24a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07a9d57640 1 --2- 172.21.15.19:0/1207756981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07a9d57640 1 -- 172.21.15.19:0/1207756981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 msgr2=0x55baf24b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07a9d57640 1 --2- 172.21.15.19:0/1207756981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07a9d57640 1 -- 172.21.15.19:0/1207756981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baf24cbdf0 con 0x55baf24ae490 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.915+0000 7f07aa558640 1 --2- 172.21.15.19:0/1207756981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f07a9d57640 1 --2- 172.21.15.19:0/1207756981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b0d00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f07a0000c00 tx=0x7f07a0007fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85f72c8a356d36a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f078f7fe640 1 -- 172.21.15.19:0/1207756981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0028070 con 0x55baf24ae490 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f078f7fe640 1 -- 172.21.15.19:0/1207756981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a0011a70 con 0x55baf24ae490 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f078f7fe640 1 -- 172.21.15.19:0/1207756981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0005ca0 con 0x55baf24ae490 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f07aedab740 1 -- 172.21.15.19:0/1207756981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 msgr2=0x55baf24b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f07aedab740 1 --2- 172.21.15.19:0/1207756981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b0d00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f07a0000c00 tx=0x7f07a0007fc0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f07aedab740 1 -- 172.21.15.19:0/1207756981 shutdown_connections 2024-04-23T18:29:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f07aedab740 1 --2- 172.21.15.19:0/1207756981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24b9480 0x55baf24b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f07aedab740 1 --2- 172.21.15.19:0/1207756981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24aedb0 0x55baf24a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f07aedab740 1 --2- 172.21.15.19:0/1207756981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24b0d00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f07aedab740 1 -- 172.21.15.19:0/1207756981 >> 172.21.15.19:0/1207756981 conn(0x55baf24aa7b0 msgr2=0x55baf24a9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f07aedab740 1 -- 172.21.15.19:0/1207756981 shutdown_connections 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.916+0000 7f07aedab740 1 -- 172.21.15.19:0/1207756981 wait complete. 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07aedab740 1 -- start start 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07aedab740 1 Processor -- start 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07aedab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07aedab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24aedb0 0x55baf24c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07aedab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24b9480 0x55baf24bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07aedab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41aef70 con 0x55baf24ae490 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07aedab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41ae8a0 con 0x55baf24aedb0 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07aedab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41abae0 con 0x55baf24b9480 2024-04-23T18:29:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07aedab740 10 client.0 _ll_register_callbacks cb 0x7f07a000ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07a9d57640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07a9d57640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24c16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60846/0 (socket says 172.21.15.19:60846) 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07a9d57640 1 -- 172.21.15.19:0/1298082572 learned_addr learned my addr 172.21.15.19:0/1298082572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07aa558640 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24b9480 0x55baf24bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07a9556640 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24aedb0 0x55baf24c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07a9d57640 1 -- 172.21.15.19:0/1298082572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24b9480 msgr2=0x55baf24bc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07a9d57640 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24b9480 0x55baf24bc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07a9d57640 1 -- 172.21.15.19:0/1298082572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24aedb0 msgr2=0x55baf24c1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.917+0000 7f07a9d57640 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24aedb0 0x55baf24c1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07a9d57640 1 -- 172.21.15.19:0/1298082572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baf24bcc90 con 0x55baf24ae490 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07aa558640 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24b9480 0x55baf24bc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07a9556640 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24aedb0 0x55baf24c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07a9d57640 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24c16f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f07a000a2e0 tx=0x7f07a000ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78102da4fc41034 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f078d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f078d7fa640 1 -- 172.21.15.19:0/1298082572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0028040 con 0x55baf24ae490 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07aedab740 1 -- 172.21.15.19:0/1298082572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a0011a70 con 0x55baf24ae490 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f078d7fa640 1 -- 172.21.15.19:0/1298082572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f07a0013850 con 0x55baf24ae490 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f078d7fa640 1 -- 172.21.15.19:0/1298082572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07a00059e0 con 0x55baf24ae490 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07aedab740 10 client.16071 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07aedab740 1 -- 172.21.15.19:0/1298082572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55baf2534840 con 0x55baf24ae490 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f078d7fa640 1 -- 172.21.15.19:0/1298082572 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a0011a70 con 0x55baf24ae490 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f078d7fa640 1 -- 172.21.15.19:0/1298082572 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f07a003f430 con 0x55baf24ae490 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07aedab740 10 client.16071 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07aedab740 10 client.16071 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07aedab740 20 client.16071 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07aedab740 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55baf2534e40 0x55baf2537230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07aedab740 4 client.16071 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dc70017e", "release": true, "await": true}] 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.918+0000 7f07aedab740 1 -- 172.21.15.19:0/1298082572 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dc70017e", "release": true, "await": true}) -- 0x55baf24210f0 con 0x55baf2534e40 2024-04-23T18:29:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.919+0000 7f07a9556640 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55baf2534e40 0x55baf2537230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.919+0000 7f078d7fa640 10 client.16071 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.919+0000 7f07a9556640 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55baf2534e40 0x55baf2537230 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=886a620a5eee0e02 server_cookie=d6c2d84ec906609e in_seq=0 out_seq=0 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f078d7fa640 1 -- 172.21.15.19:0/1298082572 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55baf24210f0 con 0x55baf2534e40 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f078d7fa640 10 client.16071 handle_command_reply: tid=0 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 client.16071 shutdown 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 -- 172.21.15.19:0/1298082572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 msgr2=0x55baf24c16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24c16f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f07a000a2e0 tx=0x7f07a000ffc0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 -- 172.21.15.19:0/1298082572 shutdown_connections 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55baf2534e40 0x55baf2537230 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf24aedb0 0x55baf24c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55baf24b9480 0x55baf24bc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 --2- 172.21.15.19:0/1298082572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55baf24ae490 0x55baf24c16f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f078d7fa640 0 client.16071 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 -- 172.21.15.19:0/1298082572 >> 172.21.15.19:0/1298082572 conn(0x55baf24aa7b0 msgr2=0x55baf24a99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 -- 172.21.15.19:0/1298082572 shutdown_connections 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 1 -- 172.21.15.19:0/1298082572 wait complete. 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07aedab740 20 client.16071 trim_cache size 0 max 16384 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 16, 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stdout: "dc70017e": { 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 16, 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:29:19.919 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 77.7, 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:19.920 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:19.921 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:29:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07abfbb640 1 -- 172.21.15.19:0/3931259535 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f07900a3d20 msgr2=0x7f07900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07abfbb640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f07900a3d20 0x7f07900a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f079c01f000 tx=0x7f079c020000 comp rx=0 tx=0).stop 2024-04-23T18:29:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07abfbb640 1 -- 172.21.15.19:0/3931259535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a4103af0 msgr2=0x7f07a419e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.923+0000 7f07abfbb640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a4103af0 0x7f07a419e990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0794005940 tx=0x7f0794004270 comp rx=0 tx=0).stop 2024-04-23T18:29:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.924+0000 7f07abfbb640 1 -- 172.21.15.19:0/3931259535 shutdown_connections 2024-04-23T18:29:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.924+0000 7f07abfbb640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f07900a3d20 0x7f07900a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.924+0000 7f07abfbb640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a410c510 0x7f07a4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.924+0000 7f07abfbb640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a4103af0 0x7f07a419e990 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.924+0000 7f07abfbb640 1 --2- 172.21.15.19:0/3931259535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a41031d0 0x7f07a419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.924+0000 7f07abfbb640 1 -- 172.21.15.19:0/3931259535 >> 172.21.15.19:0/3931259535 conn(0x7f07a4077f50 msgr2=0x7f07a40fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.924+0000 7f07abfbb640 1 -- 172.21.15.19:0/3931259535 shutdown_connections 2024-04-23T18:29:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:19.924+0000 7f07abfbb640 1 -- 172.21.15.19:0/3931259535 wait complete. 2024-04-23T18:29:19.927 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dc70017e', seconds elapsed: 18.7 2024-04-23T18:29:19.927 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.65 2024-04-23T18:29:19.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:29:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.036+0000 7f7ec5583640 1 -- 172.21.15.19:0/1923977989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec0102bf0 msgr2=0x7f7ec0106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.036+0000 7f7ec5583640 1 --2- 172.21.15.19:0/1923977989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec0102bf0 0x7f7ec0106ba0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009900 tx=0x7f7ea802a410 comp rx=0 tx=0).stop 2024-04-23T18:29:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.036+0000 7f7ec5583640 1 -- 172.21.15.19:0/1923977989 shutdown_connections 2024-04-23T18:29:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.036+0000 7f7ec5583640 1 --2- 172.21.15.19:0/1923977989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec0107280 0x7f7ec010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.036+0000 7f7ec5583640 1 --2- 172.21.15.19:0/1923977989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec0102bf0 0x7f7ec0106ba0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.036+0000 7f7ec5583640 1 --2- 172.21.15.19:0/1923977989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec0102240 0x7f7ec0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.036+0000 7f7ec5583640 1 -- 172.21.15.19:0/1923977989 >> 172.21.15.19:0/1923977989 conn(0x7f7ec00790e0 msgr2=0x7f7ec00794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.036+0000 7f7ec5583640 1 -- 172.21.15.19:0/1923977989 shutdown_connections 2024-04-23T18:29:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.036+0000 7f7ec5583640 1 -- 172.21.15.19:0/1923977989 wait complete. 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ec5583640 1 Processor -- start 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ec5583640 1 -- start start 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ec5583640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec0102240 0x7f7ec0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ec5583640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec0107280 0x7f7ec0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ec5583640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec0074cd0 0x7f7ec01a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ec5583640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0112240 con 0x7f7ec0074cd0 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ec5583640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec01120c0 con 0x7f7ec0107280 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ec5583640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec01123c0 con 0x7f7ec0102240 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ebf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec0074cd0 0x7f7ec01a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ebf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec0074cd0 0x7f7ec01a69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60868/0 (socket says 172.21.15.19:60868) 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ebf7fe640 1 -- 172.21.15.19:0/3117347205 learned_addr learned my addr 172.21.15.19:0/3117347205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:20.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7eb7fff640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec0107280 0x7f7ec0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ebeffd640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec0102240 0x7f7ec0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ebf7fe640 1 -- 172.21.15.19:0/3117347205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec0102240 msgr2=0x7f7ec0074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ebf7fe640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec0102240 0x7f7ec0074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ebf7fe640 1 -- 172.21.15.19:0/3117347205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec0107280 msgr2=0x7f7ec0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ebf7fe640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec0107280 0x7f7ec0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ebf7fe640 1 -- 172.21.15.19:0/3117347205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec01a70f0 con 0x7f7ec0074cd0 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7eb7fff640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec0107280 0x7f7ec0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.037+0000 7f7ebeffd640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec0102240 0x7f7ec0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.038+0000 7f7ebf7fe640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec0074cd0 0x7f7ec01a69f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800eaa0 tx=0x7f7eb800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.038+0000 7f7ebcff9640 1 -- 172.21.15.19:0/3117347205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb800cdb0 con 0x7f7ec0074cd0 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.038+0000 7f7ebcff9640 1 -- 172.21.15.19:0/3117347205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eb8005020 con 0x7f7ec0074cd0 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.038+0000 7f7ec5583640 1 -- 172.21.15.19:0/3117347205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec01a73e0 con 0x7f7ec0074cd0 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.038+0000 7f7ebcff9640 1 -- 172.21.15.19:0/3117347205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80116d0 con 0x7f7ec0074cd0 2024-04-23T18:29:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.038+0000 7f7ec5583640 1 -- 172.21.15.19:0/3117347205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec01a78b0 con 0x7f7ec0074cd0 2024-04-23T18:29:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.039+0000 7f7ec5583640 1 -- 172.21.15.19:0/3117347205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec00691c0 con 0x7f7ec0074cd0 2024-04-23T18:29:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.040+0000 7f7ebcff9640 1 -- 172.21.15.19:0/3117347205 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7eb80051d0 con 0x7f7ec0074cd0 2024-04-23T18:29:20.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.041+0000 7f7ebcff9640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7eac0a3d50 0x7f7eac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.041+0000 7f7ebeffd640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7eac0a3d50 0x7f7eac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.041+0000 7f7ebcff9640 1 -- 172.21.15.19:0/3117347205 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7eb80d0a50 con 0x7f7ec0074cd0 2024-04-23T18:29:20.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.041+0000 7f7ebeffd640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7eac0a3d50 0x7f7eac0a6210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000a8b0 tx=0x7f7eb0008040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:20.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.046+0000 7f7ebcff9640 1 -- 172.21.15.19:0/3117347205 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7eb8016030 con 0x7f7ec0074cd0 2024-04-23T18:29:20.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.273+0000 7f7ec5583640 1 -- 172.21.15.19:0/3117347205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ec010a4d0 con 0x7f7ec0074cd0 2024-04-23T18:29:20.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.274+0000 7f7ebcff9640 1 -- 172.21.15.19:0/3117347205 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f7eb809d120 con 0x7f7ec0074cd0 2024-04-23T18:29:20.266 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:29:20.267 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:29:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:29:20.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 -- 172.21.15.19:0/3117347205 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7eac0a3d50 msgr2=0x7f7eac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7eac0a3d50 0x7f7eac0a6210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000a8b0 tx=0x7f7eb0008040 comp rx=0 tx=0).stop 2024-04-23T18:29:20.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 -- 172.21.15.19:0/3117347205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec0074cd0 msgr2=0x7f7ec01a69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec0074cd0 0x7f7ec01a69f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800eaa0 tx=0x7f7eb800ef60 comp rx=0 tx=0).stop 2024-04-23T18:29:20.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 -- 172.21.15.19:0/3117347205 shutdown_connections 2024-04-23T18:29:20.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f7eac0a3d50 0x7f7eac0a6210 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec0107280 0x7f7ec0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec0074cd0 0x7f7ec01a69f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 --2- 172.21.15.19:0/3117347205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec0102240 0x7f7ec0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 -- 172.21.15.19:0/3117347205 >> 172.21.15.19:0/3117347205 conn(0x7f7ec00790e0 msgr2=0x7f7ec0108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:20.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 -- 172.21.15.19:0/3117347205 shutdown_connections 2024-04-23T18:29:20.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.276+0000 7f7ec5583640 1 -- 172.21.15.19:0/3117347205 wait complete. 2024-04-23T18:29:20.279 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 77.65 --await-for 42.0' 2024-04-23T18:29:20.280 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 77.65 --await-for 42.0 2024-04-23T18:29:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.399+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2087777429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b0107270 msgr2=0x7f27b010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.399+0000 7f27b4c7c640 1 --2- 172.21.15.19:0/2087777429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b0107270 0x7f27b010ee40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009ca0 tx=0x7f27a002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.400+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2087777429 shutdown_connections 2024-04-23T18:29:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.400+0000 7f27b4c7c640 1 --2- 172.21.15.19:0/2087777429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b0102b50 0x7f27b0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.400+0000 7f27b4c7c640 1 --2- 172.21.15.19:0/2087777429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b0107270 0x7f27b010ee40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.400+0000 7f27b4c7c640 1 --2- 172.21.15.19:0/2087777429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27b01021a0 0x7f27b0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.400+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2087777429 >> 172.21.15.19:0/2087777429 conn(0x7f27b00fd9e0 msgr2=0x7f27b00ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.400+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2087777429 shutdown_connections 2024-04-23T18:29:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.400+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2087777429 wait complete. 2024-04-23T18:29:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.400+0000 7f27b4c7c640 1 Processor -- start 2024-04-23T18:29:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.400+0000 7f27b4c7c640 1 -- start start 2024-04-23T18:29:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27b4c7c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b01021a0 0x7f27b019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27b4c7c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27b0107270 0x7f27b019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27b4c7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b01985f0 0x7f27b0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27b4c7c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0112230 con 0x7f27b01985f0 2024-04-23T18:29:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27b4c7c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01120b0 con 0x7f27b01021a0 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27b4c7c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01123b0 con 0x7f27b0107270 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27aed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b01985f0 0x7f27b0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27aed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b01985f0 0x7f27b0198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60896/0 (socket says 172.21.15.19:60896) 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27aed9d640 1 -- 172.21.15.19:0/2392891195 learned_addr learned my addr 172.21.15.19:0/2392891195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27add9b640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27b0107270 0x7f27b019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27ae59c640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b01021a0 0x7f27b019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27aed9d640 1 -- 172.21.15.19:0/2392891195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27b0107270 msgr2=0x7f27b019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27aed9d640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27b0107270 0x7f27b019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27aed9d640 1 -- 172.21.15.19:0/2392891195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b01021a0 msgr2=0x7f27b019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27aed9d640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b01021a0 0x7f27b019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27aed9d640 1 -- 172.21.15.19:0/2392891195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b0199110 con 0x7f27b01985f0 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27ae59c640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b01021a0 0x7f27b019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27aed9d640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b01985f0 0x7f27b0198aa0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f27a0005cc0 tx=0x7f27a002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f279f7fe640 1 -- 172.21.15.19:0/2392891195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0003bf0 con 0x7f27b01985f0 2024-04-23T18:29:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f279f7fe640 1 -- 172.21.15.19:0/2392891195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a0003d90 con 0x7f27b01985f0 2024-04-23T18:29:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2392891195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b01993a0 con 0x7f27b01985f0 2024-04-23T18:29:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f279f7fe640 1 -- 172.21.15.19:0/2392891195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a002fbc0 con 0x7f27b01985f0 2024-04-23T18:29:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.401+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2392891195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b01a55a0 con 0x7f27b01985f0 2024-04-23T18:29:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.404+0000 7f279f7fe640 1 -- 172.21.15.19:0/2392891195 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27a002fd60 con 0x7f27b01985f0 2024-04-23T18:29:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.404+0000 7f27b7991740 1 Processor -- start 2024-04-23T18:29:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.404+0000 7f27b7991740 1 -- start start 2024-04-23T18:29:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.404+0000 7f279f7fe640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27980a3d20 0x7f27980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.404+0000 7f27b7991740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.404+0000 7f27b7991740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa7ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.404+0000 7f27b7991740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 0x55845aa81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.404+0000 7f27b7991740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55845a9d5370 con 0x55845aa70780 2024-04-23T18:29:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.404+0000 7f27b7991740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55845a9722e0 con 0x55845aa6fdd0 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.404+0000 7f27b7991740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55845a9d1330 con 0x55845aa7f4a0 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27980a3d20 0x7f27980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa7ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f279f7fe640 1 -- 172.21.15.19:0/2392891195 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f27a00f1ec0 con 0x7f27b01985f0 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa7ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60898/0 (socket says 172.21.15.19:60898) 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 -- 172.21.15.19:0/1456865674 learned_addr learned my addr 172.21.15.19:0/1456865674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27ae59c640 1 --2- 172.21.15.19:0/1456865674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27aed9d640 1 --2- 172.21.15.19:0/1456865674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 0x55845aa81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 -- 172.21.15.19:0/1456865674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 msgr2=0x55845aa81890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 --2- 172.21.15.19:0/1456865674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 0x55845aa81890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 -- 172.21.15.19:0/1456865674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 msgr2=0x55845aa701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 --2- 172.21.15.19:0/1456865674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 -- 172.21.15.19:0/1456865674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55845a7b3d80 con 0x55845aa70780 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 --2- 172.21.15.19:0/1456865674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa7ef60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f27a4014c80 tx=0x7f27a40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571910df00e25d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f27add9b640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27980a3d20 0x7f27980a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f27a4011cb0 tx=0x7f27a4016000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f279cff9640 1 -- 172.21.15.19:0/1456865674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a400bb60 con 0x55845aa70780 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f279cff9640 1 -- 172.21.15.19:0/1456865674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a400bd00 con 0x55845aa70780 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.405+0000 7f279cff9640 1 -- 172.21.15.19:0/1456865674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4003850 con 0x55845aa70780 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.406+0000 7f27b7991740 1 -- 172.21.15.19:0/1456865674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 msgr2=0x55845aa7ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.406+0000 7f27b7991740 1 --2- 172.21.15.19:0/1456865674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa7ef60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f27a4014c80 tx=0x7f27a40117b0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.406+0000 7f27b7991740 1 -- 172.21.15.19:0/1456865674 shutdown_connections 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.406+0000 7f27b7991740 1 --2- 172.21.15.19:0/1456865674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 0x55845aa81890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.406+0000 7f27b7991740 1 --2- 172.21.15.19:0/1456865674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa7ef60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.406+0000 7f27b7991740 1 --2- 172.21.15.19:0/1456865674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.406+0000 7f27b7991740 1 -- 172.21.15.19:0/1456865674 >> 172.21.15.19:0/1456865674 conn(0x55845aa6c3c0 msgr2=0x55845aa6c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.406+0000 7f27b7991740 1 -- 172.21.15.19:0/1456865674 shutdown_connections 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.406+0000 7f27b7991740 1 -- 172.21.15.19:0/1456865674 wait complete. 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.406+0000 7f27b7991740 1 -- start start 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27b7991740 1 Processor -- start 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27b7991740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa7c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27b7991740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa70780 0x55845aa7ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27b7991740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa7cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27b7991740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55845aa82200 con 0x55845aa7f4a0 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27b7991740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55845a96ffb0 con 0x55845aa6fdd0 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27b7991740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55845aa82380 con 0x55845aa70780 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27b7991740 10 client.0 _ll_register_callbacks cb 0x55845a96f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27aed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa7cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27aed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa7cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60908/0 (socket says 172.21.15.19:60908) 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27aed9d640 1 -- 172.21.15.19:0/1425015202 learned_addr learned my addr 172.21.15.19:0/1425015202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27ae59c640 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa7c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27aed9d640 1 -- 172.21.15.19:0/1425015202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa70780 msgr2=0x55845aa7ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27aed9d640 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa70780 0x55845aa7ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27aed9d640 1 -- 172.21.15.19:0/1425015202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 msgr2=0x55845aa7c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27aed9d640 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa7c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27aed9d640 1 -- 172.21.15.19:0/1425015202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55845a814510 con 0x55845aa7f4a0 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f27aed9d640 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa7cf40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f27a00ba3a0 tx=0x7f27a0044680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201435089cfb5229 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.407+0000 7f2795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f2795ffb640 1 -- 172.21.15.19:0/1425015202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a00098b0 con 0x55845aa7f4a0 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f2795ffb640 1 -- 172.21.15.19:0/1425015202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f27a0004610 con 0x55845aa7f4a0 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f27b7991740 1 -- 172.21.15.19:0/1425015202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a400bd00 con 0x55845aa7f4a0 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f2795ffb640 1 -- 172.21.15.19:0/1425015202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27a00a4170 con 0x55845aa7f4a0 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f27b7991740 10 client.16089 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f27b7991740 1 -- 172.21.15.19:0/1425015202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55845a6a7670 con 0x55845aa7f4a0 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f2795ffb640 1 -- 172.21.15.19:0/1425015202 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a400bd00 con 0x55845aa7f4a0 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f2795ffb640 1 -- 172.21.15.19:0/1425015202 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f27a00ef860 con 0x55845aa7f4a0 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f27b7991740 10 client.16089 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f27b7991740 10 client.16089 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:29:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f27b7991740 20 client.16089 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f27b7991740 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55845aa8f8a0 0x55845aa91d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f27b7991740 4 client.16089 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f27b7991740 1 -- 172.21.15.19:0/1425015202 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55845a9e2d90 con 0x55845aa8f8a0 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.408+0000 7f27ae59c640 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55845aa8f8a0 0x55845aa91d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.409+0000 7f2795ffb640 10 client.16089 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.409+0000 7f27ae59c640 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55845aa8f8a0 0x55845aa91d40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f71143bb40c5850 server_cookie=3ad41e060cae7f60 in_seq=0 out_seq=0 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.409+0000 7f2795ffb640 1 -- 172.21.15.19:0/1425015202 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55845a9e2d90 con 0x55845aa8f8a0 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.409+0000 7f2795ffb640 10 client.16089 handle_command_reply: tid=0 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 client.16089 shutdown 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 -- 172.21.15.19:0/1425015202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 msgr2=0x55845aa7cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa7cf40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f27a00ba3a0 tx=0x7f27a0044680 comp rx=0 tx=0).stop 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 -- 172.21.15.19:0/1425015202 shutdown_connections 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55845aa8f8a0 0x55845aa91d40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa7cf40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa70780 0x55845aa7ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f2795ffb640 0 client.16089 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 --2- 172.21.15.19:0/1425015202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa7c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 -- 172.21.15.19:0/1425015202 >> 172.21.15.19:0/1425015202 conn(0x55845aa6c3c0 msgr2=0x55845aa72210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 -- 172.21.15.19:0/1425015202 shutdown_connections 2024-04-23T18:29:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 1 -- 172.21.15.19:0/1425015202 wait complete. 2024-04-23T18:29:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.410+0000 7f27b7991740 20 client.16089 trim_cache size 0 max 16384 2024-04-23T18:29:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.427+0000 7f27b7991740 1 Processor -- start 2024-04-23T18:29:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.427+0000 7f27b7991740 1 -- start start 2024-04-23T18:29:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.427+0000 7f27b7991740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa8b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.427+0000 7f27b7991740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa97650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.427+0000 7f27b7991740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 0x55845aa97b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.427+0000 7f27b7991740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ab890 con 0x55845aa70780 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.427+0000 7f27b7991740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ad3d0 con 0x55845aa6fdd0 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.427+0000 7f27b7991740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01abf60 con 0x55845aa7f4a0 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27add9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa97650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27add9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa97650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60924/0 (socket says 172.21.15.19:60924) 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27add9b640 1 -- 172.21.15.19:0/240973244 learned_addr learned my addr 172.21.15.19:0/240973244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27ae59c640 1 --2- 172.21.15.19:0/240973244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa8b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27aed9d640 1 --2- 172.21.15.19:0/240973244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 0x55845aa97b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27add9b640 1 -- 172.21.15.19:0/240973244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 msgr2=0x55845aa97b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27add9b640 1 --2- 172.21.15.19:0/240973244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 0x55845aa97b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27add9b640 1 -- 172.21.15.19:0/240973244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 msgr2=0x55845aa8b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27add9b640 1 --2- 172.21.15.19:0/240973244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa8b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27add9b640 1 -- 172.21.15.19:0/240973244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55845aa70520 con 0x55845aa70780 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27aed9d640 1 --2- 172.21.15.19:0/240973244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 0x55845aa97b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f27add9b640 1 --2- 172.21.15.19:0/240973244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa97650 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f27a40148a0 tx=0x7f27a40148d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784cef8124ce4394 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f2797fff640 1 -- 172.21.15.19:0/240973244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a40037f0 con 0x55845aa70780 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f2797fff640 1 -- 172.21.15.19:0/240973244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a400bb30 con 0x55845aa70780 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.428+0000 7f2797fff640 1 -- 172.21.15.19:0/240973244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4005270 con 0x55845aa70780 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 -- 172.21.15.19:0/240973244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 msgr2=0x55845aa97650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 --2- 172.21.15.19:0/240973244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa97650 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f27a40148a0 tx=0x7f27a40148d0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 -- 172.21.15.19:0/240973244 shutdown_connections 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 --2- 172.21.15.19:0/240973244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa7f4a0 0x55845aa97b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 --2- 172.21.15.19:0/240973244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa70780 0x55845aa97650 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 --2- 172.21.15.19:0/240973244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa6fdd0 0x55845aa8b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 -- 172.21.15.19:0/240973244 >> 172.21.15.19:0/240973244 conn(0x55845aa6c3c0 msgr2=0x55845aa7dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 -- 172.21.15.19:0/240973244 shutdown_connections 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 -- 172.21.15.19:0/240973244 wait complete. 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 -- start start 2024-04-23T18:29:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 Processor -- start 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa6fdd0 0x55845aa877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa70780 0x55845aa845a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa84ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01acd00 con 0x55845aa7f4a0 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ac630 con 0x55845aa70780 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01a9870 con 0x55845aa6fdd0 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.429+0000 7f27b7991740 10 client.0 _ll_register_callbacks cb 0x7f27a00bdaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27aed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa84ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27aed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa84ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60940/0 (socket says 172.21.15.19:60940) 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27aed9d640 1 -- 172.21.15.19:0/3973130732 learned_addr learned my addr 172.21.15.19:0/3973130732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27add9b640 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa70780 0x55845aa845a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27ae59c640 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa6fdd0 0x55845aa877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27aed9d640 1 -- 172.21.15.19:0/3973130732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa6fdd0 msgr2=0x55845aa877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27aed9d640 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa6fdd0 0x55845aa877c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27aed9d640 1 -- 172.21.15.19:0/3973130732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa70780 msgr2=0x55845aa845a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27aed9d640 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa70780 0x55845aa845a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27aed9d640 1 -- 172.21.15.19:0/3973130732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55845aa85020 con 0x55845aa7f4a0 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27aed9d640 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa84ae0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f27a0031a90 tx=0x7f27a0039f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2306f376ae98ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f2795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f2795ffb640 1 -- 172.21.15.19:0/3973130732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a00ba1e0 con 0x55845aa7f4a0 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f2795ffb640 1 -- 172.21.15.19:0/3973130732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f27a00d7270 con 0x55845aa7f4a0 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27b7991740 1 -- 172.21.15.19:0/3973130732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a400bb30 con 0x55845aa7f4a0 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f2795ffb640 1 -- 172.21.15.19:0/3973130732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27a00bd8a0 con 0x55845aa7f4a0 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27b7991740 10 client.16095 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.430+0000 7f27b7991740 1 -- 172.21.15.19:0/3973130732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55845aaf6a80 con 0x55845aa7f4a0 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f2795ffb640 1 -- 172.21.15.19:0/3973130732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a400bb30 con 0x55845aa7f4a0 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f2795ffb640 1 -- 172.21.15.19:0/3973130732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f27a00bd390 con 0x55845aa7f4a0 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f27b7991740 10 client.16095 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f27b7991740 10 client.16095 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:29:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f27b7991740 20 client.16095 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:20.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f27b7991740 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55845aaf6fd0 0x55845aaf93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:20.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f27b7991740 4 client.16095 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.65, "await_for": 42.0}] 2024-04-23T18:29:20.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f27b7991740 1 -- 172.21.15.19:0/3973130732 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.65, "await_for": 42.0}) -- 0x55845a9dcc00 con 0x55845aaf6fd0 2024-04-23T18:29:20.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f27ae59c640 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55845aaf6fd0 0x55845aaf93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:20.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f27ae59c640 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55845aaf6fd0 0x55845aaf93c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d54cc59df178cf1 server_cookie=256ea926f28c83f8 in_seq=0 out_seq=0 2024-04-23T18:29:20.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.431+0000 7f2795ffb640 10 client.16095 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f2795ffb640 1 -- 172.21.15.19:0/3973130732 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55845a9dcc00 con 0x55845aaf6fd0 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f2795ffb640 10 client.16095 handle_command_reply: tid=0 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 client.16095 shutdown 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 -- 172.21.15.19:0/3973130732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 msgr2=0x55845aa84ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa84ae0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f27a0031a90 tx=0x7f27a0039f20 comp rx=0 tx=0).stop 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 -- 172.21.15.19:0/3973130732 shutdown_connections 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55845aaf6fd0 0x55845aaf93c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55845aa70780 0x55845aa845a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55845aa7f4a0 0x55845aa84ae0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f2795ffb640 0 client.16095 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 --2- 172.21.15.19:0/3973130732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55845aa6fdd0 0x55845aa877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 -- 172.21.15.19:0/3973130732 >> 172.21.15.19:0/3973130732 conn(0x55845aa6c3c0 msgr2=0x55845aa6ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 -- 172.21.15.19:0/3973130732 shutdown_connections 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 1 -- 172.21.15.19:0/3973130732 wait complete. 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b7991740 20 client.16095 trim_cache size 0 max 16384 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 18, 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "2ca0fa3a": { 2024-04-23T18:29:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 18, 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 77.7, 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:20.442 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2392891195 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27980a3d20 msgr2=0x7f27980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b4c7c640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27980a3d20 0x7f27980a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f27a4011cb0 tx=0x7f27a4016000 comp rx=0 tx=0).stop 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2392891195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b01985f0 msgr2=0x7f27b0198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.446+0000 7f27b4c7c640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b01985f0 0x7f27b0198aa0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f27a0005cc0 tx=0x7f27a002f380 comp rx=0 tx=0).stop 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.447+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2392891195 shutdown_connections 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.447+0000 7f27b4c7c640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f27980a3d20 0x7f27980a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.447+0000 7f27b4c7c640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b01985f0 0x7f27b0198aa0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.447+0000 7f27b4c7c640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27b0107270 0x7f27b019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.447+0000 7f27b4c7c640 1 --2- 172.21.15.19:0/2392891195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b01021a0 0x7f27b019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.447+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2392891195 >> 172.21.15.19:0/2392891195 conn(0x7f27b00fd9e0 msgr2=0x7f27b00fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.447+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2392891195 shutdown_connections 2024-04-23T18:29:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:20.447+0000 7f27b4c7c640 1 -- 172.21.15.19:0/2392891195 wait complete. 2024-04-23T18:29:20.450 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2ca0fa3a', quiesce took 0.5 seconds. Will release after: 17.65 2024-04-23T18:29:20.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:20 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2509805150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:20.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:20 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2509805150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:20.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:20 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2509805150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:21.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:21 smithi019 ceph-mon[29215]: pgmap v99: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-04-23T18:29:21.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:21 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3117347205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:21.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:21 smithi044 ceph-mon[32458]: pgmap v99: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-04-23T18:29:21.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:21 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3117347205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:21.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:21 smithi155 ceph-mon[33468]: pgmap v99: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-04-23T18:29:21.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:21 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3117347205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:23.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:23 smithi019 ceph-mon[29215]: pgmap v100: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-04-23T18:29:23.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:23 smithi044 ceph-mon[32458]: pgmap v100: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-04-23T18:29:23.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:23 smithi155 ceph-mon[33468]: pgmap v100: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-04-23T18:29:25.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:25 smithi019 ceph-mon[29215]: pgmap v101: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-23T18:29:25.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:25 smithi044 ceph-mon[32458]: pgmap v101: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-23T18:29:25.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:25 smithi155 ceph-mon[33468]: pgmap v101: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-23T18:29:27.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:27 smithi019 ceph-mon[29215]: pgmap v102: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-04-23T18:29:27.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:27 smithi044 ceph-mon[32458]: pgmap v102: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-04-23T18:29:27.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:27 smithi155 ceph-mon[33468]: pgmap v102: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-04-23T18:29:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.133+0000 7fe264431640 1 -- 172.21.15.19:0/1926768758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c1010f0 msgr2=0x7fe25c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.133+0000 7fe264431640 1 --2- 172.21.15.19:0/1926768758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c1010f0 0x7fe25c1014d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe258009ca0 tx=0x7fe25802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.133+0000 7fe264431640 1 -- 172.21.15.19:0/1926768758 shutdown_connections 2024-04-23T18:29:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.133+0000 7fe264431640 1 --2- 172.21.15.19:0/1926768758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c106170 0x7fe25c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.133+0000 7fe264431640 1 --2- 172.21.15.19:0/1926768758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe25c101aa0 0x7fe25c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.133+0000 7fe264431640 1 --2- 172.21.15.19:0/1926768758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c1010f0 0x7fe25c1014d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.133+0000 7fe264431640 1 -- 172.21.15.19:0/1926768758 >> 172.21.15.19:0/1926768758 conn(0x7fe25c077fd0 msgr2=0x7fe25c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.133+0000 7fe264431640 1 -- 172.21.15.19:0/1926768758 shutdown_connections 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.133+0000 7fe264431640 1 -- 172.21.15.19:0/1926768758 wait complete. 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe264431640 1 Processor -- start 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe264431640 1 -- start start 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe264431640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c1010f0 0x7fe25c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe264431640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe25c106170 0x7fe25c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe264431640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c19cf90 0x7fe25c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe264431640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1121f0 con 0x7fe25c1010f0 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe264431640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c112070 con 0x7fe25c19cf90 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe264431640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c112370 con 0x7fe25c106170 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe2621cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c1010f0 0x7fe25c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe2621cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c1010f0 0x7fe25c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60966/0 (socket says 172.21.15.19:60966) 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe2621cd640 1 -- 172.21.15.19:0/4116936703 learned_addr learned my addr 172.21.15.19:0/4116936703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe2619cc640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe25c106170 0x7fe25c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.134+0000 7fe2629ce640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c19cf90 0x7fe25c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe2621cd640 1 -- 172.21.15.19:0/4116936703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe25c106170 msgr2=0x7fe25c198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe2621cd640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe25c106170 0x7fe25c198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe2621cd640 1 -- 172.21.15.19:0/4116936703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c19cf90 msgr2=0x7fe25c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe2621cd640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c19cf90 0x7fe25c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe2621cd640 1 -- 172.21.15.19:0/4116936703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c19da80 con 0x7fe25c1010f0 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe2629ce640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c19cf90 0x7fe25c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe2621cd640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c1010f0 0x7fe25c1986d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe25802ade0 tx=0x7fe258031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe2537fe640 1 -- 172.21.15.19:0/4116936703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258003b70 con 0x7fe25c1010f0 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe264431640 1 -- 172.21.15.19:0/4116936703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25c19dd10 con 0x7fe25c1010f0 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe2537fe640 1 -- 172.21.15.19:0/4116936703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe258003d10 con 0x7fe25c1010f0 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe264431640 1 -- 172.21.15.19:0/4116936703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25c1a5530 con 0x7fe25c1010f0 2024-04-23T18:29:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.135+0000 7fe2537fe640 1 -- 172.21.15.19:0/4116936703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25802fa10 con 0x7fe25c1010f0 2024-04-23T18:29:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.137+0000 7fe2537fe640 1 -- 172.21.15.19:0/4116936703 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe258004580 con 0x7fe25c1010f0 2024-04-23T18:29:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.138+0000 7fe267222740 1 Processor -- start 2024-04-23T18:29:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.138+0000 7fe267222740 1 -- start start 2024-04-23T18:29:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.138+0000 7fe267222740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92d620 0x55fe2e92da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.138+0000 7fe267222740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 0x55fe2e9382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.138+0000 7fe267222740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 0x55fe2e93abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.138+0000 7fe267222740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fe2e892970 con 0x55fe2e92df40 2024-04-23T18:29:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.138+0000 7fe267222740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fe2e82f4a0 con 0x55fe2e92d620 2024-04-23T18:29:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.138+0000 7fe267222740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fe2e88e930 con 0x55fe2e938800 2024-04-23T18:29:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2537fe640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe2480a3dc0 0x7fe2480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92d620 0x55fe2e92da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2621cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 0x55fe2e9382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92d620 0x55fe2e92da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:34726/0 (socket says 172.21.15.19:34726) 2024-04-23T18:29:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 -- 172.21.15.19:0/1252788770 learned_addr learned my addr 172.21.15.19:0/1252788770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe2480a3dc0 0x7fe2480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2619cc640 1 --2- 172.21.15.19:0/1252788770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 0x55fe2e93abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2537fe640 1 -- 172.21.15.19:0/4116936703 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe2580f1bf0 con 0x7fe25c1010f0 2024-04-23T18:29:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 -- 172.21.15.19:0/1252788770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 msgr2=0x55fe2e93abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 --2- 172.21.15.19:0/1252788770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 0x55fe2e93abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 -- 172.21.15.19:0/1252788770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 msgr2=0x55fe2e9382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 --2- 172.21.15.19:0/1252788770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 0x55fe2e9382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 -- 172.21.15.19:0/1252788770 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2e5a5210 con 0x55fe2e92d620 2024-04-23T18:29:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe2480a3dc0 0x7fe2480a6280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe254010860 tx=0x7fe25401d000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.139+0000 7fe2629ce640 1 --2- 172.21.15.19:0/1252788770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92d620 0x55fe2e92da00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe254012a30 tx=0x7fe254012ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=affbc469344b430e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.140+0000 7fe250bfe640 1 -- 172.21.15.19:0/1252788770 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254023070 con 0x55fe2e92d620 2024-04-23T18:29:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.140+0000 7fe250bfe640 1 -- 172.21.15.19:0/1252788770 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe254009990 con 0x55fe2e92d620 2024-04-23T18:29:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.140+0000 7fe250bfe640 1 -- 172.21.15.19:0/1252788770 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25400d9f0 con 0x55fe2e92d620 2024-04-23T18:29:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.140+0000 7fe267222740 1 -- 172.21.15.19:0/1252788770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92d620 msgr2=0x55fe2e92da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.140+0000 7fe267222740 1 --2- 172.21.15.19:0/1252788770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92d620 0x55fe2e92da00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe254012a30 tx=0x7fe254012ef0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.140+0000 7fe267222740 1 -- 172.21.15.19:0/1252788770 shutdown_connections 2024-04-23T18:29:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.140+0000 7fe267222740 1 --2- 172.21.15.19:0/1252788770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 0x55fe2e93abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.140+0000 7fe267222740 1 --2- 172.21.15.19:0/1252788770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 0x55fe2e9382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.140+0000 7fe267222740 1 --2- 172.21.15.19:0/1252788770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92d620 0x55fe2e92da00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.140+0000 7fe267222740 1 -- 172.21.15.19:0/1252788770 >> 172.21.15.19:0/1252788770 conn(0x55fe2e929b30 msgr2=0x55fe2e929f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 1 -- 172.21.15.19:0/1252788770 shutdown_connections 2024-04-23T18:29:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 1 -- 172.21.15.19:0/1252788770 wait complete. 2024-04-23T18:29:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 1 -- start start 2024-04-23T18:29:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 1 Processor -- start 2024-04-23T18:29:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92d620 0x55fe2e938150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 0x55fe2e935440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 0x55fe2e9359a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fe2e93b3d0 con 0x55fe2e92d620 2024-04-23T18:29:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fe2e82d0b0 con 0x55fe2e92df40 2024-04-23T18:29:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fe2e93b550 con 0x55fe2e938800 2024-04-23T18:29:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.141+0000 7fe267222740 10 client.0 _ll_register_callbacks cb 0x55fe2e81ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2621cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92d620 0x55fe2e938150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2621cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92d620 0x55fe2e938150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60980/0 (socket says 172.21.15.19:60980) 2024-04-23T18:29:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2621cd640 1 -- 172.21.15.19:0/1278070894 learned_addr learned my addr 172.21.15.19:0/1278070894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2619cc640 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 0x55fe2e935440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2629ce640 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 0x55fe2e9359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2621cd640 1 -- 172.21.15.19:0/1278070894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 msgr2=0x55fe2e9359a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2621cd640 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 0x55fe2e9359a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2621cd640 1 -- 172.21.15.19:0/1278070894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 msgr2=0x55fe2e935440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2621cd640 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 0x55fe2e935440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2621cd640 1 -- 172.21.15.19:0/1278070894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe2e671620 con 0x55fe2e92d620 2024-04-23T18:29:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2629ce640 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 0x55fe2e9359a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe2621cd640 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92d620 0x55fe2e938150 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe2580ba640 tx=0x7fe25808d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510891a0cf00d542 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe245ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe245ffb640 1 -- 172.21.15.19:0/1278070894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580ef790 con 0x55fe2e92d620 2024-04-23T18:29:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe245ffb640 1 -- 172.21.15.19:0/1278070894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe2580de070 con 0x55fe2e92d620 2024-04-23T18:29:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe267222740 1 -- 172.21.15.19:0/1278070894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe254009990 con 0x55fe2e92d620 2024-04-23T18:29:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe245ffb640 1 -- 172.21.15.19:0/1278070894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe2580be370 con 0x55fe2e92d620 2024-04-23T18:29:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe267222740 10 client.16107 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.142+0000 7fe267222740 1 -- 172.21.15.19:0/1278070894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fe2e93b3d0 con 0x55fe2e92d620 2024-04-23T18:29:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.143+0000 7fe245ffb640 1 -- 172.21.15.19:0/1278070894 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe254009990 con 0x55fe2e92d620 2024-04-23T18:29:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.143+0000 7fe245ffb640 1 -- 172.21.15.19:0/1278070894 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe2580bded0 con 0x55fe2e92d620 2024-04-23T18:29:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.143+0000 7fe267222740 10 client.16107 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.143+0000 7fe267222740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.143+0000 7fe267222740 20 client.16107 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.143+0000 7fe267222740 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fe2e9487e0 0x55fe2e94ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.143+0000 7fe267222740 4 client.16107 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.143+0000 7fe267222740 1 -- 172.21.15.19:0/1278070894 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe2e8a0390 con 0x55fe2e9487e0 2024-04-23T18:29:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.143+0000 7fe2619cc640 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fe2e9487e0 0x55fe2e94ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe2619cc640 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fe2e9487e0 0x55fe2e94ac30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fd44a597569febb server_cookie=71b6d230ec69920d in_seq=0 out_seq=0 2024-04-23T18:29:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe245ffb640 10 client.16107 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe245ffb640 1 -- 172.21.15.19:0/1278070894 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe2e8a0390 con 0x55fe2e9487e0 2024-04-23T18:29:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe245ffb640 10 client.16107 handle_command_reply: tid=0 2024-04-23T18:29:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe267222740 1 client.16107 shutdown 2024-04-23T18:29:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe267222740 1 -- 172.21.15.19:0/1278070894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92d620 msgr2=0x55fe2e938150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe267222740 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92d620 0x55fe2e938150 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe2580ba640 tx=0x7fe25808d550 comp rx=0 tx=0).stop 2024-04-23T18:29:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe267222740 1 -- 172.21.15.19:0/1278070894 shutdown_connections 2024-04-23T18:29:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe267222740 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fe2e9487e0 0x55fe2e94ac30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe267222740 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 0x55fe2e935440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe267222740 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e938800 0x55fe2e9359a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe245ffb640 0 client.16107 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.144+0000 7fe267222740 1 --2- 172.21.15.19:0/1278070894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92d620 0x55fe2e938150 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.145+0000 7fe267222740 1 -- 172.21.15.19:0/1278070894 >> 172.21.15.19:0/1278070894 conn(0x55fe2e929b30 msgr2=0x55fe2e92f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.145+0000 7fe267222740 1 -- 172.21.15.19:0/1278070894 shutdown_connections 2024-04-23T18:29:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.145+0000 7fe267222740 1 -- 172.21.15.19:0/1278070894 wait complete. 2024-04-23T18:29:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.145+0000 7fe267222740 20 client.16107 trim_cache size 0 max 16384 2024-04-23T18:29:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe267222740 1 Processor -- start 2024-04-23T18:29:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe267222740 1 -- start start 2024-04-23T18:29:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe267222740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e92d620 0x55fe2e944290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe267222740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 0x55fe2e930850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe267222740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e938800 0x55fe2e937c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe267222740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1adab0 con 0x55fe2e938800 2024-04-23T18:29:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe267222740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1af5f0 con 0x55fe2e92df40 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe267222740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1ae180 con 0x55fe2e92d620 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2629ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e938800 0x55fe2e937c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2629ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e938800 0x55fe2e937c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60992/0 (socket says 172.21.15.19:60992) 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2629ce640 1 -- 172.21.15.19:0/295823426 learned_addr learned my addr 172.21.15.19:0/295823426 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2621cd640 1 --2- 172.21.15.19:0/295823426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e92d620 0x55fe2e944290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2629ce640 1 -- 172.21.15.19:0/295823426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e92d620 msgr2=0x55fe2e944290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2629ce640 1 --2- 172.21.15.19:0/295823426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e92d620 0x55fe2e944290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2619cc640 1 --2- 172.21.15.19:0/295823426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 0x55fe2e930850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2629ce640 1 -- 172.21.15.19:0/295823426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 msgr2=0x55fe2e930850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2629ce640 1 --2- 172.21.15.19:0/295823426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 0x55fe2e930850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2629ce640 1 -- 172.21.15.19:0/295823426 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2e92dd70 con 0x55fe2e938800 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.160+0000 7fe2619cc640 1 --2- 172.21.15.19:0/295823426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 0x55fe2e930850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe2629ce640 1 --2- 172.21.15.19:0/295823426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e938800 0x55fe2e937c80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe254009250 tx=0x7fe254009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cc6b863622bec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe247fff640 1 -- 172.21.15.19:0/295823426 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25401b040 con 0x55fe2e938800 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe247fff640 1 -- 172.21.15.19:0/295823426 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe25402b070 con 0x55fe2e938800 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe247fff640 1 -- 172.21.15.19:0/295823426 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254023040 con 0x55fe2e938800 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe267222740 1 -- 172.21.15.19:0/295823426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e938800 msgr2=0x55fe2e937c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe267222740 1 --2- 172.21.15.19:0/295823426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e938800 0x55fe2e937c80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe254009250 tx=0x7fe254009ea0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe267222740 1 -- 172.21.15.19:0/295823426 shutdown_connections 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe267222740 1 --2- 172.21.15.19:0/295823426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e92df40 0x55fe2e930850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe267222740 1 --2- 172.21.15.19:0/295823426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e938800 0x55fe2e937c80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe267222740 1 --2- 172.21.15.19:0/295823426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e92d620 0x55fe2e944290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.161+0000 7fe267222740 1 -- 172.21.15.19:0/295823426 >> 172.21.15.19:0/295823426 conn(0x55fe2e929b30 msgr2=0x55fe2e930140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 1 -- 172.21.15.19:0/295823426 shutdown_connections 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 1 -- 172.21.15.19:0/295823426 wait complete. 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 1 -- start start 2024-04-23T18:29:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 1 Processor -- start 2024-04-23T18:29:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e92d620 0x55fe2e9407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 0x55fe2e940ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e938800 0x55fe2e93bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1aef20 con 0x55fe2e92df40 2024-04-23T18:29:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1ae850 con 0x55fe2e938800 2024-04-23T18:29:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1abab0 con 0x55fe2e92d620 2024-04-23T18:29:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe267222740 10 client.0 _ll_register_callbacks cb 0x7fe258009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe2619cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 0x55fe2e940ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe2619cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 0x55fe2e940ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32776/0 (socket says 172.21.15.19:32776) 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe2621cd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e92d620 0x55fe2e9407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.162+0000 7fe2619cc640 1 -- 172.21.15.19:0/505130783 learned_addr learned my addr 172.21.15.19:0/505130783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe2629ce640 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e938800 0x55fe2e93bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe2619cc640 1 -- 172.21.15.19:0/505130783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e92d620 msgr2=0x55fe2e9407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe2619cc640 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e92d620 0x55fe2e9407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe2619cc640 1 -- 172.21.15.19:0/505130783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e938800 msgr2=0x55fe2e93bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe2619cc640 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e938800 0x55fe2e93bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe2619cc640 1 -- 172.21.15.19:0/505130783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe2e93c6f0 con 0x55fe2e92df40 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe2619cc640 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 0x55fe2e940ce0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00abb0 tx=0x7fe24c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94326def24d777a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe245ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe245ffb640 1 -- 172.21.15.19:0/505130783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c01d680 con 0x55fe2e92df40 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe245ffb640 1 -- 172.21.15.19:0/505130783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe24c01d820 con 0x55fe2e92df40 2024-04-23T18:29:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe267222740 1 -- 172.21.15.19:0/505130783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe25402b070 con 0x55fe2e92df40 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe245ffb640 1 -- 172.21.15.19:0/505130783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe24c004020 con 0x55fe2e92df40 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe267222740 10 client.16113 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe267222740 1 -- 172.21.15.19:0/505130783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fe2e9aea30 con 0x55fe2e92df40 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.163+0000 7fe245ffb640 1 -- 172.21.15.19:0/505130783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe25402b070 con 0x55fe2e92df40 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.164+0000 7fe245ffb640 1 -- 172.21.15.19:0/505130783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe24c027430 con 0x55fe2e92df40 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.164+0000 7fe267222740 10 client.16113 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.164+0000 7fe267222740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.164+0000 7fe267222740 20 client.16113 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.164+0000 7fe267222740 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fe2e9af1b0 0x55fe2e9b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.164+0000 7fe267222740 4 client.16113 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.164+0000 7fe267222740 1 -- 172.21.15.19:0/505130783 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe2e89a200 con 0x55fe2e9af1b0 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.164+0000 7fe2621cd640 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fe2e9af1b0 0x55fe2e9b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.164+0000 7fe245ffb640 10 client.16113 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.164+0000 7fe2621cd640 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fe2e9af1b0 0x55fe2e9b15d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa46cc1672e93ddb server_cookie=f034ee520544a23b in_seq=0 out_seq=0 2024-04-23T18:29:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe245ffb640 1 -- 172.21.15.19:0/505130783 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe2e89a200 con 0x55fe2e9af1b0 2024-04-23T18:29:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe245ffb640 10 client.16113 handle_command_reply: tid=0 2024-04-23T18:29:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 client.16113 shutdown 2024-04-23T18:29:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 -- 172.21.15.19:0/505130783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 msgr2=0x55fe2e940ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 0x55fe2e940ce0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00abb0 tx=0x7fe24c011810 comp rx=0 tx=0).stop 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 -- 172.21.15.19:0/505130783 shutdown_connections 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55fe2e9af1b0 0x55fe2e9b15d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe2e938800 0x55fe2e93bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe2e92df40 0x55fe2e940ce0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 --2- 172.21.15.19:0/505130783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe2e92d620 0x55fe2e9407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe245ffb640 0 client.16113 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 -- 172.21.15.19:0/505130783 >> 172.21.15.19:0/505130783 conn(0x55fe2e929b30 msgr2=0x55fe2e9309d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 -- 172.21.15.19:0/505130783 shutdown_connections 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.176+0000 7fe267222740 1 -- 172.21.15.19:0/505130783 wait complete. 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe267222740 20 client.16113 trim_cache size 0 max 16384 2024-04-23T18:29:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:29:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:29:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:29:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:29:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 -- 172.21.15.19:0/4116936703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe2480a3dc0 msgr2=0x7fe2480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe2480a3dc0 0x7fe2480a6280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe254010860 tx=0x7fe25401d000 comp rx=0 tx=0).stop 2024-04-23T18:29:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 -- 172.21.15.19:0/4116936703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c1010f0 msgr2=0x7fe25c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c1010f0 0x7fe25c1986d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe25802ade0 tx=0x7fe258031770 comp rx=0 tx=0).stop 2024-04-23T18:29:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 -- 172.21.15.19:0/4116936703 shutdown_connections 2024-04-23T18:29:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe2480a3dc0 0x7fe2480a6280 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c19cf90 0x7fe25c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe25c106170 0x7fe25c198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 --2- 172.21.15.19:0/4116936703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c1010f0 0x7fe25c1986d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 -- 172.21.15.19:0/4116936703 >> 172.21.15.19:0/4116936703 conn(0x7fe25c077fd0 msgr2=0x7fe25c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 -- 172.21.15.19:0/4116936703 shutdown_connections 2024-04-23T18:29:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:28.177+0000 7fe264431640 1 -- 172.21.15.19:0/4116936703 wait complete. 2024-04-23T18:29:29.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:29 smithi044 ceph-mon[32458]: pgmap v103: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-04-23T18:29:29.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:29 smithi155 ceph-mon[33468]: pgmap v103: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-04-23T18:29:29.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:29 smithi019 ceph-mon[29215]: pgmap v103: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-04-23T18:29:31.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:31 smithi044 ceph-mon[32458]: pgmap v104: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-23T18:29:31.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:31 smithi155 ceph-mon[33468]: pgmap v104: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-23T18:29:31.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:31 smithi019 ceph-mon[29215]: pgmap v104: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-23T18:29:33.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:33 smithi044 ceph-mon[32458]: pgmap v105: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-04-23T18:29:33.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:33 smithi155 ceph-mon[33468]: pgmap v105: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-04-23T18:29:33.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:33 smithi019 ceph-mon[29215]: pgmap v105: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-04-23T18:29:35.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:35 smithi044 ceph-mon[32458]: pgmap v106: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-04-23T18:29:35.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:35 smithi155 ceph-mon[33468]: pgmap v106: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-04-23T18:29:35.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:35 smithi019 ceph-mon[29215]: pgmap v106: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-04-23T18:29:37.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:37 smithi044 ceph-mon[32458]: pgmap v107: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-04-23T18:29:37.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:37 smithi155 ceph-mon[33468]: pgmap v107: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-04-23T18:29:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:37 smithi019 ceph-mon[29215]: pgmap v107: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-04-23T18:29:38.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:29:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.224+0000 7f464c937640 1 -- 172.21.15.19:0/3773285433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648103230 msgr2=0x7f4648103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.224+0000 7f464c937640 1 --2- 172.21.15.19:0/3773285433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648103230 0x7f4648103610 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4638009900 tx=0x7f463802a480 comp rx=0 tx=0).stop 2024-04-23T18:29:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.224+0000 7f464c937640 1 -- 172.21.15.19:0/3773285433 shutdown_connections 2024-04-23T18:29:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.224+0000 7f464c937640 1 --2- 172.21.15.19:0/3773285433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464810c570 0x7f464810e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.224+0000 7f464c937640 1 --2- 172.21.15.19:0/3773285433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4648103b50 0x7f464810c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.224+0000 7f464c937640 1 --2- 172.21.15.19:0/3773285433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648103230 0x7f4648103610 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.224+0000 7f464c937640 1 -- 172.21.15.19:0/3773285433 >> 172.21.15.19:0/3773285433 conn(0x7f4648077fb0 msgr2=0x7f46480783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.224+0000 7f464c937640 1 -- 172.21.15.19:0/3773285433 shutdown_connections 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.224+0000 7f464c937640 1 -- 172.21.15.19:0/3773285433 wait complete. 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.225+0000 7f464c937640 1 Processor -- start 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.225+0000 7f464c937640 1 -- start start 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.225+0000 7f464c937640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4648103230 0x7f4648198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.225+0000 7f464c937640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648103b50 0x7f4648198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.225+0000 7f464c937640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464810c570 0x7f464819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.225+0000 7f464c937640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4648112200 con 0x7f4648103b50 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.225+0000 7f464c937640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4648112080 con 0x7f464810c570 2024-04-23T18:29:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.225+0000 7f464c937640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4648112380 con 0x7f4648103230 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f4645d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648103b50 0x7f4648198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f4645d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648103b50 0x7f4648198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35134/0 (socket says 172.21.15.19:35134) 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f4645d9b640 1 -- 172.21.15.19:0/1770771686 learned_addr learned my addr 172.21.15.19:0/1770771686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f4646d9d640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464810c570 0x7f464819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f4645d9b640 1 -- 172.21.15.19:0/1770771686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4648103230 msgr2=0x7f4648198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f4645d9b640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4648103230 0x7f4648198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f4645d9b640 1 -- 172.21.15.19:0/1770771686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464810c570 msgr2=0x7f464819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f4645d9b640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464810c570 0x7f464819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f4645d9b640 1 -- 172.21.15.19:0/1770771686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464819d730 con 0x7f4648103b50 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f4645d9b640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648103b50 0x7f4648198c80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f463c00d950 tx=0x7f463c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f46377fe640 1 -- 172.21.15.19:0/1770771686 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c017070 con 0x7f4648103b50 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f464c937640 1 -- 172.21.15.19:0/1770771686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464819da20 con 0x7f4648103b50 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f46377fe640 1 -- 172.21.15.19:0/1770771686 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f463c009c30 con 0x7f4648103b50 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f464c937640 1 -- 172.21.15.19:0/1770771686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46481a52b0 con 0x7f4648103b50 2024-04-23T18:29:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.226+0000 7f46377fe640 1 -- 172.21.15.19:0/1770771686 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c020060 con 0x7f4648103b50 2024-04-23T18:29:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.227+0000 7f464c937640 1 -- 172.21.15.19:0/1770771686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46480691c0 con 0x7f4648103b50 2024-04-23T18:29:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.228+0000 7f46377fe640 1 -- 172.21.15.19:0/1770771686 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f463c010460 con 0x7f4648103b50 2024-04-23T18:29:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.229+0000 7f46377fe640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46300a3d20 0x7f46300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.229+0000 7f46377fe640 1 -- 172.21.15.19:0/1770771686 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f463c0d0780 con 0x7f4648103b50 2024-04-23T18:29:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.229+0000 7f464659c640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46300a3d20 0x7f46300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.230+0000 7f464659c640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46300a3d20 0x7f46300a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f463802a9a0 tx=0x7f4638031400 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:38.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.234+0000 7f46377fe640 1 -- 172.21.15.19:0/1770771686 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f463c09ce50 con 0x7f4648103b50 2024-04-23T18:29:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.462+0000 7f464c937640 1 -- 172.21.15.19:0/1770771686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4648107750 con 0x7f4648103b50 2024-04-23T18:29:38.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.462+0000 7f46377fe640 1 -- 172.21.15.19:0/1770771686 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f463c009de0 con 0x7f4648103b50 2024-04-23T18:29:38.455 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:29:38.455 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:29:38.456 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:29:38.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.464+0000 7f464c937640 1 -- 172.21.15.19:0/1770771686 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46300a3d20 msgr2=0x7f46300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.464+0000 7f464c937640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46300a3d20 0x7f46300a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f463802a9a0 tx=0x7f4638031400 comp rx=0 tx=0).stop 2024-04-23T18:29:38.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.464+0000 7f464c937640 1 -- 172.21.15.19:0/1770771686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648103b50 msgr2=0x7f4648198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.464+0000 7f464c937640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648103b50 0x7f4648198c80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f463c00d950 tx=0x7f463c00de10 comp rx=0 tx=0).stop 2024-04-23T18:29:38.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.465+0000 7f464c937640 1 -- 172.21.15.19:0/1770771686 shutdown_connections 2024-04-23T18:29:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.465+0000 7f464c937640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f46300a3d20 0x7f46300a61e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.465+0000 7f464c937640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464810c570 0x7f464819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.465+0000 7f464c937640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648103b50 0x7f4648198c80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.465+0000 7f464c937640 1 --2- 172.21.15.19:0/1770771686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4648103230 0x7f4648198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.465+0000 7f464c937640 1 -- 172.21.15.19:0/1770771686 >> 172.21.15.19:0/1770771686 conn(0x7f4648077fb0 msgr2=0x7f46480fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.465+0000 7f464c937640 1 -- 172.21.15.19:0/1770771686 shutdown_connections 2024-04-23T18:29:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.465+0000 7f464c937640 1 -- 172.21.15.19:0/1770771686 wait complete. 2024-04-23T18:29:38.468 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id 2ca0fa3a --release --await' 2024-04-23T18:29:38.468 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id 2ca0fa3a --release --await 2024-04-23T18:29:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.585+0000 7f42c1813640 1 -- 172.21.15.19:0/3644666434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42bc1031d0 msgr2=0x7f42bc1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.585+0000 7f42c1813640 1 --2- 172.21.15.19:0/3644666434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42bc1031d0 0x7f42bc1035b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009860 tx=0x7f42ac02a460 comp rx=0 tx=0).stop 2024-04-23T18:29:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.585+0000 7f42c1813640 1 -- 172.21.15.19:0/3644666434 shutdown_connections 2024-04-23T18:29:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.585+0000 7f42c1813640 1 --2- 172.21.15.19:0/3644666434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42bc10c510 0x7f42bc10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.585+0000 7f42c1813640 1 --2- 172.21.15.19:0/3644666434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc103af0 0x7f42bc10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.585+0000 7f42c1813640 1 --2- 172.21.15.19:0/3644666434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42bc1031d0 0x7f42bc1035b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.585+0000 7f42c1813640 1 -- 172.21.15.19:0/3644666434 >> 172.21.15.19:0/3644666434 conn(0x7f42bc077f50 msgr2=0x7f42bc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.585+0000 7f42c1813640 1 -- 172.21.15.19:0/3644666434 shutdown_connections 2024-04-23T18:29:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.585+0000 7f42c1813640 1 -- 172.21.15.19:0/3644666434 wait complete. 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42c1813640 1 Processor -- start 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42c1813640 1 -- start start 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42c1813640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42bc1031d0 0x7f42bc198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42c1813640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc103af0 0x7f42bc198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42c1813640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42bc10c510 0x7f42bc19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42c1813640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1121f0 con 0x7f42bc103af0 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42c1813640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc112070 con 0x7f42bc1031d0 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42c1813640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc112370 con 0x7f42bc10c510 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42ba7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc103af0 0x7f42bc198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42ba7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc103af0 0x7f42bc198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35150/0 (socket says 172.21.15.19:35150) 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42ba7fc640 1 -- 172.21.15.19:0/2434326304 learned_addr learned my addr 172.21.15.19:0/2434326304 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42baffd640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42bc1031d0 0x7f42bc198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.586+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42bc10c510 0x7f42bc19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42ba7fc640 1 -- 172.21.15.19:0/2434326304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42bc10c510 msgr2=0x7f42bc19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42ba7fc640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42bc10c510 0x7f42bc19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42ba7fc640 1 -- 172.21.15.19:0/2434326304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42bc1031d0 msgr2=0x7f42bc198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42ba7fc640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42bc1031d0 0x7f42bc198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42ba7fc640 1 -- 172.21.15.19:0/2434326304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc19d740 con 0x7f42bc103af0 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42ba7fc640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc103af0 0x7f42bc198c90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f42b000e9c0 tx=0x7f42b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42c0811640 1 -- 172.21.15.19:0/2434326304 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b000cde0 con 0x7f42bc103af0 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42c1813640 1 -- 172.21.15.19:0/2434326304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc19da30 con 0x7f42bc103af0 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42c0811640 1 -- 172.21.15.19:0/2434326304 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b00035f0 con 0x7f42bc103af0 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42c0811640 1 -- 172.21.15.19:0/2434326304 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0010690 con 0x7f42bc103af0 2024-04-23T18:29:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.587+0000 7f42c1813640 1 -- 172.21.15.19:0/2434326304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc1a52c0 con 0x7f42bc103af0 2024-04-23T18:29:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.589+0000 7f42c0811640 1 -- 172.21.15.19:0/2434326304 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f42b0004000 con 0x7f42bc103af0 2024-04-23T18:29:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.589+0000 7f42c4609740 1 Processor -- start 2024-04-23T18:29:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.589+0000 7f42c4609740 1 -- start start 2024-04-23T18:29:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42c4609740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece19770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42c4609740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19cb0 0x5620ece28350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42c4609740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece28890 0x5620ece2ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42c4609740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5620ecd7e6d0 con 0x5620ece19390 2024-04-23T18:29:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42c4609740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5620ecd1b1a0 con 0x5620ece28890 2024-04-23T18:29:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42c4609740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620ecd7a690 con 0x5620ece19cb0 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42c0811640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42a40a3d20 0x7f42a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece19770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece19770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35160/0 (socket says 172.21.15.19:35160) 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42bb7fe640 1 -- 172.21.15.19:0/82001141 learned_addr learned my addr 172.21.15.19:0/82001141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42a40a3d20 0x7f42a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42ba7fc640 1 --2- 172.21.15.19:0/82001141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece28890 0x5620ece2ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42c0811640 1 -- 172.21.15.19:0/2434326304 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f42b00d0b30 con 0x7f42bc103af0 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42baffd640 1 --2- 172.21.15.19:0/82001141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19cb0 0x5620ece28350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42bb7fe640 1 -- 172.21.15.19:0/82001141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19cb0 msgr2=0x5620ece28350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/82001141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19cb0 0x5620ece28350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42bb7fe640 1 -- 172.21.15.19:0/82001141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece28890 msgr2=0x5620ece2ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/82001141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece28890 0x5620ece2ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42bb7fe640 1 -- 172.21.15.19:0/82001141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ecd255b0 con 0x5620ece19390 2024-04-23T18:29:38.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42baffd640 1 --2- 172.21.15.19:0/82001141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19cb0 0x5620ece28350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.590+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/82001141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece19770 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f42b4011da0 tx=0x7f42b400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0fed132cedff33 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42a40a3d20 0x7f42a40a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f42b400fc90 tx=0x7f42b401e000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42a1ffb640 1 -- 172.21.15.19:0/82001141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4022070 con 0x5620ece19390 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42a1ffb640 1 -- 172.21.15.19:0/82001141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b4009dc0 con 0x5620ece19390 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42a1ffb640 1 -- 172.21.15.19:0/82001141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4004050 con 0x5620ece19390 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42c4609740 1 -- 172.21.15.19:0/82001141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 msgr2=0x5620ece19770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42c4609740 1 --2- 172.21.15.19:0/82001141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece19770 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f42b4011da0 tx=0x7f42b400f790 comp rx=0 tx=0).stop 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42c4609740 1 -- 172.21.15.19:0/82001141 shutdown_connections 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42c4609740 1 --2- 172.21.15.19:0/82001141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece28890 0x5620ece2ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42c4609740 1 --2- 172.21.15.19:0/82001141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19cb0 0x5620ece28350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42c4609740 1 --2- 172.21.15.19:0/82001141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece19770 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42c4609740 1 -- 172.21.15.19:0/82001141 >> 172.21.15.19:0/82001141 conn(0x5620ece157b0 msgr2=0x5620ece15b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42c4609740 1 -- 172.21.15.19:0/82001141 shutdown_connections 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.591+0000 7f42c4609740 1 -- 172.21.15.19:0/82001141 wait complete. 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42c4609740 1 -- start start 2024-04-23T18:29:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42c4609740 1 Processor -- start 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42c4609740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19390 0x5620ece28320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42c4609740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece25610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42c4609740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece25d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42c4609740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5620ece2b690 con 0x5620ece28890 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42c4609740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5620ecd18b20 con 0x5620ece19cb0 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42c4609740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620ece2b810 con 0x5620ece19390 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42c4609740 10 client.0 _ll_register_callbacks cb 0x5620ecd18530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece25d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece25d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35174/0 (socket says 172.21.15.19:35174) 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42bb7fe640 1 -- 172.21.15.19:0/2264424242 learned_addr learned my addr 172.21.15.19:0/2264424242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.592+0000 7f42ba7fc640 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece25610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f42baffd640 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19390 0x5620ece28320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f42bb7fe640 1 -- 172.21.15.19:0/2264424242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19390 msgr2=0x5620ece28320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19390 0x5620ece28320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f42bb7fe640 1 -- 172.21.15.19:0/2264424242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 msgr2=0x5620ece25610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece25610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f42bb7fe640 1 -- 172.21.15.19:0/2264424242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620eca90920 con 0x5620ece28890 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece25d00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f42b40099b0 tx=0x7f42b4011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a326b1cbf63de71 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f429affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f429affd640 1 -- 172.21.15.19:0/2264424242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4022030 con 0x5620ece28890 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f429affd640 1 -- 172.21.15.19:0/2264424242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f42b4011910 con 0x5620ece28890 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f42c4609740 1 -- 172.21.15.19:0/2264424242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b4009dc0 con 0x5620ece28890 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f429affd640 1 -- 172.21.15.19:0/2264424242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f42b403c050 con 0x5620ece28890 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f42c4609740 10 client.16131 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f42c4609740 1 -- 172.21.15.19:0/2264424242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620ecb5cd30 con 0x5620ece28890 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f429affd640 1 -- 172.21.15.19:0/2264424242 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b4009dc0 con 0x5620ece28890 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.593+0000 7f429affd640 1 -- 172.21.15.19:0/2264424242 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f42b4005000 con 0x5620ece28890 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.594+0000 7f42c4609740 10 client.16131 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.594+0000 7f42c4609740 10 client.16131 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:29:38.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.594+0000 7f42c4609740 20 client.16131 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.594+0000 7f42c4609740 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5620ece38920 0x5620ece3ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.594+0000 7f42c4609740 4 client.16131 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.594+0000 7f42c4609740 1 -- 172.21.15.19:0/2264424242 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620ecd3c500 con 0x5620ece38920 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.594+0000 7f42baffd640 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5620ece38920 0x5620ece3ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.594+0000 7f42baffd640 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5620ece38920 0x5620ece3ad10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3856ed03446a0a9a server_cookie=edfa8029fa025062 in_seq=0 out_seq=0 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.594+0000 7f429affd640 10 client.16131 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f429affd640 1 -- 172.21.15.19:0/2264424242 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620ecd3c500 con 0x5620ece38920 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f429affd640 10 client.16131 handle_command_reply: tid=0 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 client.16131 shutdown 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 -- 172.21.15.19:0/2264424242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 msgr2=0x5620ece25d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece25d00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f42b40099b0 tx=0x7f42b4011f70 comp rx=0 tx=0).stop 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 -- 172.21.15.19:0/2264424242 shutdown_connections 2024-04-23T18:29:38.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5620ece38920 0x5620ece3ad10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece25610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece25d00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 --2- 172.21.15.19:0/2264424242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19390 0x5620ece28320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 -- 172.21.15.19:0/2264424242 >> 172.21.15.19:0/2264424242 conn(0x5620ece157b0 msgr2=0x5620ece1b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:38.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f429affd640 0 client.16131 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:38.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 -- 172.21.15.19:0/2264424242 shutdown_connections 2024-04-23T18:29:38.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 1 -- 172.21.15.19:0/2264424242 wait complete. 2024-04-23T18:29:38.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.595+0000 7f42c4609740 20 client.16131 trim_cache size 0 max 16384 2024-04-23T18:29:38.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42c4609740 1 Processor -- start 2024-04-23T18:29:38.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42c4609740 1 -- start start 2024-04-23T18:29:38.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42c4609740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece24770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42c4609740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece14850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42c4609740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece28890 0x5620ece27e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42c4609740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1ab9f0 con 0x5620ece19390 2024-04-23T18:29:38.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42c4609740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1ad530 con 0x5620ece19cb0 2024-04-23T18:29:38.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42c4609740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1ac0c0 con 0x5620ece28890 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42baffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece24770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42baffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece24770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35186/0 (socket says 172.21.15.19:35186) 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42baffd640 1 -- 172.21.15.19:0/1446151229 learned_addr learned my addr 172.21.15.19:0/1446151229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/1446151229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece28890 0x5620ece27e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42baffd640 1 -- 172.21.15.19:0/1446151229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece28890 msgr2=0x5620ece27e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42baffd640 1 --2- 172.21.15.19:0/1446151229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece28890 0x5620ece27e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42baffd640 1 -- 172.21.15.19:0/1446151229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 msgr2=0x5620ece14850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42baffd640 1 --2- 172.21.15.19:0/1446151229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece14850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.613+0000 7f42baffd640 1 -- 172.21.15.19:0/1446151229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ece23b60 con 0x5620ece19390 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42baffd640 1 --2- 172.21.15.19:0/1446151229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece24770 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009250 tx=0x7f42ac0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23008b5765add77 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42a17fa640 1 -- 172.21.15.19:0/1446151229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac049030 con 0x5620ece19390 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42a17fa640 1 -- 172.21.15.19:0/1446151229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac045bc0 con 0x5620ece19390 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42a17fa640 1 -- 172.21.15.19:0/1446151229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac045340 con 0x5620ece19390 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42c4609740 1 -- 172.21.15.19:0/1446151229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 msgr2=0x5620ece24770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42c4609740 1 --2- 172.21.15.19:0/1446151229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece24770 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009250 tx=0x7f42ac0058b0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42c4609740 1 -- 172.21.15.19:0/1446151229 shutdown_connections 2024-04-23T18:29:38.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42c4609740 1 --2- 172.21.15.19:0/1446151229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece14850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42c4609740 1 --2- 172.21.15.19:0/1446151229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece28890 0x5620ece27e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42c4609740 1 --2- 172.21.15.19:0/1446151229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece19390 0x5620ece24770 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42c4609740 1 -- 172.21.15.19:0/1446151229 >> 172.21.15.19:0/1446151229 conn(0x5620ece157b0 msgr2=0x5620ece14140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:38.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42c4609740 1 -- 172.21.15.19:0/1446151229 shutdown_connections 2024-04-23T18:29:38.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.614+0000 7f42c4609740 1 -- 172.21.15.19:0/1446151229 wait complete. 2024-04-23T18:29:38.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42c4609740 1 -- start start 2024-04-23T18:29:38.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42c4609740 1 Processor -- start 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42c4609740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19390 0x5620ece30e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42c4609740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42c4609740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece29900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42c4609740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1ace60 con 0x5620ece28890 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42c4609740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1ac790 con 0x5620ece19cb0 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42c4609740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1a99d0 con 0x5620ece19390 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42c4609740 10 client.0 _ll_register_callbacks cb 0x7f42b4004050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece29900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece29900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35188/0 (socket says 172.21.15.19:35188) 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42bb7fe640 1 -- 172.21.15.19:0/2141488833 learned_addr learned my addr 172.21.15.19:0/2141488833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42baffd640 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19390 0x5620ece30e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42bb7fe640 1 -- 172.21.15.19:0/2141488833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19390 msgr2=0x5620ece30e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19390 0x5620ece30e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42bb7fe640 1 -- 172.21.15.19:0/2141488833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 msgr2=0x5620ece293c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.615+0000 7f42ba7fc640 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece293c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42bb7fe640 1 -- 172.21.15.19:0/2141488833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ece2a110 con 0x5620ece28890 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42ba7fc640 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42bb7fe640 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece29900 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f42b4013010 tx=0x7f42b4009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7717eb4faa7f844 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f429affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f429affd640 1 -- 172.21.15.19:0/2141488833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4022040 con 0x5620ece28890 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f429affd640 1 -- 172.21.15.19:0/2141488833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f42b402b040 con 0x5620ece28890 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42c4609740 1 -- 172.21.15.19:0/2141488833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ac045bc0 con 0x5620ece28890 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f429affd640 1 -- 172.21.15.19:0/2141488833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f42b4003970 con 0x5620ece28890 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42c4609740 10 client.16137 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42c4609740 1 -- 172.21.15.19:0/2141488833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620ece9ee60 con 0x5620ece28890 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f429affd640 1 -- 172.21.15.19:0/2141488833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac045bc0 con 0x5620ece28890 2024-04-23T18:29:38.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f429affd640 1 -- 172.21.15.19:0/2141488833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f42b4038450 con 0x5620ece28890 2024-04-23T18:29:38.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42c4609740 10 client.16137 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:38.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42c4609740 10 client.16137 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:29:38.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42c4609740 20 client.16137 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:38.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42c4609740 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5620ece9f560 0x5620ecea1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:38.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42c4609740 4 client.16137 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2ca0fa3a", "release": true, "await": true}] 2024-04-23T18:29:38.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.616+0000 7f42c4609740 1 -- 172.21.15.19:0/2141488833 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2ca0fa3a", "release": true, "await": true}) -- 0x5620ecd8c0f0 con 0x5620ece9f560 2024-04-23T18:29:38.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.617+0000 7f42baffd640 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5620ece9f560 0x5620ecea1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:38.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.617+0000 7f42baffd640 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5620ece9f560 0x5620ecea1980 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8db417381e1c048 server_cookie=2d3c6fe680bbfdd4 in_seq=0 out_seq=0 2024-04-23T18:29:38.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.617+0000 7f429affd640 10 client.16137 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:38.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f429affd640 1 -- 172.21.15.19:0/2141488833 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5620ecd8c0f0 con 0x5620ece9f560 2024-04-23T18:29:38.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f429affd640 10 client.16137 handle_command_reply: tid=0 2024-04-23T18:29:38.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 client.16137 shutdown 2024-04-23T18:29:38.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 -- 172.21.15.19:0/2141488833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 msgr2=0x5620ece29900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece29900 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f42b4013010 tx=0x7f42b4009fd0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 -- 172.21.15.19:0/2141488833 shutdown_connections 2024-04-23T18:29:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5620ece9f560 0x5620ecea1980 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5620ece19cb0 0x5620ece293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f429affd640 0 client.16137 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620ece28890 0x5620ece29900 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 --2- 172.21.15.19:0/2141488833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620ece19390 0x5620ece30e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 -- 172.21.15.19:0/2141488833 >> 172.21.15.19:0/2141488833 conn(0x5620ece157b0 msgr2=0x5620ece1b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 -- 172.21.15.19:0/2141488833 shutdown_connections 2024-04-23T18:29:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 1 -- 172.21.15.19:0/2141488833 wait complete. 2024-04-23T18:29:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.621+0000 7f42c4609740 20 client.16137 trim_cache size 0 max 16384 2024-04-23T18:29:38.616 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 20, 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "2ca0fa3a": { 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 20, 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 77.7, 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:29:38.617 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:29:38.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:29:38.618 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:29:38.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:38.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:38.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:38.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:38.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:29:38.618 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 -- 172.21.15.19:0/2434326304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42a40a3d20 msgr2=0x7f42a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42a40a3d20 0x7f42a40a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f42b400fc90 tx=0x7f42b401e000 comp rx=0 tx=0).stop 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 -- 172.21.15.19:0/2434326304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc103af0 msgr2=0x7f42bc198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc103af0 0x7f42bc198c90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f42b000e9c0 tx=0x7f42b000ee80 comp rx=0 tx=0).stop 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 -- 172.21.15.19:0/2434326304 shutdown_connections 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42a40a3d20 0x7f42a40a61e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42bc10c510 0x7f42bc19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc103af0 0x7f42bc198c90 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 --2- 172.21.15.19:0/2434326304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42bc1031d0 0x7f42bc198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 -- 172.21.15.19:0/2434326304 >> 172.21.15.19:0/2434326304 conn(0x7f42bc077f50 msgr2=0x7f42bc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 -- 172.21.15.19:0/2434326304 shutdown_connections 2024-04-23T18:29:38.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:29:38.622+0000 7f42c1813640 1 -- 172.21.15.19:0/2434326304 wait complete. 2024-04-23T18:29:38.625 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2ca0fa3a', seconds elapsed: 18.7 2024-04-23T18:29:39.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:39 smithi044 ceph-mon[32458]: pgmap v108: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-04-23T18:29:39.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:39 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1770771686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:39.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:39 smithi155 ceph-mon[33468]: pgmap v108: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-04-23T18:29:39.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:39 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1770771686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:39.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:39 smithi019 ceph-mon[29215]: pgmap v108: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-04-23T18:29:39.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:39 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1770771686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 -- 172.21.15.19:0/3789239620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6810c520 msgr2=0x7f3f6810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 --2- 172.21.15.19:0/3789239620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6810c520 0x7f3f6810e9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009ca0 tx=0x7f3f6002aa00 comp rx=0 tx=0).stop 2024-04-23T18:29:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 -- 172.21.15.19:0/3789239620 shutdown_connections 2024-04-23T18:29:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 --2- 172.21.15.19:0/3789239620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f68103b00 0x7f3f6810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 --2- 172.21.15.19:0/3789239620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6810c520 0x7f3f6810e9c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 --2- 172.21.15.19:0/3789239620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f681031e0 0x7f3f681035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 -- 172.21.15.19:0/3789239620 >> 172.21.15.19:0/3789239620 conn(0x7f3f68077fd0 msgr2=0x7f3f680783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 -- 172.21.15.19:0/3789239620 shutdown_connections 2024-04-23T18:29:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 -- 172.21.15.19:0/3789239620 wait complete. 2024-04-23T18:29:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 Processor -- start 2024-04-23T18:29:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.279+0000 7f3f6e435640 1 -- start start 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f6e435640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f681031e0 0x7f3f68100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f6e435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f68103b00 0x7f3f68101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f6e435640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f6810c520 0x7f3f681018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f6e435640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68112230 con 0x7f3f68103b00 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f6e435640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681120b0 con 0x7f3f681031e0 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f6e435640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681123b0 con 0x7f3f6810c520 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f68103b00 0x7f3f68101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f68103b00 0x7f3f68101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36792/0 (socket says 172.21.15.19:36792) 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f677fe640 1 -- 172.21.15.19:0/1959504768 learned_addr learned my addr 172.21.15.19:0/1959504768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f67fff640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f681031e0 0x7f3f68100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f677fe640 1 -- 172.21.15.19:0/1959504768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f6810c520 msgr2=0x7f3f681018b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T18:29:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f677fe640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f6810c520 0x7f3f681018b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f677fe640 1 -- 172.21.15.19:0/1959504768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f681031e0 msgr2=0x7f3f68100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f677fe640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f681031e0 0x7f3f68100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f677fe640 1 -- 172.21.15.19:0/1959504768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f681a6bf0 con 0x7f3f68103b00 2024-04-23T18:29:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.280+0000 7f3f67fff640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f681031e0 0x7f3f68100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.281+0000 7f3f677fe640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f68103b00 0x7f3f68101370 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00d950 tx=0x7f3f5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.281+0000 7f3f657fa640 1 -- 172.21.15.19:0/1959504768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c017070 con 0x7f3f68103b00 2024-04-23T18:29:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.281+0000 7f3f657fa640 1 -- 172.21.15.19:0/1959504768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5c009c30 con 0x7f3f68103b00 2024-04-23T18:29:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.281+0000 7f3f657fa640 1 -- 172.21.15.19:0/1959504768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00f620 con 0x7f3f68103b00 2024-04-23T18:29:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.281+0000 7f3f6e435640 1 -- 172.21.15.19:0/1959504768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f681a6ee0 con 0x7f3f68103b00 2024-04-23T18:29:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.281+0000 7f3f6e435640 1 -- 172.21.15.19:0/1959504768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f681a73a0 con 0x7f3f68103b00 2024-04-23T18:29:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.283+0000 7f3f657fa640 1 -- 172.21.15.19:0/1959504768 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3f5c010460 con 0x7f3f68103b00 2024-04-23T18:29:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f7122d740 1 Processor -- start 2024-04-23T18:29:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f7122d740 1 -- start start 2024-04-23T18:29:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e85875a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85875f40 0x563e858802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85880800 0x563e85882bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f7122d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563e857da970 con 0x563e85875f40 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f7122d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563e857774a0 con 0x563e85875620 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f7122d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e857d6930 con 0x563e85880800 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85875f40 0x563e858802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f657fa640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3f500a3df0 0x7f3f500a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85875f40 0x563e858802c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36808/0 (socket says 172.21.15.19:36808) 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f67fff640 1 -- 172.21.15.19:0/1815241364 learned_addr learned my addr 172.21.15.19:0/1815241364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/1815241364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e85875a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3f500a3df0 0x7f3f500a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.284+0000 7f3f657fa640 1 -- 172.21.15.19:0/1959504768 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3f5c0d04e0 con 0x7f3f68103b00 2024-04-23T18:29:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.285+0000 7f3f67fff640 1 -- 172.21.15.19:0/1815241364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85880800 msgr2=0x563e85882bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.285+0000 7f3f67fff640 1 --2- 172.21.15.19:0/1815241364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85880800 0x563e85882bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.285+0000 7f3f67fff640 1 -- 172.21.15.19:0/1815241364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 msgr2=0x563e85875a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.285+0000 7f3f67fff640 1 --2- 172.21.15.19:0/1815241364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e85875a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.285+0000 7f3f67fff640 1 -- 172.21.15.19:0/1815241364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e854ed210 con 0x563e85875f40 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.285+0000 7f3f67fff640 1 --2- 172.21.15.19:0/1815241364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85875f40 0x563e858802c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800a9c0 tx=0x7f3f5800ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dbad151d148bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.285+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3f500a3df0 0x7f3f500a62b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3f6002af20 tx=0x7f3f6002aa00 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.285+0000 7f3f4e7fc640 1 -- 172.21.15.19:0/1815241364 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58015070 con 0x563e85875f40 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.285+0000 7f3f4e7fc640 1 -- 172.21.15.19:0/1815241364 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5800cc60 con 0x563e85875f40 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.285+0000 7f3f4e7fc640 1 -- 172.21.15.19:0/1815241364 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f580185c0 con 0x563e85875f40 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 -- 172.21.15.19:0/1815241364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85875f40 msgr2=0x563e858802c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 --2- 172.21.15.19:0/1815241364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85875f40 0x563e858802c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800a9c0 tx=0x7f3f5800ae80 comp rx=0 tx=0).stop 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 -- 172.21.15.19:0/1815241364 shutdown_connections 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 --2- 172.21.15.19:0/1815241364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85880800 0x563e85882bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 --2- 172.21.15.19:0/1815241364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85875f40 0x563e858802c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 --2- 172.21.15.19:0/1815241364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e85875a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 -- 172.21.15.19:0/1815241364 >> 172.21.15.19:0/1815241364 conn(0x563e85871b30 msgr2=0x563e85871f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 -- 172.21.15.19:0/1815241364 shutdown_connections 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 -- 172.21.15.19:0/1815241364 wait complete. 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 -- start start 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.286+0000 7f3f7122d740 1 Processor -- start 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e858801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875f40 0x563e8587d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e8587dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f7122d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563e85883430 con 0x563e85880800 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f7122d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563e857750b0 con 0x563e85875620 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f7122d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e858835b0 con 0x563e85875f40 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f7122d740 10 client.0 _ll_register_callbacks cb 0x563e85762d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f6c9d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e8587dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f6c9d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e8587dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36814/0 (socket says 172.21.15.19:36814) 2024-04-23T18:29:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/4046079111 learned_addr learned my addr 172.21.15.19:0/4046079111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f67fff640 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e858801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/4046079111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875f40 msgr2=0x563e8587d4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875f40 0x563e8587d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/4046079111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 msgr2=0x563e858801e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e858801e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/4046079111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e855b9620 con 0x563e85880800 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e8587dae0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3f60035000 tx=0x7f3f600095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34bdf330728526f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.287+0000 7f3f477fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f477fe640 1 -- 172.21.15.19:0/4046079111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60033040 con 0x563e85880800 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f477fe640 1 -- 172.21.15.19:0/4046079111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3f60003b70 con 0x563e85880800 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f7122d740 1 -- 172.21.15.19:0/4046079111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f5800cc60 con 0x563e85880800 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f477fe640 1 -- 172.21.15.19:0/4046079111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f6005a050 con 0x563e85880800 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f7122d740 10 client.16149 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f7122d740 1 -- 172.21.15.19:0/4046079111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e8588f3e0 con 0x563e85880800 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f477fe640 1 -- 172.21.15.19:0/4046079111 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5800cc60 con 0x563e85880800 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f477fe640 1 -- 172.21.15.19:0/4046079111 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3f60046050 con 0x563e85880800 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f7122d740 10 client.16149 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f7122d740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f7122d740 20 client.16149 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f7122d740 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563e858908a0 0x563e85892cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f7122d740 4 client.16149 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.288+0000 7f3f7122d740 1 -- 172.21.15.19:0/4046079111 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e857e8390 con 0x563e858908a0 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.289+0000 7f3f67fff640 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563e858908a0 0x563e85892cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.289+0000 7f3f67fff640 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563e858908a0 0x563e85892cf0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec627658b71592d server_cookie=641522a43ed79e4f in_seq=0 out_seq=0 2024-04-23T18:29:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.289+0000 7f3f477fe640 10 client.16149 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f477fe640 1 -- 172.21.15.19:0/4046079111 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e857e8390 con 0x563e858908a0 2024-04-23T18:29:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f477fe640 10 client.16149 handle_command_reply: tid=0 2024-04-23T18:29:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 client.16149 shutdown 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 -- 172.21.15.19:0/4046079111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 msgr2=0x563e8587dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e8587dae0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3f60035000 tx=0x7f3f600095c0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 -- 172.21.15.19:0/4046079111 shutdown_connections 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563e858908a0 0x563e85892cf0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e8587dae0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875f40 0x563e8587d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f477fe640 0 client.16149 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 --2- 172.21.15.19:0/4046079111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e858801e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 -- 172.21.15.19:0/4046079111 >> 172.21.15.19:0/4046079111 conn(0x563e85871b30 msgr2=0x563e858778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 -- 172.21.15.19:0/4046079111 shutdown_connections 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 1 -- 172.21.15.19:0/4046079111 wait complete. 2024-04-23T18:29:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.290+0000 7f3f7122d740 20 client.16149 trim_cache size 0 max 16384 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f7122d740 1 Processor -- start 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f7122d740 1 -- start start 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e8588c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875f40 0x563e8587fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e85898600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f7122d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681afcc0 con 0x563e85880800 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f7122d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681b1800 con 0x563e85875620 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f7122d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681b0390 con 0x563e85875f40 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f6c9d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e85898600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f6c9d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e85898600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36816/0 (socket says 172.21.15.19:36816) 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.306+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/438222939 learned_addr learned my addr 172.21.15.19:0/438222939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f67fff640 1 --2- 172.21.15.19:0/438222939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e8588c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/438222939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875f40 msgr2=0x563e8587fd10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/438222939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875f40 0x563e8587fd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/438222939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 msgr2=0x563e8588c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/438222939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e8588c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/438222939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e85882e20 con 0x563e85880800 2024-04-23T18:29:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f67fff640 1 --2- 172.21.15.19:0/438222939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e8588c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/438222939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e85898600 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3f6002b1f0 tx=0x7f3f6002b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526690744d92ba4c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f4dffb640 1 -- 172.21.15.19:0/438222939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60033070 con 0x563e85880800 2024-04-23T18:29:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f4dffb640 1 -- 172.21.15.19:0/438222939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f60036e10 con 0x563e85880800 2024-04-23T18:29:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.307+0000 7f3f4dffb640 1 -- 172.21.15.19:0/438222939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600043d0 con 0x563e85880800 2024-04-23T18:29:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 -- 172.21.15.19:0/438222939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 msgr2=0x563e85898600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 --2- 172.21.15.19:0/438222939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e85898600 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3f6002b1f0 tx=0x7f3f6002b180 comp rx=0 tx=0).stop 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 -- 172.21.15.19:0/438222939 shutdown_connections 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 --2- 172.21.15.19:0/438222939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e85898600 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 --2- 172.21.15.19:0/438222939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875f40 0x563e8587fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 --2- 172.21.15.19:0/438222939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875620 0x563e8588c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 -- 172.21.15.19:0/438222939 >> 172.21.15.19:0/438222939 conn(0x563e85871b30 msgr2=0x563e85878af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 -- 172.21.15.19:0/438222939 shutdown_connections 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 -- 172.21.15.19:0/438222939 wait complete. 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 -- start start 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.308+0000 7f3f7122d740 1 Processor -- start 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875620 0x563e85888d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875f40 0x563e858840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f7122d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e858847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f7122d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681b1130 con 0x563e85880800 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f7122d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681b0a60 con 0x563e85875f40 2024-04-23T18:29:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f7122d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681adca0 con 0x563e85875620 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f7122d740 10 client.0 _ll_register_callbacks cb 0x7f3f6002e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f6c9d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e858847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f6c9d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e858847c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36830/0 (socket says 172.21.15.19:36830) 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f677fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875f40 0x563e858840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/3135962157 learned_addr learned my addr 172.21.15.19:0/3135962157 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/3135962157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875620 msgr2=0x563e85888d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875620 0x563e85888d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/3135962157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875f40 msgr2=0x563e858840d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875f40 0x563e858840d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f6c9d2640 1 -- 172.21.15.19:0/3135962157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e858f5e70 con 0x563e85880800 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f677fe640 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875f40 0x563e858840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f6c9d2640 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e858847c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3f600366f0 tx=0x7f3f60003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc85b0f2283a12f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.309+0000 7f3f477fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f477fe640 1 -- 172.21.15.19:0/3135962157 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60033040 con 0x563e85880800 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f477fe640 1 -- 172.21.15.19:0/3135962157 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3f60004170 con 0x563e85880800 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f7122d740 1 -- 172.21.15.19:0/3135962157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f60036e10 con 0x563e85880800 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f477fe640 1 -- 172.21.15.19:0/3135962157 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f60003b20 con 0x563e85880800 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f7122d740 10 client.16155 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f7122d740 1 -- 172.21.15.19:0/3135962157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e858f7020 con 0x563e85880800 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f477fe640 1 -- 172.21.15.19:0/3135962157 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f60036e10 con 0x563e85880800 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f477fe640 1 -- 172.21.15.19:0/3135962157 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3f60046040 con 0x563e85880800 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f7122d740 10 client.16155 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f7122d740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f7122d740 20 client.16155 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f7122d740 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563e858f76d0 0x563e858f9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f7122d740 4 client.16155 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.310+0000 7f3f7122d740 1 -- 172.21.15.19:0/3135962157 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e857e2200 con 0x563e858f76d0 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.311+0000 7f3f67fff640 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563e858f76d0 0x563e858f9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.311+0000 7f3f67fff640 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563e858f76d0 0x563e858f9ac0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92804be8fff08e2b server_cookie=c9d73476f8116eed in_seq=0 out_seq=0 2024-04-23T18:29:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.311+0000 7f3f477fe640 10 client.16155 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.322+0000 7f3f477fe640 1 -- 172.21.15.19:0/3135962157 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e857e2200 con 0x563e858f76d0 2024-04-23T18:29:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.322+0000 7f3f477fe640 10 client.16155 handle_command_reply: tid=0 2024-04-23T18:29:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.322+0000 7f3f7122d740 1 client.16155 shutdown 2024-04-23T18:29:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.322+0000 7f3f7122d740 1 -- 172.21.15.19:0/3135962157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 msgr2=0x563e858847c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.322+0000 7f3f7122d740 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e858847c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3f600366f0 tx=0x7f3f60003970 comp rx=0 tx=0).stop 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.322+0000 7f3f7122d740 1 -- 172.21.15.19:0/3135962157 shutdown_connections 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.322+0000 7f3f7122d740 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563e858f76d0 0x563e858f9ac0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f7122d740 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e85875f40 0x563e858840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f477fe640 0 client.16155 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f7122d740 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e85880800 0x563e858847c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f7122d740 1 --2- 172.21.15.19:0/3135962157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e85875620 0x563e85888d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f7122d740 1 -- 172.21.15.19:0/3135962157 >> 172.21.15.19:0/3135962157 conn(0x563e85871b30 msgr2=0x563e8588b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f7122d740 1 -- 172.21.15.19:0/3135962157 shutdown_connections 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f7122d740 1 -- 172.21.15.19:0/3135962157 wait complete. 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f7122d740 20 client.16155 trim_cache size 0 max 16384 2024-04-23T18:29:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:29:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:29:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:29:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:29:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 -- 172.21.15.19:0/1959504768 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3f500a3df0 msgr2=0x7f3f500a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3f500a3df0 0x7f3f500a62b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3f6002af20 tx=0x7f3f6002aa00 comp rx=0 tx=0).stop 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 -- 172.21.15.19:0/1959504768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f68103b00 msgr2=0x7f3f68101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f68103b00 0x7f3f68101370 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00d950 tx=0x7f3f5c00de10 comp rx=0 tx=0).stop 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 -- 172.21.15.19:0/1959504768 shutdown_connections 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3f500a3df0 0x7f3f500a62b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f6810c520 0x7f3f681018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f68103b00 0x7f3f68101370 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 --2- 172.21.15.19:0/1959504768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f681031e0 0x7f3f68100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 -- 172.21.15.19:0/1959504768 >> 172.21.15.19:0/1959504768 conn(0x7f3f68077fd0 msgr2=0x7f3f680fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.323+0000 7f3f6e435640 1 -- 172.21.15.19:0/1959504768 shutdown_connections 2024-04-23T18:29:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:41.324+0000 7f3f6e435640 1 -- 172.21.15.19:0/1959504768 wait complete. 2024-04-23T18:29:41.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:41 smithi044 ceph-mon[32458]: pgmap v109: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-23T18:29:41.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:41 smithi155 ceph-mon[33468]: pgmap v109: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-23T18:29:41.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:41 smithi019 ceph-mon[29215]: pgmap v109: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-23T18:29:43.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:43 smithi044 ceph-mon[32458]: pgmap v110: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-23T18:29:43.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:43 smithi155 ceph-mon[33468]: pgmap v110: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-23T18:29:43.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:43 smithi019 ceph-mon[29215]: pgmap v110: 97 pgs: 97 active+clean; 3.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-23T18:29:45.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:45 smithi044 ceph-mon[32458]: pgmap v111: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-23T18:29:45.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:45 smithi155 ceph-mon[33468]: pgmap v111: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-23T18:29:45.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:45 smithi019 ceph-mon[29215]: pgmap v111: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-23T18:29:47.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:47 smithi044 ceph-mon[32458]: pgmap v112: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:29:47.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:47 smithi155 ceph-mon[33468]: pgmap v112: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:29:47.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:47 smithi019 ceph-mon[29215]: pgmap v112: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:29:49.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:49 smithi044 ceph-mon[32458]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:29:49.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:49 smithi155 ceph-mon[33468]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:29:49.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:49 smithi019 ceph-mon[29215]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:29:51.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:51 smithi044 ceph-mon[32458]: pgmap v114: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 19 op/s 2024-04-23T18:29:51.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:51 smithi155 ceph-mon[33468]: pgmap v114: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 19 op/s 2024-04-23T18:29:51.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:51 smithi019 ceph-mon[29215]: pgmap v114: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 19 op/s 2024-04-23T18:29:53.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:53 smithi044 ceph-mon[32458]: pgmap v115: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-23T18:29:53.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:53 smithi155 ceph-mon[33468]: pgmap v115: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-23T18:29:53.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:53 smithi019 ceph-mon[29215]: pgmap v115: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-23T18:29:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.423+0000 7fa64364e640 1 -- 172.21.15.19:0/2773596207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa63c1010c0 msgr2=0x7fa63c1014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.423+0000 7fa64364e640 1 --2- 172.21.15.19:0/2773596207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa63c1010c0 0x7fa63c1014a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa638009900 tx=0x7fa63802a410 comp rx=0 tx=0).stop 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.423+0000 7fa64364e640 1 -- 172.21.15.19:0/2773596207 shutdown_connections 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.423+0000 7fa64364e640 1 --2- 172.21.15.19:0/2773596207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c106190 0x7fa63c10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.423+0000 7fa64364e640 1 --2- 172.21.15.19:0/2773596207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63c101a70 0x7fa63c105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.423+0000 7fa64364e640 1 --2- 172.21.15.19:0/2773596207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa63c1010c0 0x7fa63c1014a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.423+0000 7fa64364e640 1 -- 172.21.15.19:0/2773596207 >> 172.21.15.19:0/2773596207 conn(0x7fa63c077fd0 msgr2=0x7fa63c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.424+0000 7fa64364e640 1 -- 172.21.15.19:0/2773596207 shutdown_connections 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.424+0000 7fa64364e640 1 -- 172.21.15.19:0/2773596207 wait complete. 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.424+0000 7fa64364e640 1 Processor -- start 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.424+0000 7fa64364e640 1 -- start start 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.424+0000 7fa64364e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63c1010c0 0x7fa63c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.424+0000 7fa64364e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa63c106190 0x7fa63c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.424+0000 7fa64364e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c198590 0x7fa63c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.424+0000 7fa64364e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c112250 con 0x7fa63c106190 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.424+0000 7fa64364e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c1120d0 con 0x7fa63c198590 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.424+0000 7fa64364e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c1123d0 con 0x7fa63c1010c0 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa640be9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa63c106190 0x7fa63c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa640be9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa63c106190 0x7fa63c19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51296/0 (socket says 172.21.15.19:51296) 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa640be9640 1 -- 172.21.15.19:0/3402289479 learned_addr learned my addr 172.21.15.19:0/3402289479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa641beb640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c198590 0x7fa63c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa6413ea640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63c1010c0 0x7fa63c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa640be9640 1 -- 172.21.15.19:0/3402289479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63c1010c0 msgr2=0x7fa63c19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa640be9640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63c1010c0 0x7fa63c19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa640be9640 1 -- 172.21.15.19:0/3402289479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c198590 msgr2=0x7fa63c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa640be9640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c198590 0x7fa63c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa640be9640 1 -- 172.21.15.19:0/3402289479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63c1990b0 con 0x7fa63c106190 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa640be9640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa63c106190 0x7fa63c19e920 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00b450 tx=0x7fa62c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa6327fc640 1 -- 172.21.15.19:0/3402289479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c015020 con 0x7fa63c106190 2024-04-23T18:29:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa6327fc640 1 -- 172.21.15.19:0/3402289479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62c011d40 con 0x7fa63c106190 2024-04-23T18:29:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa64364e640 1 -- 172.21.15.19:0/3402289479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63c1993a0 con 0x7fa63c106190 2024-04-23T18:29:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa6327fc640 1 -- 172.21.15.19:0/3402289479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00f340 con 0x7fa63c106190 2024-04-23T18:29:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.425+0000 7fa64364e640 1 -- 172.21.15.19:0/3402289479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63c1a55f0 con 0x7fa63c106190 2024-04-23T18:29:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.428+0000 7fa6327fc640 1 -- 172.21.15.19:0/3402289479 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa62c018020 con 0x7fa63c106190 2024-04-23T18:29:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.428+0000 7fa646367740 1 Processor -- start 2024-04-23T18:29:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.428+0000 7fa646367740 1 -- start start 2024-04-23T18:29:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.428+0000 7fa646367740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1d620 0x557a86f1da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.428+0000 7fa646367740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1df40 0x557a86f282c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.428+0000 7fa646367740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f2abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.428+0000 7fa646367740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a86e82970 con 0x557a86f1df40 2024-04-23T18:29:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.428+0000 7fa646367740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557a86e1f4a0 con 0x557a86f28800 2024-04-23T18:29:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.428+0000 7fa646367740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557a86e7e930 con 0x557a86f1d620 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.428+0000 7fa6327fc640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6280a3e40 0x7fa6280a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6413ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1df40 0x557a86f282c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6413ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1df40 0x557a86f282c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51312/0 (socket says 172.21.15.19:51312) 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6413ea640 1 -- 172.21.15.19:0/3743000473 learned_addr learned my addr 172.21.15.19:0/3743000473 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa640be9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f2abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa641beb640 1 --2- 172.21.15.19:0/3743000473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1d620 0x557a86f1da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa641beb640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6280a3e40 0x7fa6280a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6327fc640 1 -- 172.21.15.19:0/3402289479 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa62c0d0110 con 0x7fa63c106190 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6413ea640 1 -- 172.21.15.19:0/3743000473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1d620 msgr2=0x557a86f1da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6413ea640 1 --2- 172.21.15.19:0/3743000473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1d620 0x557a86f1da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6413ea640 1 -- 172.21.15.19:0/3743000473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 msgr2=0x557a86f2abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6413ea640 1 --2- 172.21.15.19:0/3743000473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f2abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6413ea640 1 -- 172.21.15.19:0/3743000473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a86b95210 con 0x557a86f1df40 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa640be9640 1 --2- 172.21.15.19:0/3743000473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f2abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa641beb640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6280a3e40 0x7fa6280a6300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa634009fd0 tx=0x7fa634009600 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6413ea640 1 --2- 172.21.15.19:0/3743000473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1df40 0x557a86f282c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa63802a930 tx=0x7fa638032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90213fcead7ff47 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6277fe640 1 -- 172.21.15.19:0/3743000473 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63803f070 con 0x557a86f1df40 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6277fe640 1 -- 172.21.15.19:0/3743000473 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6380316a0 con 0x557a86f1df40 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.429+0000 7fa6277fe640 1 -- 172.21.15.19:0/3743000473 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638035070 con 0x557a86f1df40 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.430+0000 7fa646367740 1 -- 172.21.15.19:0/3743000473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1df40 msgr2=0x557a86f282c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.430+0000 7fa646367740 1 --2- 172.21.15.19:0/3743000473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1df40 0x557a86f282c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa63802a930 tx=0x7fa638032ce0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.430+0000 7fa646367740 1 -- 172.21.15.19:0/3743000473 shutdown_connections 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.430+0000 7fa646367740 1 --2- 172.21.15.19:0/3743000473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f2abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.430+0000 7fa646367740 1 --2- 172.21.15.19:0/3743000473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1df40 0x557a86f282c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.430+0000 7fa646367740 1 --2- 172.21.15.19:0/3743000473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1d620 0x557a86f1da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.430+0000 7fa646367740 1 -- 172.21.15.19:0/3743000473 >> 172.21.15.19:0/3743000473 conn(0x557a86f19b30 msgr2=0x557a86f19f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.430+0000 7fa646367740 1 -- 172.21.15.19:0/3743000473 shutdown_connections 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.430+0000 7fa646367740 1 -- 172.21.15.19:0/3743000473 wait complete. 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa646367740 1 -- start start 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa646367740 1 Processor -- start 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa646367740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa646367740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1df40 0x557a86f254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa646367740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f25a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa646367740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a86f2b3d0 con 0x557a86f1d620 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa646367740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557a86e1d0b0 con 0x557a86f28800 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa646367740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557a86f2b550 con 0x557a86f1df40 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa646367740 10 client.0 _ll_register_callbacks cb 0x557a86e0ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa6413ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa6413ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f281b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51318/0 (socket says 172.21.15.19:51318) 2024-04-23T18:29:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa6413ea640 1 -- 172.21.15.19:0/2384803792 learned_addr learned my addr 172.21.15.19:0/2384803792 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.431+0000 7fa640be9640 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1df40 0x557a86f254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa641beb640 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f25a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa6413ea640 1 -- 172.21.15.19:0/2384803792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1df40 msgr2=0x557a86f254a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa6413ea640 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1df40 0x557a86f254a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa6413ea640 1 -- 172.21.15.19:0/2384803792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 msgr2=0x557a86f25a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa6413ea640 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f25a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa6413ea640 1 -- 172.21.15.19:0/2384803792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a86c61620 con 0x557a86f1d620 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa641beb640 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f25a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa6413ea640 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f281b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa63802a450 tx=0x7fa638032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbdb1affec3318e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa624ff9640 1 -- 172.21.15.19:0/2384803792 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63804b060 con 0x557a86f1d620 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa624ff9640 1 -- 172.21.15.19:0/2384803792 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa638004440 con 0x557a86f1d620 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa646367740 1 -- 172.21.15.19:0/2384803792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6380316a0 con 0x557a86f1d620 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa624ff9640 1 -- 172.21.15.19:0/2384803792 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa63803f030 con 0x557a86f1d620 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa646367740 10 client.16167 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa646367740 1 -- 172.21.15.19:0/2384803792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a86f37340 con 0x557a86f1d620 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.432+0000 7fa624ff9640 1 -- 172.21.15.19:0/2384803792 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6380316a0 con 0x557a86f1d620 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.433+0000 7fa624ff9640 1 -- 172.21.15.19:0/2384803792 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa638004a60 con 0x557a86f1d620 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.433+0000 7fa646367740 10 client.16167 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.433+0000 7fa646367740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.433+0000 7fa646367740 20 client.16167 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.433+0000 7fa646367740 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a86f38690 0x557a86f3aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.433+0000 7fa646367740 4 client.16167 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.433+0000 7fa646367740 1 -- 172.21.15.19:0/2384803792 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a86e90390 con 0x557a86f38690 2024-04-23T18:29:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.433+0000 7fa640be9640 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a86f38690 0x557a86f3aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.433+0000 7fa624ff9640 10 client.16167 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.433+0000 7fa640be9640 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a86f38690 0x557a86f3aa80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9fa925d9b578732 server_cookie=3d120499df90e475 in_seq=0 out_seq=0 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa624ff9640 1 -- 172.21.15.19:0/2384803792 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a86e90390 con 0x557a86f38690 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa624ff9640 10 client.16167 handle_command_reply: tid=0 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 client.16167 shutdown 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 -- 172.21.15.19:0/2384803792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 msgr2=0x557a86f281b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f281b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa63802a450 tx=0x7fa638032f40 comp rx=0 tx=0).stop 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 -- 172.21.15.19:0/2384803792 shutdown_connections 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a86f38690 0x557a86f3aa80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f25a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa624ff9640 0 client.16167 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1df40 0x557a86f254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 --2- 172.21.15.19:0/2384803792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f281b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 -- 172.21.15.19:0/2384803792 >> 172.21.15.19:0/2384803792 conn(0x557a86f19b30 msgr2=0x557a86f1f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 -- 172.21.15.19:0/2384803792 shutdown_connections 2024-04-23T18:29:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.434+0000 7fa646367740 1 -- 172.21.15.19:0/2384803792 wait complete. 2024-04-23T18:29:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.435+0000 7fa646367740 20 client.16167 trim_cache size 0 max 16384 2024-04-23T18:29:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa646367740 1 Processor -- start 2024-04-23T18:29:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa646367740 1 -- start start 2024-04-23T18:29:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa646367740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f34220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa646367740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f1df40 0x557a86f30b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa646367740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f28800 0x557a86f40020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa646367740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c1ada20 con 0x557a86f1d620 2024-04-23T18:29:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa646367740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c1af560 con 0x557a86f1df40 2024-04-23T18:29:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa646367740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c1ae0f0 con 0x557a86f28800 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa6413ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f34220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa6413ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f34220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51334/0 (socket says 172.21.15.19:51334) 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa6413ea640 1 -- 172.21.15.19:0/2870884450 learned_addr learned my addr 172.21.15.19:0/2870884450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa640be9640 1 --2- 172.21.15.19:0/2870884450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f1df40 0x557a86f30b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa6413ea640 1 -- 172.21.15.19:0/2870884450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f28800 msgr2=0x557a86f40020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa6413ea640 1 --2- 172.21.15.19:0/2870884450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f28800 0x557a86f40020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa6413ea640 1 -- 172.21.15.19:0/2870884450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f1df40 msgr2=0x557a86f30b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa6413ea640 1 --2- 172.21.15.19:0/2870884450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f1df40 0x557a86f30b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa6413ea640 1 -- 172.21.15.19:0/2870884450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a86f19660 con 0x557a86f1d620 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.450+0000 7fa640be9640 1 --2- 172.21.15.19:0/2870884450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f1df40 0x557a86f30b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa6413ea640 1 --2- 172.21.15.19:0/2870884450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f34220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa63802a930 tx=0x7fa638032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99e74f89dabb482 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa626ffd640 1 -- 172.21.15.19:0/2870884450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63804b040 con 0x557a86f1d620 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa626ffd640 1 -- 172.21.15.19:0/2870884450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa638004b70 con 0x557a86f1d620 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa626ffd640 1 -- 172.21.15.19:0/2870884450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638035040 con 0x557a86f1d620 2024-04-23T18:29:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa646367740 1 -- 172.21.15.19:0/2870884450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 msgr2=0x557a86f34220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa646367740 1 --2- 172.21.15.19:0/2870884450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f34220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa63802a930 tx=0x7fa638032ce0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa646367740 1 -- 172.21.15.19:0/2870884450 shutdown_connections 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa646367740 1 --2- 172.21.15.19:0/2870884450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f1df40 0x557a86f30b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa646367740 1 --2- 172.21.15.19:0/2870884450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f28800 0x557a86f40020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa646367740 1 --2- 172.21.15.19:0/2870884450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f34220 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa646367740 1 -- 172.21.15.19:0/2870884450 >> 172.21.15.19:0/2870884450 conn(0x557a86f19b30 msgr2=0x557a86f33b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa646367740 1 -- 172.21.15.19:0/2870884450 shutdown_connections 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.451+0000 7fa646367740 1 -- 172.21.15.19:0/2870884450 wait complete. 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa646367740 1 -- start start 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa646367740 1 Processor -- start 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa646367740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f9b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa646367740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1df40 0x557a86f9b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa646367740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f22370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa646367740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c1aee90 con 0x557a86f1d620 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa646367740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c1ae7c0 con 0x557a86f28800 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa646367740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c1aba20 con 0x557a86f1df40 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa646367740 10 client.0 _ll_register_callbacks cb 0x7fa638032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa6413ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f9b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa6413ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f9b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51350/0 (socket says 172.21.15.19:51350) 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa6413ea640 1 -- 172.21.15.19:0/2825551866 learned_addr learned my addr 172.21.15.19:0/2825551866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.452+0000 7fa641beb640 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f22370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa6413ea640 1 -- 172.21.15.19:0/2825551866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1df40 msgr2=0x557a86f9b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa6413ea640 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1df40 0x557a86f9b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa6413ea640 1 -- 172.21.15.19:0/2825551866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 msgr2=0x557a86f22370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa6413ea640 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f22370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa6413ea640 1 -- 172.21.15.19:0/2825551866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a86f22a60 con 0x557a86f1d620 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa6413ea640 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f9b0a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa6380057c0 tx=0x7fa638004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0705b96bf5c24f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa624ff9640 1 -- 172.21.15.19:0/2825551866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63805e030 con 0x557a86f1d620 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa624ff9640 1 -- 172.21.15.19:0/2825551866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa6380048d0 con 0x557a86f1d620 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa646367740 1 -- 172.21.15.19:0/2825551866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa638004b70 con 0x557a86f1d620 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa624ff9640 1 -- 172.21.15.19:0/2825551866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa638068050 con 0x557a86f1d620 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa646367740 10 client.16173 fetch_fsmap learned FSMap version 27 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa646367740 1 -- 172.21.15.19:0/2825551866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a86fa2820 con 0x557a86f1d620 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa624ff9640 1 -- 172.21.15.19:0/2825551866 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa638004b70 con 0x557a86f1d620 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.453+0000 7fa624ff9640 1 -- 172.21.15.19:0/2825551866 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa638058020 con 0x557a86f1d620 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.454+0000 7fa646367740 10 client.16173 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.454+0000 7fa646367740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.454+0000 7fa646367740 20 client.16173 populate_metadata read hostname 'smithi019' 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.454+0000 7fa646367740 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a86fa2c80 0x557a86fa50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.454+0000 7fa646367740 4 client.16173 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.454+0000 7fa646367740 1 -- 172.21.15.19:0/2825551866 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a86e8a200 con 0x557a86fa2c80 2024-04-23T18:29:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.454+0000 7fa640be9640 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a86fa2c80 0x557a86fa50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.454+0000 7fa624ff9640 10 client.16173 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.454+0000 7fa640be9640 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a86fa2c80 0x557a86fa50a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc786442b629ef2 server_cookie=b850310c504a5d5d in_seq=0 out_seq=0 2024-04-23T18:29:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa624ff9640 1 -- 172.21.15.19:0/2825551866 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a86e8a200 con 0x557a86fa2c80 2024-04-23T18:29:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa624ff9640 10 client.16173 handle_command_reply: tid=0 2024-04-23T18:29:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 client.16173 shutdown 2024-04-23T18:29:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 -- 172.21.15.19:0/2825551866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 msgr2=0x557a86f9b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f9b0a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa6380057c0 tx=0x7fa638004d20 comp rx=0 tx=0).stop 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 -- 172.21.15.19:0/2825551866 shutdown_connections 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a86fa2c80 0x557a86fa50a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a86f28800 0x557a86f22370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a86f1df40 0x557a86f9b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa624ff9640 0 client.16173 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 --2- 172.21.15.19:0/2825551866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a86f1d620 0x557a86f9b0a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 -- 172.21.15.19:0/2825551866 >> 172.21.15.19:0/2825551866 conn(0x557a86f19b30 msgr2=0x557a86f2fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 -- 172.21.15.19:0/2825551866 shutdown_connections 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 1 -- 172.21.15.19:0/2825551866 wait complete. 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.470+0000 7fa646367740 20 client.16173 trim_cache size 0 max 16384 2024-04-23T18:29:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:29:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:29:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:29:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:29:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 -- 172.21.15.19:0/3402289479 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6280a3e40 msgr2=0x7fa6280a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6280a3e40 0x7fa6280a6300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa634009fd0 tx=0x7fa634009600 comp rx=0 tx=0).stop 2024-04-23T18:29:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 -- 172.21.15.19:0/3402289479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa63c106190 msgr2=0x7fa63c19e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa63c106190 0x7fa63c19e920 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00b450 tx=0x7fa62c00b910 comp rx=0 tx=0).stop 2024-04-23T18:29:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 -- 172.21.15.19:0/3402289479 shutdown_connections 2024-04-23T18:29:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6280a3e40 0x7fa6280a6300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c198590 0x7fa63c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa63c106190 0x7fa63c19e920 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 --2- 172.21.15.19:0/3402289479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63c1010c0 0x7fa63c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 -- 172.21.15.19:0/3402289479 >> 172.21.15.19:0/3402289479 conn(0x7fa63c077fd0 msgr2=0x7fa63c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 -- 172.21.15.19:0/3402289479 shutdown_connections 2024-04-23T18:29:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:29:54.471+0000 7fa64364e640 1 -- 172.21.15.19:0/3402289479 wait complete. 2024-04-23T18:29:55.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:55 smithi044 ceph-mon[32458]: pgmap v116: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-23T18:29:55.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:55 smithi155 ceph-mon[33468]: pgmap v116: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-23T18:29:55.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:55 smithi019 ceph-mon[29215]: pgmap v116: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-23T18:29:57.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:57 smithi155 ceph-mon[33468]: pgmap v117: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:29:57.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:57 smithi019 ceph-mon[29215]: pgmap v117: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:29:57.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:57 smithi044 ceph-mon[32458]: pgmap v117: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:29:59.572 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:29:59 smithi155 ceph-mon[33468]: pgmap v118: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:29:59.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:29:59 smithi019 ceph-mon[29215]: pgmap v118: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:29:59.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:29:59 smithi044 ceph-mon[32458]: pgmap v118: 97 pgs: 97 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:30:00.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:00 smithi019 ceph-mon[29215]: overall HEALTH_OK 2024-04-23T18:30:00.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:00 smithi044 ceph-mon[32458]: overall HEALTH_OK 2024-04-23T18:30:00.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:00 smithi155 ceph-mon[33468]: overall HEALTH_OK 2024-04-23T18:30:01.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:01 smithi019 ceph-mon[29215]: pgmap v119: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-04-23T18:30:01.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:01 smithi044 ceph-mon[32458]: pgmap v119: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-04-23T18:30:01.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:01 smithi155 ceph-mon[33468]: pgmap v119: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-04-23T18:30:03.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:03 smithi019 ceph-mon[29215]: pgmap v120: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-23T18:30:03.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:03 smithi044 ceph-mon[32458]: pgmap v120: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-23T18:30:03.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:03 smithi155 ceph-mon[33468]: pgmap v120: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-23T18:30:05.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:05 smithi019 ceph-mon[29215]: pgmap v121: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 62 KiB/s wr, 37 op/s 2024-04-23T18:30:05.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:05 smithi044 ceph-mon[32458]: pgmap v121: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 62 KiB/s wr, 37 op/s 2024-04-23T18:30:05.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:05 smithi155 ceph-mon[33468]: pgmap v121: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 62 KiB/s wr, 37 op/s 2024-04-23T18:30:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.571+0000 7fe846908640 1 -- 172.21.15.19:0/1927515831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe840106fb0 msgr2=0x7fe840107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.571+0000 7fe846908640 1 --2- 172.21.15.19:0/1927515831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe840106fb0 0x7fe840107390 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe830009900 tx=0x7fe83002a400 comp rx=0 tx=0).stop 2024-04-23T18:30:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.571+0000 7fe846908640 1 -- 172.21.15.19:0/1927515831 shutdown_connections 2024-04-23T18:30:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.571+0000 7fe846908640 1 --2- 172.21.15.19:0/1927515831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8401011a0 0x7fe840101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.571+0000 7fe846908640 1 --2- 172.21.15.19:0/1927515831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe840101b60 0x7fe84010dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.571+0000 7fe846908640 1 --2- 172.21.15.19:0/1927515831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe840106fb0 0x7fe840107390 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.571+0000 7fe846908640 1 -- 172.21.15.19:0/1927515831 >> 172.21.15.19:0/1927515831 conn(0x7fe840076c50 msgr2=0x7fe840077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.571+0000 7fe846908640 1 -- 172.21.15.19:0/1927515831 shutdown_connections 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.571+0000 7fe846908640 1 -- 172.21.15.19:0/1927515831 wait complete. 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.572+0000 7fe846908640 1 Processor -- start 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.572+0000 7fe846908640 1 -- start start 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.572+0000 7fe846908640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8401011a0 0x7fe840102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.572+0000 7fe846908640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe840101b60 0x7fe840103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.572+0000 7fe846908640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe840106fb0 0x7fe840103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.572+0000 7fe846908640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe840111090 con 0x7fe840101b60 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.572+0000 7fe846908640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe840110f10 con 0x7fe8401011a0 2024-04-23T18:30:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.572+0000 7fe846908640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe840111210 con 0x7fe840106fb0 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe840101b60 0x7fe840103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe840101b60 0x7fe840103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34128/0 (socket says 172.21.15.19:34128) 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83f7fe640 1 -- 172.21.15.19:0/3641945497 learned_addr learned my addr 172.21.15.19:0/3641945497 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8401011a0 0x7fe840102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe844ea5640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe840106fb0 0x7fe840103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83f7fe640 1 -- 172.21.15.19:0/3641945497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe840106fb0 msgr2=0x7fe840103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe840106fb0 0x7fe840103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83f7fe640 1 -- 172.21.15.19:0/3641945497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8401011a0 msgr2=0x7fe840102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8401011a0 0x7fe840102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83f7fe640 1 -- 172.21.15.19:0/3641945497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8401a6a80 con 0x7fe840101b60 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe840101b60 0x7fe840103050 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe83400c920 tx=0x7fe83400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83d7fa640 1 -- 172.21.15.19:0/3641945497 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834007c70 con 0x7fe840101b60 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe846908640 1 -- 172.21.15.19:0/3641945497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8401a6d70 con 0x7fe840101b60 2024-04-23T18:30:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83d7fa640 1 -- 172.21.15.19:0/3641945497 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe834007e10 con 0x7fe840101b60 2024-04-23T18:30:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe83d7fa640 1 -- 172.21.15.19:0/3641945497 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834018820 con 0x7fe840101b60 2024-04-23T18:30:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.573+0000 7fe846908640 1 -- 172.21.15.19:0/3641945497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8401a7260 con 0x7fe840101b60 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.576+0000 7fe83d7fa640 1 -- 172.21.15.19:0/3641945497 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe83400f450 con 0x7fe840101b60 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.576+0000 7fe849625740 1 Processor -- start 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.576+0000 7fe849625740 1 -- start start 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.576+0000 7fe83d7fa640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8280a3dc0 0x7fe8280a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.576+0000 7fe849625740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fe4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.576+0000 7fe849625740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 0x558ad7fef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.576+0000 7fe849625740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fef800 0x558ad7ff1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.576+0000 7fe849625740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ad7f49970 con 0x558ad7fe4620 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.576+0000 7fe849625740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558ad7ee64a0 con 0x558ad7fef800 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.576+0000 7fe849625740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558ad7f45930 con 0x558ad7fe4f40 2024-04-23T18:30:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe844ea5640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8280a3dc0 0x7fe8280a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fe4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83d7fa640 1 -- 172.21.15.19:0/3641945497 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe8340d0900 con 0x7fe840101b60 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fe4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34142/0 (socket says 172.21.15.19:34142) 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83ffff640 1 -- 172.21.15.19:0/3698829490 learned_addr learned my addr 172.21.15.19:0/3698829490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe844ea5640 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fef800 0x558ad7ff1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 0x558ad7fef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83ffff640 1 -- 172.21.15.19:0/3698829490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 msgr2=0x558ad7fef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83ffff640 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 0x558ad7fef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83ffff640 1 -- 172.21.15.19:0/3698829490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fef800 msgr2=0x558ad7ff1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83ffff640 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fef800 0x558ad7ff1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83ffff640 1 -- 172.21.15.19:0/3698829490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad7c5c210 con 0x558ad7fe4620 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 0x558ad7fef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe844ea5640 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fef800 0x558ad7ff1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe844ea5640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8280a3dc0 0x7fe8280a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe8380092f0 tx=0x7fe8380099b0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe83ffff640 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fe4a00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe830009900 tx=0x7fe830032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6235ef15fcf10b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe8267fc640 1 -- 172.21.15.19:0/3698829490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830040070 con 0x558ad7fe4620 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe8267fc640 1 -- 172.21.15.19:0/3698829490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8300316a0 con 0x558ad7fe4620 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.577+0000 7fe8267fc640 1 -- 172.21.15.19:0/3698829490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830035040 con 0x558ad7fe4620 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.578+0000 7fe849625740 1 -- 172.21.15.19:0/3698829490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 msgr2=0x558ad7fe4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.578+0000 7fe849625740 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fe4a00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe830009900 tx=0x7fe830032e60 comp rx=0 tx=0).stop 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.578+0000 7fe849625740 1 -- 172.21.15.19:0/3698829490 shutdown_connections 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.578+0000 7fe849625740 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fef800 0x558ad7ff1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.578+0000 7fe849625740 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 0x558ad7fef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.578+0000 7fe849625740 1 --2- 172.21.15.19:0/3698829490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fe4a00 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.578+0000 7fe849625740 1 -- 172.21.15.19:0/3698829490 >> 172.21.15.19:0/3698829490 conn(0x558ad7fe0b30 msgr2=0x558ad7fe0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.578+0000 7fe849625740 1 -- 172.21.15.19:0/3698829490 shutdown_connections 2024-04-23T18:30:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.578+0000 7fe849625740 1 -- 172.21.15.19:0/3698829490 wait complete. 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe849625740 1 -- start start 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe849625740 1 Processor -- start 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe849625740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe849625740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4f40 0x558ad7fec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe849625740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fef800 0x558ad7fec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe849625740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ad7ff23d0 con 0x558ad7fe4620 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe849625740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558ad7ee40b0 con 0x558ad7fe4f40 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe849625740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558ad7ff2550 con 0x558ad7fef800 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe849625740 10 client.0 _ll_register_callbacks cb 0x558ad7ed1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe83ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe83ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fef0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34158/0 (socket says 172.21.15.19:34158) 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe83ffff640 1 -- 172.21.15.19:0/958325920 learned_addr learned my addr 172.21.15.19:0/958325920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4f40 0x558ad7fec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.579+0000 7fe844ea5640 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fef800 0x558ad7fec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe83ffff640 1 -- 172.21.15.19:0/958325920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fef800 msgr2=0x558ad7fec900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe83ffff640 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fef800 0x558ad7fec900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe83ffff640 1 -- 172.21.15.19:0/958325920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4f40 msgr2=0x558ad7fec3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe83ffff640 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4f40 0x558ad7fec3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe83ffff640 1 -- 172.21.15.19:0/958325920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad7d28620 con 0x558ad7fe4620 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe844ea5640 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fef800 0x558ad7fec900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe83ffff640 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fef0d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe830031a80 tx=0x7fe830031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d1b6cf7f2e61e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe81f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe81f7fe640 1 -- 172.21.15.19:0/958325920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83004b020 con 0x558ad7fe4620 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe81f7fe640 1 -- 172.21.15.19:0/958325920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe83003c640 con 0x558ad7fe4620 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe849625740 1 -- 172.21.15.19:0/958325920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8300316a0 con 0x558ad7fe4620 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe81f7fe640 1 -- 172.21.15.19:0/958325920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe830056050 con 0x558ad7fe4620 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe849625740 10 client.16185 fetch_fsmap learned FSMap version 27 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe849625740 1 -- 172.21.15.19:0/958325920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ad7ffe2b0 con 0x558ad7fe4620 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.580+0000 7fe81f7fe640 1 -- 172.21.15.19:0/958325920 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8300316a0 con 0x558ad7fe4620 2024-04-23T18:30:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.581+0000 7fe81f7fe640 1 -- 172.21.15.19:0/958325920 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe830044820 con 0x558ad7fe4620 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.581+0000 7fe849625740 10 client.16185 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.581+0000 7fe849625740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.581+0000 7fe849625740 20 client.16185 populate_metadata read hostname 'smithi019' 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.581+0000 7fe849625740 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ad7fff600 0x558ad80019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.581+0000 7fe849625740 4 client.16185 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.581+0000 7fe849625740 1 -- 172.21.15.19:0/958325920 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ad7f57390 con 0x558ad7fff600 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.581+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ad7fff600 0x558ad80019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.581+0000 7fe81f7fe640 10 client.16185 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.581+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ad7fff600 0x558ad80019f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=289b4affb445194e server_cookie=6b00cd50d74d826b in_seq=0 out_seq=0 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe81f7fe640 1 -- 172.21.15.19:0/958325920 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ad7f57390 con 0x558ad7fff600 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe81f7fe640 10 client.16185 handle_command_reply: tid=0 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 client.16185 shutdown 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 -- 172.21.15.19:0/958325920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 msgr2=0x558ad7fef0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fef0d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe830031a80 tx=0x7fe830031400 comp rx=0 tx=0).stop 2024-04-23T18:30:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 -- 172.21.15.19:0/958325920 shutdown_connections 2024-04-23T18:30:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ad7fff600 0x558ad80019f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4f40 0x558ad7fec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fef800 0x558ad7fec900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe81f7fe640 0 client.16185 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 --2- 172.21.15.19:0/958325920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4620 0x558ad7fef0d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 -- 172.21.15.19:0/958325920 >> 172.21.15.19:0/958325920 conn(0x558ad7fe0b30 msgr2=0x558ad7fe6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 -- 172.21.15.19:0/958325920 shutdown_connections 2024-04-23T18:30:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 1 -- 172.21.15.19:0/958325920 wait complete. 2024-04-23T18:30:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.582+0000 7fe849625740 20 client.16185 trim_cache size 0 max 16384 2024-04-23T18:30:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.597+0000 7fe849625740 1 Processor -- start 2024-04-23T18:30:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe849625740 1 -- start start 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe849625740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 0x558ad7ffb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe849625740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4f40 0x558ad7feec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe849625740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fef800 0x558ad8006fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe849625740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401afb80 con 0x558ad7fe4f40 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe849625740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401b16c0 con 0x558ad7fe4620 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe849625740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401b0250 con 0x558ad7fef800 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4f40 0x558ad7feec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4f40 0x558ad7feec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34166/0 (socket says 172.21.15.19:34166) 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83f7fe640 1 -- 172.21.15.19:0/2133306866 learned_addr learned my addr 172.21.15.19:0/2133306866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83ffff640 1 --2- 172.21.15.19:0/2133306866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 0x558ad7ffb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83f7fe640 1 -- 172.21.15.19:0/2133306866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fef800 msgr2=0x558ad8006fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/2133306866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fef800 0x558ad8006fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83f7fe640 1 -- 172.21.15.19:0/2133306866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 msgr2=0x558ad7ffb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/2133306866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 0x558ad7ffb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83f7fe640 1 -- 172.21.15.19:0/2133306866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad7fea910 con 0x558ad7fe4f40 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83ffff640 1 --2- 172.21.15.19:0/2133306866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 0x558ad7ffb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/2133306866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4f40 0x558ad7feec00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe8340ce130 tx=0x7fe8340bd050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbaee5e4005b027 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.598+0000 7fe825ffb640 1 -- 172.21.15.19:0/2133306866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83409a970 con 0x558ad7fe4f40 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe825ffb640 1 -- 172.21.15.19:0/2133306866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe83409abd0 con 0x558ad7fe4f40 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe825ffb640 1 -- 172.21.15.19:0/2133306866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834098f40 con 0x558ad7fe4f40 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe849625740 1 -- 172.21.15.19:0/2133306866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4f40 msgr2=0x558ad7feec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe849625740 1 --2- 172.21.15.19:0/2133306866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4f40 0x558ad7feec00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe8340ce130 tx=0x7fe8340bd050 comp rx=0 tx=0).stop 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe849625740 1 -- 172.21.15.19:0/2133306866 shutdown_connections 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe849625740 1 --2- 172.21.15.19:0/2133306866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fef800 0x558ad8006fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe849625740 1 --2- 172.21.15.19:0/2133306866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fe4f40 0x558ad7feec00 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe849625740 1 --2- 172.21.15.19:0/2133306866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 0x558ad7ffb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe849625740 1 -- 172.21.15.19:0/2133306866 >> 172.21.15.19:0/2133306866 conn(0x558ad7fe0b30 msgr2=0x558ad7fdfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe849625740 1 -- 172.21.15.19:0/2133306866 shutdown_connections 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe849625740 1 -- 172.21.15.19:0/2133306866 wait complete. 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.599+0000 7fe849625740 1 -- start start 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe849625740 1 Processor -- start 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe849625740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 0x558ad7ff3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe849625740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 0x558ad7ff3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe849625740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fef800 0x558ad8064830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe849625740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401b0ff0 con 0x558ad7fef800 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe849625740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401b0920 con 0x558ad7fe4620 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe849625740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401adb60 con 0x558ad7fe4f40 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe849625740 10 client.0 _ll_register_callbacks cb 0x7fe830032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe844ea5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fef800 0x558ad8064830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe844ea5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fef800 0x558ad8064830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34182/0 (socket says 172.21.15.19:34182) 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe844ea5640 1 -- 172.21.15.19:0/1053827659 learned_addr learned my addr 172.21.15.19:0/1053827659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe83ffff640 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 0x558ad7ff3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe83f7fe640 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 0x558ad7ff3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe844ea5640 1 -- 172.21.15.19:0/1053827659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 msgr2=0x558ad7ff3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe844ea5640 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 0x558ad7ff3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe844ea5640 1 -- 172.21.15.19:0/1053827659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 msgr2=0x558ad7ff3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe844ea5640 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 0x558ad7ff3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.600+0000 7fe844ea5640 1 -- 172.21.15.19:0/1053827659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad8064da0 con 0x558ad7fef800 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe844ea5640 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fef800 0x558ad8064830 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x558ad7fe1cc0 tx=0x7fe838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c984c66331ee81d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe81f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe81f7fe640 1 -- 172.21.15.19:0/1053827659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838027070 con 0x558ad7fef800 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe81f7fe640 1 -- 172.21.15.19:0/1053827659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe838004d20 con 0x558ad7fef800 2024-04-23T18:30:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe849625740 1 -- 172.21.15.19:0/1053827659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe83409abd0 con 0x558ad7fef800 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe81f7fe640 1 -- 172.21.15.19:0/1053827659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe838031050 con 0x558ad7fef800 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe849625740 10 client.16191 fetch_fsmap learned FSMap version 27 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe849625740 1 -- 172.21.15.19:0/1053827659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ad8066020 con 0x558ad7fef800 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe81f7fe640 1 -- 172.21.15.19:0/1053827659 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe83409abd0 con 0x558ad7fef800 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe81f7fe640 1 -- 172.21.15.19:0/1053827659 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe83800f040 con 0x558ad7fef800 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe849625740 10 client.16191 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe849625740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe849625740 20 client.16191 populate_metadata read hostname 'smithi019' 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe849625740 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ad8066810 0x558ad8068c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe849625740 4 client.16191 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.601+0000 7fe849625740 1 -- 172.21.15.19:0/1053827659 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ad7f51200 con 0x558ad8066810 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.602+0000 7fe83ffff640 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ad8066810 0x558ad8068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.602+0000 7fe81f7fe640 10 client.16191 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.602+0000 7fe83ffff640 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ad8066810 0x558ad8068c30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae23ece8b8b3703 server_cookie=652ccb67ab0a7271 in_seq=0 out_seq=0 2024-04-23T18:30:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.617+0000 7fe81f7fe640 1 -- 172.21.15.19:0/1053827659 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ad7f51200 con 0x558ad8066810 2024-04-23T18:30:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.617+0000 7fe81f7fe640 10 client.16191 handle_command_reply: tid=0 2024-04-23T18:30:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.617+0000 7fe849625740 1 client.16191 shutdown 2024-04-23T18:30:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.617+0000 7fe849625740 1 -- 172.21.15.19:0/1053827659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fef800 msgr2=0x558ad8064830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.617+0000 7fe849625740 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fef800 0x558ad8064830 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x558ad7fe1cc0 tx=0x7fe838004a40 comp rx=0 tx=0).stop 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.617+0000 7fe849625740 1 -- 172.21.15.19:0/1053827659 shutdown_connections 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.617+0000 7fe849625740 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ad8066810 0x558ad8068c30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe849625740 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ad7fef800 0x558ad8064830 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe849625740 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad7fe4f40 0x558ad7ff3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe849625740 1 --2- 172.21.15.19:0/1053827659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ad7fe4620 0x558ad7ff3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe849625740 1 -- 172.21.15.19:0/1053827659 >> 172.21.15.19:0/1053827659 conn(0x558ad7fe0b30 msgr2=0x558ad7fe5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe81f7fe640 0 client.16191 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe849625740 1 -- 172.21.15.19:0/1053827659 shutdown_connections 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe849625740 1 -- 172.21.15.19:0/1053827659 wait complete. 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe849625740 20 client.16191 trim_cache size 0 max 16384 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:30:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:30:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:30:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:30:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 -- 172.21.15.19:0/3641945497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8280a3dc0 msgr2=0x7fe8280a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8280a3dc0 0x7fe8280a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe8380092f0 tx=0x7fe8380099b0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 -- 172.21.15.19:0/3641945497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe840101b60 msgr2=0x7fe840103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe840101b60 0x7fe840103050 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe83400c920 tx=0x7fe83400cde0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 -- 172.21.15.19:0/3641945497 shutdown_connections 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8280a3dc0 0x7fe8280a6280 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe840106fb0 0x7fe840103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe840101b60 0x7fe840103050 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 --2- 172.21.15.19:0/3641945497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8401011a0 0x7fe840102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 -- 172.21.15.19:0/3641945497 >> 172.21.15.19:0/3641945497 conn(0x7fe840076c50 msgr2=0x7fe840100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.618+0000 7fe846908640 1 -- 172.21.15.19:0/3641945497 shutdown_connections 2024-04-23T18:30:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:07.619+0000 7fe846908640 1 -- 172.21.15.19:0/3641945497 wait complete. 2024-04-23T18:30:07.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:07 smithi019 ceph-mon[29215]: pgmap v122: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-23T18:30:07.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:07 smithi044 ceph-mon[32458]: pgmap v122: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-23T18:30:07.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:07 smithi155 ceph-mon[33468]: pgmap v122: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-23T18:30:08.879 INFO:tasks.workunit.client.0.smithi019.stdout:64/600: open fd = 3 2024-04-23T18:30:08.879 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.879 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:65/600: open fd = 3 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:66/600: open fd = 3 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:67/600: open fd = 3 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.880 INFO:tasks.workunit.client.0.smithi019.stdout:68/600: open fd = 3 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:69/600: open fd = 3 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:70/600: open fd = 3 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:71/600: open fd = 3 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.881 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:72/600: open fd = 3 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:73/600: open fd = 3 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:74/600: open fd = 3 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.882 INFO:tasks.workunit.client.0.smithi019.stdout:75/600: open fd = 3 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:76/600: open fd = 3 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:77/600: open fd = 3 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:78/600: open fd = 3 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.883 INFO:tasks.workunit.client.0.smithi019.stdout:79/600: open fd = 3 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:80/600: open fd = 3 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:81/600: open fd = 3 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:82/600: open fd = 3 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.884 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:83/600: open fd = 3 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:84/600: open fd = 3 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:85/600: open fd = 3 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:86/600: open fd = 3 2024-04-23T18:30:08.885 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:87/600: open fd = 3 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:88/600: open fd = 3 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:89/600: open fd = 3 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.886 INFO:tasks.workunit.client.0.smithi019.stdout:90/600: open fd = 3 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:91/600: open fd = 3 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:92/600: open fd = 3 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:93/600: open fd = 3 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:94/600: open fd = 3 2024-04-23T18:30:08.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:95/600: open fd = 3 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:96/600: open fd = 3 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:97/600: open fd = 3 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.888 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:98/600: open fd = 3 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:99/600: open fd = 3 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:100/600: open fd = 3 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:101/600: open fd = 3 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.889 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:102/600: open fd = 3 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:103/600: open fd = 3 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:104/600: open fd = 3 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:105/600: open fd = 3 2024-04-23T18:30:08.890 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:106/600: open fd = 3 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:107/600: open fd = 3 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:108/600: open fd = 3 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.891 INFO:tasks.workunit.client.0.smithi019.stdout:109/600: open fd = 3 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:110/600: open fd = 3 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:111/600: open fd = 3 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:112/600: open fd = 3 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.892 INFO:tasks.workunit.client.0.smithi019.stdout:113/600: open fd = 3 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:114/600: open fd = 3 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:115/600: open fd = 3 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:116/600: open fd = 3 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.893 INFO:tasks.workunit.client.0.smithi019.stdout:117/600: open fd = 3 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:118/600: open fd = 3 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:119/600: open fd = 3 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:120/600: open fd = 3 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.894 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:121/600: open fd = 3 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:122/600: open fd = 3 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:123/600: open fd = 3 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:124/600: open fd = 3 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.896 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.896 INFO:tasks.workunit.client.0.smithi019.stdout:125/600: open fd = 3 2024-04-23T18:30:08.896 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.896 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:08.896 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:30:08.896 INFO:tasks.workunit.client.0.smithi019.stdout:126/600: open fd = 3 2024-04-23T18:30:08.896 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:30:09.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:09 smithi019 ceph-mon[29215]: pgmap v123: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-23T18:30:09.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:09 smithi044 ceph-mon[32458]: pgmap v123: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-23T18:30:09.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:09 smithi155 ceph-mon[33468]: pgmap v123: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-23T18:30:11.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:11 smithi019 ceph-mon[29215]: pgmap v124: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 68 KiB/s wr, 39 op/s 2024-04-23T18:30:11.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:11 smithi044 ceph-mon[32458]: pgmap v124: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 68 KiB/s wr, 39 op/s 2024-04-23T18:30:11.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:11 smithi155 ceph-mon[33468]: pgmap v124: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 68 KiB/s wr, 39 op/s 2024-04-23T18:30:13.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:13 smithi019 ceph-mon[29215]: pgmap v125: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-23T18:30:13.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:13 smithi044 ceph-mon[32458]: pgmap v125: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-23T18:30:13.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:13 smithi155 ceph-mon[33468]: pgmap v125: 97 pgs: 97 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-23T18:30:15.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:15 smithi019 ceph-mon[29215]: pgmap v126: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 65 KiB/s wr, 36 op/s 2024-04-23T18:30:15.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:15 smithi044 ceph-mon[32458]: pgmap v126: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 65 KiB/s wr, 36 op/s 2024-04-23T18:30:15.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:15 smithi155 ceph-mon[33468]: pgmap v126: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 65 KiB/s wr, 36 op/s 2024-04-23T18:30:17.624 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:17 smithi155 ceph-mon[33468]: pgmap v127: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T18:30:17.624 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:17.624 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:17.624 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:17.625 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:30:17.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:17 smithi019 ceph-mon[29215]: pgmap v127: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T18:30:17.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:17.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:17.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:17.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:30:17.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:17 smithi044 ceph-mon[32458]: pgmap v127: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T18:30:17.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:17.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:17.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:17.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:30:19.666 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:19 smithi044 ceph-mon[32458]: pgmap v128: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T18:30:19.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:19 smithi019 ceph-mon[29215]: pgmap v128: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T18:30:19.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:19 smithi155 ceph-mon[33468]: pgmap v128: 97 pgs: 97 active+clean; 4.2 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T18:30:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.719+0000 7efe6dd8e640 1 -- 172.21.15.19:0/3428506666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6810c520 msgr2=0x7efe6810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.719+0000 7efe6dd8e640 1 --2- 172.21.15.19:0/3428506666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6810c520 0x7efe6810e9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efe60009ca0 tx=0x7efe6002a9e0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.719+0000 7efe6dd8e640 1 -- 172.21.15.19:0/3428506666 shutdown_connections 2024-04-23T18:30:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.719+0000 7efe6dd8e640 1 --2- 172.21.15.19:0/3428506666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6810c520 0x7efe6810e9c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.719+0000 7efe6dd8e640 1 --2- 172.21.15.19:0/3428506666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe68103b00 0x7efe6810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.719+0000 7efe6dd8e640 1 --2- 172.21.15.19:0/3428506666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe681031e0 0x7efe681035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.719+0000 7efe6dd8e640 1 -- 172.21.15.19:0/3428506666 >> 172.21.15.19:0/3428506666 conn(0x7efe68077fd0 msgr2=0x7efe680783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.719+0000 7efe6dd8e640 1 -- 172.21.15.19:0/3428506666 shutdown_connections 2024-04-23T18:30:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.719+0000 7efe6dd8e640 1 -- 172.21.15.19:0/3428506666 wait complete. 2024-04-23T18:30:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.719+0000 7efe6dd8e640 1 Processor -- start 2024-04-23T18:30:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe6dd8e640 1 -- start start 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe6dd8e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe681031e0 0x7efe68198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe6dd8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe68103b00 0x7efe68198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe6dd8e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6810c520 0x7efe6819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe6dd8e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe681121b0 con 0x7efe6810c520 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe6dd8e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe68112030 con 0x7efe68103b00 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe6dd8e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe68112330 con 0x7efe681031e0 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6810c520 0x7efe6819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6810c520 0x7efe6819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40092/0 (socket says 172.21.15.19:40092) 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe67fff640 1 -- 172.21.15.19:0/4221752270 learned_addr learned my addr 172.21.15.19:0/4221752270 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe677fe640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe681031e0 0x7efe68198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe66ffd640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe68103b00 0x7efe68198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe67fff640 1 -- 172.21.15.19:0/4221752270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe681031e0 msgr2=0x7efe68198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe67fff640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe681031e0 0x7efe68198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.720+0000 7efe67fff640 1 -- 172.21.15.19:0/4221752270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe68103b00 msgr2=0x7efe68198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.721+0000 7efe67fff640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe68103b00 0x7efe68198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.721+0000 7efe67fff640 1 -- 172.21.15.19:0/4221752270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6819d640 con 0x7efe6810c520 2024-04-23T18:30:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.721+0000 7efe67fff640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6810c520 0x7efe6819cf10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efe60006010 tx=0x7efe6002fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.721+0000 7efe64ff9640 1 -- 172.21.15.19:0/4221752270 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60033040 con 0x7efe6810c520 2024-04-23T18:30:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.721+0000 7efe64ff9640 1 -- 172.21.15.19:0/4221752270 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6002e7a0 con 0x7efe6810c520 2024-04-23T18:30:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.721+0000 7efe6dd8e640 1 -- 172.21.15.19:0/4221752270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6819d8d0 con 0x7efe6810c520 2024-04-23T18:30:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.721+0000 7efe64ff9640 1 -- 172.21.15.19:0/4221752270 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe600437b0 con 0x7efe6810c520 2024-04-23T18:30:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.721+0000 7efe6dd8e640 1 -- 172.21.15.19:0/4221752270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe681a51c0 con 0x7efe6810c520 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.723+0000 7efe64ff9640 1 -- 172.21.15.19:0/4221752270 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efe60036070 con 0x7efe6810c520 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.724+0000 7efe70b7e740 1 Processor -- start 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.724+0000 7efe70b7e740 1 -- start start 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.724+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 0x560db6efd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.724+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efde40 0x560db6f0c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.724+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6f0cc10 0x560db6f0f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.724+0000 7efe70b7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560db6e62970 con 0x560db6efde40 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.724+0000 7efe70b7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560db6dff4a0 con 0x560db6efd520 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.724+0000 7efe70b7e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560db6e5e930 con 0x560db6f0cc10 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.724+0000 7efe64ff9640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efe500a3dc0 0x7efe500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.724+0000 7efe677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efde40 0x560db6f0c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efde40 0x560db6f0c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40098/0 (socket says 172.21.15.19:40098) 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe677fe640 1 -- 172.21.15.19:0/1462234299 learned_addr learned my addr 172.21.15.19:0/1462234299 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe67fff640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efe500a3dc0 0x7efe500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe66ffd640 1 --2- 172.21.15.19:0/1462234299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 0x560db6efd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe66ffd640 1 --2- 172.21.15.19:0/1462234299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6f0cc10 0x560db6f0f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe64ff9640 1 -- 172.21.15.19:0/4221752270 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7efe600f1610 con 0x7efe6810c520 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe677fe640 1 -- 172.21.15.19:0/1462234299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6f0cc10 msgr2=0x560db6f0f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe677fe640 1 --2- 172.21.15.19:0/1462234299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6f0cc10 0x560db6f0f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe677fe640 1 -- 172.21.15.19:0/1462234299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 msgr2=0x560db6efd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe677fe640 1 --2- 172.21.15.19:0/1462234299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 0x560db6efd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe677fe640 1 -- 172.21.15.19:0/1462234299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db6b75210 con 0x560db6efde40 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe66ffd640 1 --2- 172.21.15.19:0/1462234299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 0x560db6efd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe677fe640 1 --2- 172.21.15.19:0/1462234299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efde40 0x560db6f0c6d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efe5800bd10 tx=0x7efe580107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00c5ec218135b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe67fff640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efe500a3dc0 0x7efe500a6280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe600bd510 tx=0x7efe600a8040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe4dffb640 1 -- 172.21.15.19:0/1462234299 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58004030 con 0x560db6efde40 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe4dffb640 1 -- 172.21.15.19:0/1462234299 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe580035f0 con 0x560db6efde40 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.725+0000 7efe4dffb640 1 -- 172.21.15.19:0/1462234299 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58011640 con 0x560db6efde40 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.726+0000 7efe70b7e740 1 -- 172.21.15.19:0/1462234299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efde40 msgr2=0x560db6f0c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.726+0000 7efe70b7e740 1 --2- 172.21.15.19:0/1462234299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efde40 0x560db6f0c6d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efe5800bd10 tx=0x7efe580107a0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.726+0000 7efe70b7e740 1 -- 172.21.15.19:0/1462234299 shutdown_connections 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.726+0000 7efe70b7e740 1 --2- 172.21.15.19:0/1462234299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6f0cc10 0x560db6f0f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.726+0000 7efe70b7e740 1 --2- 172.21.15.19:0/1462234299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efde40 0x560db6f0c6d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.726+0000 7efe70b7e740 1 --2- 172.21.15.19:0/1462234299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 0x560db6efd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.726+0000 7efe70b7e740 1 -- 172.21.15.19:0/1462234299 >> 172.21.15.19:0/1462234299 conn(0x560db6ef9b30 msgr2=0x560db6ef9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.726+0000 7efe70b7e740 1 -- 172.21.15.19:0/1462234299 shutdown_connections 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.726+0000 7efe70b7e740 1 -- 172.21.15.19:0/1462234299 wait complete. 2024-04-23T18:30:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.726+0000 7efe70b7e740 1 -- start start 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe70b7e740 1 Processor -- start 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f0c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f09850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6f0cc10 0x560db6f09db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe70b7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560db6f0f840 con 0x560db6efd520 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe70b7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560db6dfd0b0 con 0x560db6f0cc10 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe70b7e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560db6f0f9c0 con 0x560db6efde40 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe70b7e740 10 client.0 _ll_register_callbacks cb 0x560db6dead30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f0c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f0c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40102/0 (socket says 172.21.15.19:40102) 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe66ffd640 1 -- 172.21.15.19:0/2334081981 learned_addr learned my addr 172.21.15.19:0/2334081981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.727+0000 7efe677fe640 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f09850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe66ffd640 1 -- 172.21.15.19:0/2334081981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 msgr2=0x560db6f09850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe66ffd640 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f09850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe66ffd640 1 -- 172.21.15.19:0/2334081981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6f0cc10 msgr2=0x560db6f09db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe66ffd640 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6f0cc10 0x560db6f09db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe66ffd640 1 -- 172.21.15.19:0/2334081981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db6c41620 con 0x560db6efd520 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe677fe640 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f09850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe66ffd640 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f0c560 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00e5d0 tx=0x7efe5c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddda597cdd175e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe46ffd640 1 -- 172.21.15.19:0/2334081981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c011ac0 con 0x560db6efd520 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe46ffd640 1 -- 172.21.15.19:0/2334081981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7efe5c011c60 con 0x560db6efd520 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe70b7e740 1 -- 172.21.15.19:0/2334081981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe580035f0 con 0x560db6efd520 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe46ffd640 1 -- 172.21.15.19:0/2334081981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe5c011e60 con 0x560db6efd520 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe70b7e740 10 client.16203 fetch_fsmap learned FSMap version 27 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe70b7e740 1 -- 172.21.15.19:0/2334081981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560db6f1b850 con 0x560db6efd520 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.728+0000 7efe46ffd640 1 -- 172.21.15.19:0/2334081981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe580035f0 con 0x560db6efd520 2024-04-23T18:30:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.729+0000 7efe46ffd640 1 -- 172.21.15.19:0/2334081981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efe5c012550 con 0x560db6efd520 2024-04-23T18:30:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.729+0000 7efe70b7e740 10 client.16203 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:30:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.729+0000 7efe70b7e740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.729+0000 7efe70b7e740 20 client.16203 populate_metadata read hostname 'smithi019' 2024-04-23T18:30:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.729+0000 7efe70b7e740 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560db6f1cd10 0x560db6f1f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.729+0000 7efe70b7e740 4 client.16203 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.729+0000 7efe70b7e740 1 -- 172.21.15.19:0/2334081981 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560db6e70390 con 0x560db6f1cd10 2024-04-23T18:30:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.729+0000 7efe677fe640 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560db6f1cd10 0x560db6f1f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.729+0000 7efe46ffd640 10 client.16203 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.729+0000 7efe677fe640 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560db6f1cd10 0x560db6f1f100 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a24eb3b1346567 server_cookie=90dc6c102b05ccc2 in_seq=0 out_seq=0 2024-04-23T18:30:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe46ffd640 1 -- 172.21.15.19:0/2334081981 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560db6e70390 con 0x560db6f1cd10 2024-04-23T18:30:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe46ffd640 10 client.16203 handle_command_reply: tid=0 2024-04-23T18:30:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe70b7e740 1 client.16203 shutdown 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe70b7e740 1 -- 172.21.15.19:0/2334081981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 msgr2=0x560db6f0c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe70b7e740 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f0c560 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00e5d0 tx=0x7efe5c00ee60 comp rx=0 tx=0).stop 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe70b7e740 1 -- 172.21.15.19:0/2334081981 shutdown_connections 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe70b7e740 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560db6f1cd10 0x560db6f1f100 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe70b7e740 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6f0cc10 0x560db6f09db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe70b7e740 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f09850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe46ffd640 0 client.16203 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe70b7e740 1 --2- 172.21.15.19:0/2334081981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f0c560 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.730+0000 7efe70b7e740 1 -- 172.21.15.19:0/2334081981 >> 172.21.15.19:0/2334081981 conn(0x560db6ef9b30 msgr2=0x560db6eff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.731+0000 7efe70b7e740 1 -- 172.21.15.19:0/2334081981 shutdown_connections 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.731+0000 7efe70b7e740 1 -- 172.21.15.19:0/2334081981 wait complete. 2024-04-23T18:30:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.731+0000 7efe70b7e740 20 client.16203 trim_cache size 0 max 16384 2024-04-23T18:30:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe70b7e740 1 Processor -- start 2024-04-23T18:30:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe70b7e740 1 -- start start 2024-04-23T18:30:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f18740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f0c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6f0cc10 0x560db6f249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe70b7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe681ab8d0 con 0x560db6efd520 2024-04-23T18:30:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe70b7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe681ad410 con 0x560db6f0cc10 2024-04-23T18:30:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe70b7e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe681abfa0 con 0x560db6efde40 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f18740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f18740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40114/0 (socket says 172.21.15.19:40114) 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe66ffd640 1 -- 172.21.15.19:0/35026444 learned_addr learned my addr 172.21.15.19:0/35026444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe677fe640 1 --2- 172.21.15.19:0/35026444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f0c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe66ffd640 1 -- 172.21.15.19:0/35026444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 msgr2=0x560db6f0c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe66ffd640 1 --2- 172.21.15.19:0/35026444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f0c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe66ffd640 1 -- 172.21.15.19:0/35026444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6f0cc10 msgr2=0x560db6f249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe66ffd640 1 --2- 172.21.15.19:0/35026444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6f0cc10 0x560db6f249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.746+0000 7efe66ffd640 1 -- 172.21.15.19:0/35026444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db6ef9960 con 0x560db6efd520 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.747+0000 7efe66ffd640 1 --2- 172.21.15.19:0/35026444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f18740 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0058c0 tx=0x7efe5c014050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2d756714396a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.747+0000 7efe4d7fa640 1 -- 172.21.15.19:0/35026444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c02b070 con 0x560db6efd520 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.747+0000 7efe4d7fa640 1 -- 172.21.15.19:0/35026444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe5c012b60 con 0x560db6efd520 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.747+0000 7efe4d7fa640 1 -- 172.21.15.19:0/35026444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c01e960 con 0x560db6efd520 2024-04-23T18:30:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.747+0000 7efe70b7e740 1 -- 172.21.15.19:0/35026444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 msgr2=0x560db6f18740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.747+0000 7efe70b7e740 1 --2- 172.21.15.19:0/35026444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f18740 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0058c0 tx=0x7efe5c014050 comp rx=0 tx=0).stop 2024-04-23T18:30:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.747+0000 7efe70b7e740 1 -- 172.21.15.19:0/35026444 shutdown_connections 2024-04-23T18:30:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.747+0000 7efe70b7e740 1 --2- 172.21.15.19:0/35026444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6f0cc10 0x560db6f249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 --2- 172.21.15.19:0/35026444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f0c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 --2- 172.21.15.19:0/35026444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6efd520 0x560db6f18740 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 -- 172.21.15.19:0/35026444 >> 172.21.15.19:0/35026444 conn(0x560db6ef9b30 msgr2=0x560db6f04f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 -- 172.21.15.19:0/35026444 shutdown_connections 2024-04-23T18:30:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 -- 172.21.15.19:0/35026444 wait complete. 2024-04-23T18:30:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 -- start start 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 Processor -- start 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 0x560db6f06640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f06b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6f0cc10 0x560db6f070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe681acd40 con 0x560db6f0cc10 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe681ac670 con 0x560db6efd520 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe681a98d0 con 0x560db6efde40 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe70b7e740 10 client.0 _ll_register_callbacks cb 0x7efe5c014870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6f0cc10 0x560db6f070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6f0cc10 0x560db6f070c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40128/0 (socket says 172.21.15.19:40128) 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.748+0000 7efe66ffd640 1 -- 172.21.15.19:0/1558943593 learned_addr learned my addr 172.21.15.19:0/1558943593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe66ffd640 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 0x560db6f06640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe677fe640 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f06b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe66ffd640 1 -- 172.21.15.19:0/1558943593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 msgr2=0x560db6f06b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe66ffd640 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f06b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe66ffd640 1 -- 172.21.15.19:0/1558943593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 msgr2=0x560db6f06640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe66ffd640 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 0x560db6f06640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe66ffd640 1 -- 172.21.15.19:0/1558943593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db6f82340 con 0x560db6f0cc10 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe66ffd640 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6f0cc10 0x560db6f070c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005890 tx=0x7efe5c014bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899743652d3882df server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe46ffd640 1 -- 172.21.15.19:0/1558943593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c036040 con 0x560db6f0cc10 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe46ffd640 1 -- 172.21.15.19:0/1558943593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7efe5c011c20 con 0x560db6f0cc10 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe70b7e740 1 -- 172.21.15.19:0/1558943593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe5c012b60 con 0x560db6f0cc10 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe46ffd640 1 -- 172.21.15.19:0/1558943593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe5c02b040 con 0x560db6f0cc10 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe70b7e740 10 client.16209 fetch_fsmap learned FSMap version 27 2024-04-23T18:30:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.749+0000 7efe70b7e740 1 -- 172.21.15.19:0/1558943593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560db6f83630 con 0x560db6f0cc10 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe46ffd640 1 -- 172.21.15.19:0/1558943593 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe5c012b60 con 0x560db6f0cc10 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe46ffd640 1 -- 172.21.15.19:0/1558943593 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efe5c03c070 con 0x560db6f0cc10 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe70b7e740 10 client.16209 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe70b7e740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe70b7e740 20 client.16209 populate_metadata read hostname 'smithi019' 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe70b7e740 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560db6f83c40 0x560db6f86060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe70b7e740 4 client.16209 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe70b7e740 1 -- 172.21.15.19:0/1558943593 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560db6e6a200 con 0x560db6f83c40 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe677fe640 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560db6f83c40 0x560db6f86060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe677fe640 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560db6f83c40 0x560db6f86060 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e802d2be02abe50b server_cookie=cc19afdb246d4826 in_seq=0 out_seq=0 2024-04-23T18:30:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.750+0000 7efe46ffd640 10 client.16209 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe46ffd640 1 -- 172.21.15.19:0/1558943593 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560db6e6a200 con 0x560db6f83c40 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe46ffd640 10 client.16209 handle_command_reply: tid=0 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 client.16209 shutdown 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 -- 172.21.15.19:0/1558943593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6f0cc10 msgr2=0x560db6f070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6f0cc10 0x560db6f070c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005890 tx=0x7efe5c014bc0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 -- 172.21.15.19:0/1558943593 shutdown_connections 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560db6f83c40 0x560db6f86060 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560db6f0cc10 0x560db6f070c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560db6efde40 0x560db6f06b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 --2- 172.21.15.19:0/1558943593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560db6efd520 0x560db6f06640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe46ffd640 0 client.16209 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 -- 172.21.15.19:0/1558943593 >> 172.21.15.19:0/1558943593 conn(0x560db6ef9b30 msgr2=0x560db6ef8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 -- 172.21.15.19:0/1558943593 shutdown_connections 2024-04-23T18:30:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 1 -- 172.21.15.19:0/1558943593 wait complete. 2024-04-23T18:30:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.767+0000 7efe70b7e740 20 client.16209 trim_cache size 0 max 16384 2024-04-23T18:30:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:30:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:30:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:30:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:30:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 -- 172.21.15.19:0/4221752270 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efe500a3dc0 msgr2=0x7efe500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efe500a3dc0 0x7efe500a6280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe600bd510 tx=0x7efe600a8040 comp rx=0 tx=0).stop 2024-04-23T18:30:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 -- 172.21.15.19:0/4221752270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6810c520 msgr2=0x7efe6819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6810c520 0x7efe6819cf10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efe60006010 tx=0x7efe6002fc00 comp rx=0 tx=0).stop 2024-04-23T18:30:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 -- 172.21.15.19:0/4221752270 shutdown_connections 2024-04-23T18:30:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efe500a3dc0 0x7efe500a6280 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe68103b00 0x7efe68198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6810c520 0x7efe6819cf10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 --2- 172.21.15.19:0/4221752270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe681031e0 0x7efe68198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 -- 172.21.15.19:0/4221752270 >> 172.21.15.19:0/4221752270 conn(0x7efe68077fd0 msgr2=0x7efe680fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 -- 172.21.15.19:0/4221752270 shutdown_connections 2024-04-23T18:30:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:20.768+0000 7efe6dd8e640 1 -- 172.21.15.19:0/4221752270 wait complete. 2024-04-23T18:30:21.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:21 smithi019 ceph-mon[29215]: pgmap v129: 97 pgs: 97 active+clean; 4.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 21 op/s 2024-04-23T18:30:21.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:21 smithi044 ceph-mon[32458]: pgmap v129: 97 pgs: 97 active+clean; 4.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 21 op/s 2024-04-23T18:30:21.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:21 smithi155 ceph-mon[33468]: pgmap v129: 97 pgs: 97 active+clean; 4.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 21 op/s 2024-04-23T18:30:23.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:23 smithi019 ceph-mon[29215]: pgmap v130: 97 pgs: 97 active+clean; 4.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-04-23T18:30:23.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:23 smithi044 ceph-mon[32458]: pgmap v130: 97 pgs: 97 active+clean; 4.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-04-23T18:30:23.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:23 smithi155 ceph-mon[33468]: pgmap v130: 97 pgs: 97 active+clean; 4.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-04-23T18:30:25.614 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:25 smithi044 ceph-mon[32458]: pgmap v131: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T18:30:25.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:25 smithi019 ceph-mon[29215]: pgmap v131: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T18:30:25.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:25 smithi155 ceph-mon[33468]: pgmap v131: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T18:30:27.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:27 smithi019 ceph-mon[29215]: pgmap v132: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:27.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:27 smithi044 ceph-mon[32458]: pgmap v132: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:27.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:27 smithi155 ceph-mon[33468]: pgmap v132: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:29.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:29 smithi019 ceph-mon[29215]: pgmap v133: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:29.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:29 smithi044 ceph-mon[32458]: pgmap v133: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:29.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:29 smithi155 ceph-mon[33468]: pgmap v133: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:31.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:31 smithi019 ceph-mon[29215]: pgmap v134: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T18:30:31.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:31 smithi044 ceph-mon[32458]: pgmap v134: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T18:30:31.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:31 smithi155 ceph-mon[33468]: pgmap v134: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T18:30:33.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:33 smithi019 ceph-mon[29215]: pgmap v135: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T18:30:33.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:33 smithi044 ceph-mon[32458]: pgmap v135: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T18:30:33.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:33 smithi155 ceph-mon[33468]: pgmap v135: 97 pgs: 97 active+clean; 4.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T18:30:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.867+0000 7fbe21308640 1 -- 172.21.15.19:0/3059566270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1c1031e0 msgr2=0x7fbe1c1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.867+0000 7fbe21308640 1 --2- 172.21.15.19:0/3059566270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1c1031e0 0x7fbe1c1035c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009900 tx=0x7fbe0c02a510 comp rx=0 tx=0).stop 2024-04-23T18:30:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.868+0000 7fbe21308640 1 -- 172.21.15.19:0/3059566270 shutdown_connections 2024-04-23T18:30:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.868+0000 7fbe21308640 1 --2- 172.21.15.19:0/3059566270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe1c10c520 0x7fbe1c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.868+0000 7fbe21308640 1 --2- 172.21.15.19:0/3059566270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe1c103b00 0x7fbe1c10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.868+0000 7fbe21308640 1 --2- 172.21.15.19:0/3059566270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1c1031e0 0x7fbe1c1035c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.868+0000 7fbe21308640 1 -- 172.21.15.19:0/3059566270 >> 172.21.15.19:0/3059566270 conn(0x7fbe1c077fd0 msgr2=0x7fbe1c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.868+0000 7fbe21308640 1 -- 172.21.15.19:0/3059566270 shutdown_connections 2024-04-23T18:30:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.868+0000 7fbe21308640 1 -- 172.21.15.19:0/3059566270 wait complete. 2024-04-23T18:30:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.868+0000 7fbe21308640 1 Processor -- start 2024-04-23T18:30:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe21308640 1 -- start start 2024-04-23T18:30:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe21308640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe1c1031e0 0x7fbe1c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe21308640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1c103b00 0x7fbe1c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe21308640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe1c10c520 0x7fbe1c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe21308640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c112190 con 0x7fbe1c103b00 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe21308640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c112010 con 0x7fbe1c1031e0 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe21308640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c112310 con 0x7fbe1c10c520 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1c103b00 0x7fbe1c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1c103b00 0x7fbe1c198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42302/0 (socket says 172.21.15.19:42302) 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/1999669708 learned_addr learned my addr 172.21.15.19:0/1999669708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1b7fe640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe1c10c520 0x7fbe1c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1affd640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe1c1031e0 0x7fbe1c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/1999669708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe1c10c520 msgr2=0x7fbe1c19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe1c10c520 0x7fbe1c19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/1999669708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe1c1031e0 msgr2=0x7fbe1c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe1c1031e0 0x7fbe1c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/1999669708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c19d6a0 con 0x7fbe1c103b00 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.869+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1c103b00 0x7fbe1c198bf0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000d950 tx=0x7fbe1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.870+0000 7fbe03fff640 1 -- 172.21.15.19:0/1999669708 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10017070 con 0x7fbe1c103b00 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.870+0000 7fbe03fff640 1 -- 172.21.15.19:0/1999669708 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe10009c30 con 0x7fbe1c103b00 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.870+0000 7fbe21308640 1 -- 172.21.15.19:0/1999669708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c19d990 con 0x7fbe1c103b00 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.870+0000 7fbe03fff640 1 -- 172.21.15.19:0/1999669708 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000f620 con 0x7fbe1c103b00 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.870+0000 7fbe21308640 1 -- 172.21.15.19:0/1999669708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c1a5220 con 0x7fbe1c103b00 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.872+0000 7fbe03fff640 1 -- 172.21.15.19:0/1999669708 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbe10010460 con 0x7fbe1c103b00 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.872+0000 7fbe24029740 1 Processor -- start 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.872+0000 7fbe24029740 1 -- start start 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.872+0000 7fbe24029740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cc5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe24029740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe24029740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1cd2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe24029740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9a1c2a970 con 0x55c9a1cc5f40 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe24029740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c9a1bc74a0 con 0x55c9a1cc5620 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe24029740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c9a1c26930 con 0x55c9a1cd0800 2024-04-23T18:30:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe03fff640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe040a3dc0 0x7fbe040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cc5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42308/0 (socket says 172.21.15.19:42308) 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1affd640 1 -- 172.21.15.19:0/1744559155 learned_addr learned my addr 172.21.15.19:0/1744559155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1b7fe640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe040a3dc0 0x7fbe040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/1744559155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1cd2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe03fff640 1 -- 172.21.15.19:0/1999669708 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbe100d0490 con 0x7fbe1c103b00 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1affd640 1 -- 172.21.15.19:0/1744559155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 msgr2=0x55c9a1cd2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1affd640 1 --2- 172.21.15.19:0/1744559155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1cd2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1affd640 1 -- 172.21.15.19:0/1744559155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 msgr2=0x55c9a1cc5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1affd640 1 --2- 172.21.15.19:0/1744559155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cc5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1affd640 1 -- 172.21.15.19:0/1744559155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9a193d210 con 0x55c9a1cc5f40 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1b7fe640 1 --2- 172.21.15.19:0/1744559155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cc5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/1744559155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1cd2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1affd640 1 --2- 172.21.15.19:0/1744559155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd02c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009900 tx=0x7fbe0c032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29a81f3b12cf5df server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.873+0000 7fbe1b7fe640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe040a3dc0 0x7fbe040a6280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009fd0 tx=0x7fbe14009600 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe017fa640 1 -- 172.21.15.19:0/1744559155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c03f070 con 0x55c9a1cc5f40 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe017fa640 1 -- 172.21.15.19:0/1744559155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe0c0316a0 con 0x55c9a1cc5f40 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe017fa640 1 -- 172.21.15.19:0/1744559155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c035070 con 0x55c9a1cc5f40 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe24029740 1 -- 172.21.15.19:0/1744559155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 msgr2=0x55c9a1cd02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe24029740 1 --2- 172.21.15.19:0/1744559155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd02c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009900 tx=0x7fbe0c032c40 comp rx=0 tx=0).stop 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe24029740 1 -- 172.21.15.19:0/1744559155 shutdown_connections 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe24029740 1 --2- 172.21.15.19:0/1744559155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1cd2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe24029740 1 --2- 172.21.15.19:0/1744559155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd02c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe24029740 1 --2- 172.21.15.19:0/1744559155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cc5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe24029740 1 -- 172.21.15.19:0/1744559155 >> 172.21.15.19:0/1744559155 conn(0x55c9a1cc1b30 msgr2=0x55c9a1cc1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe24029740 1 -- 172.21.15.19:0/1744559155 shutdown_connections 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.874+0000 7fbe24029740 1 -- 172.21.15.19:0/1744559155 wait complete. 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe24029740 1 -- start start 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe24029740 1 Processor -- start 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe24029740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cd00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe24029740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1ccd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe24029740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1ccd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe24029740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9a1cd33d0 con 0x55c9a1cc5f40 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe24029740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c9a1bc50b0 con 0x55c9a1cc5620 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe24029740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c9a1cd3550 con 0x55c9a1cd0800 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe24029740 10 client.0 _ll_register_callbacks cb 0x55c9a1bb2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe1a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1ccd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe1a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1ccd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42318/0 (socket says 172.21.15.19:42318) 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/1864956372 learned_addr learned my addr 172.21.15.19:0/1864956372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe1affd640 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cd00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.875+0000 7fbe1b7fe640 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1ccd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/1864956372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 msgr2=0x55c9a1ccd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1ccd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/1864956372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 msgr2=0x55c9a1cd00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cd00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/1864956372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9a1a09620 con 0x55c9a1cc5f40 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbe1b7fe640 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1ccd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1ccd3a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbe10098860 tx=0x7fbe10018ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d459b9934bffd8e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbdfeff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbdfeff5640 1 -- 172.21.15.19:0/1864956372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10020040 con 0x55c9a1cc5f40 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbdfeff5640 1 -- 172.21.15.19:0/1864956372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbe10016170 con 0x55c9a1cc5f40 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbe24029740 1 -- 172.21.15.19:0/1864956372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe0c0316a0 con 0x55c9a1cc5f40 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbdfeff5640 1 -- 172.21.15.19:0/1864956372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe10009de0 con 0x55c9a1cc5f40 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbe24029740 10 client.16221 fetch_fsmap learned FSMap version 27 2024-04-23T18:30:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbe24029740 1 -- 172.21.15.19:0/1864956372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9a1cdf290 con 0x55c9a1cc5f40 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.876+0000 7fbdfeff5640 1 -- 172.21.15.19:0/1864956372 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe0c0316a0 con 0x55c9a1cc5f40 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.877+0000 7fbdfeff5640 1 -- 172.21.15.19:0/1864956372 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbe100cde30 con 0x55c9a1cc5f40 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.877+0000 7fbe24029740 10 client.16221 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.877+0000 7fbe24029740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.877+0000 7fbe24029740 20 client.16221 populate_metadata read hostname 'smithi019' 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.877+0000 7fbe24029740 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c9a1ce07a0 0x55c9a1ce2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.877+0000 7fbe24029740 4 client.16221 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.877+0000 7fbe24029740 1 -- 172.21.15.19:0/1864956372 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9a1c38390 con 0x55c9a1ce07a0 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.877+0000 7fbe1affd640 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c9a1ce07a0 0x55c9a1ce2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.877+0000 7fbe1affd640 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c9a1ce07a0 0x55c9a1ce2b90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724da0bbc3b267d6 server_cookie=5748b91c71c077de in_seq=0 out_seq=0 2024-04-23T18:30:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.877+0000 7fbdfeff5640 10 client.16221 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbdfeff5640 1 -- 172.21.15.19:0/1864956372 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9a1c38390 con 0x55c9a1ce07a0 2024-04-23T18:30:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbdfeff5640 10 client.16221 handle_command_reply: tid=0 2024-04-23T18:30:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 client.16221 shutdown 2024-04-23T18:30:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 -- 172.21.15.19:0/1864956372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 msgr2=0x55c9a1ccd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1ccd3a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbe10098860 tx=0x7fbe10018ca0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 -- 172.21.15.19:0/1864956372 shutdown_connections 2024-04-23T18:30:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c9a1ce07a0 0x55c9a1ce2b90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1ccd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1ccd3a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbdfeff5640 0 client.16221 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 --2- 172.21.15.19:0/1864956372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cd00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 -- 172.21.15.19:0/1864956372 >> 172.21.15.19:0/1864956372 conn(0x55c9a1cc1b30 msgr2=0x55c9a1cc78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 -- 172.21.15.19:0/1864956372 shutdown_connections 2024-04-23T18:30:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.878+0000 7fbe24029740 1 -- 172.21.15.19:0/1864956372 wait complete. 2024-04-23T18:30:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.879+0000 7fbe24029740 20 client.16221 trim_cache size 0 max 16384 2024-04-23T18:30:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe24029740 1 Processor -- start 2024-04-23T18:30:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe24029740 1 -- start start 2024-04-23T18:30:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe24029740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cc5620 0x55c9a1cdc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe24029740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5f40 0x55c9a1ce8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe24029740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cd0800 0x55c9a1ce89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe24029740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1ada30 con 0x55c9a1cd0800 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe24029740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1af570 con 0x55c9a1cc5f40 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe24029740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1ae100 con 0x55c9a1cc5620 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cd0800 0x55c9a1ce89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cd0800 0x55c9a1ce89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42326/0 (socket says 172.21.15.19:42326) 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1b7fe640 1 -- 172.21.15.19:0/3793026786 learned_addr learned my addr 172.21.15.19:0/3793026786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1affd640 1 --2- 172.21.15.19:0/3793026786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cc5620 0x55c9a1cdc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/3793026786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5f40 0x55c9a1ce8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1b7fe640 1 -- 172.21.15.19:0/3793026786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cc5620 msgr2=0x55c9a1cdc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1b7fe640 1 --2- 172.21.15.19:0/3793026786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cc5620 0x55c9a1cdc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1b7fe640 1 -- 172.21.15.19:0/3793026786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5f40 msgr2=0x55c9a1ce8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1b7fe640 1 --2- 172.21.15.19:0/3793026786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5f40 0x55c9a1ce8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1b7fe640 1 -- 172.21.15.19:0/3793026786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9a1ce2f00 con 0x55c9a1cd0800 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.894+0000 7fbe1affd640 1 --2- 172.21.15.19:0/3793026786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cc5620 0x55c9a1cdc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe1b7fe640 1 --2- 172.21.15.19:0/3793026786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cd0800 0x55c9a1ce89c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbe140129d0 tx=0x7fbe1400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3f3bf2b9f72b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe00ff9640 1 -- 172.21.15.19:0/3793026786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400ae50 con 0x55c9a1cd0800 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe00ff9640 1 -- 172.21.15.19:0/3793026786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe1401e8f0 con 0x55c9a1cd0800 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe00ff9640 1 -- 172.21.15.19:0/3793026786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe140052f0 con 0x55c9a1cd0800 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe24029740 1 -- 172.21.15.19:0/3793026786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cd0800 msgr2=0x55c9a1ce89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe24029740 1 --2- 172.21.15.19:0/3793026786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cd0800 0x55c9a1ce89c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbe140129d0 tx=0x7fbe1400c000 comp rx=0 tx=0).stop 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe24029740 1 -- 172.21.15.19:0/3793026786 shutdown_connections 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe24029740 1 --2- 172.21.15.19:0/3793026786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5f40 0x55c9a1ce8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe24029740 1 --2- 172.21.15.19:0/3793026786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cd0800 0x55c9a1ce89c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe24029740 1 --2- 172.21.15.19:0/3793026786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cc5620 0x55c9a1cdc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe24029740 1 -- 172.21.15.19:0/3793026786 >> 172.21.15.19:0/3793026786 conn(0x55c9a1cc1b30 msgr2=0x55c9a1cc0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe24029740 1 -- 172.21.15.19:0/3793026786 shutdown_connections 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.895+0000 7fbe24029740 1 -- 172.21.15.19:0/3793026786 wait complete. 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe24029740 1 -- start start 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe24029740 1 Processor -- start 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe24029740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cd8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe24029740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe24029740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1cd6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe24029740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1aeea0 con 0x55c9a1cc5f40 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe24029740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1ae7d0 con 0x55c9a1cc5620 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe24029740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1aba10 con 0x55c9a1cd0800 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe24029740 10 client.0 _ll_register_callbacks cb 0x7fbe1009bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe1a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe1a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42340/0 (socket says 172.21.15.19:42340) 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/3618679105 learned_addr learned my addr 172.21.15.19:0/3618679105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe1b7fe640 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1cd6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.896+0000 7fbe1affd640 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cd8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/3618679105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 msgr2=0x55c9a1cd6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1cd6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/3618679105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 msgr2=0x55c9a1cd8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cd8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe1a7fc640 1 -- 172.21.15.19:0/3618679105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9a1cd6a60 con 0x55c9a1cc5f40 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe1affd640 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cd8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe1a7fc640 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd5e30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbe10098dd0 tx=0x7fbe100a4fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d0a7655df9ace5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbdfeff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbdfeff5640 1 -- 172.21.15.19:0/3618679105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10014020 con 0x55c9a1cc5f40 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbdfeff5640 1 -- 172.21.15.19:0/3618679105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbe1009b850 con 0x55c9a1cc5f40 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe24029740 1 -- 172.21.15.19:0/3618679105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1401e8f0 con 0x55c9a1cc5f40 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbdfeff5640 1 -- 172.21.15.19:0/3618679105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe10020070 con 0x55c9a1cc5f40 2024-04-23T18:30:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe24029740 10 client.16227 fetch_fsmap learned FSMap version 27 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe24029740 1 -- 172.21.15.19:0/3618679105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9a1d46c10 con 0x55c9a1cc5f40 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbdfeff5640 1 -- 172.21.15.19:0/3618679105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe1401e8f0 con 0x55c9a1cc5f40 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbdfeff5640 1 -- 172.21.15.19:0/3618679105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbe10010750 con 0x55c9a1cc5f40 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.897+0000 7fbe24029740 10 client.16227 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.898+0000 7fbe24029740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.898+0000 7fbe24029740 20 client.16227 populate_metadata read hostname 'smithi019' 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.898+0000 7fbe24029740 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c9a1d473e0 0x55c9a1d49800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.898+0000 7fbe24029740 4 client.16227 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.898+0000 7fbe24029740 1 -- 172.21.15.19:0/3618679105 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9a1ccdc70 con 0x55c9a1d473e0 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.898+0000 7fbe1affd640 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c9a1d473e0 0x55c9a1d49800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.898+0000 7fbe1affd640 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c9a1d473e0 0x55c9a1d49800 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a58fd5e676157f7 server_cookie=5fc6b3855d4e9dee in_seq=0 out_seq=0 2024-04-23T18:30:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.898+0000 7fbdfeff5640 10 client.16227 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbdfeff5640 1 -- 172.21.15.19:0/3618679105 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9a1ccdc70 con 0x55c9a1d473e0 2024-04-23T18:30:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbdfeff5640 10 client.16227 handle_command_reply: tid=0 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 client.16227 shutdown 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 -- 172.21.15.19:0/3618679105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 msgr2=0x55c9a1cd5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd5e30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbe10098dd0 tx=0x7fbe100a4fa0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 -- 172.21.15.19:0/3618679105 shutdown_connections 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c9a1d473e0 0x55c9a1d49800 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9a1cd0800 0x55c9a1cd6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9a1cc5f40 0x55c9a1cd5e30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbdfeff5640 0 client.16227 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 --2- 172.21.15.19:0/3618679105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9a1cc5620 0x55c9a1cd8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 -- 172.21.15.19:0/3618679105 >> 172.21.15.19:0/3618679105 conn(0x55c9a1cc1b30 msgr2=0x55c9a1cc0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 -- 172.21.15.19:0/3618679105 shutdown_connections 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.915+0000 7fbe24029740 1 -- 172.21.15.19:0/3618679105 wait complete. 2024-04-23T18:30:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe24029740 20 client.16227 trim_cache size 0 max 16384 2024-04-23T18:30:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:30:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:30:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:30:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 -- 172.21.15.19:0/1999669708 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe040a3dc0 msgr2=0x7fbe040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe040a3dc0 0x7fbe040a6280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009fd0 tx=0x7fbe14009600 comp rx=0 tx=0).stop 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 -- 172.21.15.19:0/1999669708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1c103b00 msgr2=0x7fbe1c198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1c103b00 0x7fbe1c198bf0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000d950 tx=0x7fbe1000de10 comp rx=0 tx=0).stop 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 -- 172.21.15.19:0/1999669708 shutdown_connections 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbe040a3dc0 0x7fbe040a6280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe1c10c520 0x7fbe1c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1c103b00 0x7fbe1c198bf0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 --2- 172.21.15.19:0/1999669708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe1c1031e0 0x7fbe1c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 -- 172.21.15.19:0/1999669708 >> 172.21.15.19:0/1999669708 conn(0x7fbe1c077fd0 msgr2=0x7fbe1c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 -- 172.21.15.19:0/1999669708 shutdown_connections 2024-04-23T18:30:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:33.916+0000 7fbe21308640 1 -- 172.21.15.19:0/1999669708 wait complete. 2024-04-23T18:30:35.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:35 smithi019 ceph-mon[29215]: pgmap v136: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-04-23T18:30:35.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:35 smithi044 ceph-mon[32458]: pgmap v136: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-04-23T18:30:35.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:35 smithi155 ceph-mon[33468]: pgmap v136: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-04-23T18:30:37.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:37 smithi019 ceph-mon[29215]: pgmap v137: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:37.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:37 smithi044 ceph-mon[32458]: pgmap v137: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:37.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:37 smithi155 ceph-mon[33468]: pgmap v137: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:39.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:39 smithi019 ceph-mon[29215]: pgmap v138: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:39.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:39 smithi044 ceph-mon[32458]: pgmap v138: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:39.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:39 smithi155 ceph-mon[33468]: pgmap v138: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:41.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:41 smithi019 ceph-mon[29215]: pgmap v139: 97 pgs: 97 active+clean; 4.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T18:30:41.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:41 smithi044 ceph-mon[32458]: pgmap v139: 97 pgs: 97 active+clean; 4.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T18:30:41.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:41 smithi155 ceph-mon[33468]: pgmap v139: 97 pgs: 97 active+clean; 4.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T18:30:43.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:43 smithi019 ceph-mon[29215]: pgmap v140: 97 pgs: 97 active+clean; 4.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:43.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:43 smithi044 ceph-mon[32458]: pgmap v140: 97 pgs: 97 active+clean; 4.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:43.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:43 smithi155 ceph-mon[33468]: pgmap v140: 97 pgs: 97 active+clean; 4.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T18:30:44.183 INFO:tasks.quiescer.fs.[cephfs]:Total duration (47.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:30:44.183 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.6 2024-04-23T18:30:44.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:30:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.308+0000 7f47e267c640 1 -- 172.21.15.19:0/1872954282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc102bf0 msgr2=0x7f47dc106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.308+0000 7f47e267c640 1 --2- 172.21.15.19:0/1872954282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc102bf0 0x7f47dc106ba0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009900 tx=0x7f47d002a410 comp rx=0 tx=0).stop 2024-04-23T18:30:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.308+0000 7f47e267c640 1 -- 172.21.15.19:0/1872954282 shutdown_connections 2024-04-23T18:30:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.308+0000 7f47e267c640 1 --2- 172.21.15.19:0/1872954282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47dc107280 0x7f47dc10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.308+0000 7f47e267c640 1 --2- 172.21.15.19:0/1872954282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc102bf0 0x7f47dc106ba0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.308+0000 7f47e267c640 1 --2- 172.21.15.19:0/1872954282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47dc102240 0x7f47dc102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.308+0000 7f47e267c640 1 -- 172.21.15.19:0/1872954282 >> 172.21.15.19:0/1872954282 conn(0x7f47dc0790e0 msgr2=0x7f47dc0794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.308+0000 7f47e267c640 1 -- 172.21.15.19:0/1872954282 shutdown_connections 2024-04-23T18:30:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.308+0000 7f47e267c640 1 -- 172.21.15.19:0/1872954282 wait complete. 2024-04-23T18:30:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e267c640 1 Processor -- start 2024-04-23T18:30:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e267c640 1 -- start start 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e267c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47dc102240 0x7f47dc074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e267c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47dc107280 0x7f47dc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e267c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc074cd0 0x7f47dc1a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e267c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc1121c0 con 0x7f47dc074cd0 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e267c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc112040 con 0x7f47dc107280 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e267c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc112340 con 0x7f47dc102240 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e0c19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc074cd0 0x7f47dc1a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e0c19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc074cd0 0x7f47dc1a69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54802/0 (socket says 172.21.15.19:54802) 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47e0c19640 1 -- 172.21.15.19:0/372685456 learned_addr learned my addr 172.21.15.19:0/372685456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47db7fe640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47dc107280 0x7f47dc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.309+0000 7f47dbfff640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47dc102240 0x7f47dc074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47e0c19640 1 -- 172.21.15.19:0/372685456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47dc102240 msgr2=0x7f47dc074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47e0c19640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47dc102240 0x7f47dc074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47e0c19640 1 -- 172.21.15.19:0/372685456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47dc107280 msgr2=0x7f47dc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47e0c19640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47dc107280 0x7f47dc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47e0c19640 1 -- 172.21.15.19:0/372685456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc1a70d0 con 0x7f47dc074cd0 2024-04-23T18:30:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47e0c19640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc074cd0 0x7f47dc1a69d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f47d400eaa0 tx=0x7f47d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47d97fa640 1 -- 172.21.15.19:0/372685456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d400cdb0 con 0x7f47dc074cd0 2024-04-23T18:30:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47d97fa640 1 -- 172.21.15.19:0/372685456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d4005020 con 0x7f47dc074cd0 2024-04-23T18:30:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47d97fa640 1 -- 172.21.15.19:0/372685456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d40116d0 con 0x7f47dc074cd0 2024-04-23T18:30:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47e267c640 1 -- 172.21.15.19:0/372685456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47dc1a73c0 con 0x7f47dc074cd0 2024-04-23T18:30:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.310+0000 7f47e267c640 1 -- 172.21.15.19:0/372685456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47dc1a7890 con 0x7f47dc074cd0 2024-04-23T18:30:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.311+0000 7f47e267c640 1 -- 172.21.15.19:0/372685456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47dc0691c0 con 0x7f47dc074cd0 2024-04-23T18:30:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.313+0000 7f47d97fa640 1 -- 172.21.15.19:0/372685456 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f47d40051d0 con 0x7f47dc074cd0 2024-04-23T18:30:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.314+0000 7f47d97fa640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47c40a3d50 0x7f47c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.314+0000 7f47dbfff640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47c40a3d50 0x7f47c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.314+0000 7f47d97fa640 1 -- 172.21.15.19:0/372685456 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f47d40d0c70 con 0x7f47dc074cd0 2024-04-23T18:30:44.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.314+0000 7f47dbfff640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47c40a3d50 0x7f47c40a6210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47cc005fd0 tx=0x7f47cc00b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.317+0000 7f47d97fa640 1 -- 172.21.15.19:0/372685456 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47d4016030 con 0x7f47dc074cd0 2024-04-23T18:30:44.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.547+0000 7f47e267c640 1 -- 172.21.15.19:0/372685456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47dc075780 con 0x7f47dc074cd0 2024-04-23T18:30:44.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.548+0000 7f47d97fa640 1 -- 172.21.15.19:0/372685456 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f47dc075780 con 0x7f47dc074cd0 2024-04-23T18:30:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:30:44.541 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:30:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:30:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.550+0000 7f47e267c640 1 -- 172.21.15.19:0/372685456 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47c40a3d50 msgr2=0x7f47c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.550+0000 7f47e267c640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47c40a3d50 0x7f47c40a6210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47cc005fd0 tx=0x7f47cc00b390 comp rx=0 tx=0).stop 2024-04-23T18:30:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.550+0000 7f47e267c640 1 -- 172.21.15.19:0/372685456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc074cd0 msgr2=0x7f47dc1a69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.550+0000 7f47e267c640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc074cd0 0x7f47dc1a69d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f47d400eaa0 tx=0x7f47d400ef60 comp rx=0 tx=0).stop 2024-04-23T18:30:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.551+0000 7f47e267c640 1 -- 172.21.15.19:0/372685456 shutdown_connections 2024-04-23T18:30:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.551+0000 7f47e267c640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47c40a3d50 0x7f47c40a6210 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.551+0000 7f47e267c640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47dc107280 0x7f47dc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.551+0000 7f47e267c640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc074cd0 0x7f47dc1a69d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.551+0000 7f47e267c640 1 --2- 172.21.15.19:0/372685456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47dc102240 0x7f47dc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.551+0000 7f47e267c640 1 -- 172.21.15.19:0/372685456 >> 172.21.15.19:0/372685456 conn(0x7f47dc0790e0 msgr2=0x7f47dc100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:44.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.551+0000 7f47e267c640 1 -- 172.21.15.19:0/372685456 shutdown_connections 2024-04-23T18:30:44.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.551+0000 7f47e267c640 1 -- 172.21.15.19:0/372685456 wait complete. 2024-04-23T18:30:44.554 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 83.6 --await-for 42.0' 2024-04-23T18:30:44.554 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 83.6 --await-for 42.0 2024-04-23T18:30:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.673+0000 7f7568435640 1 -- 172.21.15.19:0/3633251734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7560101110 msgr2=0x7f75601014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.673+0000 7f7568435640 1 --2- 172.21.15.19:0/3633251734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7560101110 0x7f75601014f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f755c009900 tx=0x7f755c02a410 comp rx=0 tx=0).stop 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.673+0000 7f7568435640 1 -- 172.21.15.19:0/3633251734 shutdown_connections 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.673+0000 7f7568435640 1 --2- 172.21.15.19:0/3633251734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7560106190 0x7f756010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.673+0000 7f7568435640 1 --2- 172.21.15.19:0/3633251734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7560101ac0 0x7f7560105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.673+0000 7f7568435640 1 --2- 172.21.15.19:0/3633251734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7560101110 0x7f75601014f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.673+0000 7f7568435640 1 -- 172.21.15.19:0/3633251734 >> 172.21.15.19:0/3633251734 conn(0x7f7560077fb0 msgr2=0x7f75600783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.673+0000 7f7568435640 1 -- 172.21.15.19:0/3633251734 shutdown_connections 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.673+0000 7f7568435640 1 -- 172.21.15.19:0/3633251734 wait complete. 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f7568435640 1 Processor -- start 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f7568435640 1 -- start start 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f7568435640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7560101110 0x7f7560102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f7568435640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7560106190 0x7f7560103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f7568435640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601035d0 0x7f75601a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f7568435640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7560111150 con 0x7f75601035d0 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f7568435640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7560110fd0 con 0x7f7560101110 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f7568435640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75601112d0 con 0x7f7560106190 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75669d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601035d0 0x7f75601a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75669d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601035d0 0x7f75601a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54818/0 (socket says 172.21.15.19:54818) 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75669d2640 1 -- 172.21.15.19:0/2184998670 learned_addr learned my addr 172.21.15.19:0/2184998670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75659d0640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7560106190 0x7f7560103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75661d1640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7560101110 0x7f7560102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75669d2640 1 -- 172.21.15.19:0/2184998670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7560106190 msgr2=0x7f7560103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75669d2640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7560106190 0x7f7560103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75669d2640 1 -- 172.21.15.19:0/2184998670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7560101110 msgr2=0x7f7560102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75669d2640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7560101110 0x7f7560102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75669d2640 1 -- 172.21.15.19:0/2184998670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75601a7110 con 0x7f75601035d0 2024-04-23T18:30:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75661d1640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7560101110 0x7f7560102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.674+0000 7f75669d2640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601035d0 0x7f75601a6a10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f755800cab0 tx=0x7f755800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.675+0000 7f75577fe640 1 -- 172.21.15.19:0/2184998670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755800ad00 con 0x7f75601035d0 2024-04-23T18:30:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.675+0000 7f75577fe640 1 -- 172.21.15.19:0/2184998670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755800f900 con 0x7f75601035d0 2024-04-23T18:30:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.675+0000 7f75577fe640 1 -- 172.21.15.19:0/2184998670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558018940 con 0x7f75601035d0 2024-04-23T18:30:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.675+0000 7f7568435640 1 -- 172.21.15.19:0/2184998670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75601a7400 con 0x7f75601035d0 2024-04-23T18:30:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.675+0000 7f7568435640 1 -- 172.21.15.19:0/2184998670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75601a7900 con 0x7f75601035d0 2024-04-23T18:30:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.677+0000 7f75577fe640 1 -- 172.21.15.19:0/2184998670 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f755800f450 con 0x7f75601035d0 2024-04-23T18:30:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.677+0000 7f756b22e740 1 Processor -- start 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.677+0000 7f756b22e740 1 -- start start 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f756b22e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c70a0 0x56481e1c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f756b22e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f756b22e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1d2090 0x56481e1d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f756b22e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56481e12c370 con 0x56481e1c70a0 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f756b22e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56481e0c92e0 con 0x56481e1c79c0 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f756b22e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56481e128330 con 0x56481e1d2090 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75577fe640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f754c0a3d50 0x7f754c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75659d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c70a0 0x56481e1c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75659d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c70a0 0x56481e1c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54822/0 (socket says 172.21.15.19:54822) 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75659d0640 1 -- 172.21.15.19:0/2760184312 learned_addr learned my addr 172.21.15.19:0/2760184312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75669d2640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f754c0a3d50 0x7f754c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75661d1640 1 --2- 172.21.15.19:0/2760184312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75577fe640 1 -- 172.21.15.19:0/2184998670 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f75580d0ad0 con 0x7f75601035d0 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75659d0640 1 --2- 172.21.15.19:0/2760184312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1d2090 0x56481e1d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75659d0640 1 -- 172.21.15.19:0/2760184312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1d2090 msgr2=0x56481e1d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75659d0640 1 --2- 172.21.15.19:0/2760184312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1d2090 0x56481e1d4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75659d0640 1 -- 172.21.15.19:0/2760184312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 msgr2=0x56481e1d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75659d0640 1 --2- 172.21.15.19:0/2760184312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75659d0640 1 -- 172.21.15.19:0/2760184312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481df0ad80 con 0x56481e1c70a0 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75669d2640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f754c0a3d50 0x7f754c0a6210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f75580ce850 tx=0x7f7558091f30 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f75659d0640 1 --2- 172.21.15.19:0/2760184312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c70a0 0x56481e1c7480 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7550009c80 tx=0x7f7550009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4182278e8a1ae7e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.678+0000 7f7554bfe640 1 -- 172.21.15.19:0/2760184312 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550010e10 con 0x56481e1c70a0 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f7554bfe640 1 -- 172.21.15.19:0/2760184312 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755001a070 con 0x56481e1c70a0 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f7554bfe640 1 -- 172.21.15.19:0/2760184312 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550014750 con 0x56481e1c70a0 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f756b22e740 1 -- 172.21.15.19:0/2760184312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c70a0 msgr2=0x56481e1c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f756b22e740 1 --2- 172.21.15.19:0/2760184312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c70a0 0x56481e1c7480 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7550009c80 tx=0x7f7550009570 comp rx=0 tx=0).stop 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f756b22e740 1 -- 172.21.15.19:0/2760184312 shutdown_connections 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f756b22e740 1 --2- 172.21.15.19:0/2760184312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f756b22e740 1 --2- 172.21.15.19:0/2760184312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1d2090 0x56481e1d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f756b22e740 1 --2- 172.21.15.19:0/2760184312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c70a0 0x56481e1c7480 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f756b22e740 1 -- 172.21.15.19:0/2760184312 >> 172.21.15.19:0/2760184312 conn(0x56481e1c33c0 msgr2=0x56481e1c37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f756b22e740 1 -- 172.21.15.19:0/2760184312 shutdown_connections 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.679+0000 7f756b22e740 1 -- 172.21.15.19:0/2760184312 wait complete. 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f756b22e740 1 -- start start 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f756b22e740 1 Processor -- start 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f756b22e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f756b22e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c79c0 0x56481e1cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f756b22e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1d2090 0x56481e1cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f756b22e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56481e1d4e20 con 0x56481e1c79c0 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f756b22e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56481e0c6fb0 con 0x56481e1d2090 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f756b22e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56481e1d4fa0 con 0x56481e1c70a0 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f756b22e740 10 client.0 _ll_register_callbacks cb 0x56481e0c69a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75661d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c79c0 0x56481e1cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75661d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c79c0 0x56481e1cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54824/0 (socket says 172.21.15.19:54824) 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75661d1640 1 -- 172.21.15.19:0/781715898 learned_addr learned my addr 172.21.15.19:0/781715898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75659d0640 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1d2090 0x56481e1cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75659d0640 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75661d1640 1 -- 172.21.15.19:0/781715898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 msgr2=0x56481e1cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75661d1640 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75661d1640 1 -- 172.21.15.19:0/781715898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1d2090 msgr2=0x56481e1cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75661d1640 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1d2090 0x56481e1cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75661d1640 1 -- 172.21.15.19:0/781715898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481df6b510 con 0x56481e1c79c0 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.680+0000 7f75659d0640 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1d2090 0x56481e1cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f75661d1640 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c79c0 0x56481e1cf500 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f755c009900 tx=0x7f755c031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2349eece0893190b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f7549ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f7549ffb640 1 -- 172.21.15.19:0/781715898 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c035040 con 0x56481e1c79c0 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f7549ffb640 1 -- 172.21.15.19:0/781715898 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f755c005020 con 0x56481e1c79c0 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f756b22e740 1 -- 172.21.15.19:0/781715898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f755001a070 con 0x56481e1c79c0 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f7549ffb640 1 -- 172.21.15.19:0/781715898 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f755c005220 con 0x56481e1c79c0 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f756b22e740 10 client.16245 fetch_fsmap learned FSMap version 27 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f756b22e740 1 -- 172.21.15.19:0/781715898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56481ddfe670 con 0x56481e1c79c0 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f7549ffb640 1 -- 172.21.15.19:0/781715898 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755001a070 con 0x56481e1c79c0 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f7549ffb640 1 -- 172.21.15.19:0/781715898 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f755c0035f0 con 0x56481e1c79c0 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f756b22e740 10 client.16245 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f756b22e740 10 client.16245 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f756b22e740 20 client.16245 populate_metadata read hostname 'smithi019' 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f756b22e740 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56481e1e2340 0x56481e1e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f756b22e740 4 client.16245 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.681+0000 7f756b22e740 1 -- 172.21.15.19:0/781715898 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56481e139d90 con 0x56481e1e2340 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.682+0000 7f75659d0640 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56481e1e2340 0x56481e1e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.682+0000 7f7549ffb640 10 client.16245 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.682+0000 7f75659d0640 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56481e1e2340 0x56481e1e4760 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b080193511f02a8e server_cookie=1e482deab05ce7a5 in_seq=0 out_seq=0 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f7549ffb640 1 -- 172.21.15.19:0/781715898 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56481e139d90 con 0x56481e1e2340 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f7549ffb640 10 client.16245 handle_command_reply: tid=0 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 client.16245 shutdown 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 -- 172.21.15.19:0/781715898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c79c0 msgr2=0x56481e1cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c79c0 0x56481e1cf500 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f755c009900 tx=0x7f755c031950 comp rx=0 tx=0).stop 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 -- 172.21.15.19:0/781715898 shutdown_connections 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56481e1e2340 0x56481e1e4760 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1d2090 0x56481e1cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f7549ffb640 0 client.16245 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1c79c0 0x56481e1cf500 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 --2- 172.21.15.19:0/781715898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 -- 172.21.15.19:0/781715898 >> 172.21.15.19:0/781715898 conn(0x56481e1c33c0 msgr2=0x56481e1c91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:44.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 -- 172.21.15.19:0/781715898 shutdown_connections 2024-04-23T18:30:44.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 1 -- 172.21.15.19:0/781715898 wait complete. 2024-04-23T18:30:44.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.683+0000 7f756b22e740 20 client.16245 trim_cache size 0 max 16384 2024-04-23T18:30:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f756b22e740 1 Processor -- start 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f756b22e740 1 -- start start 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f756b22e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f756b22e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f756b22e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1ea030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f756b22e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75601afda0 con 0x56481e1d2090 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f756b22e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f75601b18e0 con 0x56481e1c79c0 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f756b22e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75601b0470 con 0x56481e1c70a0 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f75659d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1ea030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f75659d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1ea030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54832/0 (socket says 172.21.15.19:54832) 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f75659d0640 1 -- 172.21.15.19:0/1620913948 learned_addr learned my addr 172.21.15.19:0/1620913948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f75659d0640 1 --2- 172.21.15.19:0/1620913948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f75661d1640 1 --2- 172.21.15.19:0/1620913948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f75659d0640 1 -- 172.21.15.19:0/1620913948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 msgr2=0x56481e1d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f75659d0640 1 --2- 172.21.15.19:0/1620913948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f75659d0640 1 -- 172.21.15.19:0/1620913948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 msgr2=0x56481e1cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.701+0000 7f75659d0640 1 --2- 172.21.15.19:0/1620913948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f75659d0640 1 -- 172.21.15.19:0/1620913948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481e1cd510 con 0x56481e1d2090 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f75659d0640 1 --2- 172.21.15.19:0/1620913948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1ea030 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7550009c80 tx=0x7f7550015f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3389b7ba5ed44393 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f754bfff640 1 -- 172.21.15.19:0/1620913948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550030070 con 0x56481e1d2090 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f754bfff640 1 -- 172.21.15.19:0/1620913948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755001a040 con 0x56481e1d2090 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f754bfff640 1 -- 172.21.15.19:0/1620913948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550015b00 con 0x56481e1d2090 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f756b22e740 1 -- 172.21.15.19:0/1620913948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 msgr2=0x56481e1ea030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f756b22e740 1 --2- 172.21.15.19:0/1620913948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1ea030 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7550009c80 tx=0x7f7550015f30 comp rx=0 tx=0).stop 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f756b22e740 1 -- 172.21.15.19:0/1620913948 shutdown_connections 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f756b22e740 1 --2- 172.21.15.19:0/1620913948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f756b22e740 1 --2- 172.21.15.19:0/1620913948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1ea030 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f756b22e740 1 --2- 172.21.15.19:0/1620913948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f756b22e740 1 -- 172.21.15.19:0/1620913948 >> 172.21.15.19:0/1620913948 conn(0x56481e1c33c0 msgr2=0x56481e1c2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f756b22e740 1 -- 172.21.15.19:0/1620913948 shutdown_connections 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.702+0000 7f756b22e740 1 -- 172.21.15.19:0/1620913948 wait complete. 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f756b22e740 1 -- start start 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f756b22e740 1 Processor -- start 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f756b22e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1da350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f756b22e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f756b22e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1d6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f756b22e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75601b1210 con 0x56481e1d2090 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f756b22e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f75601b0b40 con 0x56481e1c79c0 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f756b22e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75601add80 con 0x56481e1c70a0 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f756b22e740 10 client.0 _ll_register_callbacks cb 0x7f755c0315e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75659d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1d6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75659d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1d6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54844/0 (socket says 172.21.15.19:54844) 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75659d0640 1 -- 172.21.15.19:0/1784859870 learned_addr learned my addr 172.21.15.19:0/1784859870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75661d1640 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75659d0640 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75659d0640 1 -- 172.21.15.19:0/1784859870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 msgr2=0x56481e1da350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75659d0640 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1da350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75659d0640 1 -- 172.21.15.19:0/1784859870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 msgr2=0x56481e1d66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75659d0640 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1d66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75659d0640 1 -- 172.21.15.19:0/1784859870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481e1d71d0 con 0x56481e1d2090 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75661d1640 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f75659d0640 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1d6c10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7550009300 tx=0x7f755002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bb1831a57a1097 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.703+0000 7f7549ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f7549ffb640 1 -- 172.21.15.19:0/1784859870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550030050 con 0x56481e1d2090 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f7549ffb640 1 -- 172.21.15.19:0/1784859870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7550015980 con 0x56481e1d2090 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f756b22e740 1 -- 172.21.15.19:0/1784859870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f755001a040 con 0x56481e1d2090 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f7549ffb640 1 -- 172.21.15.19:0/1784859870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f755002a2b0 con 0x56481e1d2090 2024-04-23T18:30:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f756b22e740 10 client.16251 fetch_fsmap learned FSMap version 27 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f756b22e740 1 -- 172.21.15.19:0/1784859870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56481e249620 con 0x56481e1d2090 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f7549ffb640 1 -- 172.21.15.19:0/1784859870 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755001a040 con 0x56481e1d2090 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f7549ffb640 1 -- 172.21.15.19:0/1784859870 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f75500358e0 con 0x56481e1d2090 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f756b22e740 10 client.16251 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f756b22e740 10 client.16251 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f756b22e740 20 client.16251 populate_metadata read hostname 'smithi019' 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f756b22e740 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56481e1e2340 0x56481e24ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f756b22e740 4 client.16251 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 83.6, "await_for": 42.0}] 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f756b22e740 1 -- 172.21.15.19:0/1784859870 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 83.6, "await_for": 42.0}) -- 0x56481e1c3210 con 0x56481e1e2340 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.704+0000 7f75661d1640 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56481e1e2340 0x56481e24ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.705+0000 7f7549ffb640 10 client.16251 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.705+0000 7f75661d1640 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56481e1e2340 0x56481e24ba20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c197ea84b43876 server_cookie=b56ec82d2277ab4d in_seq=0 out_seq=0 2024-04-23T18:30:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f7549ffb640 1 -- 172.21.15.19:0/1784859870 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56481e1c3210 con 0x56481e1e2340 2024-04-23T18:30:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f7549ffb640 10 client.16251 handle_command_reply: tid=0 2024-04-23T18:30:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 client.16251 shutdown 2024-04-23T18:30:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 -- 172.21.15.19:0/1784859870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 msgr2=0x56481e1d6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1d6c10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7550009300 tx=0x7f755002c040 comp rx=0 tx=0).stop 2024-04-23T18:30:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 -- 172.21.15.19:0/1784859870 shutdown_connections 2024-04-23T18:30:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56481e1e2340 0x56481e24ba20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56481e1c79c0 0x56481e1d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56481e1d2090 0x56481e1d6c10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 --2- 172.21.15.19:0/1784859870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56481e1c70a0 0x56481e1da350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f7549ffb640 0 client.16251 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 -- 172.21.15.19:0/1784859870 >> 172.21.15.19:0/1784859870 conn(0x56481e1c33c0 msgr2=0x56481e1c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 -- 172.21.15.19:0/1784859870 shutdown_connections 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 1 -- 172.21.15.19:0/1784859870 wait complete. 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.721+0000 7f756b22e740 20 client.16251 trim_cache size 0 max 16384 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:30:44.715 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 22, 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "20f3d723": { 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 22, 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 83.6, 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:30:44.716 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:30:44.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:30:44.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:30:44.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:30:44.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:30:44.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:30:44.717 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:30:44.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 -- 172.21.15.19:0/2184998670 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f754c0a3d50 msgr2=0x7f754c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f754c0a3d50 0x7f754c0a6210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f75580ce850 tx=0x7f7558091f30 comp rx=0 tx=0).stop 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 -- 172.21.15.19:0/2184998670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601035d0 msgr2=0x7f75601a6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601035d0 0x7f75601a6a10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f755800cab0 tx=0x7f755800ceb0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 -- 172.21.15.19:0/2184998670 shutdown_connections 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f754c0a3d50 0x7f754c0a6210 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601035d0 0x7f75601a6a10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7560106190 0x7f7560103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 --2- 172.21.15.19:0/2184998670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7560101110 0x7f7560102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 -- 172.21.15.19:0/2184998670 >> 172.21.15.19:0/2184998670 conn(0x7f7560077fb0 msgr2=0x7f756006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 -- 172.21.15.19:0/2184998670 shutdown_connections 2024-04-23T18:30:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:30:44.722+0000 7f7568435640 1 -- 172.21.15.19:0/2184998670 wait complete. 2024-04-23T18:30:44.725 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '20f3d723', quiesce took 0.5 seconds. Will release after: 23.6 2024-04-23T18:30:45.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:45 smithi019 ceph-mon[29215]: pgmap v141: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-23T18:30:45.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:45 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/372685456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:30:45.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:45 smithi044 ceph-mon[32458]: pgmap v141: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-23T18:30:45.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:45 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/372685456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:30:45.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:45 smithi155 ceph-mon[33468]: pgmap v141: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-23T18:30:45.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:45 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/372685456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.016+0000 7f370af44640 1 -- 172.21.15.19:0/2928589837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f370410c5d0 msgr2=0x7f370410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.016+0000 7f370af44640 1 --2- 172.21.15.19:0/2928589837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f370410c5d0 0x7f370410ea00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009ca0 tx=0x7f36fc02a950 comp rx=0 tx=0).stop 2024-04-23T18:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.016+0000 7f370af44640 1 -- 172.21.15.19:0/2928589837 shutdown_connections 2024-04-23T18:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.016+0000 7f370af44640 1 --2- 172.21.15.19:0/2928589837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3704103b20 0x7f370410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.016+0000 7f370af44640 1 --2- 172.21.15.19:0/2928589837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f370410c5d0 0x7f370410ea00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.016+0000 7f370af44640 1 --2- 172.21.15.19:0/2928589837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3704103200 0x7f37041035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.016+0000 7f370af44640 1 -- 172.21.15.19:0/2928589837 >> 172.21.15.19:0/2928589837 conn(0x7f37040771d0 msgr2=0x7f37040775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.016+0000 7f370af44640 1 -- 172.21.15.19:0/2928589837 shutdown_connections 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.016+0000 7f370af44640 1 -- 172.21.15.19:0/2928589837 wait complete. 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f370af44640 1 Processor -- start 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f370af44640 1 -- start start 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f370af44640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3704103200 0x7f3704100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f370af44640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3704103b20 0x7f3704101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f370af44640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370410c5d0 0x7f37041018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f370af44640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37041121f0 con 0x7f3704103200 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f370af44640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3704112070 con 0x7f3704103b20 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f370af44640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3704112370 con 0x7f370410c5d0 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f3708ce0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3704103200 0x7f3704100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f3708ce0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3704103200 0x7f3704100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54872/0 (socket says 172.21.15.19:54872) 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f3708ce0640 1 -- 172.21.15.19:0/63146788 learned_addr learned my addr 172.21.15.19:0/63146788 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.017+0000 7f37094e1640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370410c5d0 0x7f37041018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3703fff640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3704103b20 0x7f3704101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3708ce0640 1 -- 172.21.15.19:0/63146788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370410c5d0 msgr2=0x7f37041018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3708ce0640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370410c5d0 0x7f37041018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3708ce0640 1 -- 172.21.15.19:0/63146788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3704103b20 msgr2=0x7f3704101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3708ce0640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3704103b20 0x7f3704101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3708ce0640 1 -- 172.21.15.19:0/63146788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37041a6b10 con 0x7f3704103200 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3703fff640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3704103b20 0x7f3704101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3708ce0640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3704103200 0x7f3704100e50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f36f400d770 tx=0x7f36f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3701ffb640 1 -- 172.21.15.19:0/63146788 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4017070 con 0x7f3704103200 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3701ffb640 1 -- 172.21.15.19:0/63146788 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f40054d0 con 0x7f3704103200 2024-04-23T18:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f370af44640 1 -- 172.21.15.19:0/63146788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37041a6e00 con 0x7f3704103200 2024-04-23T18:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f3701ffb640 1 -- 172.21.15.19:0/63146788 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4004c30 con 0x7f3704103200 2024-04-23T18:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.018+0000 7f370af44640 1 -- 172.21.15.19:0/63146788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37041a72c0 con 0x7f3704103200 2024-04-23T18:30:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f3701ffb640 1 -- 172.21.15.19:0/63146788 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f36f4004dd0 con 0x7f3704103200 2024-04-23T18:30:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f370dc63740 1 Processor -- start 2024-04-23T18:30:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f370dc63740 1 -- start start 2024-04-23T18:30:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f3701ffb640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f36f00a3e70 0x7f36f00a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f370dc63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee387a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f370dc63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 0x564aee3922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f370dc63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee394bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f370dc63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564aee2ec970 con 0x564aee392800 2024-04-23T18:30:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f370dc63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564aee2894a0 con 0x564aee387620 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f370dc63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564aee2e8930 con 0x564aee387f40 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f3701ffb640 1 -- 172.21.15.19:0/63146788 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f36f40d0200 con 0x7f3704103200 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.021+0000 7f37094e1640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f36f00a3e70 0x7f36f00a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f37094e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee394bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f37094e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee394bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54888/0 (socket says 172.21.15.19:54888) 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f3708ce0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee387a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f37094e1640 1 -- 172.21.15.19:0/3528594625 learned_addr learned my addr 172.21.15.19:0/3528594625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f3703fff640 1 --2- 172.21.15.19:0/3528594625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 0x564aee3922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f37094e1640 1 -- 172.21.15.19:0/3528594625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 msgr2=0x564aee3922c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f37094e1640 1 --2- 172.21.15.19:0/3528594625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 0x564aee3922c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f37094e1640 1 -- 172.21.15.19:0/3528594625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 msgr2=0x564aee387a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f37094e1640 1 --2- 172.21.15.19:0/3528594625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee387a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f37094e1640 1 -- 172.21.15.19:0/3528594625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aedfff210 con 0x564aee392800 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f3703fff640 1 --2- 172.21.15.19:0/3528594625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 0x564aee3922c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f37094e1640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f36f00a3e70 0x7f36f00a6330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f36fc030040 tx=0x7f36fc0099c0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f37094e1640 1 --2- 172.21.15.19:0/3528594625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee394bf0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009ca0 tx=0x7f36fc048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8300c988455d2b3b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f36faffd640 1 -- 172.21.15.19:0/3528594625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc034040 con 0x564aee392800 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f36faffd640 1 -- 172.21.15.19:0/3528594625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36fc02f5c0 con 0x564aee392800 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.022+0000 7f36faffd640 1 -- 172.21.15.19:0/3528594625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc004860 con 0x564aee392800 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.023+0000 7f370dc63740 1 -- 172.21.15.19:0/3528594625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 msgr2=0x564aee394bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.023+0000 7f370dc63740 1 --2- 172.21.15.19:0/3528594625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee394bf0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009ca0 tx=0x7f36fc048000 comp rx=0 tx=0).stop 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.023+0000 7f370dc63740 1 -- 172.21.15.19:0/3528594625 shutdown_connections 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.023+0000 7f370dc63740 1 --2- 172.21.15.19:0/3528594625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee394bf0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.023+0000 7f370dc63740 1 --2- 172.21.15.19:0/3528594625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 0x564aee3922c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.023+0000 7f370dc63740 1 --2- 172.21.15.19:0/3528594625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee387a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.023+0000 7f370dc63740 1 -- 172.21.15.19:0/3528594625 >> 172.21.15.19:0/3528594625 conn(0x564aee383b30 msgr2=0x564aee383f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.023+0000 7f370dc63740 1 -- 172.21.15.19:0/3528594625 shutdown_connections 2024-04-23T18:30:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.023+0000 7f370dc63740 1 -- 172.21.15.19:0/3528594625 wait complete. 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.023+0000 7f370dc63740 1 -- start start 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f370dc63740 1 Processor -- start 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f370dc63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee392190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f370dc63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 0x564aee38f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f370dc63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee38fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f370dc63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564aee3953d0 con 0x564aee392800 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f370dc63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564aee2870b0 con 0x564aee387620 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f370dc63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564aee395550 con 0x564aee387f40 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f370dc63740 10 client.0 _ll_register_callbacks cb 0x564aee274d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f37094e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee38fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f37094e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee38fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54890/0 (socket says 172.21.15.19:54890) 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f37094e1640 1 -- 172.21.15.19:0/3213421866 learned_addr learned my addr 172.21.15.19:0/3213421866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f3703fff640 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 0x564aee38f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.024+0000 7f3708ce0640 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee392190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f37094e1640 1 -- 172.21.15.19:0/3213421866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 msgr2=0x564aee38f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f37094e1640 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 0x564aee38f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f37094e1640 1 -- 172.21.15.19:0/3213421866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 msgr2=0x564aee392190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f37094e1640 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee392190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f37094e1640 1 -- 172.21.15.19:0/3213421866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aee0cb620 con 0x564aee392800 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f3703fff640 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 0x564aee38f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f3708ce0640 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee392190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f37094e1640 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee38fa00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f36fc02ae70 tx=0x7f36fc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7db3a36f44f36a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f36e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f36e3fff640 1 -- 172.21.15.19:0/3213421866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc059070 con 0x564aee392800 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f36e3fff640 1 -- 172.21.15.19:0/3213421866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f36fc004b40 con 0x564aee392800 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f370dc63740 1 -- 172.21.15.19:0/3213421866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36fc02f5c0 con 0x564aee392800 2024-04-23T18:30:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f36e3fff640 1 -- 172.21.15.19:0/3213421866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36fc065050 con 0x564aee392800 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f370dc63740 10 client.16263 fetch_fsmap learned FSMap version 27 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f370dc63740 1 -- 172.21.15.19:0/3213421866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564aee3a13b0 con 0x564aee392800 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.025+0000 7f36e3fff640 1 -- 172.21.15.19:0/3213421866 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36fc02f5c0 con 0x564aee392800 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.026+0000 7f36e3fff640 1 -- 172.21.15.19:0/3213421866 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f36fc04aaa0 con 0x564aee392800 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.026+0000 7f370dc63740 10 client.16263 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.026+0000 7f370dc63740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.026+0000 7f370dc63740 20 client.16263 populate_metadata read hostname 'smithi019' 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.026+0000 7f370dc63740 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564aee3a2700 0x564aee3a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.026+0000 7f370dc63740 4 client.16263 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.026+0000 7f370dc63740 1 -- 172.21.15.19:0/3213421866 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aee2fa390 con 0x564aee3a2700 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.026+0000 7f3708ce0640 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564aee3a2700 0x564aee3a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.026+0000 7f3708ce0640 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564aee3a2700 0x564aee3a4af0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb915f9d1bf6099e server_cookie=fd870d09e4b9b915 in_seq=0 out_seq=0 2024-04-23T18:30:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.026+0000 7f36e3fff640 10 client.16263 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.027+0000 7f36e3fff640 1 -- 172.21.15.19:0/3213421866 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564aee2fa390 con 0x564aee3a2700 2024-04-23T18:30:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.027+0000 7f36e3fff640 10 client.16263 handle_command_reply: tid=0 2024-04-23T18:30:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.027+0000 7f370dc63740 1 client.16263 shutdown 2024-04-23T18:30:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.027+0000 7f370dc63740 1 -- 172.21.15.19:0/3213421866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 msgr2=0x564aee38fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.027+0000 7f370dc63740 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee38fa00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f36fc02ae70 tx=0x7f36fc004340 comp rx=0 tx=0).stop 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.028+0000 7f370dc63740 1 -- 172.21.15.19:0/3213421866 shutdown_connections 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.028+0000 7f370dc63740 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564aee3a2700 0x564aee3a4af0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.028+0000 7f370dc63740 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee392800 0x564aee38fa00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.028+0000 7f36e3fff640 0 client.16263 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.028+0000 7f370dc63740 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee387f40 0x564aee38f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.028+0000 7f370dc63740 1 --2- 172.21.15.19:0/3213421866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee392190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.028+0000 7f370dc63740 1 -- 172.21.15.19:0/3213421866 >> 172.21.15.19:0/3213421866 conn(0x564aee383b30 msgr2=0x564aee3898b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.028+0000 7f370dc63740 1 -- 172.21.15.19:0/3213421866 shutdown_connections 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.028+0000 7f370dc63740 1 -- 172.21.15.19:0/3213421866 wait complete. 2024-04-23T18:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.028+0000 7f370dc63740 20 client.16263 trim_cache size 0 max 16384 2024-04-23T18:30:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f370dc63740 1 Processor -- start 2024-04-23T18:30:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f370dc63740 1 -- start start 2024-04-23T18:30:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f370dc63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee39e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f370dc63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee38e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f370dc63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 0x564aee382b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f370dc63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37041afc50 con 0x564aee387f40 2024-04-23T18:30:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f370dc63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37041b1790 con 0x564aee387620 2024-04-23T18:30:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f370dc63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37041b0320 con 0x564aee392800 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f3703fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee38e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f3703fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee38e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54896/0 (socket says 172.21.15.19:54896) 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f3703fff640 1 -- 172.21.15.19:0/704886212 learned_addr learned my addr 172.21.15.19:0/704886212 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f37094e1640 1 --2- 172.21.15.19:0/704886212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 0x564aee382b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f3703fff640 1 -- 172.21.15.19:0/704886212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 msgr2=0x564aee382b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f3703fff640 1 --2- 172.21.15.19:0/704886212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 0x564aee382b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f3703fff640 1 -- 172.21.15.19:0/704886212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 msgr2=0x564aee39e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f3703fff640 1 --2- 172.21.15.19:0/704886212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee39e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f3703fff640 1 -- 172.21.15.19:0/704886212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aee37f500 con 0x564aee387f40 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.043+0000 7f37094e1640 1 --2- 172.21.15.19:0/704886212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 0x564aee382b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f3703fff640 1 --2- 172.21.15.19:0/704886212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee38e600 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009b80 tx=0x7f36ec012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c713b8b8a35f46d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f36fa7fc640 1 -- 172.21.15.19:0/704886212 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec009d30 con 0x564aee387f40 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f36fa7fc640 1 -- 172.21.15.19:0/704886212 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36ec0118f0 con 0x564aee387f40 2024-04-23T18:30:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f36fa7fc640 1 -- 172.21.15.19:0/704886212 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec0107b0 con 0x564aee387f40 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f370dc63740 1 -- 172.21.15.19:0/704886212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 msgr2=0x564aee38e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f370dc63740 1 --2- 172.21.15.19:0/704886212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee38e600 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009b80 tx=0x7f36ec012820 comp rx=0 tx=0).stop 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f370dc63740 1 -- 172.21.15.19:0/704886212 shutdown_connections 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f370dc63740 1 --2- 172.21.15.19:0/704886212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 0x564aee382b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f370dc63740 1 --2- 172.21.15.19:0/704886212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee38e600 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f370dc63740 1 --2- 172.21.15.19:0/704886212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee39e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f370dc63740 1 -- 172.21.15.19:0/704886212 >> 172.21.15.19:0/704886212 conn(0x564aee383b30 msgr2=0x564aee382420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f370dc63740 1 -- 172.21.15.19:0/704886212 shutdown_connections 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.044+0000 7f370dc63740 1 -- 172.21.15.19:0/704886212 wait complete. 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f370dc63740 1 -- start start 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f370dc63740 1 Processor -- start 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f370dc63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee39aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f370dc63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee397dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f370dc63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 0x564aee398300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f370dc63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37041b10c0 con 0x564aee387f40 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f370dc63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37041b09f0 con 0x564aee387620 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f370dc63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37041adc30 con 0x564aee392800 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f370dc63740 10 client.0 _ll_register_callbacks cb 0x7f36fc02ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f3703fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee397dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f3703fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee397dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54906/0 (socket says 172.21.15.19:54906) 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.045+0000 7f3703fff640 1 -- 172.21.15.19:0/3807581453 learned_addr learned my addr 172.21.15.19:0/3807581453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f37094e1640 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 0x564aee398300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f3708ce0640 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee39aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f3703fff640 1 -- 172.21.15.19:0/3807581453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 msgr2=0x564aee398300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f3703fff640 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 0x564aee398300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f3703fff640 1 -- 172.21.15.19:0/3807581453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 msgr2=0x564aee39aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f3703fff640 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee39aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f3703fff640 1 -- 172.21.15.19:0/3807581453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aee398970 con 0x564aee387f40 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f37094e1640 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 0x564aee398300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f3703fff640 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee397dc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f36ec01c000 tx=0x7f36ec011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd489813fcd194d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f36e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f36e3fff640 1 -- 172.21.15.19:0/3807581453 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec021040 con 0x564aee387f40 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f36e3fff640 1 -- 172.21.15.19:0/3807581453 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f36ec018070 con 0x564aee387f40 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f370dc63740 1 -- 172.21.15.19:0/3807581453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36ec0118f0 con 0x564aee387f40 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f36e3fff640 1 -- 172.21.15.19:0/3807581453 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36ec011c20 con 0x564aee387f40 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f370dc63740 10 client.16269 fetch_fsmap learned FSMap version 27 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.046+0000 7f370dc63740 1 -- 172.21.15.19:0/3807581453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564aee408750 con 0x564aee387f40 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.047+0000 7f36e3fff640 1 -- 172.21.15.19:0/3807581453 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36ec0118f0 con 0x564aee387f40 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.047+0000 7f36e3fff640 1 -- 172.21.15.19:0/3807581453 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f36ec024070 con 0x564aee387f40 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.047+0000 7f370dc63740 10 client.16269 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:30:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.047+0000 7f370dc63740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.047+0000 7f370dc63740 20 client.16269 populate_metadata read hostname 'smithi019' 2024-04-23T18:30:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.047+0000 7f370dc63740 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564aee3a2700 0x564aee40ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.047+0000 7f370dc63740 4 client.16269 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.047+0000 7f370dc63740 1 -- 172.21.15.19:0/3807581453 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564aee2f4200 con 0x564aee3a2700 2024-04-23T18:30:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.047+0000 7f3708ce0640 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564aee3a2700 0x564aee40ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.048+0000 7f3708ce0640 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564aee3a2700 0x564aee40ae30 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25dd157f93e91f7e server_cookie=9e61360186c058e in_seq=0 out_seq=0 2024-04-23T18:30:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.048+0000 7f36e3fff640 10 client.16269 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f36e3fff640 1 -- 172.21.15.19:0/3807581453 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564aee2f4200 con 0x564aee3a2700 2024-04-23T18:30:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f36e3fff640 10 client.16269 handle_command_reply: tid=0 2024-04-23T18:30:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 client.16269 shutdown 2024-04-23T18:30:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 -- 172.21.15.19:0/3807581453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 msgr2=0x564aee397dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee397dc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f36ec01c000 tx=0x7f36ec011850 comp rx=0 tx=0).stop 2024-04-23T18:30:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 -- 172.21.15.19:0/3807581453 shutdown_connections 2024-04-23T18:30:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564aee3a2700 0x564aee40ae30 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564aee392800 0x564aee398300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564aee387f40 0x564aee397dc0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f36e3fff640 0 client.16269 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:30:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 --2- 172.21.15.19:0/3807581453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564aee387620 0x564aee39aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 -- 172.21.15.19:0/3807581453 >> 172.21.15.19:0/3807581453 conn(0x564aee383b30 msgr2=0x564aee382420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 -- 172.21.15.19:0/3807581453 shutdown_connections 2024-04-23T18:30:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 1 -- 172.21.15.19:0/3807581453 wait complete. 2024-04-23T18:30:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370dc63740 20 client.16269 trim_cache size 0 max 16384 2024-04-23T18:30:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:30:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:30:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:30:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:30:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370af44640 1 -- 172.21.15.19:0/63146788 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f36f00a3e70 msgr2=0x7f36f00a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370af44640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f36f00a3e70 0x7f36f00a6330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f36fc030040 tx=0x7f36fc0099c0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370af44640 1 -- 172.21.15.19:0/63146788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3704103200 msgr2=0x7f3704100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370af44640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3704103200 0x7f3704100e50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f36f400d770 tx=0x7f36f400dc30 comp rx=0 tx=0).stop 2024-04-23T18:30:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370af44640 1 -- 172.21.15.19:0/63146788 shutdown_connections 2024-04-23T18:30:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370af44640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f36f00a3e70 0x7f36f00a6330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370af44640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3704103b20 0x7f3704101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370af44640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370410c5d0 0x7f37041018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370af44640 1 --2- 172.21.15.19:0/63146788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3704103200 0x7f3704100e50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.066+0000 7f370af44640 1 -- 172.21.15.19:0/63146788 >> 172.21.15.19:0/63146788 conn(0x7f37040771d0 msgr2=0x7f37040fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.067+0000 7f370af44640 1 -- 172.21.15.19:0/63146788 shutdown_connections 2024-04-23T18:30:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:30:47.067+0000 7f370af44640 1 -- 172.21.15.19:0/63146788 wait complete. 2024-04-23T18:30:47.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:47 smithi019 ceph-mon[29215]: pgmap v142: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-23T18:30:47.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:47 smithi044 ceph-mon[32458]: pgmap v142: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-23T18:30:47.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:47 smithi155 ceph-mon[33468]: pgmap v142: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-23T18:30:49.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:49 smithi019 ceph-mon[29215]: pgmap v143: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-23T18:30:49.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:49 smithi044 ceph-mon[32458]: pgmap v143: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-23T18:30:49.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:49 smithi155 ceph-mon[33468]: pgmap v143: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-23T18:30:51.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:51 smithi019 ceph-mon[29215]: pgmap v144: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 KiB/s wr, 20 op/s 2024-04-23T18:30:51.811 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:51 smithi044 ceph-mon[32458]: pgmap v144: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 KiB/s wr, 20 op/s 2024-04-23T18:30:51.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:51 smithi155 ceph-mon[33468]: pgmap v144: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 KiB/s wr, 20 op/s 2024-04-23T18:30:53.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:53 smithi019 ceph-mon[29215]: pgmap v145: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:30:53.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:53 smithi044 ceph-mon[32458]: pgmap v145: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:30:53.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:53 smithi155 ceph-mon[33468]: pgmap v145: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:30:55.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:55 smithi019 ceph-mon[29215]: pgmap v146: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-23T18:30:55.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:55 smithi044 ceph-mon[32458]: pgmap v146: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-23T18:30:55.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:55 smithi155 ceph-mon[33468]: pgmap v146: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-23T18:30:57.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:57 smithi019 ceph-mon[29215]: pgmap v147: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-04-23T18:30:57.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:57 smithi044 ceph-mon[32458]: pgmap v147: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-04-23T18:30:57.821 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:57 smithi155 ceph-mon[33468]: pgmap v147: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-04-23T18:30:59.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:30:59 smithi019 ceph-mon[29215]: pgmap v148: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-04-23T18:30:59.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:30:59 smithi044 ceph-mon[32458]: pgmap v148: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-04-23T18:30:59.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:30:59 smithi155 ceph-mon[33468]: pgmap v148: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-04-23T18:31:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.169+0000 7fb662231640 1 -- 172.21.15.19:0/2128569338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb65c106170 msgr2=0x7fb65c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.169+0000 7fb662231640 1 --2- 172.21.15.19:0/2128569338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb65c106170 0x7fb65c10dd40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb654009ca0 tx=0x7fb65402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.169+0000 7fb662231640 1 -- 172.21.15.19:0/2128569338 shutdown_connections 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.169+0000 7fb662231640 1 --2- 172.21.15.19:0/2128569338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb65c101aa0 0x7fb65c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.169+0000 7fb662231640 1 --2- 172.21.15.19:0/2128569338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb65c106170 0x7fb65c10dd40 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.169+0000 7fb662231640 1 --2- 172.21.15.19:0/2128569338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb65c1010f0 0x7fb65c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.169+0000 7fb662231640 1 -- 172.21.15.19:0/2128569338 >> 172.21.15.19:0/2128569338 conn(0x7fb65c077fd0 msgr2=0x7fb65c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.169+0000 7fb662231640 1 -- 172.21.15.19:0/2128569338 shutdown_connections 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.169+0000 7fb662231640 1 -- 172.21.15.19:0/2128569338 wait complete. 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb662231640 1 Processor -- start 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb662231640 1 -- start start 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb662231640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb65c1010f0 0x7fb65c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb662231640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb65c106170 0x7fb65c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb662231640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb65c19cfd0 0x7fb65c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb662231640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c111130 con 0x7fb65c106170 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb662231640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c110fb0 con 0x7fb65c19cfd0 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb662231640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1112b0 con 0x7fb65c1010f0 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb65affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb65c106170 0x7fb65c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb65affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb65c106170 0x7fb65c198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48498/0 (socket says 172.21.15.19:48498) 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb65affd640 1 -- 172.21.15.19:0/2501220016 learned_addr learned my addr 172.21.15.19:0/2501220016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb65bfff640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb65c19cfd0 0x7fb65c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.170+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb65c1010f0 0x7fb65c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb65affd640 1 -- 172.21.15.19:0/2501220016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb65c1010f0 msgr2=0x7fb65c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb65affd640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb65c1010f0 0x7fb65c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb65affd640 1 -- 172.21.15.19:0/2501220016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb65c19cfd0 msgr2=0x7fb65c19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb65affd640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb65c19cfd0 0x7fb65c19d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb65affd640 1 -- 172.21.15.19:0/2501220016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c19dac0 con 0x7fb65c106170 2024-04-23T18:31:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb65c1010f0 0x7fb65c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb65affd640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb65c106170 0x7fb65c198c50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb65000d770 tx=0x7fb65000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb658ff9640 1 -- 172.21.15.19:0/2501220016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650017070 con 0x7fb65c106170 2024-04-23T18:31:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb658ff9640 1 -- 172.21.15.19:0/2501220016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb650009c30 con 0x7fb65c106170 2024-04-23T18:31:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb662231640 1 -- 172.21.15.19:0/2501220016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65c19ddb0 con 0x7fb65c106170 2024-04-23T18:31:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb658ff9640 1 -- 172.21.15.19:0/2501220016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65000f620 con 0x7fb65c106170 2024-04-23T18:31:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.171+0000 7fb662231640 1 -- 172.21.15.19:0/2501220016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65c1a55a0 con 0x7fb65c106170 2024-04-23T18:31:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.173+0000 7fb658ff9640 1 -- 172.21.15.19:0/2501220016 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb65001a070 con 0x7fb65c106170 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb665029740 1 Processor -- start 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb665029740 1 -- start start 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb665029740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5eb620 0x55a86c5eba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb665029740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb665029740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c5f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb665029740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a86c550970 con 0x55a86c5ebf40 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb665029740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a86c4ed4a0 con 0x55a86c5f6800 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb665029740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a86c54c930 con 0x55a86c5eb620 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb65b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb658ff9640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb6440a3dc0 0x7fb6440a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb65b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48512/0 (socket says 172.21.15.19:48512) 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb65b7fe640 1 -- 172.21.15.19:0/3925328717 learned_addr learned my addr 172.21.15.19:0/3925328717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb65affd640 1 --2- 172.21.15.19:0/3925328717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c5f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.174+0000 7fb65bfff640 1 --2- 172.21.15.19:0/3925328717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5eb620 0x55a86c5eba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb65bfff640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb6440a3dc0 0x7fb6440a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb658ff9640 1 -- 172.21.15.19:0/2501220016 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb6500d03a0 con 0x7fb65c106170 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb65b7fe640 1 -- 172.21.15.19:0/3925328717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5eb620 msgr2=0x55a86c5eba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/3925328717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5eb620 0x55a86c5eba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb65b7fe640 1 -- 172.21.15.19:0/3925328717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 msgr2=0x55a86c5f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/3925328717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c5f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb65b7fe640 1 -- 172.21.15.19:0/3925328717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86c263210 con 0x55a86c5ebf40 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb65affd640 1 --2- 172.21.15.19:0/3925328717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c5f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/3925328717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f62c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb64c007850 tx=0x7fb64c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4c71380208bfd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb65bfff640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb6440a3dc0 0x7fb6440a6280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb654009a60 tx=0x7fb6540095c0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb641ffb640 1 -- 172.21.15.19:0/3925328717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c00d950 con 0x55a86c5ebf40 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb641ffb640 1 -- 172.21.15.19:0/3925328717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c018070 con 0x55a86c5ebf40 2024-04-23T18:31:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.175+0000 7fb641ffb640 1 -- 172.21.15.19:0/3925328717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c013e10 con 0x55a86c5ebf40 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.176+0000 7fb665029740 1 -- 172.21.15.19:0/3925328717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 msgr2=0x55a86c5f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.176+0000 7fb665029740 1 --2- 172.21.15.19:0/3925328717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f62c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb64c007850 tx=0x7fb64c007d10 comp rx=0 tx=0).stop 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.176+0000 7fb665029740 1 -- 172.21.15.19:0/3925328717 shutdown_connections 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.176+0000 7fb665029740 1 --2- 172.21.15.19:0/3925328717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c5f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.176+0000 7fb665029740 1 --2- 172.21.15.19:0/3925328717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f62c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.176+0000 7fb665029740 1 --2- 172.21.15.19:0/3925328717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5eb620 0x55a86c5eba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.176+0000 7fb665029740 1 -- 172.21.15.19:0/3925328717 >> 172.21.15.19:0/3925328717 conn(0x55a86c5e7b30 msgr2=0x55a86c5e7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.176+0000 7fb665029740 1 -- 172.21.15.19:0/3925328717 shutdown_connections 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.176+0000 7fb665029740 1 -- 172.21.15.19:0/3925328717 wait complete. 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.176+0000 7fb665029740 1 -- start start 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb665029740 1 Processor -- start 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb665029740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5eb620 0x55a86c5f6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb665029740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb665029740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c5f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb665029740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a86c5f9430 con 0x55a86c5ebf40 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb665029740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a86c4eb0b0 con 0x55a86c5f6800 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb665029740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a86c5f95b0 con 0x55a86c5eb620 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb665029740 10 client.0 _ll_register_callbacks cb 0x55a86c4d8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb65affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb65affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48528/0 (socket says 172.21.15.19:48528) 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb65affd640 1 -- 172.21.15.19:0/1051140513 learned_addr learned my addr 172.21.15.19:0/1051140513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5eb620 0x55a86c5f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb65affd640 1 -- 172.21.15.19:0/1051140513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5eb620 msgr2=0x55a86c5f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb65affd640 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5eb620 0x55a86c5f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb65bfff640 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c5f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb65affd640 1 -- 172.21.15.19:0/1051140513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 msgr2=0x55a86c5f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb65affd640 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c5f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.177+0000 7fb65affd640 1 -- 172.21.15.19:0/1051140513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86c32f620 con 0x55a86c5ebf40 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb65affd640 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f3420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb65009c2a0 tx=0x7fb65006acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5e74c595118d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb63affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb63affd640 1 -- 172.21.15.19:0/1051140513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650021040 con 0x55a86c5ebf40 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb63affd640 1 -- 172.21.15.19:0/1051140513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb65009b760 con 0x55a86c5ebf40 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb665029740 1 -- 172.21.15.19:0/1051140513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c018070 con 0x55a86c5ebf40 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb63affd640 1 -- 172.21.15.19:0/1051140513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6500cdf40 con 0x55a86c5ebf40 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb665029740 10 client.16281 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb665029740 1 -- 172.21.15.19:0/1051140513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a86c6053f0 con 0x55a86c5ebf40 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb63affd640 1 -- 172.21.15.19:0/1051140513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c018070 con 0x55a86c5ebf40 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb63affd640 1 -- 172.21.15.19:0/1051140513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb650098060 con 0x55a86c5ebf40 2024-04-23T18:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb665029740 10 client.16281 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb665029740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb665029740 20 client.16281 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb665029740 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a86c606860 0x55a86c608c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb665029740 4 client.16281 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.178+0000 7fb665029740 1 -- 172.21.15.19:0/1051140513 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86c55e390 con 0x55a86c606860 2024-04-23T18:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.179+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a86c606860 0x55a86c608c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.179+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a86c606860 0x55a86c608c50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b0236c4f0b8156 server_cookie=ca81859c50d2d34d in_seq=0 out_seq=0 2024-04-23T18:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.179+0000 7fb63affd640 10 client.16281 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb63affd640 1 -- 172.21.15.19:0/1051140513 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a86c55e390 con 0x55a86c606860 2024-04-23T18:31:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb63affd640 10 client.16281 handle_command_reply: tid=0 2024-04-23T18:31:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 client.16281 shutdown 2024-04-23T18:31:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 -- 172.21.15.19:0/1051140513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 msgr2=0x55a86c5f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f3420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb65009c2a0 tx=0x7fb65006acc0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 -- 172.21.15.19:0/1051140513 shutdown_connections 2024-04-23T18:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a86c606860 0x55a86c608c50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c5f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5f3420 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb63affd640 0 client.16281 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 --2- 172.21.15.19:0/1051140513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5eb620 0x55a86c5f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 -- 172.21.15.19:0/1051140513 >> 172.21.15.19:0/1051140513 conn(0x55a86c5e7b30 msgr2=0x55a86c5ed860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 -- 172.21.15.19:0/1051140513 shutdown_connections 2024-04-23T18:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 1 -- 172.21.15.19:0/1051140513 wait complete. 2024-04-23T18:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.180+0000 7fb665029740 20 client.16281 trim_cache size 0 max 16384 2024-04-23T18:31:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.195+0000 7fb665029740 1 Processor -- start 2024-04-23T18:31:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.195+0000 7fb665029740 1 -- start start 2024-04-23T18:31:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.195+0000 7fb665029740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5eb620 0x55a86c6022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.195+0000 7fb665029740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5ebf40 0x55a86c5e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.195+0000 7fb665029740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c60e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.195+0000 7fb665029740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1adac0 con 0x55a86c5eb620 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.195+0000 7fb665029740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1af600 con 0x55a86c5f6800 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.195+0000 7fb665029740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1ae190 con 0x55a86c5ebf40 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.195+0000 7fb65b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5eb620 0x55a86c6022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5eb620 0x55a86c6022f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48538/0 (socket says 172.21.15.19:48538) 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65b7fe640 1 -- 172.21.15.19:0/3716868031 learned_addr learned my addr 172.21.15.19:0/3716868031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65affd640 1 --2- 172.21.15.19:0/3716868031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5ebf40 0x55a86c5e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65b7fe640 1 -- 172.21.15.19:0/3716868031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5ebf40 msgr2=0x55a86c5e6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/3716868031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5ebf40 0x55a86c5e6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65b7fe640 1 -- 172.21.15.19:0/3716868031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 msgr2=0x55a86c60e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65bfff640 1 --2- 172.21.15.19:0/3716868031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c60e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/3716868031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c60e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65b7fe640 1 -- 172.21.15.19:0/3716868031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86c5e8d60 con 0x55a86c5eb620 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65affd640 1 --2- 172.21.15.19:0/3716868031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5ebf40 0x55a86c5e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/3716868031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5eb620 0x55a86c6022f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb64c007850 tx=0x7fb64c00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1f118e44f95d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb6417fa640 1 -- 172.21.15.19:0/3716868031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c02d070 con 0x55a86c5eb620 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb6417fa640 1 -- 172.21.15.19:0/3716868031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c018040 con 0x55a86c5eb620 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.196+0000 7fb6417fa640 1 -- 172.21.15.19:0/3716868031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0287d0 con 0x55a86c5eb620 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.197+0000 7fb665029740 1 -- 172.21.15.19:0/3716868031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5eb620 msgr2=0x55a86c6022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.197+0000 7fb665029740 1 --2- 172.21.15.19:0/3716868031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5eb620 0x55a86c6022f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb64c007850 tx=0x7fb64c00f040 comp rx=0 tx=0).stop 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.197+0000 7fb665029740 1 -- 172.21.15.19:0/3716868031 shutdown_connections 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.197+0000 7fb665029740 1 --2- 172.21.15.19:0/3716868031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5f6800 0x55a86c60e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.197+0000 7fb665029740 1 --2- 172.21.15.19:0/3716868031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5ebf40 0x55a86c5e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.197+0000 7fb665029740 1 --2- 172.21.15.19:0/3716868031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5eb620 0x55a86c6022f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.197+0000 7fb665029740 1 -- 172.21.15.19:0/3716868031 >> 172.21.15.19:0/3716868031 conn(0x55a86c5e7b30 msgr2=0x55a86c5e6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.197+0000 7fb665029740 1 -- 172.21.15.19:0/3716868031 shutdown_connections 2024-04-23T18:31:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.197+0000 7fb665029740 1 -- 172.21.15.19:0/3716868031 wait complete. 2024-04-23T18:31:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.197+0000 7fb665029740 1 -- start start 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb665029740 1 Processor -- start 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb665029740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5eb620 0x55a86c5fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb665029740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5fc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb665029740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5f6800 0x55a86c5fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb665029740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1aef30 con 0x55a86c5ebf40 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb665029740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1ae860 con 0x55a86c5eb620 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb665029740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1abac0 con 0x55a86c5f6800 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb665029740 10 client.0 _ll_register_callbacks cb 0x7fb65009c1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5fc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5fc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48554/0 (socket says 172.21.15.19:48554) 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65affd640 1 -- 172.21.15.19:0/1688123836 learned_addr learned my addr 172.21.15.19:0/1688123836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5eb620 0x55a86c5fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65bfff640 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5f6800 0x55a86c5fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65affd640 1 -- 172.21.15.19:0/1688123836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5f6800 msgr2=0x55a86c5fcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65affd640 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5f6800 0x55a86c5fcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65affd640 1 -- 172.21.15.19:0/1688123836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5eb620 msgr2=0x55a86c5fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65affd640 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5eb620 0x55a86c5fc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65affd640 1 -- 172.21.15.19:0/1688123836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86c66bdb0 con 0x55a86c5ebf40 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65bfff640 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5f6800 0x55a86c5fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb65affd640 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5fc710 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb6500d0310 tx=0x7fb650098bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35459d9785c4ce3c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.198+0000 7fb63affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb63affd640 1 -- 172.21.15.19:0/1688123836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500cdf50 con 0x55a86c5ebf40 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb63affd640 1 -- 172.21.15.19:0/1688123836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb6500a0150 con 0x55a86c5ebf40 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb665029740 1 -- 172.21.15.19:0/1688123836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c018040 con 0x55a86c5ebf40 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb63affd640 1 -- 172.21.15.19:0/1688123836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb65009ca10 con 0x55a86c5ebf40 2024-04-23T18:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb665029740 10 client.16287 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb665029740 1 -- 172.21.15.19:0/1688123836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a86c66d000 con 0x55a86c5ebf40 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb63affd640 1 -- 172.21.15.19:0/1688123836 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c018040 con 0x55a86c5ebf40 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb63affd640 1 -- 172.21.15.19:0/1688123836 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb6500bb0d0 con 0x55a86c5ebf40 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb665029740 10 client.16287 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb665029740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb665029740 20 client.16287 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb665029740 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a86c66d540 0x55a86c66f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb665029740 4 client.16287 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb665029740 1 -- 172.21.15.19:0/1688123836 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a86c558200 con 0x55a86c66d540 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.199+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a86c66d540 0x55a86c66f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.200+0000 7fb63affd640 10 client.16287 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.200+0000 7fb65b7fe640 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a86c66d540 0x55a86c66f960 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca6a998b0f9d476 server_cookie=acad2c5007be7d70 in_seq=0 out_seq=0 2024-04-23T18:31:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb63affd640 1 -- 172.21.15.19:0/1688123836 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a86c558200 con 0x55a86c66d540 2024-04-23T18:31:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb63affd640 10 client.16287 handle_command_reply: tid=0 2024-04-23T18:31:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 client.16287 shutdown 2024-04-23T18:31:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 -- 172.21.15.19:0/1688123836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 msgr2=0x55a86c5fc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5fc710 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb6500d0310 tx=0x7fb650098bb0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 -- 172.21.15.19:0/1688123836 shutdown_connections 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55a86c66d540 0x55a86c66f960 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a86c5f6800 0x55a86c5fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a86c5ebf40 0x55a86c5fc710 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 --2- 172.21.15.19:0/1688123836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a86c5eb620 0x55a86c5fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb63affd640 0 client.16287 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 -- 172.21.15.19:0/1688123836 >> 172.21.15.19:0/1688123836 conn(0x55a86c5e7b30 msgr2=0x55a86c5e6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 -- 172.21.15.19:0/1688123836 shutdown_connections 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.214+0000 7fb665029740 1 -- 172.21.15.19:0/1688123836 wait complete. 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb665029740 20 client.16287 trim_cache size 0 max 16384 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:31:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:31:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:31:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:31:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 -- 172.21.15.19:0/2501220016 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb6440a3dc0 msgr2=0x7fb6440a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb6440a3dc0 0x7fb6440a6280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb654009a60 tx=0x7fb6540095c0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 -- 172.21.15.19:0/2501220016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb65c106170 msgr2=0x7fb65c198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb65c106170 0x7fb65c198c50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb65000d770 tx=0x7fb65000dc30 comp rx=0 tx=0).stop 2024-04-23T18:31:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 -- 172.21.15.19:0/2501220016 shutdown_connections 2024-04-23T18:31:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb6440a3dc0 0x7fb6440a6280 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb65c19cfd0 0x7fb65c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb65c106170 0x7fb65c198c50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 --2- 172.21.15.19:0/2501220016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb65c1010f0 0x7fb65c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 -- 172.21.15.19:0/2501220016 >> 172.21.15.19:0/2501220016 conn(0x7fb65c077fd0 msgr2=0x7fb65c069960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 -- 172.21.15.19:0/2501220016 shutdown_connections 2024-04-23T18:31:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:00.215+0000 7fb662231640 1 -- 172.21.15.19:0/2501220016 wait complete. 2024-04-23T18:31:01.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:01 smithi019 ceph-mon[29215]: pgmap v149: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 16 op/s 2024-04-23T18:31:01.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:01 smithi044 ceph-mon[32458]: pgmap v149: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 16 op/s 2024-04-23T18:31:01.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:01 smithi155 ceph-mon[33468]: pgmap v149: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 16 op/s 2024-04-23T18:31:03.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:03 smithi019 ceph-mon[29215]: pgmap v150: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-23T18:31:03.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:03 smithi044 ceph-mon[32458]: pgmap v150: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-23T18:31:03.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:03 smithi155 ceph-mon[33468]: pgmap v150: 97 pgs: 97 active+clean; 5.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-23T18:31:05.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:05 smithi019 ceph-mon[29215]: pgmap v151: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 29 op/s 2024-04-23T18:31:05.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:05 smithi044 ceph-mon[32458]: pgmap v151: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 29 op/s 2024-04-23T18:31:05.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:05 smithi155 ceph-mon[33468]: pgmap v151: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 29 op/s 2024-04-23T18:31:07.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:07 smithi019 ceph-mon[29215]: pgmap v152: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:31:07.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:07 smithi044 ceph-mon[32458]: pgmap v152: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:31:07.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:07 smithi155 ceph-mon[33468]: pgmap v152: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:31:08.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:31:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.448+0000 7f3ce578e640 1 -- 172.21.15.19:0/185256780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0102bf0 msgr2=0x7f3ce0106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.448+0000 7f3ce578e640 1 --2- 172.21.15.19:0/185256780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0102bf0 0x7f3ce0106ba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009900 tx=0x7f3cc802a410 comp rx=0 tx=0).stop 2024-04-23T18:31:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.448+0000 7f3ce578e640 1 -- 172.21.15.19:0/185256780 shutdown_connections 2024-04-23T18:31:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.448+0000 7f3ce578e640 1 --2- 172.21.15.19:0/185256780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ce0107280 0x7f3ce010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.448+0000 7f3ce578e640 1 --2- 172.21.15.19:0/185256780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0102bf0 0x7f3ce0106ba0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.448+0000 7f3ce578e640 1 --2- 172.21.15.19:0/185256780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ce0102240 0x7f3ce0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.448+0000 7f3ce578e640 1 -- 172.21.15.19:0/185256780 >> 172.21.15.19:0/185256780 conn(0x7f3ce00790e0 msgr2=0x7f3ce00794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3ce578e640 1 -- 172.21.15.19:0/185256780 shutdown_connections 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3ce578e640 1 -- 172.21.15.19:0/185256780 wait complete. 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3ce578e640 1 Processor -- start 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3ce578e640 1 -- start start 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3ce578e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0102240 0x7f3ce0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3ce578e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ce0107280 0x7f3ce0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3ce578e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ce0074cd0 0x7f3ce01a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3ce578e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01121f0 con 0x7f3ce0102240 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3ce578e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0112070 con 0x7f3ce0074cd0 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3ce578e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0112370 con 0x7f3ce0107280 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.449+0000 7f3cdeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0102240 0x7f3ce0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0102240 0x7f3ce0074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48574/0 (socket says 172.21.15.19:48574) 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdeffd640 1 -- 172.21.15.19:0/2496852993 learned_addr learned my addr 172.21.15.19:0/2496852993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdf7fe640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ce0074cd0 0x7f3ce01a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cd7fff640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ce0107280 0x7f3ce0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdeffd640 1 -- 172.21.15.19:0/2496852993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ce0107280 msgr2=0x7f3ce0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdeffd640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ce0107280 0x7f3ce0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdeffd640 1 -- 172.21.15.19:0/2496852993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ce0074cd0 msgr2=0x7f3ce01a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdeffd640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ce0074cd0 0x7f3ce01a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdeffd640 1 -- 172.21.15.19:0/2496852993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce01a7060 con 0x7f3ce0102240 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cd7fff640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ce0107280 0x7f3ce0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdeffd640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0102240 0x7f3ce0074250 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000c890 tx=0x7f3cd000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdcff9640 1 -- 172.21.15.19:0/2496852993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0004220 con 0x7f3ce0102240 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdcff9640 1 -- 172.21.15.19:0/2496852993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd00043c0 con 0x7f3ce0102240 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3cdcff9640 1 -- 172.21.15.19:0/2496852993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0004a00 con 0x7f3ce0102240 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3ce578e640 1 -- 172.21.15.19:0/2496852993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce01a7350 con 0x7f3ce0102240 2024-04-23T18:31:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.450+0000 7f3ce578e640 1 -- 172.21.15.19:0/2496852993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce01a7870 con 0x7f3ce0102240 2024-04-23T18:31:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.451+0000 7f3ce578e640 1 -- 172.21.15.19:0/2496852993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce00691c0 con 0x7f3ce0102240 2024-04-23T18:31:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.453+0000 7f3cdcff9640 1 -- 172.21.15.19:0/2496852993 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3cd00079b0 con 0x7f3ce0102240 2024-04-23T18:31:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.453+0000 7f3cdcff9640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3ccc0a3d20 0x7f3ccc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.453+0000 7f3cd7fff640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3ccc0a3d20 0x7f3ccc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.453+0000 7f3cdcff9640 1 -- 172.21.15.19:0/2496852993 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3cd00d0520 con 0x7f3ce0102240 2024-04-23T18:31:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.454+0000 7f3cd7fff640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3ccc0a3d20 0x7f3ccc0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3cc802aa00 tx=0x7f3cc8000bc0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.458+0000 7f3cdcff9640 1 -- 172.21.15.19:0/2496852993 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cd0014030 con 0x7f3ce0102240 2024-04-23T18:31:08.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.686+0000 7f3ce578e640 1 -- 172.21.15.19:0/2496852993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ce010a4a0 con 0x7f3ce0102240 2024-04-23T18:31:08.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.687+0000 7f3cdcff9640 1 -- 172.21.15.19:0/2496852993 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f3cd009cbf0 con 0x7f3ce0102240 2024-04-23T18:31:08.680 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:31:08.681 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:31:08.681 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 -- 172.21.15.19:0/2496852993 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3ccc0a3d20 msgr2=0x7f3ccc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3ccc0a3d20 0x7f3ccc0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3cc802aa00 tx=0x7f3cc8000bc0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 -- 172.21.15.19:0/2496852993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0102240 msgr2=0x7f3ce0074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0102240 0x7f3ce0074250 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000c890 tx=0x7f3cd000cd50 comp rx=0 tx=0).stop 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 -- 172.21.15.19:0/2496852993 shutdown_connections 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3ccc0a3d20 0x7f3ccc0a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ce0074cd0 0x7f3ce01a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ce0107280 0x7f3ce0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 --2- 172.21.15.19:0/2496852993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0102240 0x7f3ce0074250 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 -- 172.21.15.19:0/2496852993 >> 172.21.15.19:0/2496852993 conn(0x7f3ce00790e0 msgr2=0x7f3ce0100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 -- 172.21.15.19:0/2496852993 shutdown_connections 2024-04-23T18:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.689+0000 7f3ce578e640 1 -- 172.21.15.19:0/2496852993 wait complete. 2024-04-23T18:31:08.693 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id 20f3d723 --release --await' 2024-04-23T18:31:08.694 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id 20f3d723 --release --await 2024-04-23T18:31:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 -- 172.21.15.19:0/404324668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a8102190 msgr2=0x7f22a8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 --2- 172.21.15.19:0/404324668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a8102190 0x7f22a8102570 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009900 tx=0x7f22a402a410 comp rx=0 tx=0).stop 2024-04-23T18:31:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 -- 172.21.15.19:0/404324668 shutdown_connections 2024-04-23T18:31:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 --2- 172.21.15.19:0/404324668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a8107260 0x7f22a810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 --2- 172.21.15.19:0/404324668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22a8102b40 0x7f22a8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 --2- 172.21.15.19:0/404324668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a8102190 0x7f22a8102570 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 -- 172.21.15.19:0/404324668 >> 172.21.15.19:0/404324668 conn(0x7f22a80793a0 msgr2=0x7f22a80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 -- 172.21.15.19:0/404324668 shutdown_connections 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 -- 172.21.15.19:0/404324668 wait complete. 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 Processor -- start 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 -- start start 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22a8102190 0x7f22a81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.810+0000 7f22b0259640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a8107260 0x7f22a81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22b0259640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a819a7c0 0x7f22a819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22b0259640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81121c0 con 0x7f22a819a7c0 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22b0259640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22a8112040 con 0x7f22a8107260 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22b0259640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22a8112340 con 0x7f22a8102190 2024-04-23T18:31:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22ae7f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a819a7c0 0x7f22a819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22ae7f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a819a7c0 0x7f22a819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48598/0 (socket says 172.21.15.19:48598) 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22ae7f6640 1 -- 172.21.15.19:0/3129023983 learned_addr learned my addr 172.21.15.19:0/3129023983 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a8107260 0x7f22a81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22adff5640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22a8102190 0x7f22a81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22ae7f6640 1 -- 172.21.15.19:0/3129023983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22a8102190 msgr2=0x7f22a81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22ae7f6640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22a8102190 0x7f22a81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22ae7f6640 1 -- 172.21.15.19:0/3129023983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a8107260 msgr2=0x7f22a81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22ae7f6640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a8107260 0x7f22a81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22ae7f6640 1 -- 172.21.15.19:0/3129023983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a819b350 con 0x7f22a819a7c0 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22ae7f6640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a819a7c0 0x7f22a819ac70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f22a000eb10 tx=0x7f22a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f229effd640 1 -- 172.21.15.19:0/3129023983 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0004b20 con 0x7f22a819a7c0 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22b0259640 1 -- 172.21.15.19:0/3129023983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a819b640 con 0x7f22a819a7c0 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f229effd640 1 -- 172.21.15.19:0/3129023983 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a0004cc0 con 0x7f22a819a7c0 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f229effd640 1 -- 172.21.15.19:0/3129023983 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0010670 con 0x7f22a819a7c0 2024-04-23T18:31:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.811+0000 7f22b0259640 1 -- 172.21.15.19:0/3129023983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a81a77d0 con 0x7f22a819a7c0 2024-04-23T18:31:08.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.814+0000 7f229effd640 1 -- 172.21.15.19:0/3129023983 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f22a0005020 con 0x7f22a819a7c0 2024-04-23T18:31:08.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.814+0000 7f22b2f6c740 1 Processor -- start 2024-04-23T18:31:08.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.814+0000 7f22b2f6c740 1 -- start start 2024-04-23T18:31:08.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.814+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f317870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.814+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f317db0 0x563b1f321f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.814+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f324870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.814+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563b1f27c6d0 con 0x563b1f322480 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.814+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563b1f2191a0 con 0x563b1f317db0 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.814+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563b1f278690 con 0x563b1f317490 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ad7f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f324870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ad7f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f324870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48600/0 (socket says 172.21.15.19:48600) 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ad7f4640 1 -- 172.21.15.19:0/3021317203 learned_addr learned my addr 172.21.15.19:0/3021317203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f229effd640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f22940a3d20 0x7f22940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22adff5640 1 --2- 172.21.15.19:0/3021317203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f317db0 0x563b1f321f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/3021317203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f317870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ae7f6640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f22940a3d20 0x7f22940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ad7f4640 1 -- 172.21.15.19:0/3021317203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 msgr2=0x563b1f317870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/3021317203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f317870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ad7f4640 1 -- 172.21.15.19:0/3021317203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f317db0 msgr2=0x563b1f321f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/3021317203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f317db0 0x563b1f321f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ad7f4640 1 -- 172.21.15.19:0/3021317203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1f2235b0 con 0x563b1f322480 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f229effd640 1 -- 172.21.15.19:0/3129023983 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f22a00d0d80 con 0x7f22a819a7c0 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/3021317203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f324870 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f22980058c0 tx=0x7f2298005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b700acaebe92407e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f22ae7f6640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f22940a3d20 0x7f22940a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f22a009ca50 tx=0x7f22a0067040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f2293fff640 1 -- 172.21.15.19:0/3021317203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298005020 con 0x563b1f322480 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f2293fff640 1 -- 172.21.15.19:0/3021317203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22980047a0 con 0x563b1f322480 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.815+0000 7f2293fff640 1 -- 172.21.15.19:0/3021317203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229800fb20 con 0x563b1f322480 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.816+0000 7f22b2f6c740 1 -- 172.21.15.19:0/3021317203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 msgr2=0x563b1f324870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.816+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/3021317203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f324870 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f22980058c0 tx=0x7f2298005850 comp rx=0 tx=0).stop 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.816+0000 7f22b2f6c740 1 -- 172.21.15.19:0/3021317203 shutdown_connections 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.816+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/3021317203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f317db0 0x563b1f321f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.816+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/3021317203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f324870 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.816+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/3021317203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f317870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.816+0000 7f22b2f6c740 1 -- 172.21.15.19:0/3021317203 >> 172.21.15.19:0/3021317203 conn(0x563b1f3137b0 msgr2=0x563b1f313b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.816+0000 7f22b2f6c740 1 -- 172.21.15.19:0/3021317203 shutdown_connections 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.816+0000 7f22b2f6c740 1 -- 172.21.15.19:0/3021317203 wait complete. 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.816+0000 7f22b2f6c740 1 -- start start 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22b2f6c740 1 Processor -- start 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317490 0x563b1f31f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317db0 0x563b1f31f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f322480 0x563b1f31fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563b1f325310 con 0x563b1f317490 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563b1f216b20 con 0x563b1f322480 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563b1f325490 con 0x563b1f317db0 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22b2f6c740 10 client.0 _ll_register_callbacks cb 0x563b1f216530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22ad7f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317490 0x563b1f31f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22ad7f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317490 0x563b1f31f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48602/0 (socket says 172.21.15.19:48602) 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22ad7f4640 1 -- 172.21.15.19:0/150664005 learned_addr learned my addr 172.21.15.19:0/150664005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22adff5640 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317db0 0x563b1f31f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22ad7f4640 1 -- 172.21.15.19:0/150664005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317db0 msgr2=0x563b1f31f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317db0 0x563b1f31f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22ad7f4640 1 -- 172.21.15.19:0/150664005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f322480 msgr2=0x563b1f31fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f322480 0x563b1f31fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.817+0000 7f22ad7f4640 1 -- 172.21.15.19:0/150664005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1ef8e920 con 0x563b1f317490 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317490 0x563b1f31f260 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2298001eb0 tx=0x7f2298004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93db4c6df582c501 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22917fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22917fa640 1 -- 172.21.15.19:0/150664005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298004ca0 con 0x563b1f317490 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22917fa640 1 -- 172.21.15.19:0/150664005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2298004040 con 0x563b1f317490 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22b2f6c740 1 -- 172.21.15.19:0/150664005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22980047a0 con 0x563b1f317490 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22917fa640 1 -- 172.21.15.19:0/150664005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2298004240 con 0x563b1f317490 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22b2f6c740 10 client.16305 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22b2f6c740 1 -- 172.21.15.19:0/150664005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b1f05ad30 con 0x563b1f317490 2024-04-23T18:31:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22917fa640 1 -- 172.21.15.19:0/150664005 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22980047a0 con 0x563b1f317490 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22917fa640 1 -- 172.21.15.19:0/150664005 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2298027070 con 0x563b1f317490 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22b2f6c740 10 client.16305 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22b2f6c740 10 client.16305 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22b2f6c740 20 client.16305 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563b1f332580 0x563b1f334970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22b2f6c740 4 client.16305 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.818+0000 7f22b2f6c740 1 -- 172.21.15.19:0/150664005 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b1f23a500 con 0x563b1f332580 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.819+0000 7f22adff5640 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563b1f332580 0x563b1f334970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.819+0000 7f22adff5640 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563b1f332580 0x563b1f334970 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd11f65c187c9ea server_cookie=eb18ad2f95c52184 in_seq=0 out_seq=0 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.819+0000 7f22917fa640 10 client.16305 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22917fa640 1 -- 172.21.15.19:0/150664005 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b1f23a500 con 0x563b1f332580 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22917fa640 10 client.16305 handle_command_reply: tid=0 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 client.16305 shutdown 2024-04-23T18:31:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 -- 172.21.15.19:0/150664005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317490 msgr2=0x563b1f31f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317490 0x563b1f31f260 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2298001eb0 tx=0x7f2298004730 comp rx=0 tx=0).stop 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 -- 172.21.15.19:0/150664005 shutdown_connections 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563b1f332580 0x563b1f334970 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f322480 0x563b1f31fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317db0 0x563b1f31f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/150664005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317490 0x563b1f31f260 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 -- 172.21.15.19:0/150664005 >> 172.21.15.19:0/150664005 conn(0x563b1f3137b0 msgr2=0x563b1f3195b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22917fa640 0 client.16305 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 -- 172.21.15.19:0/150664005 shutdown_connections 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 1 -- 172.21.15.19:0/150664005 wait complete. 2024-04-23T18:31:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.820+0000 7f22b2f6c740 20 client.16305 trim_cache size 0 max 16384 2024-04-23T18:31:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22b2f6c740 1 Processor -- start 2024-04-23T18:31:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22b2f6c740 1 -- start start 2024-04-23T18:31:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f321a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317db0 0x563b1f32e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f322480 0x563b1f31f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81afdc0 con 0x563b1f317db0 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b1900 con 0x563b1f322480 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b0490 con 0x563b1f317490 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22adff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317db0 0x563b1f32e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22adff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317db0 0x563b1f32e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48610/0 (socket says 172.21.15.19:48610) 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.838+0000 7f22adff5640 1 -- 172.21.15.19:0/2616386507 learned_addr learned my addr 172.21.15.19:0/2616386507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.839+0000 7f22adff5640 1 -- 172.21.15.19:0/2616386507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 msgr2=0x563b1f321a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.839+0000 7f22adff5640 1 --2- 172.21.15.19:0/2616386507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f321a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.839+0000 7f22adff5640 1 -- 172.21.15.19:0/2616386507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f322480 msgr2=0x563b1f31f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.839+0000 7f22adff5640 1 --2- 172.21.15.19:0/2616386507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f322480 0x563b1f31f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.839+0000 7f22adff5640 1 -- 172.21.15.19:0/2616386507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1f317be0 con 0x563b1f317db0 2024-04-23T18:31:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.839+0000 7f22adff5640 1 --2- 172.21.15.19:0/2616386507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317db0 0x563b1f32e1d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ec0 tx=0x7f22a4005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b624ba917567ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.839+0000 7f22937fe640 1 -- 172.21.15.19:0/2616386507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4049070 con 0x563b1f317db0 2024-04-23T18:31:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.839+0000 7f22937fe640 1 -- 172.21.15.19:0/2616386507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a4044dc0 con 0x563b1f317db0 2024-04-23T18:31:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.839+0000 7f22937fe640 1 -- 172.21.15.19:0/2616386507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4035040 con 0x563b1f317db0 2024-04-23T18:31:08.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2616386507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317db0 msgr2=0x563b1f32e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/2616386507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317db0 0x563b1f32e1d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ec0 tx=0x7f22a4005e30 comp rx=0 tx=0).stop 2024-04-23T18:31:08.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2616386507 shutdown_connections 2024-04-23T18:31:08.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/2616386507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f322480 0x563b1f31f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/2616386507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f317db0 0x563b1f32e1d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/2616386507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f321a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2616386507 >> 172.21.15.19:0/2616386507 conn(0x563b1f3137b0 msgr2=0x563b1f321320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2616386507 shutdown_connections 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2616386507 wait complete. 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 -- start start 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 Processor -- start 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f31c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f317db0 0x563b1f31ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f328cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b1230 con 0x563b1f322480 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b0b60 con 0x563b1f317db0 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81adda0 con 0x563b1f317490 2024-04-23T18:31:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.840+0000 7f22b2f6c740 10 client.0 _ll_register_callbacks cb 0x7f22980116a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22ad7f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f328cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22ad7f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f328cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48616/0 (socket says 172.21.15.19:48616) 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22ad7f4640 1 -- 172.21.15.19:0/2356352023 learned_addr learned my addr 172.21.15.19:0/2356352023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f31c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22adff5640 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f317db0 0x563b1f31ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22ad7f4640 1 -- 172.21.15.19:0/2356352023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 msgr2=0x563b1f31c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f31c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22ad7f4640 1 -- 172.21.15.19:0/2356352023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f317db0 msgr2=0x563b1f31ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f317db0 0x563b1f31ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22ad7f4640 1 -- 172.21.15.19:0/2356352023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1f329230 con 0x563b1f322480 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22ad7f4640 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f328cf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x563b1f3209c0 tx=0x7f2298011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d775435e49624250 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22917fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22917fa640 1 -- 172.21.15.19:0/2356352023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298036040 con 0x563b1f322480 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22917fa640 1 -- 172.21.15.19:0/2356352023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f22980043e0 con 0x563b1f322480 2024-04-23T18:31:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2356352023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a4044dc0 con 0x563b1f322480 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22917fa640 1 -- 172.21.15.19:0/2356352023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2298040050 con 0x563b1f322480 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22b2f6c740 10 client.16311 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2356352023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b1f399080 con 0x563b1f322480 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.841+0000 7f22917fa640 1 -- 172.21.15.19:0/2356352023 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a4044dc0 con 0x563b1f322480 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.842+0000 7f22917fa640 1 -- 172.21.15.19:0/2356352023 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2298027040 con 0x563b1f322480 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.842+0000 7f22b2f6c740 10 client.16311 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.842+0000 7f22b2f6c740 10 client.16311 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.842+0000 7f22b2f6c740 20 client.16311 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.842+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563b1f399740 0x563b1f39bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.842+0000 7f22b2f6c740 4 client.16311 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "20f3d723", "release": true, "await": true}] 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.842+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2356352023 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "20f3d723", "release": true, "await": true}) -- 0x563b1f28a0f0 con 0x563b1f399740 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.842+0000 7f22adff5640 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563b1f399740 0x563b1f39bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.843+0000 7f22adff5640 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563b1f399740 0x563b1f39bb60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28383902656d6ec7 server_cookie=bed14a8fc1aa0e8c in_seq=0 out_seq=0 2024-04-23T18:31:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.843+0000 7f22917fa640 10 client.16311 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22917fa640 1 -- 172.21.15.19:0/2356352023 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563b1f28a0f0 con 0x563b1f399740 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22917fa640 10 client.16311 handle_command_reply: tid=0 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 client.16311 shutdown 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2356352023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 msgr2=0x563b1f328cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f328cf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x563b1f3209c0 tx=0x7f2298011440 comp rx=0 tx=0).stop 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2356352023 shutdown_connections 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x563b1f399740 0x563b1f39bb60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b1f317db0 0x563b1f31ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b1f322480 0x563b1f328cf0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22917fa640 0 client.16311 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 --2- 172.21.15.19:0/2356352023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b1f317490 0x563b1f31c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2356352023 >> 172.21.15.19:0/2356352023 conn(0x563b1f3137b0 msgr2=0x563b1f32d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2356352023 shutdown_connections 2024-04-23T18:31:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 1 -- 172.21.15.19:0/2356352023 wait complete. 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b2f6c740 20 client.16311 trim_cache size 0 max 16384 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 24, 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: "20f3d723": { 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 24, 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:31:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 83.6, 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:08.844 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:31:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 -- 172.21.15.19:0/3129023983 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f22940a3d20 msgr2=0x7f22940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f22940a3d20 0x7f22940a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f22a009ca50 tx=0x7f22a0067040 comp rx=0 tx=0).stop 2024-04-23T18:31:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 -- 172.21.15.19:0/3129023983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a819a7c0 msgr2=0x7f22a819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a819a7c0 0x7f22a819ac70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f22a000eb10 tx=0x7f22a000c2d0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 -- 172.21.15.19:0/3129023983 shutdown_connections 2024-04-23T18:31:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f22940a3d20 0x7f22940a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22a8107260 0x7f22a81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a819a7c0 0x7f22a819ac70 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 --2- 172.21.15.19:0/3129023983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22a8102190 0x7f22a81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 -- 172.21.15.19:0/3129023983 >> 172.21.15.19:0/3129023983 conn(0x7f22a80793a0 msgr2=0x7f22a8100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 -- 172.21.15.19:0/3129023983 shutdown_connections 2024-04-23T18:31:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.847+0000 7f22b0259640 1 -- 172.21.15.19:0/3129023983 wait complete. 2024-04-23T18:31:08.851 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '20f3d723', seconds elapsed: 24.7 2024-04-23T18:31:08.851 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.6 2024-04-23T18:31:08.852 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:31:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 -- 172.21.15.19:0/1951645831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47a0101a50 msgr2=0x7f47a0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 --2- 172.21.15.19:0/1951645831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47a0101a50 0x7f47a0105a90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f478c009900 tx=0x7f478c02a410 comp rx=0 tx=0).stop 2024-04-23T18:31:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 -- 172.21.15.19:0/1951645831 shutdown_connections 2024-04-23T18:31:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 --2- 172.21.15.19:0/1951645831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47a0106170 0x7f47a010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 --2- 172.21.15.19:0/1951645831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47a0101a50 0x7f47a0105a90 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 --2- 172.21.15.19:0/1951645831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a01010a0 0x7f47a0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 -- 172.21.15.19:0/1951645831 >> 172.21.15.19:0/1951645831 conn(0x7f47a0077f50 msgr2=0x7f47a0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 -- 172.21.15.19:0/1951645831 shutdown_connections 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 -- 172.21.15.19:0/1951645831 wait complete. 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 Processor -- start 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.960+0000 7f47a6df9640 1 -- start start 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f47a6df9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a01010a0 0x7f47a0102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f47a6df9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47a0106170 0x7f47a0103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f47a6df9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47a01035b0 0x7f47a01a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f47a6df9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0111090 con 0x7f47a0106170 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f47a6df9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0110f10 con 0x7f47a01035b0 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f47a6df9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0111210 con 0x7f47a01010a0 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47a0106170 0x7f47a0103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47a0106170 0x7f47a0103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48628/0 (socket says 172.21.15.19:48628) 2024-04-23T18:31:08.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479ffff640 1 -- 172.21.15.19:0/879859343 learned_addr learned my addr 172.21.15.19:0/879859343 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f47a5396640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47a01035b0 0x7f47a01a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f47a4b95640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a01010a0 0x7f47a0102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479ffff640 1 -- 172.21.15.19:0/879859343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a01010a0 msgr2=0x7f47a0102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479ffff640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a01010a0 0x7f47a0102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479ffff640 1 -- 172.21.15.19:0/879859343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47a01035b0 msgr2=0x7f47a01a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479ffff640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47a01035b0 0x7f47a01a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479ffff640 1 -- 172.21.15.19:0/879859343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a01a7010 con 0x7f47a0106170 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f47a4b95640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a01010a0 0x7f47a0102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479ffff640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47a0106170 0x7f47a0103070 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f478c032fd0 tx=0x7f478c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479dffb640 1 -- 172.21.15.19:0/879859343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c03f030 con 0x7f47a0106170 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479dffb640 1 -- 172.21.15.19:0/879859343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c004490 con 0x7f47a0106170 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f479dffb640 1 -- 172.21.15.19:0/879859343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c035070 con 0x7f47a0106170 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.961+0000 7f47a6df9640 1 -- 172.21.15.19:0/879859343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a01a72a0 con 0x7f47a0106170 2024-04-23T18:31:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.962+0000 7f47a6df9640 1 -- 172.21.15.19:0/879859343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a01a7740 con 0x7f47a0106170 2024-04-23T18:31:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.963+0000 7f47a6df9640 1 -- 172.21.15.19:0/879859343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a0068d70 con 0x7f47a0106170 2024-04-23T18:31:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.964+0000 7f479dffb640 1 -- 172.21.15.19:0/879859343 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f478c042030 con 0x7f47a0106170 2024-04-23T18:31:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.964+0000 7f479dffb640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47900a3dd0 0x7f47900a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.964+0000 7f479dffb640 1 -- 172.21.15.19:0/879859343 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f478c0f0ca0 con 0x7f47a0106170 2024-04-23T18:31:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.964+0000 7f47a4b95640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47900a3dd0 0x7f47900a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.965+0000 7f47a4b95640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47900a3dd0 0x7f47900a6290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f47940058c0 tx=0x7f4794005d30 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:08.969+0000 7f479dffb640 1 -- 172.21.15.19:0/879859343 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f478c0bd370 con 0x7f47a0106170 2024-04-23T18:31:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.195+0000 7f47a6df9640 1 -- 172.21.15.19:0/879859343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47a010a3b0 con 0x7f47a0106170 2024-04-23T18:31:09.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.196+0000 7f479dffb640 1 -- 172.21.15.19:0/879859343 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f478c0bcc70 con 0x7f47a0106170 2024-04-23T18:31:09.189 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:31:09.189 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:31:09.190 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:31:09.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 -- 172.21.15.19:0/879859343 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47900a3dd0 msgr2=0x7f47900a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47900a3dd0 0x7f47900a6290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f47940058c0 tx=0x7f4794005d30 comp rx=0 tx=0).stop 2024-04-23T18:31:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 -- 172.21.15.19:0/879859343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47a0106170 msgr2=0x7f47a0103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47a0106170 0x7f47a0103070 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f478c032fd0 tx=0x7f478c004270 comp rx=0 tx=0).stop 2024-04-23T18:31:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 -- 172.21.15.19:0/879859343 shutdown_connections 2024-04-23T18:31:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f47900a3dd0 0x7f47900a6290 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47a01035b0 0x7f47a01a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47a0106170 0x7f47a0103070 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 --2- 172.21.15.19:0/879859343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a01010a0 0x7f47a0102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 -- 172.21.15.19:0/879859343 >> 172.21.15.19:0/879859343 conn(0x7f47a0077f50 msgr2=0x7f47a0069520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 -- 172.21.15.19:0/879859343 shutdown_connections 2024-04-23T18:31:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.198+0000 7f47a6df9640 1 -- 172.21.15.19:0/879859343 wait complete. 2024-04-23T18:31:09.202 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 83.6 --await-for 42.0' 2024-04-23T18:31:09.202 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 83.6 --await-for 42.0 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.321+0000 7fb1a1a13640 1 -- 172.21.15.19:0/3562512210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb19c103230 msgr2=0x7fb19c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.321+0000 7fb1a1a13640 1 --2- 172.21.15.19:0/3562512210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb19c103230 0x7fb19c103610 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009900 tx=0x7fb18c02a480 comp rx=0 tx=0).stop 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.321+0000 7fb1a1a13640 1 -- 172.21.15.19:0/3562512210 shutdown_connections 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.321+0000 7fb1a1a13640 1 --2- 172.21.15.19:0/3562512210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb19c10c570 0x7fb19c10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.321+0000 7fb1a1a13640 1 --2- 172.21.15.19:0/3562512210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb19c103b50 0x7fb19c10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.321+0000 7fb1a1a13640 1 --2- 172.21.15.19:0/3562512210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb19c103230 0x7fb19c103610 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.321+0000 7fb1a1a13640 1 -- 172.21.15.19:0/3562512210 >> 172.21.15.19:0/3562512210 conn(0x7fb19c077fb0 msgr2=0x7fb19c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.321+0000 7fb1a1a13640 1 -- 172.21.15.19:0/3562512210 shutdown_connections 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.321+0000 7fb1a1a13640 1 -- 172.21.15.19:0/3562512210 wait complete. 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb1a1a13640 1 Processor -- start 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb1a1a13640 1 -- start start 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb1a1a13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb19c103230 0x7fb19c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb1a1a13640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb19c103b50 0x7fb19c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb1a1a13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb19c10c570 0x7fb19c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb1a1a13640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1121d0 con 0x7fb19c103230 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb1a1a13640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c112050 con 0x7fb19c10c570 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb1a1a13640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c112350 con 0x7fb19c103b50 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb19affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb19c103230 0x7fb19c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb19affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb19c103230 0x7fb19c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48646/0 (socket says 172.21.15.19:48646) 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb19affd640 1 -- 172.21.15.19:0/651304260 learned_addr learned my addr 172.21.15.19:0/651304260 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.322+0000 7fb19b7fe640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb19c10c570 0x7fb19c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb1927fc640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb19c103b50 0x7fb19c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb19affd640 1 -- 172.21.15.19:0/651304260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb19c103b50 msgr2=0x7fb19c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb19affd640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb19c103b50 0x7fb19c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb19affd640 1 -- 172.21.15.19:0/651304260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb19c10c570 msgr2=0x7fb19c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb19affd640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb19c10c570 0x7fb19c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb19affd640 1 -- 172.21.15.19:0/651304260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19c19b040 con 0x7fb19c103230 2024-04-23T18:31:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb19affd640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb19c103230 0x7fb19c1a06b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb18c005940 tx=0x7fb18c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb1a0a11640 1 -- 172.21.15.19:0/651304260 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c03f030 con 0x7fb19c103230 2024-04-23T18:31:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb1a0a11640 1 -- 172.21.15.19:0/651304260 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb18c003cb0 con 0x7fb19c103230 2024-04-23T18:31:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb1a1a13640 1 -- 172.21.15.19:0/651304260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19c19b2d0 con 0x7fb19c103230 2024-04-23T18:31:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb1a0a11640 1 -- 172.21.15.19:0/651304260 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c035070 con 0x7fb19c103230 2024-04-23T18:31:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.323+0000 7fb1a1a13640 1 -- 172.21.15.19:0/651304260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19c1a7500 con 0x7fb19c103230 2024-04-23T18:31:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.325+0000 7fb1a0a11640 1 -- 172.21.15.19:0/651304260 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb18c042030 con 0x7fb19c103230 2024-04-23T18:31:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1a4808740 1 Processor -- start 2024-04-23T18:31:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1a4808740 1 -- start start 2024-04-23T18:31:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27290a0 0x5596f2729480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27299c0 0x5596f2733b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f2734090 0x5596f2736480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1a4808740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596f268e370 con 0x5596f2734090 2024-04-23T18:31:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1a4808740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5596f262b2e0 con 0x5596f27299c0 2024-04-23T18:31:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1a4808740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5596f268a330 con 0x5596f27290a0 2024-04-23T18:31:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f2734090 0x5596f2736480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1a0a11640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb1880a3d20 0x7fb1880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f2734090 0x5596f2736480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48650/0 (socket says 172.21.15.19:48650) 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1927fc640 1 -- 172.21.15.19:0/2207356891 learned_addr learned my addr 172.21.15.19:0/2207356891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb19b7fe640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb1880a3d20 0x7fb1880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb19b7fe640 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27290a0 0x5596f2729480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1a0a11640 1 -- 172.21.15.19:0/651304260 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb18c0f0fa0 con 0x7fb19c103230 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb19affd640 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27299c0 0x5596f2733b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1927fc640 1 -- 172.21.15.19:0/2207356891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27290a0 msgr2=0x5596f2729480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1927fc640 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27290a0 0x5596f2729480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1927fc640 1 -- 172.21.15.19:0/2207356891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27299c0 msgr2=0x5596f2733b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1927fc640 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27299c0 0x5596f2733b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb1927fc640 1 -- 172.21.15.19:0/2207356891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f246cd80 con 0x5596f2734090 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb19b7fe640 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27290a0 0x5596f2729480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.326+0000 7fb19affd640 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27299c0 0x5596f2733b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb1927fc640 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f2734090 0x5596f2736480 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b6c0 tx=0x7fb18400bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ba6de4e4235de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb19b7fe640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb1880a3d20 0x7fb1880a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb194005fd0 tx=0x7fb194004a40 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb191ffb640 1 -- 172.21.15.19:0/2207356891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184003bd0 con 0x5596f2734090 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb191ffb640 1 -- 172.21.15.19:0/2207356891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb184015070 con 0x5596f2734090 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb191ffb640 1 -- 172.21.15.19:0/2207356891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184004cf0 con 0x5596f2734090 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb1a4808740 1 -- 172.21.15.19:0/2207356891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f2734090 msgr2=0x5596f2736480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb1a4808740 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f2734090 0x5596f2736480 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b6c0 tx=0x7fb18400bb80 comp rx=0 tx=0).stop 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb1a4808740 1 -- 172.21.15.19:0/2207356891 shutdown_connections 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb1a4808740 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27299c0 0x5596f2733b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb1a4808740 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f2734090 0x5596f2736480 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb1a4808740 1 --2- 172.21.15.19:0/2207356891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27290a0 0x5596f2729480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb1a4808740 1 -- 172.21.15.19:0/2207356891 >> 172.21.15.19:0/2207356891 conn(0x5596f27253c0 msgr2=0x5596f27257a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb1a4808740 1 -- 172.21.15.19:0/2207356891 shutdown_connections 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.327+0000 7fb1a4808740 1 -- 172.21.15.19:0/2207356891 wait complete. 2024-04-23T18:31:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb1a4808740 1 -- start start 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb1a4808740 1 Processor -- start 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27290a0 0x5596f2730f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27299c0 0x5596f27314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f2734090 0x5596f27319f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb1a4808740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596f2736e80 con 0x5596f27290a0 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb1a4808740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5596f2628fb0 con 0x5596f2734090 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb1a4808740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5596f2737000 con 0x5596f27299c0 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb1a4808740 10 client.0 _ll_register_callbacks cb 0x5596f26289a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb19affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27290a0 0x5596f2730f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb19affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27290a0 0x5596f2730f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48666/0 (socket says 172.21.15.19:48666) 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.328+0000 7fb19affd640 1 -- 172.21.15.19:0/3198466464 learned_addr learned my addr 172.21.15.19:0/3198466464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb1927fc640 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27299c0 0x5596f27314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb19b7fe640 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f2734090 0x5596f27319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb19affd640 1 -- 172.21.15.19:0/3198466464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27299c0 msgr2=0x5596f27314b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb19affd640 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27299c0 0x5596f27314b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb19affd640 1 -- 172.21.15.19:0/3198466464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f2734090 msgr2=0x5596f27319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb19affd640 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f2734090 0x5596f27319f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb19affd640 1 -- 172.21.15.19:0/3198466464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f24cd510 con 0x5596f27290a0 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb1927fc640 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27299c0 0x5596f27314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb19affd640 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27290a0 0x5596f2730f70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0ee780 tx=0x7fb18c0b93b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfe390dd8b10437 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb17affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb17affd640 1 -- 172.21.15.19:0/3198466464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c049070 con 0x5596f27290a0 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb17affd640 1 -- 172.21.15.19:0/3198466464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb18c0b9690 con 0x5596f27290a0 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb1a4808740 1 -- 172.21.15.19:0/3198466464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb184015070 con 0x5596f27290a0 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb17affd640 1 -- 172.21.15.19:0/3198466464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb18c005d50 con 0x5596f27290a0 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb1a4808740 10 client.16329 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb1a4808740 1 -- 172.21.15.19:0/3198466464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596f2360670 con 0x5596f27290a0 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb17affd640 1 -- 172.21.15.19:0/3198466464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb184015070 con 0x5596f27290a0 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb17affd640 1 -- 172.21.15.19:0/3198466464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb18c0f30b0 con 0x5596f27290a0 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb1a4808740 10 client.16329 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb1a4808740 10 client.16329 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.329+0000 7fb1a4808740 20 client.16329 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.330+0000 7fb1a4808740 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596f27441d0 0x5596f2746640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.330+0000 7fb1a4808740 4 client.16329 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.330+0000 7fb1a4808740 1 -- 172.21.15.19:0/3198466464 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596f269bd90 con 0x5596f27441d0 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.330+0000 7fb1927fc640 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596f27441d0 0x5596f2746640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.330+0000 7fb1927fc640 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596f27441d0 0x5596f2746640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e37792e3f2ddc0 server_cookie=8dd31d79871e0ea4 in_seq=0 out_seq=0 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.330+0000 7fb17affd640 10 client.16329 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb17affd640 1 -- 172.21.15.19:0/3198466464 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596f269bd90 con 0x5596f27441d0 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb17affd640 10 client.16329 handle_command_reply: tid=0 2024-04-23T18:31:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 client.16329 shutdown 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 -- 172.21.15.19:0/3198466464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27290a0 msgr2=0x5596f2730f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27290a0 0x5596f2730f70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0ee780 tx=0x7fb18c0b93b0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 -- 172.21.15.19:0/3198466464 shutdown_connections 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596f27441d0 0x5596f2746640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f2734090 0x5596f27319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27299c0 0x5596f27314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb17affd640 0 client.16329 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 --2- 172.21.15.19:0/3198466464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27290a0 0x5596f2730f70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 -- 172.21.15.19:0/3198466464 >> 172.21.15.19:0/3198466464 conn(0x5596f27253c0 msgr2=0x5596f272b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 -- 172.21.15.19:0/3198466464 shutdown_connections 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 1 -- 172.21.15.19:0/3198466464 wait complete. 2024-04-23T18:31:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.331+0000 7fb1a4808740 20 client.16329 trim_cache size 0 max 16384 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.348+0000 7fb1a4808740 1 Processor -- start 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.348+0000 7fb1a4808740 1 -- start start 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27290a0 0x5596f273fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f274bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f2734090 0x5596f274c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1a4808740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1afc90 con 0x5596f27299c0 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1a4808740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1b17d0 con 0x5596f2734090 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1a4808740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1b0360 con 0x5596f27290a0 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f274bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f274bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48682/0 (socket says 172.21.15.19:48682) 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1927fc640 1 -- 172.21.15.19:0/1083782231 learned_addr learned my addr 172.21.15.19:0/1083782231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb19b7fe640 1 --2- 172.21.15.19:0/1083782231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f2734090 0x5596f274c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1927fc640 1 -- 172.21.15.19:0/1083782231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27290a0 msgr2=0x5596f273fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1927fc640 1 --2- 172.21.15.19:0/1083782231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27290a0 0x5596f273fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1927fc640 1 -- 172.21.15.19:0/1083782231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f2734090 msgr2=0x5596f274c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1927fc640 1 --2- 172.21.15.19:0/1083782231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f2734090 0x5596f274c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1927fc640 1 -- 172.21.15.19:0/1083782231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f272bd20 con 0x5596f27299c0 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1927fc640 1 --2- 172.21.15.19:0/1083782231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f274bee0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb1840045c0 tx=0x7fb184013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a898248b96b0acf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1917fa640 1 -- 172.21.15.19:0/1083782231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184003cd0 con 0x5596f27299c0 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1917fa640 1 -- 172.21.15.19:0/1083782231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb184015040 con 0x5596f27299c0 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.349+0000 7fb1917fa640 1 -- 172.21.15.19:0/1083782231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184011b00 con 0x5596f27299c0 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.350+0000 7fb1a4808740 1 -- 172.21.15.19:0/1083782231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 msgr2=0x5596f274bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.350+0000 7fb1a4808740 1 --2- 172.21.15.19:0/1083782231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f274bee0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb1840045c0 tx=0x7fb184013040 comp rx=0 tx=0).stop 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.350+0000 7fb1a4808740 1 -- 172.21.15.19:0/1083782231 shutdown_connections 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.350+0000 7fb1a4808740 1 --2- 172.21.15.19:0/1083782231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f2734090 0x5596f274c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.350+0000 7fb1a4808740 1 --2- 172.21.15.19:0/1083782231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f274bee0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.350+0000 7fb1a4808740 1 --2- 172.21.15.19:0/1083782231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f27290a0 0x5596f273fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.350+0000 7fb1a4808740 1 -- 172.21.15.19:0/1083782231 >> 172.21.15.19:0/1083782231 conn(0x5596f27253c0 msgr2=0x5596f2724600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.350+0000 7fb1a4808740 1 -- 172.21.15.19:0/1083782231 shutdown_connections 2024-04-23T18:31:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.350+0000 7fb1a4808740 1 -- 172.21.15.19:0/1083782231 wait complete. 2024-04-23T18:31:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.350+0000 7fb1a4808740 1 -- start start 2024-04-23T18:31:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1a4808740 1 Processor -- start 2024-04-23T18:31:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27290a0 0x5596f2738540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f2738a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1a4808740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f2734090 0x5596f27a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1a4808740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1b1100 con 0x5596f27299c0 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1a4808740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1b0a30 con 0x5596f27290a0 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1a4808740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1adc70 con 0x5596f2734090 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1a4808740 10 client.0 _ll_register_callbacks cb 0x7fb18c0b8bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f2738a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f2738a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48694/0 (socket says 172.21.15.19:48694) 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1927fc640 1 -- 172.21.15.19:0/570226529 learned_addr learned my addr 172.21.15.19:0/570226529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb19affd640 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27290a0 0x5596f2738540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1927fc640 1 -- 172.21.15.19:0/570226529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f2734090 msgr2=0x5596f27a3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1927fc640 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f2734090 0x5596f27a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1927fc640 1 -- 172.21.15.19:0/570226529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27290a0 msgr2=0x5596f2738540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1927fc640 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27290a0 0x5596f2738540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1927fc640 1 -- 172.21.15.19:0/570226529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f27a3dd0 con 0x5596f27299c0 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb1927fc640 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f2738a80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b690 tx=0x7fb18400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784612641fac822b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.351+0000 7fb17affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb17affd640 1 -- 172.21.15.19:0/570226529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184028650 con 0x5596f27299c0 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb17affd640 1 -- 172.21.15.19:0/570226529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb1840287f0 con 0x5596f27299c0 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb1a4808740 1 -- 172.21.15.19:0/570226529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb184015040 con 0x5596f27299c0 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb17affd640 1 -- 172.21.15.19:0/570226529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb184011580 con 0x5596f27299c0 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb1a4808740 10 client.16335 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb1a4808740 1 -- 172.21.15.19:0/570226529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596f27ab160 con 0x5596f27299c0 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb17affd640 1 -- 172.21.15.19:0/570226529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb184015040 con 0x5596f27299c0 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb17affd640 1 -- 172.21.15.19:0/570226529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb184032750 con 0x5596f27299c0 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb1a4808740 10 client.16335 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb1a4808740 10 client.16335 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb1a4808740 20 client.16335 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb1a4808740 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596f27ab640 0x5596f27ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb1a4808740 4 client.16335 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 83.6, "await_for": 42.0}] 2024-04-23T18:31:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.352+0000 7fb1a4808740 1 -- 172.21.15.19:0/570226529 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 83.6, "await_for": 42.0}) -- 0x5596f2695c00 con 0x5596f27ab640 2024-04-23T18:31:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.353+0000 7fb19affd640 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596f27ab640 0x5596f27ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.353+0000 7fb19affd640 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596f27ab640 0x5596f27ada30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219cd50878595216 server_cookie=af48d4666fc09a1e in_seq=0 out_seq=0 2024-04-23T18:31:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.353+0000 7fb17affd640 10 client.16335 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb17affd640 1 -- 172.21.15.19:0/570226529 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5596f2695c00 con 0x5596f27ab640 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb17affd640 10 client.16335 handle_command_reply: tid=0 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 client.16335 shutdown 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 -- 172.21.15.19:0/570226529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 msgr2=0x5596f2738a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f2738a80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b690 tx=0x7fb18400bdd0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 -- 172.21.15.19:0/570226529 shutdown_connections 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596f27ab640 0x5596f27ada30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596f2734090 0x5596f27a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596f27299c0 0x5596f2738a80 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 --2- 172.21.15.19:0/570226529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596f27290a0 0x5596f2738540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb17affd640 0 client.16335 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 -- 172.21.15.19:0/570226529 >> 172.21.15.19:0/570226529 conn(0x5596f27253c0 msgr2=0x5596f2730d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 -- 172.21.15.19:0/570226529 shutdown_connections 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 1 -- 172.21.15.19:0/570226529 wait complete. 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a4808740 20 client.16335 trim_cache size 0 max 16384 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 26, 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stdout: "41396e18": { 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 26, 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:31:09.363 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 83.6, 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:09.365 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:09.365 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:31:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 -- 172.21.15.19:0/651304260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb1880a3d20 msgr2=0x7fb1880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb1880a3d20 0x7fb1880a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb194005fd0 tx=0x7fb194004a40 comp rx=0 tx=0).stop 2024-04-23T18:31:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 -- 172.21.15.19:0/651304260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb19c103230 msgr2=0x7fb19c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb19c103230 0x7fb19c1a06b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb18c005940 tx=0x7fb18c0042f0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 -- 172.21.15.19:0/651304260 shutdown_connections 2024-04-23T18:31:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb1880a3d20 0x7fb1880a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb19c10c570 0x7fb19c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb19c103b50 0x7fb19c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 --2- 172.21.15.19:0/651304260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb19c103230 0x7fb19c1a06b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 -- 172.21.15.19:0/651304260 >> 172.21.15.19:0/651304260 conn(0x7fb19c077fb0 msgr2=0x7fb19c108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 -- 172.21.15.19:0/651304260 shutdown_connections 2024-04-23T18:31:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:09.369+0000 7fb1a1a13640 1 -- 172.21.15.19:0/651304260 wait complete. 2024-04-23T18:31:09.373 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '41396e18', quiesce took 0.5 seconds. Will release after: 23.6 2024-04-23T18:31:09.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:09 smithi019 ceph-mon[29215]: pgmap v153: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:31:09.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:09 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2496852993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:09.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:09 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/879859343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:09.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:09 smithi044 ceph-mon[32458]: pgmap v153: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:31:09.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:09 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2496852993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:09.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:09 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/879859343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:09.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:09 smithi155 ceph-mon[33468]: pgmap v153: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:31:09.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:09 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2496852993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:09.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:09 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/879859343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:11.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:11 smithi019 ceph-mon[29215]: pgmap v154: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 32 KiB/s wr, 30 op/s 2024-04-23T18:31:11.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:11 smithi044 ceph-mon[32458]: pgmap v154: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 32 KiB/s wr, 30 op/s 2024-04-23T18:31:11.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:11 smithi155 ceph-mon[33468]: pgmap v154: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 32 KiB/s wr, 30 op/s 2024-04-23T18:31:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.315+0000 7f42a8ccb640 1 -- 172.21.15.19:0/3889938816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a4107280 msgr2=0x7f42a410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.315+0000 7f42a8ccb640 1 --2- 172.21.15.19:0/3889938816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a4107280 0x7f42a410ee50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f429c009ca0 tx=0x7f429c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.315+0000 7f42a8ccb640 1 -- 172.21.15.19:0/3889938816 shutdown_connections 2024-04-23T18:31:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.315+0000 7f42a8ccb640 1 --2- 172.21.15.19:0/3889938816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42a4101070 0x7f42a4106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.315+0000 7f42a8ccb640 1 --2- 172.21.15.19:0/3889938816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a4107280 0x7f42a410ee50 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.315+0000 7f42a8ccb640 1 --2- 172.21.15.19:0/3889938816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42a41006c0 0x7f42a4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.315+0000 7f42a8ccb640 1 -- 172.21.15.19:0/3889938816 >> 172.21.15.19:0/3889938816 conn(0x7f42a4077390 msgr2=0x7f42a40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.315+0000 7f42a8ccb640 1 -- 172.21.15.19:0/3889938816 shutdown_connections 2024-04-23T18:31:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.315+0000 7f42a8ccb640 1 -- 172.21.15.19:0/3889938816 wait complete. 2024-04-23T18:31:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.315+0000 7f42a8ccb640 1 Processor -- start 2024-04-23T18:31:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f42a8ccb640 1 -- start start 2024-04-23T18:31:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f42a8ccb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42a41006c0 0x7f42a4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f42a8ccb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a4107280 0x7f42a4198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f42a8ccb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42a419cf10 0x7f42a419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f42a8ccb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42a41121f0 con 0x7f42a4107280 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f42a8ccb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42a4112070 con 0x7f42a41006c0 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f42a8ccb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42a4112370 con 0x7f42a419cf10 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f429bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a4107280 0x7f42a4198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f429bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a4107280 0x7f42a4198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55498/0 (socket says 172.21.15.19:55498) 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f429bfff640 1 -- 172.21.15.19:0/677278133 learned_addr learned my addr 172.21.15.19:0/677278133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f42a259c640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42a41006c0 0x7f42a4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f42a2d9d640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42a419cf10 0x7f42a419d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f429bfff640 1 -- 172.21.15.19:0/677278133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42a419cf10 msgr2=0x7f42a419d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f429bfff640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42a419cf10 0x7f42a419d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f429bfff640 1 -- 172.21.15.19:0/677278133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42a41006c0 msgr2=0x7f42a4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f429bfff640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42a41006c0 0x7f42a4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f429bfff640 1 -- 172.21.15.19:0/677278133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a419da00 con 0x7f42a4107280 2024-04-23T18:31:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f42a259c640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42a41006c0 0x7f42a4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.316+0000 7f429bfff640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a4107280 0x7f42a4198b90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f428c00e9c0 tx=0x7f428c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.317+0000 7f429b7fe640 1 -- 172.21.15.19:0/677278133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c00cd80 con 0x7f42a4107280 2024-04-23T18:31:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.317+0000 7f429b7fe640 1 -- 172.21.15.19:0/677278133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c0054d0 con 0x7f42a4107280 2024-04-23T18:31:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.317+0000 7f42a8ccb640 1 -- 172.21.15.19:0/677278133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a419dcf0 con 0x7f42a4107280 2024-04-23T18:31:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.317+0000 7f429b7fe640 1 -- 172.21.15.19:0/677278133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c010430 con 0x7f42a4107280 2024-04-23T18:31:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.317+0000 7f42a8ccb640 1 -- 172.21.15.19:0/677278133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a41a54e0 con 0x7f42a4107280 2024-04-23T18:31:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.319+0000 7f429b7fe640 1 -- 172.21.15.19:0/677278133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f428c004fd0 con 0x7f42a4107280 2024-04-23T18:31:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.319+0000 7f42ab9da740 1 Processor -- start 2024-04-23T18:31:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f42ab9da740 1 -- start start 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9620 0x56160b5d9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 0x56160b5e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5e4800 0x56160b5e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f42ab9da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56160b53e970 con 0x56160b5e4800 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f42ab9da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56160b4db4a0 con 0x56160b5d9f40 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f42ab9da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56160b53a930 con 0x56160b5d9620 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f429b7fe640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42900a3dc0 0x7f42900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f429bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5e4800 0x56160b5e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f429bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5e4800 0x56160b5e6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55510/0 (socket says 172.21.15.19:55510) 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f429bfff640 1 -- 172.21.15.19:0/3327956894 learned_addr learned my addr 172.21.15.19:0/3327956894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f42a2d9d640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42900a3dc0 0x7f42900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f42a2d9d640 1 --2- 172.21.15.19:0/3327956894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9620 0x56160b5d9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f429b7fe640 1 -- 172.21.15.19:0/677278133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f428c0d0c80 con 0x7f42a4107280 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f42a259c640 1 --2- 172.21.15.19:0/3327956894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 0x56160b5e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f429bfff640 1 -- 172.21.15.19:0/3327956894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9620 msgr2=0x56160b5d9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f429bfff640 1 --2- 172.21.15.19:0/3327956894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9620 0x56160b5d9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f429bfff640 1 -- 172.21.15.19:0/3327956894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 msgr2=0x56160b5e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f429bfff640 1 --2- 172.21.15.19:0/3327956894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 0x56160b5e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.320+0000 7f429bfff640 1 -- 172.21.15.19:0/3327956894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56160b251210 con 0x56160b5e4800 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.321+0000 7f429bfff640 1 --2- 172.21.15.19:0/3327956894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5e4800 0x56160b5e6bf0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f428c0a5390 tx=0x7f428c0049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db937dc3f2011860 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.321+0000 7f42a2d9d640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42900a3dc0 0x7f42900a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f429c005cc0 tx=0x7f429c005c50 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.321+0000 7f4298ff9640 1 -- 172.21.15.19:0/3327956894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c0046c0 con 0x56160b5e4800 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.321+0000 7f4298ff9640 1 -- 172.21.15.19:0/3327956894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c09d350 con 0x56160b5e4800 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.321+0000 7f4298ff9640 1 -- 172.21.15.19:0/3327956894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c0992c0 con 0x56160b5e4800 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 -- 172.21.15.19:0/3327956894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5e4800 msgr2=0x56160b5e6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 --2- 172.21.15.19:0/3327956894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5e4800 0x56160b5e6bf0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f428c0a5390 tx=0x7f428c0049b0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 -- 172.21.15.19:0/3327956894 shutdown_connections 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 --2- 172.21.15.19:0/3327956894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 0x56160b5e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 --2- 172.21.15.19:0/3327956894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5e4800 0x56160b5e6bf0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 --2- 172.21.15.19:0/3327956894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9620 0x56160b5d9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 -- 172.21.15.19:0/3327956894 >> 172.21.15.19:0/3327956894 conn(0x56160b5d5b30 msgr2=0x56160b5d5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 -- 172.21.15.19:0/3327956894 shutdown_connections 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 -- 172.21.15.19:0/3327956894 wait complete. 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 -- start start 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 Processor -- start 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5e4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9f40 0x56160b5e1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5e4800 0x56160b5e19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56160b5e7370 con 0x56160b5d9620 2024-04-23T18:31:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56160b4d90b0 con 0x56160b5e4800 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.322+0000 7f42ab9da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56160b5e74f0 con 0x56160b5d9f40 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42ab9da740 10 client.0 _ll_register_callbacks cb 0x56160b4c6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5e4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5e4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55522/0 (socket says 172.21.15.19:55522) 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42a259c640 1 -- 172.21.15.19:0/256905093 learned_addr learned my addr 172.21.15.19:0/256905093 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42a2d9d640 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5e4800 0x56160b5e19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f429bfff640 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9f40 0x56160b5e1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42a259c640 1 -- 172.21.15.19:0/256905093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9f40 msgr2=0x56160b5e1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42a259c640 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9f40 0x56160b5e1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42a259c640 1 -- 172.21.15.19:0/256905093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5e4800 msgr2=0x56160b5e19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42a259c640 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5e4800 0x56160b5e19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42a259c640 1 -- 172.21.15.19:0/256905093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56160b31d620 con 0x56160b5d9620 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f429bfff640 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9f40 0x56160b5e1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42a259c640 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5e4170 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f429400bf80 tx=0x7f4294011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec03c3162617b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f4289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f4289ffb640 1 -- 172.21.15.19:0/256905093 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294005220 con 0x56160b5d9620 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f4289ffb640 1 -- 172.21.15.19:0/256905093 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f42940053c0 con 0x56160b5d9620 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f42ab9da740 1 -- 172.21.15.19:0/256905093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f428c09d350 con 0x56160b5d9620 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.323+0000 7f4289ffb640 1 -- 172.21.15.19:0/256905093 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f42940055c0 con 0x56160b5d9620 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f42ab9da740 10 client.16347 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f42ab9da740 1 -- 172.21.15.19:0/256905093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56160b5f3300 con 0x56160b5d9620 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f4289ffb640 1 -- 172.21.15.19:0/256905093 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c09d350 con 0x56160b5d9620 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f4289ffb640 1 -- 172.21.15.19:0/256905093 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4294012920 con 0x56160b5d9620 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f42ab9da740 10 client.16347 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f42ab9da740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f42ab9da740 20 client.16347 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f42ab9da740 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56160b5f4770 0x56160b5f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f42ab9da740 4 client.16347 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f42ab9da740 1 -- 172.21.15.19:0/256905093 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56160b54c390 con 0x56160b5f4770 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.324+0000 7f429bfff640 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56160b5f4770 0x56160b5f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.325+0000 7f429bfff640 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56160b5f4770 0x56160b5f6b80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c453324952e1cf1 server_cookie=500742675915811b in_seq=0 out_seq=0 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.325+0000 7f4289ffb640 10 client.16347 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.325+0000 7f4289ffb640 1 -- 172.21.15.19:0/256905093 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56160b54c390 con 0x56160b5f4770 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.325+0000 7f4289ffb640 10 client.16347 handle_command_reply: tid=0 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 client.16347 shutdown 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 -- 172.21.15.19:0/256905093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 msgr2=0x56160b5e4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5e4170 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f429400bf80 tx=0x7f4294011870 comp rx=0 tx=0).stop 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 -- 172.21.15.19:0/256905093 shutdown_connections 2024-04-23T18:31:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56160b5f4770 0x56160b5f6b80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5e4800 0x56160b5e19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9f40 0x56160b5e1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 --2- 172.21.15.19:0/256905093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5e4170 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f4289ffb640 0 client.16347 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 -- 172.21.15.19:0/256905093 >> 172.21.15.19:0/256905093 conn(0x56160b5d5b30 msgr2=0x56160b5db860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 -- 172.21.15.19:0/256905093 shutdown_connections 2024-04-23T18:31:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 1 -- 172.21.15.19:0/256905093 wait complete. 2024-04-23T18:31:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.326+0000 7f42ab9da740 20 client.16347 trim_cache size 0 max 16384 2024-04-23T18:31:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42ab9da740 1 Processor -- start 2024-04-23T18:31:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42ab9da740 1 -- start start 2024-04-23T18:31:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9f40 0x56160b5e1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5e4800 0x56160b5e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42ab9da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42a41adaa0 con 0x56160b5d9620 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42ab9da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42a41af5e0 con 0x56160b5e4800 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42ab9da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42a41ae170 con 0x56160b5d9f40 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5f6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55536/0 (socket says 172.21.15.19:55536) 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42a259c640 1 -- 172.21.15.19:0/4036885454 learned_addr learned my addr 172.21.15.19:0/4036885454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42a259c640 1 -- 172.21.15.19:0/4036885454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9f40 msgr2=0x56160b5e1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42a259c640 1 --2- 172.21.15.19:0/4036885454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9f40 0x56160b5e1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42a259c640 1 -- 172.21.15.19:0/4036885454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5e4800 msgr2=0x56160b5e3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42a259c640 1 --2- 172.21.15.19:0/4036885454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5e4800 0x56160b5e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.341+0000 7f42a259c640 1 -- 172.21.15.19:0/4036885454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56160b5d9d70 con 0x56160b5d9620 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f42a259c640 1 --2- 172.21.15.19:0/4036885454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5f6b60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4294006010 tx=0x7f4294000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f901aa07d52050 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f428bfff640 1 -- 172.21.15.19:0/4036885454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294033070 con 0x56160b5d9620 2024-04-23T18:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f428bfff640 1 -- 172.21.15.19:0/4036885454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f429400b670 con 0x56160b5d9620 2024-04-23T18:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f428bfff640 1 -- 172.21.15.19:0/4036885454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429402a040 con 0x56160b5d9620 2024-04-23T18:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f42ab9da740 1 -- 172.21.15.19:0/4036885454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 msgr2=0x56160b5f6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f42ab9da740 1 --2- 172.21.15.19:0/4036885454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5f6b60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4294006010 tx=0x7f4294000bc0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f42ab9da740 1 -- 172.21.15.19:0/4036885454 shutdown_connections 2024-04-23T18:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f42ab9da740 1 --2- 172.21.15.19:0/4036885454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5e4800 0x56160b5e3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f42ab9da740 1 --2- 172.21.15.19:0/4036885454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5d9f40 0x56160b5e1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f42ab9da740 1 --2- 172.21.15.19:0/4036885454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5f6b60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.342+0000 7f42ab9da740 1 -- 172.21.15.19:0/4036885454 >> 172.21.15.19:0/4036885454 conn(0x56160b5d5b30 msgr2=0x56160b5e0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 1 -- 172.21.15.19:0/4036885454 shutdown_connections 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 1 -- 172.21.15.19:0/4036885454 wait complete. 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 1 -- start start 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 1 Processor -- start 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5ec9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 0x56160b5e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5e4800 0x56160b5ea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42a41aef10 con 0x56160b5d9620 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42a41ae840 con 0x56160b5d9f40 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42a41aba80 con 0x56160b5e4800 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42ab9da740 10 client.0 _ll_register_callbacks cb 0x7f429400bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5ec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5ec9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55540/0 (socket says 172.21.15.19:55540) 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42a259c640 1 -- 172.21.15.19:0/4178748879 learned_addr learned my addr 172.21.15.19:0/4178748879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.343+0000 7f42a2d9d640 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5e4800 0x56160b5ea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f429bfff640 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 0x56160b5e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f42a259c640 1 -- 172.21.15.19:0/4178748879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5e4800 msgr2=0x56160b5ea360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f42a259c640 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5e4800 0x56160b5ea360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f42a259c640 1 -- 172.21.15.19:0/4178748879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 msgr2=0x56160b5e9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f42a259c640 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 0x56160b5e9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f42a259c640 1 -- 172.21.15.19:0/4178748879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56160b5eaa50 con 0x56160b5d9620 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f429bfff640 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 0x56160b5e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f42a259c640 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5ec9f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4294031040 tx=0x7f4294011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639f43654b87d8a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f4289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f4289ffb640 1 -- 172.21.15.19:0/4178748879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429403c060 con 0x56160b5d9620 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f4289ffb640 1 -- 172.21.15.19:0/4178748879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f429401bb60 con 0x56160b5d9620 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f42ab9da740 1 -- 172.21.15.19:0/4178748879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f429400b670 con 0x56160b5d9620 2024-04-23T18:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f4289ffb640 1 -- 172.21.15.19:0/4178748879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4294033070 con 0x56160b5d9620 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f42ab9da740 10 client.16353 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f42ab9da740 1 -- 172.21.15.19:0/4178748879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56160b65ac20 con 0x56160b5d9620 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.344+0000 7f4289ffb640 1 -- 172.21.15.19:0/4178748879 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f429400b670 con 0x56160b5d9620 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.345+0000 7f4289ffb640 1 -- 172.21.15.19:0/4178748879 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4294024450 con 0x56160b5d9620 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.345+0000 7f42ab9da740 10 client.16353 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.345+0000 7f42ab9da740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.345+0000 7f42ab9da740 20 client.16353 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.345+0000 7f42ab9da740 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56160b65b280 0x56160b65d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.345+0000 7f42ab9da740 4 client.16353 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.345+0000 7f42ab9da740 1 -- 172.21.15.19:0/4178748879 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56160b546200 con 0x56160b65b280 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.345+0000 7f429bfff640 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56160b65b280 0x56160b65d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.345+0000 7f4289ffb640 10 client.16353 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.345+0000 7f429bfff640 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56160b65b280 0x56160b65d6a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b9fddf3eacab494 server_cookie=bca8f5a9fe8fcfd9 in_seq=0 out_seq=0 2024-04-23T18:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f4289ffb640 1 -- 172.21.15.19:0/4178748879 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56160b546200 con 0x56160b65b280 2024-04-23T18:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f4289ffb640 10 client.16353 handle_command_reply: tid=0 2024-04-23T18:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 client.16353 shutdown 2024-04-23T18:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 -- 172.21.15.19:0/4178748879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 msgr2=0x56160b5ec9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5ec9f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4294031040 tx=0x7f4294011870 comp rx=0 tx=0).stop 2024-04-23T18:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 -- 172.21.15.19:0/4178748879 shutdown_connections 2024-04-23T18:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56160b65b280 0x56160b65d6a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56160b5d9f40 0x56160b5e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56160b5e4800 0x56160b5ea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f4289ffb640 0 client.16353 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 --2- 172.21.15.19:0/4178748879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56160b5d9620 0x56160b5ec9f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 -- 172.21.15.19:0/4178748879 >> 172.21.15.19:0/4178748879 conn(0x56160b5d5b30 msgr2=0x56160b5d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 -- 172.21.15.19:0/4178748879 shutdown_connections 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 1 -- 172.21.15.19:0/4178748879 wait complete. 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.359+0000 7f42ab9da740 20 client.16353 trim_cache size 0 max 16384 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:31:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:31:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 -- 172.21.15.19:0/677278133 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42900a3dc0 msgr2=0x7f42900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42900a3dc0 0x7f42900a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f429c005cc0 tx=0x7f429c005c50 comp rx=0 tx=0).stop 2024-04-23T18:31:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 -- 172.21.15.19:0/677278133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a4107280 msgr2=0x7f42a4198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a4107280 0x7f42a4198b90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f428c00e9c0 tx=0x7f428c00ee80 comp rx=0 tx=0).stop 2024-04-23T18:31:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 -- 172.21.15.19:0/677278133 shutdown_connections 2024-04-23T18:31:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f42900a3dc0 0x7f42900a6280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42a419cf10 0x7f42a419d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a4107280 0x7f42a4198b90 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 --2- 172.21.15.19:0/677278133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42a41006c0 0x7f42a4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 -- 172.21.15.19:0/677278133 >> 172.21.15.19:0/677278133 conn(0x7f42a4077390 msgr2=0x7f42a41059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 -- 172.21.15.19:0/677278133 shutdown_connections 2024-04-23T18:31:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:13.360+0000 7f42a8ccb640 1 -- 172.21.15.19:0/677278133 wait complete. 2024-04-23T18:31:13.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:13 smithi019 ceph-mon[29215]: pgmap v155: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 KiB/s wr, 25 op/s 2024-04-23T18:31:13.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:13 smithi044 ceph-mon[32458]: pgmap v155: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 KiB/s wr, 25 op/s 2024-04-23T18:31:13.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:13 smithi155 ceph-mon[33468]: pgmap v155: 97 pgs: 97 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 KiB/s wr, 25 op/s 2024-04-23T18:31:15.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:15 smithi019 ceph-mon[29215]: pgmap v156: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 36 KiB/s wr, 32 op/s 2024-04-23T18:31:15.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:15 smithi044 ceph-mon[32458]: pgmap v156: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 36 KiB/s wr, 32 op/s 2024-04-23T18:31:15.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:15 smithi155 ceph-mon[33468]: pgmap v156: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 36 KiB/s wr, 32 op/s 2024-04-23T18:31:17.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:17 smithi019 ceph-mon[29215]: pgmap v157: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-23T18:31:17.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:17.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:17 smithi044 ceph-mon[32458]: pgmap v157: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-23T18:31:17.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:17.821 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:17 smithi155 ceph-mon[33468]: pgmap v157: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-23T18:31:17.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:18.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:18 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:18.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:18 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:18.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:18 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:31:18.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:18 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:18.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:18 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:18.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:18 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:31:18.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:18 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:18.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:18 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:18.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:18 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:31:19.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:19 smithi019 ceph-mon[29215]: pgmap v158: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-23T18:31:19.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:19 smithi044 ceph-mon[32458]: pgmap v158: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-23T18:31:19.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:19 smithi155 ceph-mon[33468]: pgmap v158: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-23T18:31:21.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:21 smithi019 ceph-mon[29215]: pgmap v159: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-23T18:31:21.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:21 smithi044 ceph-mon[32458]: pgmap v159: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-23T18:31:21.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:21 smithi155 ceph-mon[33468]: pgmap v159: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-23T18:31:23.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:23 smithi019 ceph-mon[29215]: pgmap v160: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-23T18:31:23.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:23 smithi044 ceph-mon[32458]: pgmap v160: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-23T18:31:23.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:23 smithi155 ceph-mon[33468]: pgmap v160: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-23T18:31:25.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:25 smithi019 ceph-mon[29215]: pgmap v161: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 KiB/s wr, 15 op/s 2024-04-23T18:31:25.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:25 smithi044 ceph-mon[32458]: pgmap v161: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 KiB/s wr, 15 op/s 2024-04-23T18:31:25.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:25 smithi155 ceph-mon[33468]: pgmap v161: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 KiB/s wr, 15 op/s 2024-04-23T18:31:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.461+0000 7f5e3e08a640 1 -- 172.21.15.19:0/3690670204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e381021c0 msgr2=0x7f5e381025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.461+0000 7f5e3e08a640 1 --2- 172.21.15.19:0/3690670204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e381021c0 0x7f5e381025a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009900 tx=0x7f5e2802a410 comp rx=0 tx=0).stop 2024-04-23T18:31:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.461+0000 7f5e3e08a640 1 -- 172.21.15.19:0/3690670204 shutdown_connections 2024-04-23T18:31:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.461+0000 7f5e3e08a640 1 --2- 172.21.15.19:0/3690670204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e38107290 0x7f5e3810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.461+0000 7f5e3e08a640 1 --2- 172.21.15.19:0/3690670204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e38102b70 0x7f5e38106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.461+0000 7f5e3e08a640 1 --2- 172.21.15.19:0/3690670204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e381021c0 0x7f5e381025a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.461+0000 7f5e3e08a640 1 -- 172.21.15.19:0/3690670204 >> 172.21.15.19:0/3690670204 conn(0x7f5e380fda20 msgr2=0x7f5e380ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.461+0000 7f5e3e08a640 1 -- 172.21.15.19:0/3690670204 shutdown_connections 2024-04-23T18:31:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.461+0000 7f5e3e08a640 1 -- 172.21.15.19:0/3690670204 wait complete. 2024-04-23T18:31:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.461+0000 7f5e3e08a640 1 Processor -- start 2024-04-23T18:31:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e3e08a640 1 -- start start 2024-04-23T18:31:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e3e08a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e381021c0 0x7f5e38198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e3e08a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e38107290 0x7f5e38198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e3e08a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e3819cfe0 0x7f5e3819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e3e08a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381121d0 con 0x7f5e381021c0 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e3e08a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e38112050 con 0x7f5e38107290 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e3e08a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e38112350 con 0x7f5e3819cfe0 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e381021c0 0x7f5e38198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e381021c0 0x7f5e38198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57810/0 (socket says 172.21.15.19:57810) 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e377fe640 1 -- 172.21.15.19:0/2213098937 learned_addr learned my addr 172.21.15.19:0/2213098937 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e37fff640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e3819cfe0 0x7f5e3819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e36ffd640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e38107290 0x7f5e38198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e377fe640 1 -- 172.21.15.19:0/2213098937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e3819cfe0 msgr2=0x7f5e3819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e377fe640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e3819cfe0 0x7f5e3819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e377fe640 1 -- 172.21.15.19:0/2213098937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e38107290 msgr2=0x7f5e38198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e377fe640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e38107290 0x7f5e38198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e377fe640 1 -- 172.21.15.19:0/2213098937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3819dad0 con 0x7f5e381021c0 2024-04-23T18:31:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.462+0000 7f5e36ffd640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e38107290 0x7f5e38198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.463+0000 7f5e377fe640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e381021c0 0x7f5e38198770 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5e28005940 tx=0x7f5e280040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.463+0000 7f5e34ff9640 1 -- 172.21.15.19:0/2213098937 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2803f030 con 0x7f5e381021c0 2024-04-23T18:31:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.463+0000 7f5e34ff9640 1 -- 172.21.15.19:0/2213098937 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e280044c0 con 0x7f5e381021c0 2024-04-23T18:31:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.463+0000 7f5e34ff9640 1 -- 172.21.15.19:0/2213098937 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28035070 con 0x7f5e381021c0 2024-04-23T18:31:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.463+0000 7f5e3e08a640 1 -- 172.21.15.19:0/2213098937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3819dd60 con 0x7f5e381021c0 2024-04-23T18:31:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.463+0000 7f5e3e08a640 1 -- 172.21.15.19:0/2213098937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e381a54d0 con 0x7f5e381021c0 2024-04-23T18:31:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.466+0000 7f5e34ff9640 1 -- 172.21.15.19:0/2213098937 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e28042030 con 0x7f5e381021c0 2024-04-23T18:31:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.466+0000 7f5e40da1740 1 Processor -- start 2024-04-23T18:31:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.466+0000 7f5e40da1740 1 -- start start 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.466+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972a620 0x5596e972aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.466+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 0x5596e97352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.466+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 0x5596e9737bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.466+0000 7f5e40da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596e968f970 con 0x5596e972a620 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.466+0000 7f5e40da1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5596e962c4a0 con 0x5596e9735800 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.466+0000 7f5e40da1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5596e968b930 con 0x5596e972af40 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972a620 0x5596e972aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972a620 0x5596e972aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57822/0 (socket says 172.21.15.19:57822) 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e37fff640 1 -- 172.21.15.19:0/330637760 learned_addr learned my addr 172.21.15.19:0/330637760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e36ffd640 1 --2- 172.21.15.19:0/330637760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 0x5596e9737bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e377fe640 1 --2- 172.21.15.19:0/330637760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 0x5596e97352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e37fff640 1 -- 172.21.15.19:0/330637760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 msgr2=0x5596e97352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e34ff9640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5e200a3e40 0x7f5e200a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e37fff640 1 --2- 172.21.15.19:0/330637760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 0x5596e97352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e37fff640 1 -- 172.21.15.19:0/330637760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 msgr2=0x5596e9737bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e37fff640 1 --2- 172.21.15.19:0/330637760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 0x5596e9737bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e37fff640 1 -- 172.21.15.19:0/330637760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e93a2210 con 0x5596e972a620 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e34ff9640 1 -- 172.21.15.19:0/2213098937 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5e280f0b00 con 0x7f5e381021c0 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e37fff640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5e200a3e40 0x7f5e200a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.467+0000 7f5e37fff640 1 --2- 172.21.15.19:0/330637760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972a620 0x5596e972aa00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009f50 tx=0x7f5e300118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aefea0b8e1ff44b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.468+0000 7f5e1dffb640 1 -- 172.21.15.19:0/330637760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000f680 con 0x5596e972a620 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.468+0000 7f5e1dffb640 1 -- 172.21.15.19:0/330637760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e30003cb0 con 0x5596e972a620 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.468+0000 7f5e1dffb640 1 -- 172.21.15.19:0/330637760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3001bac0 con 0x5596e972a620 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 -- 172.21.15.19:0/330637760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972a620 msgr2=0x5596e972aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 --2- 172.21.15.19:0/330637760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972a620 0x5596e972aa00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009f50 tx=0x7f5e300118d0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e37fff640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5e200a3e40 0x7f5e200a6300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5e30012fd0 tx=0x7f5e30021000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 -- 172.21.15.19:0/330637760 shutdown_connections 2024-04-23T18:31:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 --2- 172.21.15.19:0/330637760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 0x5596e9737bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 --2- 172.21.15.19:0/330637760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 0x5596e97352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 --2- 172.21.15.19:0/330637760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972a620 0x5596e972aa00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 -- 172.21.15.19:0/330637760 >> 172.21.15.19:0/330637760 conn(0x5596e9726b30 msgr2=0x5596e9726f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 -- 172.21.15.19:0/330637760 shutdown_connections 2024-04-23T18:31:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 -- 172.21.15.19:0/330637760 wait complete. 2024-04-23T18:31:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 -- start start 2024-04-23T18:31:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.469+0000 7f5e40da1740 1 Processor -- start 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972a620 0x5596e9735190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e972af40 0x5596e9732480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e9732a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e40da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596e9738430 con 0x5596e9735800 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e40da1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5596e962a0b0 con 0x5596e972af40 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e40da1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5596e97385b0 con 0x5596e972a620 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e40da1740 10 client.0 _ll_register_callbacks cb 0x5596e9617d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e9732a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e9732a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57838/0 (socket says 172.21.15.19:57838) 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e37fff640 1 -- 172.21.15.19:0/1424712441 learned_addr learned my addr 172.21.15.19:0/1424712441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e36ffd640 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e972af40 0x5596e9732480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e377fe640 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972a620 0x5596e9735190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e37fff640 1 -- 172.21.15.19:0/1424712441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972a620 msgr2=0x5596e9735190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e37fff640 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972a620 0x5596e9735190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e37fff640 1 -- 172.21.15.19:0/1424712441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e972af40 msgr2=0x5596e9732480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e37fff640 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e972af40 0x5596e9732480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e37fff640 1 -- 172.21.15.19:0/1424712441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e946e620 con 0x5596e9735800 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e37fff640 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e9732a00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5e3001b700 tx=0x7f5e3000f8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0668fc1793ebdd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1424712441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30009be0 con 0x5596e9735800 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1424712441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5e3000fad0 con 0x5596e9735800 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.470+0000 7f5e40da1740 1 -- 172.21.15.19:0/1424712441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596e969d390 con 0x5596e9735800 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1424712441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5e3003e050 con 0x5596e9735800 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e40da1740 10 client.16365 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e40da1740 1 -- 172.21.15.19:0/1424712441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596e9744380 con 0x5596e9735800 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1424712441 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5596e969d390 con 0x5596e9735800 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1424712441 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5e300309a0 con 0x5596e9735800 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e40da1740 10 client.16365 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e40da1740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e40da1740 20 client.16365 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596e9745720 0x5596e9747b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e40da1740 4 client.16365 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e40da1740 1 -- 172.21.15.19:0/1424712441 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596e9697200 con 0x5596e9745720 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.471+0000 7f5e377fe640 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596e9745720 0x5596e9747b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.472+0000 7f5e16ffd640 10 client.16365 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.472+0000 7f5e377fe640 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596e9745720 0x5596e9747b10 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f062d4bd8f723ffd server_cookie=783d2e05103da006 in_seq=0 out_seq=0 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.472+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1424712441 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596e9697200 con 0x5596e9745720 2024-04-23T18:31:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.472+0000 7f5e16ffd640 10 client.16365 handle_command_reply: tid=0 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 client.16365 shutdown 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 -- 172.21.15.19:0/1424712441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 msgr2=0x5596e9732a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e9732a00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5e3001b700 tx=0x7f5e3000f8e0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 -- 172.21.15.19:0/1424712441 shutdown_connections 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596e9745720 0x5596e9747b10 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e972af40 0x5596e9732480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e9732a00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e16ffd640 0 client.16365 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1424712441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972a620 0x5596e9735190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 -- 172.21.15.19:0/1424712441 >> 172.21.15.19:0/1424712441 conn(0x5596e9726b30 msgr2=0x5596e972c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 -- 172.21.15.19:0/1424712441 shutdown_connections 2024-04-23T18:31:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 1 -- 172.21.15.19:0/1424712441 wait complete. 2024-04-23T18:31:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.473+0000 7f5e40da1740 20 client.16365 trim_cache size 0 max 16384 2024-04-23T18:31:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e40da1740 1 Processor -- start 2024-04-23T18:31:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e40da1740 1 -- start start 2024-04-23T18:31:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e972a620 0x5596e97412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 0x5596e9734cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e974d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e40da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381ada90 con 0x5596e9735800 2024-04-23T18:31:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e40da1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381af5d0 con 0x5596e972a620 2024-04-23T18:31:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e40da1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381ae160 con 0x5596e972af40 2024-04-23T18:31:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e974d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e974d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57842/0 (socket says 172.21.15.19:57842) 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e37fff640 1 -- 172.21.15.19:0/990041609 learned_addr learned my addr 172.21.15.19:0/990041609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e36ffd640 1 --2- 172.21.15.19:0/990041609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 0x5596e9734cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e37fff640 1 -- 172.21.15.19:0/990041609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 msgr2=0x5596e9734cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e37fff640 1 --2- 172.21.15.19:0/990041609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 0x5596e9734cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e37fff640 1 -- 172.21.15.19:0/990041609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e972a620 msgr2=0x5596e97412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e37fff640 1 --2- 172.21.15.19:0/990041609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e972a620 0x5596e97412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e37fff640 1 -- 172.21.15.19:0/990041609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e9738010 con 0x5596e9735800 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.488+0000 7f5e36ffd640 1 --2- 172.21.15.19:0/990041609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 0x5596e9734cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e37fff640 1 --2- 172.21.15.19:0/990041609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e974d310 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5e30011910 tx=0x7f5e30005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b6ba21de123f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e1d7fa640 1 -- 172.21.15.19:0/990041609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3001b430 con 0x5596e9735800 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e1d7fa640 1 -- 172.21.15.19:0/990041609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e30035070 con 0x5596e9735800 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e1d7fa640 1 -- 172.21.15.19:0/990041609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300258e0 con 0x5596e9735800 2024-04-23T18:31:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e40da1740 1 -- 172.21.15.19:0/990041609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 msgr2=0x5596e974d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e40da1740 1 --2- 172.21.15.19:0/990041609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e974d310 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5e30011910 tx=0x7f5e30005bd0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e40da1740 1 -- 172.21.15.19:0/990041609 shutdown_connections 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e40da1740 1 --2- 172.21.15.19:0/990041609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e9735800 0x5596e974d310 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e40da1740 1 --2- 172.21.15.19:0/990041609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972af40 0x5596e9734cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e40da1740 1 --2- 172.21.15.19:0/990041609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e972a620 0x5596e97412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e40da1740 1 -- 172.21.15.19:0/990041609 >> 172.21.15.19:0/990041609 conn(0x5596e9726b30 msgr2=0x5596e9725cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e40da1740 1 -- 172.21.15.19:0/990041609 shutdown_connections 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.489+0000 7f5e40da1740 1 -- 172.21.15.19:0/990041609 wait complete. 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e40da1740 1 -- start start 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e40da1740 1 Processor -- start 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972a620 0x5596e973da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972af40 0x5596e9738e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e40da1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 0x5596e9739340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e40da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381aef00 con 0x5596e972af40 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e40da1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381ae830 con 0x5596e9735800 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e40da1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381aba90 con 0x5596e972a620 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e40da1740 10 client.0 _ll_register_callbacks cb 0x7f5e30009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e36ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972af40 0x5596e9738e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e36ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972af40 0x5596e9738e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57858/0 (socket says 172.21.15.19:57858) 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e36ffd640 1 -- 172.21.15.19:0/1381423763 learned_addr learned my addr 172.21.15.19:0/1381423763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.490+0000 7f5e377fe640 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972a620 0x5596e973da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e37fff640 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 0x5596e9739340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e36ffd640 1 -- 172.21.15.19:0/1381423763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972a620 msgr2=0x5596e973da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e36ffd640 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972a620 0x5596e973da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e36ffd640 1 -- 172.21.15.19:0/1381423763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 msgr2=0x5596e9739340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e36ffd640 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 0x5596e9739340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e36ffd640 1 -- 172.21.15.19:0/1381423763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e9739a30 con 0x5596e972af40 2024-04-23T18:31:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e37fff640 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 0x5596e9739340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e36ffd640 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972af40 0x5596e9738e00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5e381087a0 tx=0x7f5e2c012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9820f4acd6bda899 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1381423763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c011d80 con 0x5596e972af40 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e40da1740 1 -- 172.21.15.19:0/1381423763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e30035070 con 0x5596e972af40 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1381423763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5e2c004030 con 0x5596e972af40 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1381423763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5e2c004230 con 0x5596e972af40 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e40da1740 10 client.16371 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e40da1740 1 -- 172.21.15.19:0/1381423763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596e97aba50 con 0x5596e972af40 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.491+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1381423763 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e30035070 con 0x5596e972af40 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.492+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1381423763 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5e2c025030 con 0x5596e972af40 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.492+0000 7f5e40da1740 10 client.16371 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.492+0000 7f5e40da1740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.492+0000 7f5e40da1740 20 client.16371 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.492+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596e97ac280 0x5596e97ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.492+0000 7f5e40da1740 4 client.16371 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.492+0000 7f5e40da1740 1 -- 172.21.15.19:0/1381423763 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596e96135f0 con 0x5596e97ac280 2024-04-23T18:31:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.492+0000 7f5e377fe640 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596e97ac280 0x5596e97ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.492+0000 7f5e16ffd640 10 client.16371 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.492+0000 7f5e377fe640 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596e97ac280 0x5596e97ae6a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3297efb85fd32c2 server_cookie=58fffab4b74a7936 in_seq=0 out_seq=0 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.503+0000 7f5e16ffd640 1 -- 172.21.15.19:0/1381423763 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596e96135f0 con 0x5596e97ac280 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e16ffd640 10 client.16371 handle_command_reply: tid=0 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 client.16371 shutdown 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 -- 172.21.15.19:0/1381423763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972af40 msgr2=0x5596e9738e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972af40 0x5596e9738e00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5e381087a0 tx=0x7f5e2c012ca0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 -- 172.21.15.19:0/1381423763 shutdown_connections 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5596e97ac280 0x5596e97ae6a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596e9735800 0x5596e9739340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e972af40 0x5596e9738e00 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 --2- 172.21.15.19:0/1381423763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e972a620 0x5596e973da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e16ffd640 0 client.16371 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:26.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 -- 172.21.15.19:0/1381423763 >> 172.21.15.19:0/1381423763 conn(0x5596e9726b30 msgr2=0x5596e972c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 -- 172.21.15.19:0/1381423763 shutdown_connections 2024-04-23T18:31:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 1 -- 172.21.15.19:0/1381423763 wait complete. 2024-04-23T18:31:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.504+0000 7f5e40da1740 20 client.16371 trim_cache size 0 max 16384 2024-04-23T18:31:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:31:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:31:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:31:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:31:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 -- 172.21.15.19:0/2213098937 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5e200a3e40 msgr2=0x7f5e200a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5e200a3e40 0x7f5e200a6300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5e30012fd0 tx=0x7f5e30021000 comp rx=0 tx=0).stop 2024-04-23T18:31:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 -- 172.21.15.19:0/2213098937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e381021c0 msgr2=0x7f5e38198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e381021c0 0x7f5e38198770 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5e28005940 tx=0x7f5e280040c0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 -- 172.21.15.19:0/2213098937 shutdown_connections 2024-04-23T18:31:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5e200a3e40 0x7f5e200a6300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e38107290 0x7f5e38198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e3819cfe0 0x7f5e3819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 --2- 172.21.15.19:0/2213098937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e381021c0 0x7f5e38198770 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 -- 172.21.15.19:0/2213098937 >> 172.21.15.19:0/2213098937 conn(0x7f5e380fda20 msgr2=0x7f5e380ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 -- 172.21.15.19:0/2213098937 shutdown_connections 2024-04-23T18:31:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:26.505+0000 7f5e3e08a640 1 -- 172.21.15.19:0/2213098937 wait complete. 2024-04-23T18:31:27.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:27 smithi019 ceph-mon[29215]: pgmap v162: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:31:27.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:27 smithi044 ceph-mon[32458]: pgmap v162: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:31:27.821 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:27 smithi155 ceph-mon[33468]: pgmap v162: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:31:29.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:29 smithi019 ceph-mon[29215]: pgmap v163: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:31:29.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:29 smithi044 ceph-mon[32458]: pgmap v163: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:31:29.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:29 smithi155 ceph-mon[33468]: pgmap v163: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:31:31.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:31 smithi019 ceph-mon[29215]: pgmap v164: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-04-23T18:31:31.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:31 smithi044 ceph-mon[32458]: pgmap v164: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-04-23T18:31:31.822 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:31 smithi155 ceph-mon[33468]: pgmap v164: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-04-23T18:31:32.974 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:31:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.097+0000 7fbfdcb68640 1 -- 172.21.15.19:0/3552761224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd810c520 msgr2=0x7fbfd810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.097+0000 7fbfdcb68640 1 --2- 172.21.15.19:0/3552761224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd810c520 0x7fbfd810e9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009900 tx=0x7fbfd002a510 comp rx=0 tx=0).stop 2024-04-23T18:31:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.097+0000 7fbfdcb68640 1 -- 172.21.15.19:0/3552761224 shutdown_connections 2024-04-23T18:31:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.097+0000 7fbfdcb68640 1 --2- 172.21.15.19:0/3552761224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd810c520 0x7fbfd810e9c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.097+0000 7fbfdcb68640 1 --2- 172.21.15.19:0/3552761224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfd8103b00 0x7fbfd810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.097+0000 7fbfdcb68640 1 --2- 172.21.15.19:0/3552761224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd81031e0 0x7fbfd81035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.097+0000 7fbfdcb68640 1 -- 172.21.15.19:0/3552761224 >> 172.21.15.19:0/3552761224 conn(0x7fbfd8077f50 msgr2=0x7fbfd8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.097+0000 7fbfdcb68640 1 -- 172.21.15.19:0/3552761224 shutdown_connections 2024-04-23T18:31:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.097+0000 7fbfdcb68640 1 -- 172.21.15.19:0/3552761224 wait complete. 2024-04-23T18:31:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfdcb68640 1 Processor -- start 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfdcb68640 1 -- start start 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfdcb68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd81031e0 0x7fbfd8100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfdcb68640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfd8103b00 0x7fbfd8101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfdcb68640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd810c520 0x7fbfd81018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfdcb68640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd81121b0 con 0x7fbfd81031e0 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfdcb68640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd8112030 con 0x7fbfd810c520 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfdcb68640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd8112330 con 0x7fbfd8103b00 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfd659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd81031e0 0x7fbfd8100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfd659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd81031e0 0x7fbfd8100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39806/0 (socket says 172.21.15.19:39806) 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfd659c640 1 -- 172.21.15.19:0/4074324528 learned_addr learned my addr 172.21.15.19:0/4074324528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.098+0000 7fbfd5d9b640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfd8103b00 0x7fbfd8101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfd6d9d640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd810c520 0x7fbfd81018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfd659c640 1 -- 172.21.15.19:0/4074324528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfd8103b00 msgr2=0x7fbfd8101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfd659c640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfd8103b00 0x7fbfd8101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfd659c640 1 -- 172.21.15.19:0/4074324528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd810c520 msgr2=0x7fbfd81018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfd659c640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd810c520 0x7fbfd81018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfd659c640 1 -- 172.21.15.19:0/4074324528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd81a6bd0 con 0x7fbfd81031e0 2024-04-23T18:31:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfd659c640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd81031e0 0x7fbfd8100e30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800db80 tx=0x7fbfc800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfc77fe640 1 -- 172.21.15.19:0/4074324528 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8017070 con 0x7fbfd81031e0 2024-04-23T18:31:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfc77fe640 1 -- 172.21.15.19:0/4074324528 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc8003b70 con 0x7fbfd81031e0 2024-04-23T18:31:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfdcb68640 1 -- 172.21.15.19:0/4074324528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd81a6ec0 con 0x7fbfd81031e0 2024-04-23T18:31:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfc77fe640 1 -- 172.21.15.19:0/4074324528 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80106c0 con 0x7fbfd81031e0 2024-04-23T18:31:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.099+0000 7fbfdcb68640 1 -- 172.21.15.19:0/4074324528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd81a7380 con 0x7fbfd81031e0 2024-04-23T18:31:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.100+0000 7fbfdcb68640 1 -- 172.21.15.19:0/4074324528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd8068d70 con 0x7fbfd81031e0 2024-04-23T18:31:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.101+0000 7fbfc77fe640 1 -- 172.21.15.19:0/4074324528 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbfc8004020 con 0x7fbfd81031e0 2024-04-23T18:31:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.102+0000 7fbfc77fe640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfc00a3d20 0x7fbfc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.102+0000 7fbfd5d9b640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfc00a3d20 0x7fbfc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.102+0000 7fbfc77fe640 1 -- 172.21.15.19:0/4074324528 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbfc80d0810 con 0x7fbfd81031e0 2024-04-23T18:31:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.103+0000 7fbfd5d9b640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfc00a3d20 0x7fbfc00a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbfd81028c0 tx=0x7fbfcc005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.107+0000 7fbfc77fe640 1 -- 172.21.15.19:0/4074324528 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfc809cee0 con 0x7fbfd81031e0 2024-04-23T18:31:33.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.334+0000 7fbfdcb68640 1 -- 172.21.15.19:0/4074324528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfd8107110 con 0x7fbfd81031e0 2024-04-23T18:31:33.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.335+0000 7fbfc77fe640 1 -- 172.21.15.19:0/4074324528 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7fbfc809c7e0 con 0x7fbfd81031e0 2024-04-23T18:31:33.327 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:31:33.327 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:31:33.327 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:31:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.336+0000 7fbfdcb68640 1 -- 172.21.15.19:0/4074324528 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfc00a3d20 msgr2=0x7fbfc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.336+0000 7fbfdcb68640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfc00a3d20 0x7fbfc00a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbfd81028c0 tx=0x7fbfcc005f90 comp rx=0 tx=0).stop 2024-04-23T18:31:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.336+0000 7fbfdcb68640 1 -- 172.21.15.19:0/4074324528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd81031e0 msgr2=0x7fbfd8100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.336+0000 7fbfdcb68640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd81031e0 0x7fbfd8100e30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800db80 tx=0x7fbfc800b630 comp rx=0 tx=0).stop 2024-04-23T18:31:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.337+0000 7fbfdcb68640 1 -- 172.21.15.19:0/4074324528 shutdown_connections 2024-04-23T18:31:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.337+0000 7fbfdcb68640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbfc00a3d20 0x7fbfc00a61e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.337+0000 7fbfdcb68640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd810c520 0x7fbfd81018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.337+0000 7fbfdcb68640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfd8103b00 0x7fbfd8101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.337+0000 7fbfdcb68640 1 --2- 172.21.15.19:0/4074324528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd81031e0 0x7fbfd8100e30 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.337+0000 7fbfdcb68640 1 -- 172.21.15.19:0/4074324528 >> 172.21.15.19:0/4074324528 conn(0x7fbfd8077f50 msgr2=0x7fbfd80fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.337+0000 7fbfdcb68640 1 -- 172.21.15.19:0/4074324528 shutdown_connections 2024-04-23T18:31:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.337+0000 7fbfdcb68640 1 -- 172.21.15.19:0/4074324528 wait complete. 2024-04-23T18:31:33.340 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id 41396e18 --release --await' 2024-04-23T18:31:33.340 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id 41396e18 --release --await 2024-04-23T18:31:33.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.457+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/586655495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8f41010a0 msgr2=0x7fc8f4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.457+0000 7fc8fb1f7640 1 --2- 172.21.15.19:0/586655495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8f41010a0 0x7fc8f4101480 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009900 tx=0x7fc8f002a410 comp rx=0 tx=0).stop 2024-04-23T18:31:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.457+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/586655495 shutdown_connections 2024-04-23T18:31:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.457+0000 7fc8fb1f7640 1 --2- 172.21.15.19:0/586655495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8f4106170 0x7fc8f410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.457+0000 7fc8fb1f7640 1 --2- 172.21.15.19:0/586655495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4101a50 0x7fc8f4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.457+0000 7fc8fb1f7640 1 --2- 172.21.15.19:0/586655495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8f41010a0 0x7fc8f4101480 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.457+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/586655495 >> 172.21.15.19:0/586655495 conn(0x7fc8f4077eb0 msgr2=0x7fc8f40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.457+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/586655495 shutdown_connections 2024-04-23T18:31:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.457+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/586655495 wait complete. 2024-04-23T18:31:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.457+0000 7fc8fb1f7640 1 Processor -- start 2024-04-23T18:31:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8fb1f7640 1 -- start start 2024-04-23T18:31:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8fb1f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8f41010a0 0x7fc8f41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8fb1f7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4106170 0x7fc8f4198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8fb1f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8f419cfa0 0x7fc8f419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8fb1f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41110b0 con 0x7fc8f41010a0 2024-04-23T18:31:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8fb1f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4110f30 con 0x7fc8f419cfa0 2024-04-23T18:31:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8fb1f7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4111230 con 0x7fc8f4106170 2024-04-23T18:31:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f8f93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8f41010a0 0x7fc8f41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f8f93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8f41010a0 0x7fc8f41986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39818/0 (socket says 172.21.15.19:39818) 2024-04-23T18:31:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f8f93640 1 -- 172.21.15.19:0/1061055603 learned_addr learned my addr 172.21.15.19:0/1061055603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8ebfff640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4106170 0x7fc8f4198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f9794640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8f419cfa0 0x7fc8f419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f8f93640 1 -- 172.21.15.19:0/1061055603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4106170 msgr2=0x7fc8f4198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4106170 0x7fc8f4198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f8f93640 1 -- 172.21.15.19:0/1061055603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8f419cfa0 msgr2=0x7fc8f419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8f419cfa0 0x7fc8f419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f8f93640 1 -- 172.21.15.19:0/1061055603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f419da90 con 0x7fc8f41010a0 2024-04-23T18:31:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f9794640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8f419cfa0 0x7fc8f419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8f41010a0 0x7fc8f41986e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0005940 tx=0x7fc8f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.458+0000 7fc8e9ffb640 1 -- 172.21.15.19:0/1061055603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f003f030 con 0x7fc8f41010a0 2024-04-23T18:31:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.459+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/1061055603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f419dd20 con 0x7fc8f41010a0 2024-04-23T18:31:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.459+0000 7fc8e9ffb640 1 -- 172.21.15.19:0/1061055603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f0003aa0 con 0x7fc8f41010a0 2024-04-23T18:31:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.459+0000 7fc8e9ffb640 1 -- 172.21.15.19:0/1061055603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0035070 con 0x7fc8f41010a0 2024-04-23T18:31:33.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.459+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/1061055603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41a54e0 con 0x7fc8f41010a0 2024-04-23T18:31:33.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.461+0000 7fc8e9ffb640 1 -- 172.21.15.19:0/1061055603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc8f0042030 con 0x7fc8f41010a0 2024-04-23T18:31:33.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.461+0000 7fc8fdfe2740 1 Processor -- start 2024-04-23T18:31:33.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.461+0000 7fc8fdfe2740 1 -- start start 2024-04-23T18:31:33.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.461+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5f490 0x561836d5f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.461+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5fdb0 0x561836d69f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.461+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d6a480 0x561836d6c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.461+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561836cc46d0 con 0x561836d5fdb0 2024-04-23T18:31:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.461+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561836c611a0 con 0x561836d6a480 2024-04-23T18:31:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.461+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561836cc0690 con 0x561836d5f490 2024-04-23T18:31:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f8f93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5fdb0 0x561836d69f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5f490 0x561836d5f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8e9ffb640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8dc0a3d20 0x7fc8dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5f490 0x561836d5f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:34020/0 (socket says 172.21.15.19:34020) 2024-04-23T18:31:33.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 -- 172.21.15.19:0/3081114498 learned_addr learned my addr 172.21.15.19:0/3081114498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:33.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8ebfff640 1 --2- 172.21.15.19:0/3081114498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d6a480 0x561836d6c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8dc0a3d20 0x7fc8dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8e9ffb640 1 -- 172.21.15.19:0/1061055603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc8f00f0f20 con 0x7fc8f41010a0 2024-04-23T18:31:33.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 -- 172.21.15.19:0/3081114498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d6a480 msgr2=0x561836d6c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:33.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 --2- 172.21.15.19:0/3081114498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d6a480 0x561836d6c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 -- 172.21.15.19:0/3081114498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5fdb0 msgr2=0x561836d69f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:33.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 --2- 172.21.15.19:0/3081114498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5fdb0 0x561836d69f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 -- 172.21.15.19:0/3081114498 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561836c6b5b0 con 0x561836d5f490 2024-04-23T18:31:33.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 --2- 172.21.15.19:0/3081114498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5f490 0x561836d5f870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec012940 tx=0x7fc8ec012e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4f838429493185 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:33.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8f9794640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8dc0a3d20 0x7fc8dc0a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec019000 tx=0x7fc8ec01b000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:33.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8daffd640 1 -- 172.21.15.19:0/3081114498 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec004d20 con 0x561836d5f490 2024-04-23T18:31:33.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8daffd640 1 -- 172.21.15.19:0/3081114498 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ec0054e0 con 0x561836d5f490 2024-04-23T18:31:33.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.462+0000 7fc8daffd640 1 -- 172.21.15.19:0/3081114498 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec0040f0 con 0x561836d5f490 2024-04-23T18:31:33.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.463+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/3081114498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5f490 msgr2=0x561836d5f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.463+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/3081114498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5f490 0x561836d5f870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec012940 tx=0x7fc8ec012e00 comp rx=0 tx=0).stop 2024-04-23T18:31:33.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.463+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/3081114498 shutdown_connections 2024-04-23T18:31:33.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.463+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/3081114498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d6a480 0x561836d6c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.463+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/3081114498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5fdb0 0x561836d69f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.463+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/3081114498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5f490 0x561836d5f870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.463+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/3081114498 >> 172.21.15.19:0/3081114498 conn(0x561836d5b7b0 msgr2=0x561836d5bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.463+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/3081114498 shutdown_connections 2024-04-23T18:31:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.463+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/3081114498 wait complete. 2024-04-23T18:31:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.463+0000 7fc8fdfe2740 1 -- start start 2024-04-23T18:31:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8fdfe2740 1 Processor -- start 2024-04-23T18:31:33.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 0x561836d69eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 0x561836d671a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561836d6d250 con 0x561836d6a480 2024-04-23T18:31:33.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561836c5eb20 con 0x561836d5f490 2024-04-23T18:31:33.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561836d6d3d0 con 0x561836d5fdb0 2024-04-23T18:31:33.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8fdfe2740 10 client.0 _ll_register_callbacks cb 0x561836c5e530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:33.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8f9794640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8f9794640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d678e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39838/0 (socket says 172.21.15.19:39838) 2024-04-23T18:31:33.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8f9794640 1 -- 172.21.15.19:0/4109884692 learned_addr learned my addr 172.21.15.19:0/4109884692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:33.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8ebfff640 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 0x561836d671a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 0x561836d69eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8f9794640 1 -- 172.21.15.19:0/4109884692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 msgr2=0x561836d671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:33.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8f9794640 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 0x561836d671a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8f9794640 1 -- 172.21.15.19:0/4109884692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 msgr2=0x561836d69eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:33.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8f9794640 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 0x561836d69eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8f9794640 1 -- 172.21.15.19:0/4109884692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618369d6920 con 0x561836d6a480 2024-04-23T18:31:33.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.464+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 0x561836d69eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:33.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8f9794640 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d678e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec010fa0 tx=0x7fc8ec004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6f87fff6d9a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:33.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:33.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8d3fff640 1 -- 172.21.15.19:0/4109884692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec005690 con 0x561836d6a480 2024-04-23T18:31:33.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8d3fff640 1 -- 172.21.15.19:0/4109884692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc8ec005830 con 0x561836d6a480 2024-04-23T18:31:33.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/4109884692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ec0054e0 con 0x561836d6a480 2024-04-23T18:31:33.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8d3fff640 1 -- 172.21.15.19:0/4109884692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8ec005a30 con 0x561836d6a480 2024-04-23T18:31:33.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8fdfe2740 10 client.16389 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:33.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/4109884692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561836aa2d30 con 0x561836d6a480 2024-04-23T18:31:33.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8d3fff640 1 -- 172.21.15.19:0/4109884692 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ec0054e0 con 0x561836d6a480 2024-04-23T18:31:33.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8d3fff640 1 -- 172.21.15.19:0/4109884692 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8ec02e020 con 0x561836d6a480 2024-04-23T18:31:33.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8fdfe2740 10 client.16389 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:33.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8fdfe2740 10 client.16389 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:31:33.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8fdfe2740 20 client.16389 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:33.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561836d7a560 0x561836d7c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8fdfe2740 4 client.16389 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.465+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/4109884692 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561836c82500 con 0x561836d7a560 2024-04-23T18:31:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.466+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561836d7a560 0x561836d7c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.466+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561836d7a560 0x561836d7c950 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9faf3fe4d65cd4f server_cookie=497f2b4dd1ec7ee3 in_seq=0 out_seq=0 2024-04-23T18:31:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.466+0000 7fc8d3fff640 10 client.16389 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8d3fff640 1 -- 172.21.15.19:0/4109884692 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561836c82500 con 0x561836d7a560 2024-04-23T18:31:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8d3fff640 10 client.16389 handle_command_reply: tid=0 2024-04-23T18:31:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 client.16389 shutdown 2024-04-23T18:31:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/4109884692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 msgr2=0x561836d678e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d678e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec010fa0 tx=0x7fc8ec004120 comp rx=0 tx=0).stop 2024-04-23T18:31:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/4109884692 shutdown_connections 2024-04-23T18:31:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561836d7a560 0x561836d7c950 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d678e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 0x561836d671a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8d3fff640 0 client.16389 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:33.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/4109884692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 0x561836d69eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/4109884692 >> 172.21.15.19:0/4109884692 conn(0x561836d5b7b0 msgr2=0x561836d615b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:33.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/4109884692 shutdown_connections 2024-04-23T18:31:33.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/4109884692 wait complete. 2024-04-23T18:31:33.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.467+0000 7fc8fdfe2740 20 client.16389 trim_cache size 0 max 16384 2024-04-23T18:31:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8fdfe2740 1 Processor -- start 2024-04-23T18:31:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8fdfe2740 1 -- start start 2024-04-23T18:31:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 0x561836d699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 0x561836d69f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d820e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41ada40 con 0x561836d6a480 2024-04-23T18:31:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41af580 con 0x561836d5f490 2024-04-23T18:31:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41ae110 con 0x561836d5fdb0 2024-04-23T18:31:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f9794640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d820e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f9794640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d820e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39850/0 (socket says 172.21.15.19:39850) 2024-04-23T18:31:33.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f9794640 1 -- 172.21.15.19:0/3947616806 learned_addr learned my addr 172.21.15.19:0/3947616806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:33.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 0x561836d699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8ebfff640 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 0x561836d69f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f9794640 1 -- 172.21.15.19:0/3947616806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 msgr2=0x561836d69f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:33.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f9794640 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 0x561836d69f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f9794640 1 -- 172.21.15.19:0/3947616806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 msgr2=0x561836d699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:33.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f9794640 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 0x561836d699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f9794640 1 -- 172.21.15.19:0/3947616806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561836d5fbe0 con 0x561836d6a480 2024-04-23T18:31:33.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8ebfff640 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 0x561836d69f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:33.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 0x561836d699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:33.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.485+0000 7fc8f9794640 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d820e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec012e40 tx=0x7fc8ec007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdf49ca7151b81e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:33.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8da7fc640 1 -- 172.21.15.19:0/3947616806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec005240 con 0x561836d6a480 2024-04-23T18:31:33.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8da7fc640 1 -- 172.21.15.19:0/3947616806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ec030440 con 0x561836d6a480 2024-04-23T18:31:33.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8da7fc640 1 -- 172.21.15.19:0/3947616806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec005890 con 0x561836d6a480 2024-04-23T18:31:33.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/3947616806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 msgr2=0x561836d820e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d820e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec012e40 tx=0x7fc8ec007df0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/3947616806 shutdown_connections 2024-04-23T18:31:33.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d6a480 0x561836d820e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d5fdb0 0x561836d69f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/3947616806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5f490 0x561836d699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/3947616806 >> 172.21.15.19:0/3947616806 conn(0x561836d5b7b0 msgr2=0x561836d5a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:33.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/3947616806 shutdown_connections 2024-04-23T18:31:33.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/3947616806 wait complete. 2024-04-23T18:31:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.486+0000 7fc8fdfe2740 1 -- start start 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8fdfe2740 1 Processor -- start 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5f490 0x561836d72630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5fdb0 0x561836d72b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8fdfe2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d6a480 0x561836d6d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41aeeb0 con 0x561836d5f490 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41ae7e0 con 0x561836d5fdb0 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8fdfe2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41aba40 con 0x561836d6a480 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8fdfe2740 10 client.0 _ll_register_callbacks cb 0x7fc8ec012740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8f8f93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5f490 0x561836d72630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8f8f93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5f490 0x561836d72630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39858/0 (socket says 172.21.15.19:39858) 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8f8f93640 1 -- 172.21.15.19:0/2815180823 learned_addr learned my addr 172.21.15.19:0/2815180823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8f9794640 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d6a480 0x561836d6d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8ebfff640 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5fdb0 0x561836d72b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8f8f93640 1 -- 172.21.15.19:0/2815180823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d6a480 msgr2=0x561836d6d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d6a480 0x561836d6d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8f8f93640 1 -- 172.21.15.19:0/2815180823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5fdb0 msgr2=0x561836d72b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5fdb0 0x561836d72b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8f8f93640 1 -- 172.21.15.19:0/2815180823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561836d6e010 con 0x561836d5f490 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.487+0000 7fc8ebfff640 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5fdb0 0x561836d72b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8f8f93640 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5f490 0x561836d72630 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00b9860 tx=0x7fc8f00b8b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbc605751c6023 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8d3fff640 1 -- 172.21.15.19:0/2815180823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00db060 con 0x561836d5f490 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8d3fff640 1 -- 172.21.15.19:0/2815180823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc8f00b9500 con 0x561836d5f490 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/2815180823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ec030440 con 0x561836d5f490 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8d3fff640 1 -- 172.21.15.19:0/2815180823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8f0049040 con 0x561836d5f490 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8fdfe2740 10 client.16395 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/2815180823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561836de15f0 con 0x561836d5f490 2024-04-23T18:31:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8d3fff640 1 -- 172.21.15.19:0/2815180823 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ec030440 con 0x561836d5f490 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8d3fff640 1 -- 172.21.15.19:0/2815180823 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8f00ee8c0 con 0x561836d5f490 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8fdfe2740 10 client.16395 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8fdfe2740 10 client.16395 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8fdfe2740 20 client.16395 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561836de1d10 0x561836de4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8fdfe2740 4 client.16395 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "41396e18", "release": true, "await": true}] 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.488+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/2815180823 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "41396e18", "release": true, "await": true}) -- 0x561836cd20f0 con 0x561836de1d10 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.489+0000 7fc8ebfff640 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561836de1d10 0x561836de4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.489+0000 7fc8ebfff640 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561836de1d10 0x561836de4130 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f770be1b37e2d666 server_cookie=9e96dee50badc9d in_seq=0 out_seq=0 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.489+0000 7fc8d3fff640 10 client.16395 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8d3fff640 1 -- 172.21.15.19:0/2815180823 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x561836cd20f0 con 0x561836de1d10 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8d3fff640 10 client.16395 handle_command_reply: tid=0 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 client.16395 shutdown 2024-04-23T18:31:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/2815180823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5f490 msgr2=0x561836d72630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5f490 0x561836d72630 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00b9860 tx=0x7fc8f00b8b20 comp rx=0 tx=0).stop 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/2815180823 shutdown_connections 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561836de1d10 0x561836de4130 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561836d5fdb0 0x561836d72b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561836d6a480 0x561836d6d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8d3fff640 0 client.16395 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 --2- 172.21.15.19:0/2815180823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561836d5f490 0x561836d72630 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/2815180823 >> 172.21.15.19:0/2815180823 conn(0x561836d5b7b0 msgr2=0x561836d60be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/2815180823 shutdown_connections 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 1 -- 172.21.15.19:0/2815180823 wait complete. 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.493+0000 7fc8fdfe2740 20 client.16395 trim_cache size 0 max 16384 2024-04-23T18:31:33.691 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 28, 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "41396e18": { 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 28, 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 83.6, 2024-04-23T18:31:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:31:33.693 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:31:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/1061055603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8dc0a3d20 msgr2=0x7fc8dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8dc0a3d20 0x7fc8dc0a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec019000 tx=0x7fc8ec01b000 comp rx=0 tx=0).stop 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/1061055603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8f41010a0 msgr2=0x7fc8f41986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8f41010a0 0x7fc8f41986e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0005940 tx=0x7fc8f0004270 comp rx=0 tx=0).stop 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/1061055603 shutdown_connections 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc8dc0a3d20 0x7fc8dc0a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8f419cfa0 0x7fc8f419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4106170 0x7fc8f4198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 --2- 172.21.15.19:0/1061055603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8f41010a0 0x7fc8f41986e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/1061055603 >> 172.21.15.19:0/1061055603 conn(0x7fc8f4077eb0 msgr2=0x7fc8f406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/1061055603 shutdown_connections 2024-04-23T18:31:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:31:33.494+0000 7fc8fb1f7640 1 -- 172.21.15.19:0/1061055603 wait complete. 2024-04-23T18:31:33.696 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '41396e18', seconds elapsed: 24.8 2024-04-23T18:31:34.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:33 smithi019 ceph-mon[29215]: pgmap v165: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-23T18:31:34.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:33 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4074324528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:34.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:33 smithi044 ceph-mon[32458]: pgmap v165: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-23T18:31:34.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:33 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4074324528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:34.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:33 smithi155 ceph-mon[33468]: pgmap v165: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-23T18:31:34.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:33 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4074324528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:36.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:35 smithi019 ceph-mon[29215]: pgmap v166: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-23T18:31:36.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:35 smithi044 ceph-mon[32458]: pgmap v166: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-23T18:31:36.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:35 smithi155 ceph-mon[33468]: pgmap v166: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-23T18:31:38.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:37 smithi019 ceph-mon[29215]: pgmap v167: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-23T18:31:38.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:37 smithi044 ceph-mon[32458]: pgmap v167: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-23T18:31:38.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:37 smithi155 ceph-mon[33468]: pgmap v167: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-23T18:31:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 -- 172.21.15.19:0/720288993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c102b70 msgr2=0x7f1c4c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 --2- 172.21.15.19:0/720288993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c102b70 0x7f1c4c106bb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009900 tx=0x7f1c3c02a410 comp rx=0 tx=0).stop 2024-04-23T18:31:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 -- 172.21.15.19:0/720288993 shutdown_connections 2024-04-23T18:31:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 --2- 172.21.15.19:0/720288993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c4c107290 0x7f1c4c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 --2- 172.21.15.19:0/720288993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c102b70 0x7f1c4c106bb0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 --2- 172.21.15.19:0/720288993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c1021c0 0x7f1c4c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 -- 172.21.15.19:0/720288993 >> 172.21.15.19:0/720288993 conn(0x7f1c4c0fda20 msgr2=0x7f1c4c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 -- 172.21.15.19:0/720288993 shutdown_connections 2024-04-23T18:31:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 -- 172.21.15.19:0/720288993 wait complete. 2024-04-23T18:31:39.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 Processor -- start 2024-04-23T18:31:39.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.605+0000 7f1c523f7640 1 -- start start 2024-04-23T18:31:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c523f7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c1021c0 0x7f1c4c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c523f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c107290 0x7f1c4c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c523f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c4c19cfb0 0x7f1c4c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c523f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1121d0 con 0x7f1c4c107290 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c523f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c112050 con 0x7f1c4c19cfb0 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c523f7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c112350 con 0x7f1c4c1021c0 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c107290 0x7f1c4c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c107290 0x7f1c4c198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39880/0 (socket says 172.21.15.19:39880) 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c4b7fe640 1 -- 172.21.15.19:0/205447841 learned_addr learned my addr 172.21.15.19:0/205447841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c50994640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c4c19cfb0 0x7f1c4c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c1021c0 0x7f1c4c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c4b7fe640 1 -- 172.21.15.19:0/205447841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c1021c0 msgr2=0x7f1c4c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c1021c0 0x7f1c4c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c4b7fe640 1 -- 172.21.15.19:0/205447841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c4c19cfb0 msgr2=0x7f1c4c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c4c19cfb0 0x7f1c4c19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c4b7fe640 1 -- 172.21.15.19:0/205447841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c19daa0 con 0x7f1c4c107290 2024-04-23T18:31:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.606+0000 7f1c50994640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c4c19cfb0 0x7f1c4c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.607+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c107290 0x7f1c4c198c30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c006010 tx=0x7f1c3c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.607+0000 7f1c497fa640 1 -- 172.21.15.19:0/205447841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c03f030 con 0x7f1c4c107290 2024-04-23T18:31:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.607+0000 7f1c497fa640 1 -- 172.21.15.19:0/205447841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c3c003aa0 con 0x7f1c4c107290 2024-04-23T18:31:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.607+0000 7f1c497fa640 1 -- 172.21.15.19:0/205447841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c035070 con 0x7f1c4c107290 2024-04-23T18:31:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.607+0000 7f1c523f7640 1 -- 172.21.15.19:0/205447841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4c19dd30 con 0x7f1c4c107290 2024-04-23T18:31:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.607+0000 7f1c523f7640 1 -- 172.21.15.19:0/205447841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4c1a54f0 con 0x7f1c4c107290 2024-04-23T18:31:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.609+0000 7f1c497fa640 1 -- 172.21.15.19:0/205447841 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c3c042030 con 0x7f1c4c107290 2024-04-23T18:31:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.609+0000 7f1c551e3740 1 Processor -- start 2024-04-23T18:31:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.609+0000 7f1c551e3740 1 -- start start 2024-04-23T18:31:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d838d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d839c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d839cc10 0x5565d839f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c551e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5565d82f2970 con 0x5565d838de20 2024-04-23T18:31:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c551e3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5565d828f4a0 con 0x5565d839cc10 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c551e3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5565d82ee930 con 0x5565d838d500 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c497fa640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1c380a3dc0 0x7f1c380a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c4bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d839c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c4bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d839c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39890/0 (socket says 172.21.15.19:39890) 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c4bfff640 1 -- 172.21.15.19:0/4126764699 learned_addr learned my addr 172.21.15.19:0/4126764699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c50994640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1c380a3dc0 0x7f1c380a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c497fa640 1 -- 172.21.15.19:0/205447841 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f1c3c0f0cd0 con 0x7f1c4c107290 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c50994640 1 --2- 172.21.15.19:0/4126764699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d838d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/4126764699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d839cc10 0x5565d839f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c4bfff640 1 -- 172.21.15.19:0/4126764699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 msgr2=0x5565d838d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/4126764699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d838d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c4bfff640 1 -- 172.21.15.19:0/4126764699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d839cc10 msgr2=0x5565d839f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/4126764699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d839cc10 0x5565d839f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.610+0000 7f1c4bfff640 1 -- 172.21.15.19:0/4126764699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565d8005210 con 0x5565d838de20 2024-04-23T18:31:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c50994640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1c380a3dc0 0x7f1c380a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c44005fd0 tx=0x7f1c440094c0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/4126764699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d839c6d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009c80 tx=0x7f1c34010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cd106c86b1b1af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c427fc640 1 -- 172.21.15.19:0/4126764699 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3401b480 con 0x5565d838de20 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c427fc640 1 -- 172.21.15.19:0/4126764699 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c34012900 con 0x5565d838de20 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c427fc640 1 -- 172.21.15.19:0/4126764699 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c340118e0 con 0x5565d838de20 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c551e3740 1 -- 172.21.15.19:0/4126764699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 msgr2=0x5565d839c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c551e3740 1 --2- 172.21.15.19:0/4126764699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d839c6d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009c80 tx=0x7f1c34010740 comp rx=0 tx=0).stop 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c551e3740 1 -- 172.21.15.19:0/4126764699 shutdown_connections 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c551e3740 1 --2- 172.21.15.19:0/4126764699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d839cc10 0x5565d839f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c551e3740 1 --2- 172.21.15.19:0/4126764699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d839c6d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c551e3740 1 --2- 172.21.15.19:0/4126764699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d838d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.611+0000 7f1c551e3740 1 -- 172.21.15.19:0/4126764699 >> 172.21.15.19:0/4126764699 conn(0x5565d8389b30 msgr2=0x5565d8389f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 1 -- 172.21.15.19:0/4126764699 shutdown_connections 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 1 -- 172.21.15.19:0/4126764699 wait complete. 2024-04-23T18:31:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 1 -- start start 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 1 Processor -- start 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d839c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d8399880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d839cc10 0x5565d8399e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5565d839f840 con 0x5565d838de20 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5565d828d0b0 con 0x5565d839cc10 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5565d839f9c0 con 0x5565d838d500 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.612+0000 7f1c551e3740 10 client.0 _ll_register_callbacks cb 0x5565d827ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d8399880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d8399880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39904/0 (socket says 172.21.15.19:39904) 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c4b7fe640 1 -- 172.21.15.19:0/4174107886 learned_addr learned my addr 172.21.15.19:0/4174107886 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d839c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c50994640 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d839cc10 0x5565d8399e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c4b7fe640 1 -- 172.21.15.19:0/4174107886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 msgr2=0x5565d839c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d839c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c4b7fe640 1 -- 172.21.15.19:0/4174107886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d839cc10 msgr2=0x5565d8399e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d839cc10 0x5565d8399e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c4b7fe640 1 -- 172.21.15.19:0/4174107886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565d80d1620 con 0x5565d838de20 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d8399880 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c032af0 tx=0x7f1c3c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb502672a8fc5b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/4174107886 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c049040 con 0x5565d838de20 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/4174107886 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f1c3c003c50 con 0x5565d838de20 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c551e3740 1 -- 172.21.15.19:0/4174107886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c34012900 con 0x5565d838de20 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/4174107886 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c3c0dbe60 con 0x5565d838de20 2024-04-23T18:31:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c551e3740 10 client.16407 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.613+0000 7f1c551e3740 1 -- 172.21.15.19:0/4174107886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565d83ab810 con 0x5565d838de20 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.614+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/4174107886 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c34012900 con 0x5565d838de20 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.614+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/4174107886 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1c3c0ee6d0 con 0x5565d838de20 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.614+0000 7f1c551e3740 10 client.16407 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.614+0000 7f1c551e3740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.614+0000 7f1c551e3740 20 client.16407 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.614+0000 7f1c551e3740 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5565d83acc70 0x5565d83af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.614+0000 7f1c551e3740 4 client.16407 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.614+0000 7f1c551e3740 1 -- 172.21.15.19:0/4174107886 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565d8300390 con 0x5565d83acc70 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.614+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5565d83acc70 0x5565d83af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.615+0000 7f1c2b7fe640 10 client.16407 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.615+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5565d83acc70 0x5565d83af0a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebe6dbff53531a78 server_cookie=9b538ba8a6a2105 in_seq=0 out_seq=0 2024-04-23T18:31:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.615+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/4174107886 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565d8300390 con 0x5565d83acc70 2024-04-23T18:31:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.615+0000 7f1c2b7fe640 10 client.16407 handle_command_reply: tid=0 2024-04-23T18:31:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 client.16407 shutdown 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 -- 172.21.15.19:0/4174107886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 msgr2=0x5565d8399880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d8399880 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c032af0 tx=0x7f1c3c009260 comp rx=0 tx=0).stop 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 -- 172.21.15.19:0/4174107886 shutdown_connections 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5565d83acc70 0x5565d83af0a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d839cc10 0x5565d8399e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d838de20 0x5565d8399880 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 --2- 172.21.15.19:0/4174107886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d839c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c2b7fe640 0 client.16407 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 -- 172.21.15.19:0/4174107886 >> 172.21.15.19:0/4174107886 conn(0x5565d8389b30 msgr2=0x5565d838f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 -- 172.21.15.19:0/4174107886 shutdown_connections 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 1 -- 172.21.15.19:0/4174107886 wait complete. 2024-04-23T18:31:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.616+0000 7f1c551e3740 20 client.16407 trim_cache size 0 max 16384 2024-04-23T18:31:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.631+0000 7f1c551e3740 1 Processor -- start 2024-04-23T18:31:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.631+0000 7f1c551e3740 1 -- start start 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.631+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d83a8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838de20 0x5565d839c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d83b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c551e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1ab8d0 con 0x5565d839cc10 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c551e3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1ad410 con 0x5565d838de20 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c551e3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1abfa0 con 0x5565d838d500 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c50994640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d83b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c50994640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d83b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39910/0 (socket says 172.21.15.19:39910) 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c50994640 1 -- 172.21.15.19:0/1889332161 learned_addr learned my addr 172.21.15.19:0/1889332161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/1889332161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d83a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/1889332161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838de20 0x5565d839c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c50994640 1 -- 172.21.15.19:0/1889332161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 msgr2=0x5565d83a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c50994640 1 --2- 172.21.15.19:0/1889332161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d83a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c50994640 1 -- 172.21.15.19:0/1889332161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838de20 msgr2=0x5565d839c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c50994640 1 --2- 172.21.15.19:0/1889332161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838de20 0x5565d839c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c50994640 1 -- 172.21.15.19:0/1889332161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565d838dc50 con 0x5565d839cc10 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/1889332161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838de20 0x5565d839c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c50994640 1 --2- 172.21.15.19:0/1889332161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d83b4970 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1c44003be0 tx=0x7f1c44009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8edc3547fb274fd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c41ffb640 1 -- 172.21.15.19:0/1889332161 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44004d20 con 0x5565d839cc10 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c41ffb640 1 -- 172.21.15.19:0/1889332161 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c4400ee90 con 0x5565d839cc10 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.632+0000 7f1c41ffb640 1 -- 172.21.15.19:0/1889332161 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c440054c0 con 0x5565d839cc10 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.633+0000 7f1c551e3740 1 -- 172.21.15.19:0/1889332161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 msgr2=0x5565d83b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.633+0000 7f1c551e3740 1 --2- 172.21.15.19:0/1889332161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d83b4970 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1c44003be0 tx=0x7f1c44009240 comp rx=0 tx=0).stop 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.633+0000 7f1c551e3740 1 -- 172.21.15.19:0/1889332161 shutdown_connections 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.633+0000 7f1c551e3740 1 --2- 172.21.15.19:0/1889332161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838de20 0x5565d839c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.633+0000 7f1c551e3740 1 --2- 172.21.15.19:0/1889332161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d83b4970 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.633+0000 7f1c551e3740 1 --2- 172.21.15.19:0/1889332161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838d500 0x5565d83a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.633+0000 7f1c551e3740 1 -- 172.21.15.19:0/1889332161 >> 172.21.15.19:0/1889332161 conn(0x5565d8389b30 msgr2=0x5565d8394f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.633+0000 7f1c551e3740 1 -- 172.21.15.19:0/1889332161 shutdown_connections 2024-04-23T18:31:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.633+0000 7f1c551e3740 1 -- 172.21.15.19:0/1889332161 wait complete. 2024-04-23T18:31:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.633+0000 7f1c551e3740 1 -- start start 2024-04-23T18:31:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c551e3740 1 Processor -- start 2024-04-23T18:31:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838d500 0x5565d83967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838de20 0x5565d8396ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c551e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d8397220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c551e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1acd40 con 0x5565d839cc10 2024-04-23T18:31:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c551e3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1ac670 con 0x5565d838d500 2024-04-23T18:31:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c551e3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1a98d0 con 0x5565d838de20 2024-04-23T18:31:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c551e3740 10 client.0 _ll_register_callbacks cb 0x7f1c3c0bc7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c50994640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d8397220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c50994640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d8397220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39916/0 (socket says 172.21.15.19:39916) 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c50994640 1 -- 172.21.15.19:0/1957997333 learned_addr learned my addr 172.21.15.19:0/1957997333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838d500 0x5565d83967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838de20 0x5565d8396ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c50994640 1 -- 172.21.15.19:0/1957997333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838de20 msgr2=0x5565d8396ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c50994640 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838de20 0x5565d8396ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c50994640 1 -- 172.21.15.19:0/1957997333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838d500 msgr2=0x5565d83967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c50994640 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838d500 0x5565d83967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c50994640 1 -- 172.21.15.19:0/1957997333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565d8412430 con 0x5565d839cc10 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.634+0000 7f1c4b7fe640 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838de20 0x5565d8396ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c50994640 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d8397220 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1c44004f90 tx=0x7f1c44005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1137be5180810b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/1957997333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4402d040 con 0x5565d839cc10 2024-04-23T18:31:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/1957997333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f1c440041e0 con 0x5565d839cc10 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c551e3740 1 -- 172.21.15.19:0/1957997333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c4400ee90 con 0x5565d839cc10 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/1957997333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c4403f050 con 0x5565d839cc10 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c551e3740 10 client.16413 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c551e3740 1 -- 172.21.15.19:0/1957997333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565d8413690 con 0x5565d839cc10 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/1957997333 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c4400ee90 con 0x5565d839cc10 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/1957997333 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1c44030070 con 0x5565d839cc10 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c551e3740 10 client.16413 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c551e3740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c551e3740 20 client.16413 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c551e3740 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5565d8413d70 0x5565d8416190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c551e3740 4 client.16413 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.635+0000 7f1c551e3740 1 -- 172.21.15.19:0/1957997333 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565d82fa200 con 0x5565d8413d70 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.636+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5565d8413d70 0x5565d8416190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.636+0000 7f1c4bfff640 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5565d8413d70 0x5565d8416190 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b68b337dffaca23 server_cookie=71bd933ee3278a20 in_seq=0 out_seq=0 2024-04-23T18:31:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.636+0000 7f1c2b7fe640 10 client.16413 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c2b7fe640 1 -- 172.21.15.19:0/1957997333 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565d82fa200 con 0x5565d8413d70 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c2b7fe640 10 client.16413 handle_command_reply: tid=0 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 client.16413 shutdown 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 -- 172.21.15.19:0/1957997333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 msgr2=0x5565d8397220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d8397220 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1c44004f90 tx=0x7f1c44005f10 comp rx=0 tx=0).stop 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 -- 172.21.15.19:0/1957997333 shutdown_connections 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5565d8413d70 0x5565d8416190 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565d839cc10 0x5565d8397220 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5565d838de20 0x5565d8396ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 --2- 172.21.15.19:0/1957997333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5565d838d500 0x5565d83967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c2b7fe640 0 client.16413 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 -- 172.21.15.19:0/1957997333 >> 172.21.15.19:0/1957997333 conn(0x5565d8389b30 msgr2=0x5565d838efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 -- 172.21.15.19:0/1957997333 shutdown_connections 2024-04-23T18:31:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 1 -- 172.21.15.19:0/1957997333 wait complete. 2024-04-23T18:31:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.648+0000 7f1c551e3740 20 client.16413 trim_cache size 0 max 16384 2024-04-23T18:31:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:31:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:31:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:31:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:31:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 -- 172.21.15.19:0/205447841 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1c380a3dc0 msgr2=0x7f1c380a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1c380a3dc0 0x7f1c380a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c44005fd0 tx=0x7f1c440094c0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 -- 172.21.15.19:0/205447841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c107290 msgr2=0x7f1c4c198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c107290 0x7f1c4c198c30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c006010 tx=0x7f1c3c004060 comp rx=0 tx=0).stop 2024-04-23T18:31:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 -- 172.21.15.19:0/205447841 shutdown_connections 2024-04-23T18:31:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1c380a3dc0 0x7f1c380a6280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c4c19cfb0 0x7f1c4c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c4c107290 0x7f1c4c198c30 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 --2- 172.21.15.19:0/205447841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c1021c0 0x7f1c4c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 -- 172.21.15.19:0/205447841 >> 172.21.15.19:0/205447841 conn(0x7f1c4c0fda20 msgr2=0x7f1c4c0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 -- 172.21.15.19:0/205447841 shutdown_connections 2024-04-23T18:31:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:39.649+0000 7f1c523f7640 1 -- 172.21.15.19:0/205447841 wait complete. 2024-04-23T18:31:40.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:39 smithi019 ceph-mon[29215]: pgmap v168: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-23T18:31:40.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:39 smithi044 ceph-mon[32458]: pgmap v168: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-23T18:31:40.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:39 smithi155 ceph-mon[33468]: pgmap v168: 97 pgs: 97 active+clean; 5.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-23T18:31:42.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:41 smithi019 ceph-mon[29215]: pgmap v169: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-23T18:31:42.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:41 smithi044 ceph-mon[32458]: pgmap v169: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-23T18:31:42.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:41 smithi155 ceph-mon[33468]: pgmap v169: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-23T18:31:44.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:43 smithi019 ceph-mon[29215]: pgmap v170: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-04-23T18:31:44.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:43 smithi044 ceph-mon[32458]: pgmap v170: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-04-23T18:31:44.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:43 smithi155 ceph-mon[33468]: pgmap v170: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-04-23T18:31:46.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:45 smithi019 ceph-mon[29215]: pgmap v171: 97 pgs: 97 active+clean; 1.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 19 op/s 2024-04-23T18:31:46.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:45 smithi044 ceph-mon[32458]: pgmap v171: 97 pgs: 97 active+clean; 1.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 19 op/s 2024-04-23T18:31:46.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:45 smithi155 ceph-mon[33468]: pgmap v171: 97 pgs: 97 active+clean; 1.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 19 op/s 2024-04-23T18:31:48.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:47 smithi019 ceph-mon[29215]: pgmap v172: 97 pgs: 97 active+clean; 1.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:31:48.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:47 smithi044 ceph-mon[32458]: pgmap v172: 97 pgs: 97 active+clean; 1.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:31:48.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:47 smithi155 ceph-mon[33468]: pgmap v172: 97 pgs: 97 active+clean; 1.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:31:50.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:49 smithi019 ceph-mon[29215]: pgmap v173: 97 pgs: 97 active+clean; 1.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:31:50.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:49 smithi044 ceph-mon[32458]: pgmap v173: 97 pgs: 97 active+clean; 1.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:31:50.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:49 smithi155 ceph-mon[33468]: pgmap v173: 97 pgs: 97 active+clean; 1.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:31:52.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:51 smithi019 ceph-mon[29215]: pgmap v174: 97 pgs: 97 active+clean; 1.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 21 op/s 2024-04-23T18:31:52.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:51 smithi044 ceph-mon[32458]: pgmap v174: 97 pgs: 97 active+clean; 1.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 21 op/s 2024-04-23T18:31:52.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:51 smithi155 ceph-mon[33468]: pgmap v174: 97 pgs: 97 active+clean; 1.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 21 op/s 2024-04-23T18:31:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.749+0000 7f634b835640 1 -- 172.21.15.19:0/1482801427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f634410e8d0 msgr2=0x7f6344110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.749+0000 7f634b835640 1 --2- 172.21.15.19:0/1482801427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f634410e8d0 0x7f6344110cc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f633c009ca0 tx=0x7f633c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.749+0000 7f634b835640 1 -- 172.21.15.19:0/1482801427 shutdown_connections 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.749+0000 7f634b835640 1 --2- 172.21.15.19:0/1482801427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f634410e8d0 0x7f6344110cc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.749+0000 7f634b835640 1 --2- 172.21.15.19:0/1482801427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63441016d0 0x7f634410e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.749+0000 7f634b835640 1 --2- 172.21.15.19:0/1482801427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6344100d20 0x7f6344101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.749+0000 7f634b835640 1 -- 172.21.15.19:0/1482801427 >> 172.21.15.19:0/1482801427 conn(0x7f6344078060 msgr2=0x7f6344078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.749+0000 7f634b835640 1 -- 172.21.15.19:0/1482801427 shutdown_connections 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.749+0000 7f634b835640 1 -- 172.21.15.19:0/1482801427 wait complete. 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.749+0000 7f634b835640 1 Processor -- start 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f634b835640 1 -- start start 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f634b835640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6344100d20 0x7f63441a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f634b835640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63441016d0 0x7f63441a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f634b835640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634410e8d0 0x7f634419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f634b835640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63441143e0 con 0x7f6344100d20 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f634b835640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6344114260 con 0x7f634410e8d0 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f634b835640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6344114560 con 0x7f63441016d0 2024-04-23T18:31:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f63495d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6344100d20 0x7f63441a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f63495d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6344100d20 0x7f63441a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57834/0 (socket says 172.21.15.19:57834) 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f63495d1640 1 -- 172.21.15.19:0/3099971730 learned_addr learned my addr 172.21.15.19:0/3099971730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f6348dd0640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63441016d0 0x7f63441a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f6349dd2640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634410e8d0 0x7f634419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f63495d1640 1 -- 172.21.15.19:0/3099971730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63441016d0 msgr2=0x7f63441a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f63495d1640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63441016d0 0x7f63441a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f63495d1640 1 -- 172.21.15.19:0/3099971730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634410e8d0 msgr2=0x7f634419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f63495d1640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634410e8d0 0x7f634419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.750+0000 7f63495d1640 1 -- 172.21.15.19:0/3099971730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634419af60 con 0x7f6344100d20 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.751+0000 7f63495d1640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6344100d20 0x7f63441a05e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f634000d950 tx=0x7f634000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.751+0000 7f633a7fc640 1 -- 172.21.15.19:0/3099971730 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340017030 con 0x7f6344100d20 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.751+0000 7f633a7fc640 1 -- 172.21.15.19:0/3099971730 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6340004020 con 0x7f6344100d20 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.751+0000 7f634b835640 1 -- 172.21.15.19:0/3099971730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634419b250 con 0x7f6344100d20 2024-04-23T18:31:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.751+0000 7f633a7fc640 1 -- 172.21.15.19:0/3099971730 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400047a0 con 0x7f6344100d20 2024-04-23T18:31:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.751+0000 7f634b835640 1 -- 172.21.15.19:0/3099971730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63441a7560 con 0x7f6344100d20 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.753+0000 7f633a7fc640 1 -- 172.21.15.19:0/3099971730 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6340009a30 con 0x7f6344100d20 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.754+0000 7f634e62e740 1 Processor -- start 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.754+0000 7f634e62e740 1 -- start start 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.754+0000 7f634e62e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbc620 0x55c8acfbca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.754+0000 7f634e62e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbcf40 0x55c8acfc72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.754+0000 7f634e62e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 0x55c8acfc9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.754+0000 7f634e62e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c8acf21970 con 0x55c8acfbcf40 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.754+0000 7f634e62e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c8acebe4a0 con 0x55c8acfbc620 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.754+0000 7f633a7fc640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f63300a3e40 0x7f63300a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.754+0000 7f634e62e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c8acf1d930 con 0x55c8acfc7800 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f63495d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbcf40 0x55c8acfc72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f6349dd2640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f63300a3e40 0x7f63300a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f6348dd0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 0x55c8acfc9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f63495d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbcf40 0x55c8acfc72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57848/0 (socket says 172.21.15.19:57848) 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f6348dd0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 0x55c8acfc9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:56202/0 (socket says 172.21.15.19:56202) 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f63495d1640 1 -- 172.21.15.19:0/3832870936 learned_addr learned my addr 172.21.15.19:0/3832870936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f6349dd2640 1 --2- 172.21.15.19:0/3832870936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbc620 0x55c8acfbca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f633a7fc640 1 -- 172.21.15.19:0/3099971730 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f63400d0150 con 0x7f6344100d20 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f63495d1640 1 -- 172.21.15.19:0/3832870936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 msgr2=0x55c8acfc9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f63495d1640 1 --2- 172.21.15.19:0/3832870936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 0x55c8acfc9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f63495d1640 1 -- 172.21.15.19:0/3832870936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbc620 msgr2=0x55c8acfbca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f63495d1640 1 --2- 172.21.15.19:0/3832870936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbc620 0x55c8acfbca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f63495d1640 1 -- 172.21.15.19:0/3832870936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8acc34210 con 0x55c8acfbcf40 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f6349dd2640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f63300a3e40 0x7f63300a6300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f633c006010 tx=0x7f633c009a20 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f63495d1640 1 --2- 172.21.15.19:0/3832870936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbcf40 0x55c8acfc72c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f63400cd8d0 tx=0x7f634006b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb57bb434c5ef76 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f632f7fe640 1 -- 172.21.15.19:0/3832870936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634009c7c0 con 0x55c8acfbcf40 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f632f7fe640 1 -- 172.21.15.19:0/3832870936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f634009c0c0 con 0x55c8acfbcf40 2024-04-23T18:31:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.755+0000 7f632f7fe640 1 -- 172.21.15.19:0/3832870936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400041d0 con 0x55c8acfbcf40 2024-04-23T18:31:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.756+0000 7f634e62e740 1 -- 172.21.15.19:0/3832870936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbcf40 msgr2=0x55c8acfc72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.756+0000 7f634e62e740 1 --2- 172.21.15.19:0/3832870936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbcf40 0x55c8acfc72c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f63400cd8d0 tx=0x7f634006b9a0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.756+0000 7f634e62e740 1 -- 172.21.15.19:0/3832870936 shutdown_connections 2024-04-23T18:31:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.756+0000 7f634e62e740 1 --2- 172.21.15.19:0/3832870936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 0x55c8acfc9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.756+0000 7f634e62e740 1 --2- 172.21.15.19:0/3832870936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbcf40 0x55c8acfc72c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.756+0000 7f634e62e740 1 --2- 172.21.15.19:0/3832870936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbc620 0x55c8acfbca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.756+0000 7f634e62e740 1 -- 172.21.15.19:0/3832870936 >> 172.21.15.19:0/3832870936 conn(0x55c8acfb8b30 msgr2=0x55c8acfb8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.756+0000 7f634e62e740 1 -- 172.21.15.19:0/3832870936 shutdown_connections 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.756+0000 7f634e62e740 1 -- 172.21.15.19:0/3832870936 wait complete. 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f634e62e740 1 -- start start 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f634e62e740 1 Processor -- start 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f634e62e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfc7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f634e62e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbcf40 0x55c8acfc4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f634e62e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 0x55c8acfc49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f634e62e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c8acfca370 con 0x55c8acfbc620 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f634e62e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c8acebc0b0 con 0x55c8acfbcf40 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f634e62e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c8acfca4f0 con 0x55c8acfc7800 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f634e62e740 10 client.0 _ll_register_callbacks cb 0x55c8acea9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f63495d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfc7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f63495d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfc7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57860/0 (socket says 172.21.15.19:57860) 2024-04-23T18:31:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f63495d1640 1 -- 172.21.15.19:0/2708927387 learned_addr learned my addr 172.21.15.19:0/2708927387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f6348dd0640 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbcf40 0x55c8acfc4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.757+0000 7f6349dd2640 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 0x55c8acfc49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f63495d1640 1 -- 172.21.15.19:0/2708927387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 msgr2=0x55c8acfc49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f63495d1640 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 0x55c8acfc49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f63495d1640 1 -- 172.21.15.19:0/2708927387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbcf40 msgr2=0x55c8acfc4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f63495d1640 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbcf40 0x55c8acfc4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f63495d1640 1 -- 172.21.15.19:0/2708927387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8acd00620 con 0x55c8acfbc620 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f6348dd0640 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbcf40 0x55c8acfc4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f63495d1640 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfc7170 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f634009c050 tx=0x7f634000df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff00a0f0088d6eba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f632cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f632cff9640 1 -- 172.21.15.19:0/2708927387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400b4070 con 0x55c8acfbc620 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f632cff9640 1 -- 172.21.15.19:0/2708927387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f6340098730 con 0x55c8acfbc620 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f634e62e740 1 -- 172.21.15.19:0/2708927387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f634009c0c0 con 0x55c8acfbc620 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f632cff9640 1 -- 172.21.15.19:0/2708927387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f63400a4d30 con 0x55c8acfbc620 2024-04-23T18:31:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f634e62e740 10 client.16425 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f634e62e740 1 -- 172.21.15.19:0/2708927387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8acfd6310 con 0x55c8acfbc620 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.758+0000 7f632cff9640 1 -- 172.21.15.19:0/2708927387 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f634009c0c0 con 0x55c8acfbc620 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.759+0000 7f632cff9640 1 -- 172.21.15.19:0/2708927387 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f63400a48a0 con 0x55c8acfbc620 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.759+0000 7f634e62e740 10 client.16425 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.759+0000 7f634e62e740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.759+0000 7f634e62e740 20 client.16425 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.759+0000 7f634e62e740 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c8acfd7660 0x55c8acfd9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.759+0000 7f634e62e740 4 client.16425 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.759+0000 7f634e62e740 1 -- 172.21.15.19:0/2708927387 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8acf2f390 con 0x55c8acfd7660 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.759+0000 7f6348dd0640 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c8acfd7660 0x55c8acfd9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.759+0000 7f6348dd0640 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c8acfd7660 0x55c8acfd9a50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af839d13156ee1b7 server_cookie=3265b9961f601e5a in_seq=0 out_seq=0 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.759+0000 7f632cff9640 10 client.16425 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f632cff9640 1 -- 172.21.15.19:0/2708927387 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8acf2f390 con 0x55c8acfd7660 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f632cff9640 10 client.16425 handle_command_reply: tid=0 2024-04-23T18:31:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f634e62e740 1 client.16425 shutdown 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f634e62e740 1 -- 172.21.15.19:0/2708927387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 msgr2=0x55c8acfc7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f634e62e740 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfc7170 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f634009c050 tx=0x7f634000df40 comp rx=0 tx=0).stop 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f634e62e740 1 -- 172.21.15.19:0/2708927387 shutdown_connections 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f634e62e740 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c8acfd7660 0x55c8acfd9a50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f634e62e740 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbcf40 0x55c8acfc4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f634e62e740 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfc7800 0x55c8acfc49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f632cff9640 0 client.16425 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f634e62e740 1 --2- 172.21.15.19:0/2708927387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfc7170 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.760+0000 7f634e62e740 1 -- 172.21.15.19:0/2708927387 >> 172.21.15.19:0/2708927387 conn(0x55c8acfb8b30 msgr2=0x55c8acfbe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.761+0000 7f634e62e740 1 -- 172.21.15.19:0/2708927387 shutdown_connections 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.761+0000 7f634e62e740 1 -- 172.21.15.19:0/2708927387 wait complete. 2024-04-23T18:31:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.761+0000 7f634e62e740 20 client.16425 trim_cache size 0 max 16384 2024-04-23T18:31:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f634e62e740 1 Processor -- start 2024-04-23T18:31:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f634e62e740 1 -- start start 2024-04-23T18:31:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f634e62e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfd3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f634e62e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfbcf40 0x55c8acfdf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f634e62e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfc7800 0x55c8acfdf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f634e62e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63441afce0 con 0x55c8acfbc620 2024-04-23T18:31:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f634e62e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f63441b1820 con 0x55c8acfc7800 2024-04-23T18:31:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f634e62e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63441b03b0 con 0x55c8acfbcf40 2024-04-23T18:31:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f63495d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfd3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f63495d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfd3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57868/0 (socket says 172.21.15.19:57868) 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f63495d1640 1 -- 172.21.15.19:0/2433511042 learned_addr learned my addr 172.21.15.19:0/2433511042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f63495d1640 1 -- 172.21.15.19:0/2433511042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfbcf40 msgr2=0x55c8acfdf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f63495d1640 1 --2- 172.21.15.19:0/2433511042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfbcf40 0x55c8acfdf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f63495d1640 1 -- 172.21.15.19:0/2433511042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfc7800 msgr2=0x55c8acfdf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f63495d1640 1 --2- 172.21.15.19:0/2433511042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfc7800 0x55c8acfdf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.776+0000 7f63495d1640 1 -- 172.21.15.19:0/2433511042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8acfbcd70 con 0x55c8acfbc620 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f63495d1640 1 --2- 172.21.15.19:0/2433511042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfd3260 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f634009a370 tx=0x7f63400d03d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a729074450969e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f632effd640 1 -- 172.21.15.19:0/2433511042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400b4070 con 0x55c8acfbc620 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f632effd640 1 -- 172.21.15.19:0/2433511042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f634009c0c0 con 0x55c8acfbc620 2024-04-23T18:31:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f632effd640 1 -- 172.21.15.19:0/2433511042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400ba080 con 0x55c8acfbc620 2024-04-23T18:31:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f634e62e740 1 -- 172.21.15.19:0/2433511042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 msgr2=0x55c8acfd3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f634e62e740 1 --2- 172.21.15.19:0/2433511042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfd3260 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f634009a370 tx=0x7f63400d03d0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f634e62e740 1 -- 172.21.15.19:0/2433511042 shutdown_connections 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f634e62e740 1 --2- 172.21.15.19:0/2433511042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfc7800 0x55c8acfdf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f634e62e740 1 --2- 172.21.15.19:0/2433511042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfbcf40 0x55c8acfdf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f634e62e740 1 --2- 172.21.15.19:0/2433511042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfbc620 0x55c8acfd3260 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f634e62e740 1 -- 172.21.15.19:0/2433511042 >> 172.21.15.19:0/2433511042 conn(0x55c8acfb8b30 msgr2=0x55c8acfb7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f634e62e740 1 -- 172.21.15.19:0/2433511042 shutdown_connections 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.777+0000 7f634e62e740 1 -- 172.21.15.19:0/2433511042 wait complete. 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f634e62e740 1 -- start start 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f634e62e740 1 Processor -- start 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f634e62e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbc620 0x55c8ad035d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f634e62e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfbcf40 0x55c8ad036290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f634e62e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfc7800 0x55c8acfccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f634e62e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63441b1150 con 0x55c8acfc7800 2024-04-23T18:31:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f634e62e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f63441b0a80 con 0x55c8acfbc620 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f634e62e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63441adcc0 con 0x55c8acfbcf40 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f634e62e740 10 client.0 _ll_register_callbacks cb 0x7f634009bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f6349dd2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfc7800 0x55c8acfccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f6349dd2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfc7800 0x55c8acfccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57874/0 (socket says 172.21.15.19:57874) 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.778+0000 7f6349dd2640 1 -- 172.21.15.19:0/954278159 learned_addr learned my addr 172.21.15.19:0/954278159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f6348dd0640 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfbcf40 0x55c8ad036290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f63495d1640 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbc620 0x55c8ad035d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f6349dd2640 1 -- 172.21.15.19:0/954278159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfbcf40 msgr2=0x55c8ad036290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f6349dd2640 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfbcf40 0x55c8ad036290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f6349dd2640 1 -- 172.21.15.19:0/954278159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbc620 msgr2=0x55c8ad035d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f6349dd2640 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbc620 0x55c8ad035d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f6349dd2640 1 -- 172.21.15.19:0/954278159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8acfcd2e0 con 0x55c8acfc7800 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f6349dd2640 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfc7800 0x55c8acfccae0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f633c009ca0 tx=0x7f633c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b21797598d5f299 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f632cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:31:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f632cff9640 1 -- 172.21.15.19:0/954278159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c003b80 con 0x55c8acfc7800 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f632cff9640 1 -- 172.21.15.19:0/954278159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f633c003d20 con 0x55c8acfc7800 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f634e62e740 1 -- 172.21.15.19:0/954278159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f634009c0c0 con 0x55c8acfc7800 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f632cff9640 1 -- 172.21.15.19:0/954278159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f633c05e050 con 0x55c8acfc7800 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f634e62e740 10 client.16431 fetch_fsmap learned FSMap version 27 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f634e62e740 1 -- 172.21.15.19:0/954278159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8ad03d540 con 0x55c8acfc7800 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.779+0000 7f632cff9640 1 -- 172.21.15.19:0/954278159 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f634009c0c0 con 0x55c8acfc7800 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.780+0000 7f632cff9640 1 -- 172.21.15.19:0/954278159 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f633c02f8c0 con 0x55c8acfc7800 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.780+0000 7f634e62e740 10 client.16431 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.780+0000 7f634e62e740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.780+0000 7f634e62e740 20 client.16431 populate_metadata read hostname 'smithi019' 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.780+0000 7f634e62e740 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c8ad03dd10 0x55c8ad040130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.780+0000 7f634e62e740 4 client.16431 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.780+0000 7f634e62e740 1 -- 172.21.15.19:0/954278159 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8acf29200 con 0x55c8ad03dd10 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.780+0000 7f63495d1640 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c8ad03dd10 0x55c8ad040130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.780+0000 7f632cff9640 10 client.16431 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.780+0000 7f63495d1640 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c8ad03dd10 0x55c8ad040130 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75722fde61827288 server_cookie=41f161ab94c24850 in_seq=0 out_seq=0 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f632cff9640 1 -- 172.21.15.19:0/954278159 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8acf29200 con 0x55c8ad03dd10 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f632cff9640 10 client.16431 handle_command_reply: tid=0 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 client.16431 shutdown 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 -- 172.21.15.19:0/954278159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfc7800 msgr2=0x55c8acfccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfc7800 0x55c8acfccae0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f633c009ca0 tx=0x7f633c0095c0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 -- 172.21.15.19:0/954278159 shutdown_connections 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c8ad03dd10 0x55c8ad040130 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8acfc7800 0x55c8acfccae0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8acfbcf40 0x55c8ad036290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f632cff9640 0 client.16431 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 --2- 172.21.15.19:0/954278159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8acfbc620 0x55c8ad035d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 -- 172.21.15.19:0/954278159 >> 172.21.15.19:0/954278159 conn(0x55c8acfb8b30 msgr2=0x55c8acfc5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 -- 172.21.15.19:0/954278159 shutdown_connections 2024-04-23T18:31:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 1 -- 172.21.15.19:0/954278159 wait complete. 2024-04-23T18:31:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.794+0000 7f634e62e740 20 client.16431 trim_cache size 0 max 16384 2024-04-23T18:31:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:31:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:31:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:31:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:31:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 -- 172.21.15.19:0/3099971730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f63300a3e40 msgr2=0x7f63300a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f63300a3e40 0x7f63300a6300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f633c006010 tx=0x7f633c009a20 comp rx=0 tx=0).stop 2024-04-23T18:31:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 -- 172.21.15.19:0/3099971730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6344100d20 msgr2=0x7f63441a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6344100d20 0x7f63441a05e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f634000d950 tx=0x7f634000de10 comp rx=0 tx=0).stop 2024-04-23T18:31:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 -- 172.21.15.19:0/3099971730 shutdown_connections 2024-04-23T18:31:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f63300a3e40 0x7f63300a6300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f634410e8d0 0x7f634419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63441016d0 0x7f63441a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 --2- 172.21.15.19:0/3099971730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6344100d20 0x7f63441a05e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 -- 172.21.15.19:0/3099971730 >> 172.21.15.19:0/3099971730 conn(0x7f6344078060 msgr2=0x7f634410fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 -- 172.21.15.19:0/3099971730 shutdown_connections 2024-04-23T18:31:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:31:52.795+0000 7f634b835640 1 -- 172.21.15.19:0/3099971730 wait complete. 2024-04-23T18:31:54.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:53 smithi019 ceph-mon[29215]: pgmap v175: 97 pgs: 97 active+clean; 1.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:31:54.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:53 smithi044 ceph-mon[32458]: pgmap v175: 97 pgs: 97 active+clean; 1.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:31:54.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:53 smithi155 ceph-mon[33468]: pgmap v175: 97 pgs: 97 active+clean; 1.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:31:56.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:55 smithi019 ceph-mon[29215]: pgmap v176: 97 pgs: 97 active+clean; 2.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 24 op/s 2024-04-23T18:31:56.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:55 smithi044 ceph-mon[32458]: pgmap v176: 97 pgs: 97 active+clean; 2.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 24 op/s 2024-04-23T18:31:56.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:55 smithi155 ceph-mon[33468]: pgmap v176: 97 pgs: 97 active+clean; 2.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 24 op/s 2024-04-23T18:31:58.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:57 smithi019 ceph-mon[29215]: pgmap v177: 97 pgs: 97 active+clean; 2.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:31:58.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:57 smithi044 ceph-mon[32458]: pgmap v177: 97 pgs: 97 active+clean; 2.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:31:58.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:57 smithi155 ceph-mon[33468]: pgmap v177: 97 pgs: 97 active+clean; 2.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:31:58.687 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:127/600: open fd = 3 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:128/600: open fd = 3 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:129/600: open fd = 3 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:130/600: open fd = 3 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.688 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:131/600: open fd = 3 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:132/600: open fd = 3 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:133/600: open fd = 3 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:134/600: open fd = 3 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.689 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:135/600: open fd = 3 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:136/600: open fd = 3 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:137/600: open fd = 3 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:138/600: open fd = 3 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:139/600: open fd = 3 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:140/600: open fd = 3 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:141/600: open fd = 3 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:142/600: open fd = 3 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.691 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:143/600: open fd = 3 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:144/600: open fd = 3 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:145/600: open fd = 3 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:146/600: open fd = 3 2024-04-23T18:31:58.692 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:147/600: open fd = 3 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:148/600: open fd = 3 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:149/600: open fd = 3 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.693 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.694 INFO:tasks.workunit.client.0.smithi019.stdout:150/600: open fd = 3 2024-04-23T18:31:58.694 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.694 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.694 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.694 INFO:tasks.workunit.client.0.smithi019.stdout:151/600: open fd = 3 2024-04-23T18:31:58.694 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.694 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.695 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.695 INFO:tasks.workunit.client.0.smithi019.stdout:152/600: open fd = 3 2024-04-23T18:31:58.695 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.695 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.695 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.695 INFO:tasks.workunit.client.0.smithi019.stdout:153/600: open fd = 3 2024-04-23T18:31:58.695 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:154/600: open fd = 3 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:155/600: open fd = 3 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.696 INFO:tasks.workunit.client.0.smithi019.stdout:156/600: open fd = 3 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:157/600: open fd = 3 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:158/600: open fd = 3 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:159/600: open fd = 3 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.697 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:160/600: open fd = 3 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:161/600: open fd = 3 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:162/600: open fd = 3 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.698 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.699 INFO:tasks.workunit.client.0.smithi019.stdout:163/600: open fd = 3 2024-04-23T18:31:58.699 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.699 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.699 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.699 INFO:tasks.workunit.client.0.smithi019.stdout:164/600: open fd = 3 2024-04-23T18:31:58.699 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.700 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.700 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.700 INFO:tasks.workunit.client.0.smithi019.stdout:165/600: open fd = 3 2024-04-23T18:31:58.700 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.700 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.700 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.700 INFO:tasks.workunit.client.0.smithi019.stdout:166/600: open fd = 3 2024-04-23T18:31:58.700 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:167/600: open fd = 3 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:168/600: open fd = 3 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.701 INFO:tasks.workunit.client.0.smithi019.stdout:169/600: open fd = 3 2024-04-23T18:31:58.702 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.702 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.702 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.702 INFO:tasks.workunit.client.0.smithi019.stdout:170/600: open fd = 3 2024-04-23T18:31:58.702 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.702 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.702 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.702 INFO:tasks.workunit.client.0.smithi019.stdout:171/600: open fd = 3 2024-04-23T18:31:58.703 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.703 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.703 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.703 INFO:tasks.workunit.client.0.smithi019.stdout:172/600: open fd = 3 2024-04-23T18:31:58.703 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.703 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.703 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.703 INFO:tasks.workunit.client.0.smithi019.stdout:173/600: open fd = 3 2024-04-23T18:31:58.704 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.704 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.704 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.704 INFO:tasks.workunit.client.0.smithi019.stdout:174/600: open fd = 3 2024-04-23T18:31:58.704 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.704 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.705 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.705 INFO:tasks.workunit.client.0.smithi019.stdout:175/600: open fd = 3 2024-04-23T18:31:58.705 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.705 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.705 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.705 INFO:tasks.workunit.client.0.smithi019.stdout:176/600: open fd = 3 2024-04-23T18:31:58.705 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.705 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.706 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.706 INFO:tasks.workunit.client.0.smithi019.stdout:177/600: open fd = 3 2024-04-23T18:31:58.706 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.706 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.706 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.706 INFO:tasks.workunit.client.0.smithi019.stdout:178/600: open fd = 3 2024-04-23T18:31:58.706 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.707 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.707 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.707 INFO:tasks.workunit.client.0.smithi019.stdout:179/600: open fd = 3 2024-04-23T18:31:58.707 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.707 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.707 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.707 INFO:tasks.workunit.client.0.smithi019.stdout:180/600: open fd = 3 2024-04-23T18:31:58.707 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.708 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.708 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.708 INFO:tasks.workunit.client.0.smithi019.stdout:181/600: open fd = 3 2024-04-23T18:31:58.708 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.708 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.708 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.708 INFO:tasks.workunit.client.0.smithi019.stdout:182/600: open fd = 3 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:183/600: open fd = 3 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:184/600: open fd = 3 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.709 INFO:tasks.workunit.client.0.smithi019.stdout:185/600: open fd = 3 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:186/600: open fd = 3 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:187/600: open fd = 3 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.710 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:31:58.711 INFO:tasks.workunit.client.0.smithi019.stdout:188/600: open fd = 3 2024-04-23T18:31:58.711 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:31:58.711 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:32:00.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:31:59 smithi019 ceph-mon[29215]: pgmap v178: 97 pgs: 97 active+clean; 2.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:32:00.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:31:59 smithi044 ceph-mon[32458]: pgmap v178: 97 pgs: 97 active+clean; 2.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:32:00.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:31:59 smithi155 ceph-mon[33468]: pgmap v178: 97 pgs: 97 active+clean; 2.0 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:32:02.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:01 smithi019 ceph-mon[29215]: pgmap v179: 97 pgs: 97 active+clean; 2.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-23T18:32:02.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:01 smithi044 ceph-mon[32458]: pgmap v179: 97 pgs: 97 active+clean; 2.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-23T18:32:02.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:01 smithi155 ceph-mon[33468]: pgmap v179: 97 pgs: 97 active+clean; 2.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-23T18:32:04.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:03 smithi044 ceph-mon[32458]: pgmap v180: 97 pgs: 97 active+clean; 2.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:32:04.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:03 smithi155 ceph-mon[33468]: pgmap v180: 97 pgs: 97 active+clean; 2.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:32:04.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:03 smithi019 ceph-mon[29215]: pgmap v180: 97 pgs: 97 active+clean; 2.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:32:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.895+0000 7f57320cf640 1 -- 172.21.15.19:0/1493662185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f572c1021c0 msgr2=0x7f572c1025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.895+0000 7f57320cf640 1 --2- 172.21.15.19:0/1493662185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f572c1021c0 0x7f572c1025a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f571c009900 tx=0x7f571c02a410 comp rx=0 tx=0).stop 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.895+0000 7f57320cf640 1 -- 172.21.15.19:0/1493662185 shutdown_connections 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.895+0000 7f57320cf640 1 --2- 172.21.15.19:0/1493662185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f572c102b70 0x7f572c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.895+0000 7f57320cf640 1 --2- 172.21.15.19:0/1493662185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f572c107290 0x7f572c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.895+0000 7f57320cf640 1 --2- 172.21.15.19:0/1493662185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f572c1021c0 0x7f572c1025a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.895+0000 7f57320cf640 1 -- 172.21.15.19:0/1493662185 >> 172.21.15.19:0/1493662185 conn(0x7f572c0fda20 msgr2=0x7f572c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.895+0000 7f57320cf640 1 -- 172.21.15.19:0/1493662185 shutdown_connections 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.895+0000 7f57320cf640 1 -- 172.21.15.19:0/1493662185 wait complete. 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.895+0000 7f57320cf640 1 Processor -- start 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f57320cf640 1 -- start start 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f57320cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f572c1021c0 0x7f572c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f57320cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f572c107290 0x7f572c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f57320cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f572c19cf80 0x7f572c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f57320cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f572c112200 con 0x7f572c107290 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f57320cf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f572c112080 con 0x7f572c19cf80 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f57320cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f572c112380 con 0x7f572c1021c0 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f572c107290 0x7f572c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f572c107290 0x7f572c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50726/0 (socket says 172.21.15.19:50726) 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572affd640 1 -- 172.21.15.19:0/4276518498 learned_addr learned my addr 172.21.15.19:0/4276518498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572b7fe640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f572c1021c0 0x7f572c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572bfff640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f572c19cf80 0x7f572c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572affd640 1 -- 172.21.15.19:0/4276518498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f572c1021c0 msgr2=0x7f572c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572affd640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f572c1021c0 0x7f572c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572affd640 1 -- 172.21.15.19:0/4276518498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f572c19cf80 msgr2=0x7f572c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572affd640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f572c19cf80 0x7f572c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572affd640 1 -- 172.21.15.19:0/4276518498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572c19da70 con 0x7f572c107290 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.896+0000 7f572bfff640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f572c19cf80 0x7f572c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.897+0000 7f572affd640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f572c107290 0x7f572c198c00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f572000e9c0 tx=0x7f572000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.897+0000 7f5728ff9640 1 -- 172.21.15.19:0/4276518498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572000cd80 con 0x7f572c107290 2024-04-23T18:32:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.897+0000 7f57320cf640 1 -- 172.21.15.19:0/4276518498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572c19dd60 con 0x7f572c107290 2024-04-23T18:32:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.897+0000 7f5728ff9640 1 -- 172.21.15.19:0/4276518498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57200054d0 con 0x7f572c107290 2024-04-23T18:32:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.897+0000 7f5728ff9640 1 -- 172.21.15.19:0/4276518498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720010430 con 0x7f572c107290 2024-04-23T18:32:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.897+0000 7f57320cf640 1 -- 172.21.15.19:0/4276518498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572c1a5550 con 0x7f572c107290 2024-04-23T18:32:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.899+0000 7f5728ff9640 1 -- 172.21.15.19:0/4276518498 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5720004fb0 con 0x7f572c107290 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.899+0000 7f5734de4740 1 Processor -- start 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.899+0000 7f5734de4740 1 -- start start 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f5734de4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d93d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f5734de4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d94c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f5734de4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d94f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f5734de4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55935d8a2970 con 0x55935d93d520 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f5734de4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55935d83f4a0 con 0x55935d93de40 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f5734de4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55935d89e930 con 0x55935d94cc10 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d93d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d93d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50728/0 (socket says 172.21.15.19:50728) 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572bfff640 1 -- 172.21.15.19:0/1096182584 learned_addr learned my addr 172.21.15.19:0/1096182584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f5728ff9640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f57140a3dc0 0x7f57140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572b7fe640 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d94c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572affd640 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d94f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572bfff640 1 -- 172.21.15.19:0/1096182584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 msgr2=0x55935d94f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572bfff640 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d94f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572bfff640 1 -- 172.21.15.19:0/1096182584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 msgr2=0x55935d94c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572bfff640 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d94c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572bfff640 1 -- 172.21.15.19:0/1096182584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55935d5b5210 con 0x55935d93d520 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572b7fe640 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d94c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572affd640 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d94f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f5728ff9640 1 -- 172.21.15.19:0/4276518498 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f57200d0d60 con 0x7f572c107290 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.900+0000 7f572bfff640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f57140a3dc0 0x7f57140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.901+0000 7f572bfff640 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d93d900 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5724009fa0 tx=0x7f572400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bdba77074534a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.901+0000 7f5711ffb640 1 -- 172.21.15.19:0/1096182584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572401a610 con 0x55935d93d520 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.901+0000 7f5711ffb640 1 -- 172.21.15.19:0/1096182584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f572401ac60 con 0x55935d93d520 2024-04-23T18:32:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.901+0000 7f5711ffb640 1 -- 172.21.15.19:0/1096182584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57240108b0 con 0x55935d93d520 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 -- 172.21.15.19:0/1096182584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 msgr2=0x55935d93d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f572bfff640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f57140a3dc0 0x7f57140a6280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5724020000 tx=0x7f572401d000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d93d900 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5724009fa0 tx=0x7f572400f7f0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 -- 172.21.15.19:0/1096182584 shutdown_connections 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d94c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d94f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 --2- 172.21.15.19:0/1096182584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d93d900 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 -- 172.21.15.19:0/1096182584 >> 172.21.15.19:0/1096182584 conn(0x55935d939b30 msgr2=0x55935d939f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 -- 172.21.15.19:0/1096182584 shutdown_connections 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 -- 172.21.15.19:0/1096182584 wait complete. 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 -- start start 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.902+0000 7f5734de4740 1 Processor -- start 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f5734de4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d94c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f5734de4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d9497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f5734de4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d949d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f5734de4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55935d94f840 con 0x55935d93d520 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f5734de4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55935d83d0b0 con 0x55935d93de40 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f5734de4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55935d94f9c0 con 0x55935d94cc10 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f5734de4740 10 client.0 _ll_register_callbacks cb 0x55935d82ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d94c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d94c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50734/0 (socket says 172.21.15.19:50734) 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572b7fe640 1 -- 172.21.15.19:0/2269231877 learned_addr learned my addr 172.21.15.19:0/2269231877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572affd640 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d9497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572b7fe640 1 -- 172.21.15.19:0/2269231877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 msgr2=0x55935d949d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572bfff640 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d949d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572b7fe640 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d949d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572b7fe640 1 -- 172.21.15.19:0/2269231877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 msgr2=0x55935d9497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572b7fe640 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d9497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572b7fe640 1 -- 172.21.15.19:0/2269231877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55935d681620 con 0x55935d93d520 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572bfff640 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d949d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f572b7fe640 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d94c460 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f571c02a930 tx=0x7f571c0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8cf371ff554ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f570affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f570affd640 1 -- 172.21.15.19:0/2269231877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c035070 con 0x55935d93d520 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f570affd640 1 -- 172.21.15.19:0/2269231877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f571c004040 con 0x55935d93d520 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.903+0000 7f5734de4740 1 -- 172.21.15.19:0/2269231877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f572401ac60 con 0x55935d93d520 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f570affd640 1 -- 172.21.15.19:0/2269231877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f571c004240 con 0x55935d93d520 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f5734de4740 10 client.16443 fetch_fsmap learned FSMap version 27 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f5734de4740 1 -- 172.21.15.19:0/2269231877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55935d95b6a0 con 0x55935d93d520 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f570affd640 1 -- 172.21.15.19:0/2269231877 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f572401ac60 con 0x55935d93d520 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f570affd640 1 -- 172.21.15.19:0/2269231877 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f571c004960 con 0x55935d93d520 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f5734de4740 10 client.16443 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:32:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f5734de4740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f5734de4740 20 client.16443 populate_metadata read hostname 'smithi019' 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f5734de4740 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55935d95cb60 0x55935d95ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f5734de4740 4 client.16443 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f5734de4740 1 -- 172.21.15.19:0/2269231877 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55935d8b0390 con 0x55935d95cb60 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.904+0000 7f572affd640 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55935d95cb60 0x55935d95ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.905+0000 7f572affd640 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55935d95cb60 0x55935d95ef70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fab36d784902da4 server_cookie=281687789dd67ec0 in_seq=0 out_seq=0 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.905+0000 7f570affd640 10 client.16443 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f570affd640 1 -- 172.21.15.19:0/2269231877 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55935d8b0390 con 0x55935d95cb60 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f570affd640 10 client.16443 handle_command_reply: tid=0 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 client.16443 shutdown 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 -- 172.21.15.19:0/2269231877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 msgr2=0x55935d94c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d94c460 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f571c02a930 tx=0x7f571c0316d0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 -- 172.21.15.19:0/2269231877 shutdown_connections 2024-04-23T18:32:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55935d95cb60 0x55935d95ef70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d9497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d949d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f570affd640 0 client.16443 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 --2- 172.21.15.19:0/2269231877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d94c460 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 -- 172.21.15.19:0/2269231877 >> 172.21.15.19:0/2269231877 conn(0x55935d939b30 msgr2=0x55935d93f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 -- 172.21.15.19:0/2269231877 shutdown_connections 2024-04-23T18:32:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 1 -- 172.21.15.19:0/2269231877 wait complete. 2024-04-23T18:32:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.906+0000 7f5734de4740 20 client.16443 trim_cache size 0 max 16384 2024-04-23T18:32:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.921+0000 7f5734de4740 1 Processor -- start 2024-04-23T18:32:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.921+0000 7f5734de4740 1 -- start start 2024-04-23T18:32:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.921+0000 7f5734de4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93d520 0x55935d958700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.921+0000 7f5734de4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d93de40 0x55935d944d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.921+0000 7f5734de4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d94cc10 0x55935d94c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.921+0000 7f5734de4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1ab880 con 0x55935d94cc10 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.921+0000 7f5734de4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1ad3c0 con 0x55935d93d520 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.921+0000 7f5734de4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1abf50 con 0x55935d93de40 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d94cc10 0x55935d94c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d94cc10 0x55935d94c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50744/0 (socket says 172.21.15.19:50744) 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572bfff640 1 -- 172.21.15.19:0/2623677791 learned_addr learned my addr 172.21.15.19:0/2623677791 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572b7fe640 1 --2- 172.21.15.19:0/2623677791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93d520 0x55935d958700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572affd640 1 --2- 172.21.15.19:0/2623677791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d93de40 0x55935d944d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572bfff640 1 -- 172.21.15.19:0/2623677791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d93de40 msgr2=0x55935d944d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572bfff640 1 --2- 172.21.15.19:0/2623677791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d93de40 0x55935d944d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572bfff640 1 -- 172.21.15.19:0/2623677791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93d520 msgr2=0x55935d958700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572bfff640 1 --2- 172.21.15.19:0/2623677791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93d520 0x55935d958700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572bfff640 1 -- 172.21.15.19:0/2623677791 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55935d939960 con 0x55935d94cc10 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572b7fe640 1 --2- 172.21.15.19:0/2623677791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93d520 0x55935d958700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f572bfff640 1 --2- 172.21.15.19:0/2623677791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d94cc10 0x55935d94c000 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f57240099d0 tx=0x7f57240096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138ebd2a320fc380 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f57117fa640 1 -- 172.21.15.19:0/2623677791 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724010e10 con 0x55935d94cc10 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f57117fa640 1 -- 172.21.15.19:0/2623677791 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57240094a0 con 0x55935d94cc10 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.922+0000 7f57117fa640 1 -- 172.21.15.19:0/2623677791 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724025830 con 0x55935d94cc10 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.923+0000 7f5734de4740 1 -- 172.21.15.19:0/2623677791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d94cc10 msgr2=0x55935d94c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.923+0000 7f5734de4740 1 --2- 172.21.15.19:0/2623677791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d94cc10 0x55935d94c000 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f57240099d0 tx=0x7f57240096a0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.923+0000 7f5734de4740 1 -- 172.21.15.19:0/2623677791 shutdown_connections 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.923+0000 7f5734de4740 1 --2- 172.21.15.19:0/2623677791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d94cc10 0x55935d94c000 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.923+0000 7f5734de4740 1 --2- 172.21.15.19:0/2623677791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d93de40 0x55935d944d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.923+0000 7f5734de4740 1 --2- 172.21.15.19:0/2623677791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93d520 0x55935d958700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.923+0000 7f5734de4740 1 -- 172.21.15.19:0/2623677791 >> 172.21.15.19:0/2623677791 conn(0x55935d939b30 msgr2=0x55935d944670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.923+0000 7f5734de4740 1 -- 172.21.15.19:0/2623677791 shutdown_connections 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.923+0000 7f5734de4740 1 -- 172.21.15.19:0/2623677791 wait complete. 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.923+0000 7f5734de4740 1 -- start start 2024-04-23T18:32:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f5734de4740 1 Processor -- start 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f5734de4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d954ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f5734de4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d950250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f5734de4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d950790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f5734de4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1accf0 con 0x55935d93d520 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f5734de4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1ac620 con 0x55935d93de40 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f5734de4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1a9880 con 0x55935d94cc10 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f5734de4740 10 client.0 _ll_register_callbacks cb 0x7f571c032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f572b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d954ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f572b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d954ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50752/0 (socket says 172.21.15.19:50752) 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f572b7fe640 1 -- 172.21.15.19:0/2347838312 learned_addr learned my addr 172.21.15.19:0/2347838312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f572affd640 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d950250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f572b7fe640 1 -- 172.21.15.19:0/2347838312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 msgr2=0x55935d950790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f572b7fe640 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d950790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f572b7fe640 1 -- 172.21.15.19:0/2347838312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 msgr2=0x55935d950250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f572b7fe640 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d950250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f572b7fe640 1 -- 172.21.15.19:0/2347838312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55935d950e80 con 0x55935d93d520 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f572b7fe640 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d954ee0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f571c02a930 tx=0x7f571c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e300f7851e406e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.924+0000 7f570affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f570affd640 1 -- 172.21.15.19:0/2347838312 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c051070 con 0x55935d93d520 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f570affd640 1 -- 172.21.15.19:0/2347838312 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f571c0048c0 con 0x55935d93d520 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f5734de4740 1 -- 172.21.15.19:0/2347838312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57240094a0 con 0x55935d93d520 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f570affd640 1 -- 172.21.15.19:0/2347838312 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f571c035040 con 0x55935d93d520 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f5734de4740 10 client.16449 fetch_fsmap learned FSMap version 27 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f5734de4740 1 -- 172.21.15.19:0/2347838312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55935d9c3060 con 0x55935d93d520 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f570affd640 1 -- 172.21.15.19:0/2347838312 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57240094a0 con 0x55935d93d520 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f570affd640 1 -- 172.21.15.19:0/2347838312 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f571c0050c0 con 0x55935d93d520 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f5734de4740 10 client.16449 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f5734de4740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f5734de4740 20 client.16449 populate_metadata read hostname 'smithi019' 2024-04-23T18:32:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f5734de4740 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55935d9c3790 0x55935d9c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f5734de4740 4 client.16449 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f5734de4740 1 -- 172.21.15.19:0/2347838312 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55935d8aa200 con 0x55935d9c3790 2024-04-23T18:32:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.925+0000 7f572affd640 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55935d9c3790 0x55935d9c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.926+0000 7f570affd640 10 client.16449 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.926+0000 7f572affd640 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55935d9c3790 0x55935d9c5bb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158c35f0fa22426b server_cookie=3fe5b04281980136 in_seq=0 out_seq=0 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f570affd640 1 -- 172.21.15.19:0/2347838312 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55935d8aa200 con 0x55935d9c3790 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f570affd640 10 client.16449 handle_command_reply: tid=0 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 client.16449 shutdown 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 -- 172.21.15.19:0/2347838312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 msgr2=0x55935d954ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d954ee0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f571c02a930 tx=0x7f571c031400 comp rx=0 tx=0).stop 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 -- 172.21.15.19:0/2347838312 shutdown_connections 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55935d9c3790 0x55935d9c5bb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55935d93de40 0x55935d950250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55935d94cc10 0x55935d950790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f570affd640 0 client.16449 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 --2- 172.21.15.19:0/2347838312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55935d93d520 0x55935d954ee0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 -- 172.21.15.19:0/2347838312 >> 172.21.15.19:0/2347838312 conn(0x55935d939b30 msgr2=0x55935d938420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 -- 172.21.15.19:0/2347838312 shutdown_connections 2024-04-23T18:32:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 1 -- 172.21.15.19:0/2347838312 wait complete. 2024-04-23T18:32:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.940+0000 7f5734de4740 20 client.16449 trim_cache size 0 max 16384 2024-04-23T18:32:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:32:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:32:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:32:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:32:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 -- 172.21.15.19:0/4276518498 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f57140a3dc0 msgr2=0x7f57140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f57140a3dc0 0x7f57140a6280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5724020000 tx=0x7f572401d000 comp rx=0 tx=0).stop 2024-04-23T18:32:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 -- 172.21.15.19:0/4276518498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f572c107290 msgr2=0x7f572c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f572c107290 0x7f572c198c00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f572000e9c0 tx=0x7f572000ee80 comp rx=0 tx=0).stop 2024-04-23T18:32:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 -- 172.21.15.19:0/4276518498 shutdown_connections 2024-04-23T18:32:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f57140a3dc0 0x7f57140a6280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f572c19cf80 0x7f572c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f572c107290 0x7f572c198c00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 --2- 172.21.15.19:0/4276518498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f572c1021c0 0x7f572c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 -- 172.21.15.19:0/4276518498 >> 172.21.15.19:0/4276518498 conn(0x7f572c0fda20 msgr2=0x7f572c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 -- 172.21.15.19:0/4276518498 shutdown_connections 2024-04-23T18:32:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:05.941+0000 7f57320cf640 1 -- 172.21.15.19:0/4276518498 wait complete. 2024-04-23T18:32:06.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:05 smithi044 ceph-mon[32458]: pgmap v181: 97 pgs: 97 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 74 KiB/s wr, 37 op/s 2024-04-23T18:32:06.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:05 smithi155 ceph-mon[33468]: pgmap v181: 97 pgs: 97 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 74 KiB/s wr, 37 op/s 2024-04-23T18:32:06.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:05 smithi019 ceph-mon[29215]: pgmap v181: 97 pgs: 97 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 74 KiB/s wr, 37 op/s 2024-04-23T18:32:08.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:07 smithi044 ceph-mon[32458]: pgmap v182: 97 pgs: 97 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T18:32:08.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:07 smithi155 ceph-mon[33468]: pgmap v182: 97 pgs: 97 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T18:32:08.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:07 smithi019 ceph-mon[29215]: pgmap v182: 97 pgs: 97 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T18:32:10.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:09 smithi044 ceph-mon[32458]: pgmap v183: 97 pgs: 97 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T18:32:10.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:09 smithi155 ceph-mon[33468]: pgmap v183: 97 pgs: 97 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T18:32:10.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:09 smithi019 ceph-mon[29215]: pgmap v183: 97 pgs: 97 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T18:32:12.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:11 smithi044 ceph-mon[32458]: pgmap v184: 97 pgs: 97 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 76 KiB/s wr, 37 op/s 2024-04-23T18:32:12.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:11 smithi155 ceph-mon[33468]: pgmap v184: 97 pgs: 97 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 76 KiB/s wr, 37 op/s 2024-04-23T18:32:12.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:11 smithi019 ceph-mon[29215]: pgmap v184: 97 pgs: 97 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 76 KiB/s wr, 37 op/s 2024-04-23T18:32:14.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:13 smithi044 ceph-mon[32458]: pgmap v185: 97 pgs: 97 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-04-23T18:32:14.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:13 smithi155 ceph-mon[33468]: pgmap v185: 97 pgs: 97 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-04-23T18:32:14.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:13 smithi019 ceph-mon[29215]: pgmap v185: 97 pgs: 97 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-04-23T18:32:16.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:15 smithi044 ceph-mon[32458]: pgmap v186: 97 pgs: 97 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-04-23T18:32:16.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:15 smithi155 ceph-mon[33468]: pgmap v186: 97 pgs: 97 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-04-23T18:32:16.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:15 smithi019 ceph-mon[29215]: pgmap v186: 97 pgs: 97 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-04-23T18:32:18.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:17 smithi044 ceph-mon[32458]: pgmap v187: 97 pgs: 97 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T18:32:18.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:17 smithi155 ceph-mon[33468]: pgmap v187: 97 pgs: 97 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T18:32:18.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:17 smithi019 ceph-mon[29215]: pgmap v187: 97 pgs: 97 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T18:32:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.030+0000 7fa6503bb640 1 -- 172.21.15.19:0/68947540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6481017f0 msgr2=0x7fa648101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.030+0000 7fa6503bb640 1 --2- 172.21.15.19:0/68947540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6481017f0 0x7fa648101bd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa644009900 tx=0x7fa64402a410 comp rx=0 tx=0).stop 2024-04-23T18:32:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.030+0000 7fa6503bb640 1 -- 172.21.15.19:0/68947540 shutdown_connections 2024-04-23T18:32:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.030+0000 7fa6503bb640 1 --2- 172.21.15.19:0/68947540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6481021a0 0x7fa648105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.030+0000 7fa6503bb640 1 --2- 172.21.15.19:0/68947540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6481063a0 0x7fa64810df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.030+0000 7fa6503bb640 1 --2- 172.21.15.19:0/68947540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6481017f0 0x7fa648101bd0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.030+0000 7fa6503bb640 1 -- 172.21.15.19:0/68947540 >> 172.21.15.19:0/68947540 conn(0x7fa648077390 msgr2=0x7fa6480fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.030+0000 7fa6503bb640 1 -- 172.21.15.19:0/68947540 shutdown_connections 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.030+0000 7fa6503bb640 1 -- 172.21.15.19:0/68947540 wait complete. 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa6503bb640 1 Processor -- start 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa6503bb640 1 -- start start 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa6503bb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6481017f0 0x7fa6480fc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa6503bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6481063a0 0x7fa6480fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa6503bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6480fd2a0 0x7fa6481a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa6503bb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa648111310 con 0x7fa6480fd2a0 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa6503bb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa648111190 con 0x7fa6481063a0 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa6503bb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa648111490 con 0x7fa6481017f0 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa64e958640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6480fd2a0 0x7fa6481a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa64e958640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6480fd2a0 0x7fa6481a6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40676/0 (socket says 172.21.15.19:40676) 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa64e958640 1 -- 172.21.15.19:0/903338188 learned_addr learned my addr 172.21.15.19:0/903338188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa64e157640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6481017f0 0x7fa6480fc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa64d956640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6481063a0 0x7fa6480fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa64e958640 1 -- 172.21.15.19:0/903338188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6481017f0 msgr2=0x7fa6480fc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.031+0000 7fa64e958640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6481017f0 0x7fa6480fc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.032+0000 7fa64e958640 1 -- 172.21.15.19:0/903338188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6481063a0 msgr2=0x7fa6480fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.032+0000 7fa64e958640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6481063a0 0x7fa6480fcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.032+0000 7fa64e958640 1 -- 172.21.15.19:0/903338188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6481a71b0 con 0x7fa6480fd2a0 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.032+0000 7fa64e958640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6480fd2a0 0x7fa6481a6b10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa64000eaa0 tx=0x7fa64000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.032+0000 7fa63f7fe640 1 -- 172.21.15.19:0/903338188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64000cdb0 con 0x7fa6480fd2a0 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.032+0000 7fa63f7fe640 1 -- 172.21.15.19:0/903338188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa640005020 con 0x7fa6480fd2a0 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.032+0000 7fa6503bb640 1 -- 172.21.15.19:0/903338188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6481a74a0 con 0x7fa6480fd2a0 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.032+0000 7fa63f7fe640 1 -- 172.21.15.19:0/903338188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400116d0 con 0x7fa6480fd2a0 2024-04-23T18:32:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.032+0000 7fa6503bb640 1 -- 172.21.15.19:0/903338188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6481a79f0 con 0x7fa6480fd2a0 2024-04-23T18:32:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.034+0000 7fa63f7fe640 1 -- 172.21.15.19:0/903338188 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa6400051d0 con 0x7fa6480fd2a0 2024-04-23T18:32:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.034+0000 7fa6531ab740 1 Processor -- start 2024-04-23T18:32:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa6531ab740 1 -- start start 2024-04-23T18:32:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 0x5561ed126a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126f40 0x5561ed1312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed131800 0x5561ed133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa6531ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5561ed08b970 con 0x5561ed126f40 2024-04-23T18:32:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa6531ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5561ed0284a0 con 0x5561ed131800 2024-04-23T18:32:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa6531ab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5561ed087930 con 0x5561ed126620 2024-04-23T18:32:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64e157640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126f40 0x5561ed1312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64e157640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126f40 0x5561ed1312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40684/0 (socket says 172.21.15.19:40684) 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64e157640 1 -- 172.21.15.19:0/3946749774 learned_addr learned my addr 172.21.15.19:0/3946749774 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64d956640 1 --2- 172.21.15.19:0/3946749774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 0x5561ed126a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa63f7fe640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6340a3dc0 0x7fa6340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64d956640 1 --2- 172.21.15.19:0/3946749774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed131800 0x5561ed133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64e157640 1 -- 172.21.15.19:0/3946749774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 msgr2=0x5561ed126a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64e157640 1 --2- 172.21.15.19:0/3946749774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 0x5561ed126a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64e157640 1 -- 172.21.15.19:0/3946749774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed131800 msgr2=0x5561ed133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64e157640 1 --2- 172.21.15.19:0/3946749774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed131800 0x5561ed133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64e157640 1 -- 172.21.15.19:0/3946749774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ecd9e210 con 0x5561ed126f40 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64d956640 1 --2- 172.21.15.19:0/3946749774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 0x5561ed126a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa64e958640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6340a3dc0 0x7fa6340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.035+0000 7fa63f7fe640 1 -- 172.21.15.19:0/903338188 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa6400d08c0 con 0x7fa6480fd2a0 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.036+0000 7fa64e157640 1 --2- 172.21.15.19:0/3946749774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126f40 0x5561ed1312c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa644009900 tx=0x7fa644032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17edf3ceae8bc109 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.036+0000 7fa63cbfe640 1 -- 172.21.15.19:0/3946749774 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644040070 con 0x5561ed126f40 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.036+0000 7fa63cbfe640 1 -- 172.21.15.19:0/3946749774 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa644004030 con 0x5561ed126f40 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.036+0000 7fa63cbfe640 1 -- 172.21.15.19:0/3946749774 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644035040 con 0x5561ed126f40 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.036+0000 7fa64e958640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6340a3dc0 0x7fa6340a6280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa6400622f0 tx=0x7fa640097040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 -- 172.21.15.19:0/3946749774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126f40 msgr2=0x5561ed1312c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 --2- 172.21.15.19:0/3946749774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126f40 0x5561ed1312c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa644009900 tx=0x7fa644032bd0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 -- 172.21.15.19:0/3946749774 shutdown_connections 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 --2- 172.21.15.19:0/3946749774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed131800 0x5561ed133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 --2- 172.21.15.19:0/3946749774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126f40 0x5561ed1312c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 --2- 172.21.15.19:0/3946749774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 0x5561ed126a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 -- 172.21.15.19:0/3946749774 >> 172.21.15.19:0/3946749774 conn(0x5561ed122b30 msgr2=0x5561ed122f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 -- 172.21.15.19:0/3946749774 shutdown_connections 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 -- 172.21.15.19:0/3946749774 wait complete. 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 -- start start 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 Processor -- start 2024-04-23T18:32:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed131170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126f40 0x5561ed12e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed131800 0x5561ed12e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5561ed1343d0 con 0x5561ed126620 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5561ed0260b0 con 0x5561ed131800 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5561ed134550 con 0x5561ed126f40 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa6531ab740 10 client.0 _ll_register_callbacks cb 0x5561ed013d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa64d956640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed131170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa64d956640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed131170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40696/0 (socket says 172.21.15.19:40696) 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.037+0000 7fa64d956640 1 -- 172.21.15.19:0/4099765657 learned_addr learned my addr 172.21.15.19:0/4099765657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa64e157640 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126f40 0x5561ed12e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa64d956640 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed131800 0x5561ed12e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa64d956640 1 -- 172.21.15.19:0/4099765657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126f40 msgr2=0x5561ed12e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa64d956640 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126f40 0x5561ed12e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa64d956640 1 -- 172.21.15.19:0/4099765657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed131800 msgr2=0x5561ed12e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa64d956640 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed131800 0x5561ed12e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa64d956640 1 -- 172.21.15.19:0/4099765657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ece6a620 con 0x5561ed126620 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa64d956640 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed131170 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa638010e00 tx=0x7fa6380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ed4c28f294760f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa631ffb640 1 -- 172.21.15.19:0/4099765657 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63800fe60 con 0x5561ed126620 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa6531ab740 1 -- 172.21.15.19:0/4099765657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa644004030 con 0x5561ed126620 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa631ffb640 1 -- 172.21.15.19:0/4099765657 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa638004020 con 0x5561ed126620 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa631ffb640 1 -- 172.21.15.19:0/4099765657 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa638004220 con 0x5561ed126620 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa6531ab740 10 client.16461 fetch_fsmap learned FSMap version 27 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa6531ab740 1 -- 172.21.15.19:0/4099765657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561ed140390 con 0x5561ed126620 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.038+0000 7fa631ffb640 1 -- 172.21.15.19:0/4099765657 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa644004030 con 0x5561ed126620 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.039+0000 7fa631ffb640 1 -- 172.21.15.19:0/4099765657 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa638020040 con 0x5561ed126620 2024-04-23T18:32:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.039+0000 7fa6531ab740 10 client.16461 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.039+0000 7fa6531ab740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.039+0000 7fa6531ab740 20 client.16461 populate_metadata read hostname 'smithi019' 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.039+0000 7fa6531ab740 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5561ed141800 0x5561ed143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.039+0000 7fa6531ab740 4 client.16461 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.039+0000 7fa6531ab740 1 -- 172.21.15.19:0/4099765657 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561ed099390 con 0x5561ed141800 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.039+0000 7fa64e157640 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5561ed141800 0x5561ed143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.039+0000 7fa631ffb640 10 client.16461 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.039+0000 7fa64e157640 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5561ed141800 0x5561ed143bf0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=798664c16669e955 server_cookie=6d040e7f1698e5c7 in_seq=0 out_seq=0 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa631ffb640 1 -- 172.21.15.19:0/4099765657 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561ed099390 con 0x5561ed141800 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa631ffb640 10 client.16461 handle_command_reply: tid=0 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 client.16461 shutdown 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 -- 172.21.15.19:0/4099765657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 msgr2=0x5561ed131170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed131170 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa638010e00 tx=0x7fa6380046c0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 -- 172.21.15.19:0/4099765657 shutdown_connections 2024-04-23T18:32:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5561ed141800 0x5561ed143bf0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed131800 0x5561ed12e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa631ffb640 0 client.16461 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126f40 0x5561ed12e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 --2- 172.21.15.19:0/4099765657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed131170 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 -- 172.21.15.19:0/4099765657 >> 172.21.15.19:0/4099765657 conn(0x5561ed122b30 msgr2=0x5561ed128860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 -- 172.21.15.19:0/4099765657 shutdown_connections 2024-04-23T18:32:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.040+0000 7fa6531ab740 1 -- 172.21.15.19:0/4099765657 wait complete. 2024-04-23T18:32:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.041+0000 7fa6531ab740 20 client.16461 trim_cache size 0 max 16384 2024-04-23T18:32:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.055+0000 7fa6531ab740 1 Processor -- start 2024-04-23T18:32:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.055+0000 7fa6531ab740 1 -- start start 2024-04-23T18:32:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 0x5561ed130ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 0x5561ed1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed131800 0x5561ed149a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa6531ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481afd40 con 0x5561ed131800 2024-04-23T18:32:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa6531ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481b1880 con 0x5561ed126f40 2024-04-23T18:32:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa6531ab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481b0410 con 0x5561ed126620 2024-04-23T18:32:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64d956640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed131800 0x5561ed149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64d956640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed131800 0x5561ed149a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40702/0 (socket says 172.21.15.19:40702) 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64d956640 1 -- 172.21.15.19:0/2980291269 learned_addr learned my addr 172.21.15.19:0/2980291269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64e157640 1 --2- 172.21.15.19:0/2980291269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 0x5561ed1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64d956640 1 --2- 172.21.15.19:0/2980291269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 0x5561ed130ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64d956640 1 -- 172.21.15.19:0/2980291269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 msgr2=0x5561ed130ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64d956640 1 --2- 172.21.15.19:0/2980291269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 0x5561ed130ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64d956640 1 -- 172.21.15.19:0/2980291269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 msgr2=0x5561ed1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64d956640 1 --2- 172.21.15.19:0/2980291269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 0x5561ed1494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64d956640 1 -- 172.21.15.19:0/2980291269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ed126d70 con 0x5561ed131800 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64e157640 1 --2- 172.21.15.19:0/2980291269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 0x5561ed1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa64d956640 1 --2- 172.21.15.19:0/2980291269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed131800 0x5561ed149a00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa638010d80 tx=0x7fa638005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e96a428cd8ecca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa633fff640 1 -- 172.21.15.19:0/2980291269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638004730 con 0x5561ed131800 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa633fff640 1 -- 172.21.15.19:0/2980291269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63800fe60 con 0x5561ed131800 2024-04-23T18:32:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.056+0000 7fa633fff640 1 -- 172.21.15.19:0/2980291269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638005620 con 0x5561ed131800 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 -- 172.21.15.19:0/2980291269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed131800 msgr2=0x5561ed149a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 --2- 172.21.15.19:0/2980291269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed131800 0x5561ed149a00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa638010d80 tx=0x7fa638005970 comp rx=0 tx=0).stop 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 -- 172.21.15.19:0/2980291269 shutdown_connections 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 --2- 172.21.15.19:0/2980291269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 0x5561ed1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 --2- 172.21.15.19:0/2980291269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed131800 0x5561ed149a00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 --2- 172.21.15.19:0/2980291269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed126620 0x5561ed130ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 -- 172.21.15.19:0/2980291269 >> 172.21.15.19:0/2980291269 conn(0x5561ed122b30 msgr2=0x5561ed12e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 -- 172.21.15.19:0/2980291269 shutdown_connections 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 -- 172.21.15.19:0/2980291269 wait complete. 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 -- start start 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.057+0000 7fa6531ab740 1 Processor -- start 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed139560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 0x5561ed139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa6531ab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed131800 0x5561ed136980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa6531ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481b11b0 con 0x5561ed126620 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa6531ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481b0ae0 con 0x5561ed126f40 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa6531ab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481add20 con 0x5561ed131800 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa6531ab740 10 client.0 _ll_register_callbacks cb 0x7fa638010c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa64d956640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa64d956640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed139560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40714/0 (socket says 172.21.15.19:40714) 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa64d956640 1 -- 172.21.15.19:0/458937901 learned_addr learned my addr 172.21.15.19:0/458937901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa64e157640 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 0x5561ed139aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa64d956640 1 -- 172.21.15.19:0/458937901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed131800 msgr2=0x5561ed136980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa64d956640 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed131800 0x5561ed136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa64d956640 1 -- 172.21.15.19:0/458937901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 msgr2=0x5561ed139aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa64d956640 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 0x5561ed139aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa64d956640 1 -- 172.21.15.19:0/458937901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ed137180 con 0x5561ed126620 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa64d956640 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed139560 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa6380104f0 tx=0x7fa638010bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d74f9373f841d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa631ffb640 1 -- 172.21.15.19:0/458937901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638009b90 con 0x5561ed126620 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa631ffb640 1 -- 172.21.15.19:0/458937901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa63801a050 con 0x5561ed126620 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.058+0000 7fa6531ab740 1 -- 172.21.15.19:0/458937901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa63800fe60 con 0x5561ed126620 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa631ffb640 1 -- 172.21.15.19:0/458937901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa638004f10 con 0x5561ed126620 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa6531ab740 10 client.16467 fetch_fsmap learned FSMap version 27 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa6531ab740 1 -- 172.21.15.19:0/458937901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561ed1a77f0 con 0x5561ed126620 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa631ffb640 1 -- 172.21.15.19:0/458937901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63800fe60 con 0x5561ed126620 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa631ffb640 1 -- 172.21.15.19:0/458937901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa638039840 con 0x5561ed126620 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa6531ab740 10 client.16467 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa6531ab740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa6531ab740 20 client.16467 populate_metadata read hostname 'smithi019' 2024-04-23T18:32:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa6531ab740 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5561ed1a7e00 0x5561ed1aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa6531ab740 4 client.16467 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa6531ab740 1 -- 172.21.15.19:0/458937901 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561ed093200 con 0x5561ed1a7e00 2024-04-23T18:32:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa64e157640 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5561ed1a7e00 0x5561ed1aa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa64e157640 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5561ed1a7e00 0x5561ed1aa1f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a01892a2da47d3e server_cookie=632147a72f4ad933 in_seq=0 out_seq=0 2024-04-23T18:32:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.059+0000 7fa631ffb640 10 client.16467 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:19.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:18 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:19.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:18 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:19.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:18 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:19.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:18 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:32:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.071+0000 7fa631ffb640 1 -- 172.21.15.19:0/458937901 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561ed093200 con 0x5561ed1a7e00 2024-04-23T18:32:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.071+0000 7fa631ffb640 10 client.16467 handle_command_reply: tid=0 2024-04-23T18:32:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 client.16467 shutdown 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 -- 172.21.15.19:0/458937901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 msgr2=0x5561ed139560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed139560 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa6380104f0 tx=0x7fa638010bc0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 -- 172.21.15.19:0/458937901 shutdown_connections 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5561ed1a7e00 0x5561ed1aa1f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5561ed126f40 0x5561ed139aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561ed131800 0x5561ed136980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa631ffb640 0 client.16467 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 --2- 172.21.15.19:0/458937901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561ed126620 0x5561ed139560 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 -- 172.21.15.19:0/458937901 >> 172.21.15.19:0/458937901 conn(0x5561ed122b30 msgr2=0x5561ed121420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 -- 172.21.15.19:0/458937901 shutdown_connections 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 1 -- 172.21.15.19:0/458937901 wait complete. 2024-04-23T18:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6531ab740 20 client.16467 trim_cache size 0 max 16384 2024-04-23T18:32:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:32:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:32:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:32:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6503bb640 1 -- 172.21.15.19:0/903338188 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6340a3dc0 msgr2=0x7fa6340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6503bb640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6340a3dc0 0x7fa6340a6280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa6400622f0 tx=0x7fa640097040 comp rx=0 tx=0).stop 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6503bb640 1 -- 172.21.15.19:0/903338188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6480fd2a0 msgr2=0x7fa6481a6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6503bb640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6480fd2a0 0x7fa6481a6b10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa64000eaa0 tx=0x7fa64000ef60 comp rx=0 tx=0).stop 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6503bb640 1 -- 172.21.15.19:0/903338188 shutdown_connections 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6503bb640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6340a3dc0 0x7fa6340a6280 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6503bb640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6481063a0 0x7fa6480fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6503bb640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6480fd2a0 0x7fa6481a6b10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6503bb640 1 --2- 172.21.15.19:0/903338188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6481017f0 0x7fa6480fc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.072+0000 7fa6503bb640 1 -- 172.21.15.19:0/903338188 >> 172.21.15.19:0/903338188 conn(0x7fa648077390 msgr2=0x7fa648104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.073+0000 7fa6503bb640 1 -- 172.21.15.19:0/903338188 shutdown_connections 2024-04-23T18:32:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:19.073+0000 7fa6503bb640 1 -- 172.21.15.19:0/903338188 wait complete. 2024-04-23T18:32:19.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:18 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:19.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:18 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:19.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:18 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:19.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:18 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:32:19.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:18 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:19.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:18 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:19.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:18 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:19.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:18 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:32:20.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:19 smithi044 ceph-mon[32458]: pgmap v188: 97 pgs: 97 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T18:32:20.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:19 smithi155 ceph-mon[33468]: pgmap v188: 97 pgs: 97 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T18:32:20.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:19 smithi019 ceph-mon[29215]: pgmap v188: 97 pgs: 97 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T18:32:22.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:21 smithi044 ceph-mon[32458]: pgmap v189: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-23T18:32:22.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:21 smithi155 ceph-mon[33468]: pgmap v189: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-23T18:32:22.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:21 smithi019 ceph-mon[29215]: pgmap v189: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-23T18:32:24.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:23 smithi044 ceph-mon[32458]: pgmap v190: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:24.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:23 smithi155 ceph-mon[33468]: pgmap v190: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:24.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:23 smithi019 ceph-mon[29215]: pgmap v190: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:26.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:25 smithi044 ceph-mon[32458]: pgmap v191: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:26.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:25 smithi155 ceph-mon[33468]: pgmap v191: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:26.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:25 smithi019 ceph-mon[29215]: pgmap v191: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:28.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:27 smithi044 ceph-mon[32458]: pgmap v192: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:28.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:27 smithi155 ceph-mon[33468]: pgmap v192: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:28.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:27 smithi019 ceph-mon[29215]: pgmap v192: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:30.061 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:29 smithi019 ceph-mon[29215]: pgmap v193: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:30.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:29 smithi044 ceph-mon[32458]: pgmap v193: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:30.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:29 smithi155 ceph-mon[33468]: pgmap v193: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:32.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:31 smithi044 ceph-mon[32458]: pgmap v194: 97 pgs: 97 active+clean; 3.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:32:32.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:31 smithi155 ceph-mon[33468]: pgmap v194: 97 pgs: 97 active+clean; 3.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:32:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 -- 172.21.15.19:0/949912930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bbc107330 msgr2=0x7f0bbc10ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 --2- 172.21.15.19:0/949912930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bbc107330 0x7f0bbc10ee90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009cd0 tx=0x7f0bb402a8d0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 -- 172.21.15.19:0/949912930 shutdown_connections 2024-04-23T18:32:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 --2- 172.21.15.19:0/949912930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bbc1010d0 0x7f0bbc106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 --2- 172.21.15.19:0/949912930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bbc107330 0x7f0bbc10ee90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 --2- 172.21.15.19:0/949912930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bbc100720 0x7f0bbc100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 -- 172.21.15.19:0/949912930 >> 172.21.15.19:0/949912930 conn(0x7f0bbc077390 msgr2=0x7f0bbc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 -- 172.21.15.19:0/949912930 shutdown_connections 2024-04-23T18:32:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 -- 172.21.15.19:0/949912930 wait complete. 2024-04-23T18:32:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 Processor -- start 2024-04-23T18:32:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.172+0000 7f0bc27cd640 1 -- start start 2024-04-23T18:32:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bc27cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bbc100720 0x7f0bbc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bc27cd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bbc107330 0x7f0bbc198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bc27cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bbc19cf30 0x7f0bbc19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bc27cd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1121e0 con 0x7f0bbc100720 2024-04-23T18:32:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bc27cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc112060 con 0x7f0bbc19cf30 2024-04-23T18:32:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bc27cd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc112360 con 0x7f0bbc107330 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bbc100720 0x7f0bbc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bbc100720 0x7f0bbc1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48128/0 (socket says 172.21.15.19:48128) 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bbbfff640 1 -- 172.21.15.19:0/2073249543 learned_addr learned my addr 172.21.15.19:0/2073249543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bbc107330 0x7f0bbc198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bbbfff640 1 -- 172.21.15.19:0/2073249543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bbc107330 msgr2=0x7f0bbc198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bbc107330 0x7f0bbc198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bbc19cf30 0x7f0bbc19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bbbfff640 1 -- 172.21.15.19:0/2073249543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bbc19cf30 msgr2=0x7f0bbc19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bbc19cf30 0x7f0bbc19d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bbbfff640 1 -- 172.21.15.19:0/2073249543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc19da20 con 0x7f0bbc100720 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.173+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bbc107330 0x7f0bbc198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.174+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bbc100720 0x7f0bbc1986c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00e9f0 tx=0x7f0bac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.174+0000 7f0bb97fa640 1 -- 172.21.15.19:0/2073249543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac00cd80 con 0x7f0bbc100720 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.174+0000 7f0bb97fa640 1 -- 172.21.15.19:0/2073249543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bac0108e0 con 0x7f0bbc100720 2024-04-23T18:32:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.174+0000 7f0bc27cd640 1 -- 172.21.15.19:0/2073249543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc19dd10 con 0x7f0bbc100720 2024-04-23T18:32:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.174+0000 7f0bb97fa640 1 -- 172.21.15.19:0/2073249543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac018980 con 0x7f0bbc100720 2024-04-23T18:32:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.174+0000 7f0bc27cd640 1 -- 172.21.15.19:0/2073249543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc1a5500 con 0x7f0bbc100720 2024-04-23T18:32:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.176+0000 7f0bb97fa640 1 -- 172.21.15.19:0/2073249543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0bac010430 con 0x7f0bbc100720 2024-04-23T18:32:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bc55c3740 1 Processor -- start 2024-04-23T18:32:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bc55c3740 1 -- start start 2024-04-23T18:32:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x5557506018e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x5557506106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 0x555750613000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555750566970 con 0x555750601e20 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5557505034a0 con 0x555750601500 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555750562930 con 0x555750610c10 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bb97fa640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ba40a3dc0 0x7f0ba40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x5557506106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x5557506106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48136/0 (socket says 172.21.15.19:48136) 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bbbfff640 1 -- 172.21.15.19:0/937231842 learned_addr learned my addr 172.21.15.19:0/937231842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ba40a3dc0 0x7f0ba40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/937231842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 0x555750613000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/937231842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x5557506018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bb97fa640 1 -- 172.21.15.19:0/2073249543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0bac0d0c70 con 0x7f0bbc100720 2024-04-23T18:32:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.177+0000 7f0bbbfff640 1 -- 172.21.15.19:0/937231842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 msgr2=0x555750613000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.178+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/937231842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 0x555750613000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.178+0000 7f0bbbfff640 1 -- 172.21.15.19:0/937231842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 msgr2=0x5557506018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.178+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/937231842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x5557506018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.178+0000 7f0bbbfff640 1 -- 172.21.15.19:0/937231842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555750279210 con 0x555750601e20 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.178+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/937231842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 0x555750613000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.178+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ba40a3dc0 0x7f0ba40a6280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4005cc0 tx=0x7f0bb4009880 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.178+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/937231842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x5557506106d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0bac098ff0 tx=0x7f0bac01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ddf64cc9355dec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.178+0000 7f0ba27fc640 1 -- 172.21.15.19:0/937231842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac026040 con 0x555750601e20 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.178+0000 7f0ba27fc640 1 -- 172.21.15.19:0/937231842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bac0a0f90 con 0x555750601e20 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.178+0000 7f0ba27fc640 1 -- 172.21.15.19:0/937231842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac0ce610 con 0x555750601e20 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 -- 172.21.15.19:0/937231842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 msgr2=0x5557506106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/937231842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x5557506106d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0bac098ff0 tx=0x7f0bac01f000 comp rx=0 tx=0).stop 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 -- 172.21.15.19:0/937231842 shutdown_connections 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/937231842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 0x555750613000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/937231842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x5557506106d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/937231842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x5557506018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 -- 172.21.15.19:0/937231842 >> 172.21.15.19:0/937231842 conn(0x5557505fdb30 msgr2=0x5557505fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 -- 172.21.15.19:0/937231842 shutdown_connections 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 -- 172.21.15.19:0/937231842 wait complete. 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 -- start start 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.179+0000 7f0bc55c3740 1 Processor -- start 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x555750610570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750601e20 0x55575060d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750610c10 0x55575060ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5557506137e0 con 0x555750610c10 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5557505010b0 con 0x555750601500 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555750613960 con 0x555750601e20 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc55c3740 10 client.0 _ll_register_callbacks cb 0x5557504eed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc0d6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750610c10 0x55575060ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc0d6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750610c10 0x55575060ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48148/0 (socket says 172.21.15.19:48148) 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc0d6a640 1 -- 172.21.15.19:0/360407599 learned_addr learned my addr 172.21.15.19:0/360407599 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x555750610570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750601e20 0x55575060d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc0d6a640 1 -- 172.21.15.19:0/360407599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750601e20 msgr2=0x55575060d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750601e20 0x55575060d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc0d6a640 1 -- 172.21.15.19:0/360407599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 msgr2=0x555750610570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x555750610570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc0d6a640 1 -- 172.21.15.19:0/360407599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555750345620 con 0x555750610c10 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750610c10 0x55575060ddc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4005e00 tx=0x7f0bb4047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaf97a2115a17da server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0b9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.180+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/360407599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb404f030 con 0x555750610c10 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/360407599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0bb4046040 con 0x555750610c10 2024-04-23T18:32:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0bc55c3740 1 -- 172.21.15.19:0/360407599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bac0a0f90 con 0x555750610c10 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/360407599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0bb4003d40 con 0x555750610c10 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0bc55c3740 10 client.16479 fetch_fsmap learned FSMap version 27 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0bc55c3740 1 -- 172.21.15.19:0/360407599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55575061f810 con 0x555750610c10 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/360407599 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bac0a0f90 con 0x555750610c10 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/360407599 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0bb4049040 con 0x555750610c10 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0bc55c3740 10 client.16479 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0bc55c3740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0bc55c3740 20 client.16479 populate_metadata read hostname 'smithi019' 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555750620d40 0x555750623130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0bc55c3740 4 client.16479 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0bc55c3740 1 -- 172.21.15.19:0/360407599 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555750574390 con 0x555750620d40 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.181+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555750620d40 0x555750623130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.182+0000 7f0b9b7fe640 10 client.16479 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.182+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555750620d40 0x555750623130 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc56dedee52de86 server_cookie=f86d55d6c28ba61c in_seq=0 out_seq=0 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.182+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/360407599 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555750574390 con 0x555750620d40 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.182+0000 7f0b9b7fe640 10 client.16479 handle_command_reply: tid=0 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 client.16479 shutdown 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 -- 172.21.15.19:0/360407599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750610c10 msgr2=0x55575060ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750610c10 0x55575060ddc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4005e00 tx=0x7f0bb4047000 comp rx=0 tx=0).stop 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 -- 172.21.15.19:0/360407599 shutdown_connections 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555750620d40 0x555750623130 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750610c10 0x55575060ddc0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750601e20 0x55575060d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/360407599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x555750610570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 -- 172.21.15.19:0/360407599 >> 172.21.15.19:0/360407599 conn(0x5557505fdb30 msgr2=0x555750603940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0b9b7fe640 0 client.16479 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 -- 172.21.15.19:0/360407599 shutdown_connections 2024-04-23T18:32:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 1 -- 172.21.15.19:0/360407599 wait complete. 2024-04-23T18:32:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.183+0000 7f0bc55c3740 20 client.16479 trim_cache size 0 max 16384 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bc55c3740 1 Processor -- start 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bc55c3740 1 -- start start 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750601500 0x55575061c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750608d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750610c10 0x5557506100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1ab8d0 con 0x555750601e20 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1ad410 con 0x555750610c10 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1abfa0 con 0x555750601500 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bbb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750608d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bbb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750608d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48162/0 (socket says 172.21.15.19:48162) 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.198+0000 7f0bbb7fe640 1 -- 172.21.15.19:0/924065229 learned_addr learned my addr 172.21.15.19:0/924065229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/924065229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750601500 0x55575061c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/924065229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750610c10 0x5557506100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0bbb7fe640 1 -- 172.21.15.19:0/924065229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750601500 msgr2=0x55575061c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/924065229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750601500 0x55575061c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0bbb7fe640 1 -- 172.21.15.19:0/924065229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750610c10 msgr2=0x5557506100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/924065229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750610c10 0x5557506100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0bbb7fe640 1 -- 172.21.15.19:0/924065229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557505f92e0 con 0x555750601e20 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/924065229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750610c10 0x5557506100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/924065229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750608d80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0ffa10 tx=0x7f0bb00123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcef49388fa72b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0ba1ffb640 1 -- 172.21.15.19:0/924065229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb001b640 con 0x555750601e20 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0ba1ffb640 1 -- 172.21.15.19:0/924065229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb001a890 con 0x555750601e20 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.199+0000 7f0ba1ffb640 1 -- 172.21.15.19:0/924065229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00109c0 con 0x555750601e20 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 -- 172.21.15.19:0/924065229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 msgr2=0x555750608d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/924065229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750608d80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0ffa10 tx=0x7f0bb00123f0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 -- 172.21.15.19:0/924065229 shutdown_connections 2024-04-23T18:32:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/924065229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750610c10 0x5557506100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/924065229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750608d80 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/924065229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750601500 0x55575061c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 -- 172.21.15.19:0/924065229 >> 172.21.15.19:0/924065229 conn(0x5557505fdb30 msgr2=0x555750608670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 -- 172.21.15.19:0/924065229 shutdown_connections 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 -- 172.21.15.19:0/924065229 wait complete. 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 -- start start 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.200+0000 7f0bc55c3740 1 Processor -- start 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x555750618c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750619160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bc55c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 0x555750609c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1acd40 con 0x555750601e20 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1ac670 con 0x555750601500 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bc55c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1a98d0 con 0x555750610c10 2024-04-23T18:32:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bc55c3740 10 client.0 _ll_register_callbacks cb 0x7f0bb4031420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750619160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750619160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48174/0 (socket says 172.21.15.19:48174) 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbb7fe640 1 -- 172.21.15.19:0/1761437298 learned_addr learned my addr 172.21.15.19:0/1761437298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x555750618c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 0x555750609c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbb7fe640 1 -- 172.21.15.19:0/1761437298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 msgr2=0x555750609c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 0x555750609c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbb7fe640 1 -- 172.21.15.19:0/1761437298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 msgr2=0x555750618c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x555750618c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbb7fe640 1 -- 172.21.15.19:0/1761437298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575060a490 con 0x555750601e20 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bc0d6a640 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 0x555750609c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x555750618c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0bbb7fe640 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750619160 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0006010 tx=0x7f0bb001a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a429555f1aa5000a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.201+0000 7f0b9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:32:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/1761437298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0029020 con 0x555750601e20 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/1761437298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0bb001aa40 con 0x555750601e20 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0bc55c3740 1 -- 172.21.15.19:0/1761437298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb001a890 con 0x555750601e20 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/1761437298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0bb0034050 con 0x555750601e20 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0bc55c3740 10 client.16485 fetch_fsmap learned FSMap version 27 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0bc55c3740 1 -- 172.21.15.19:0/1761437298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555750686c10 con 0x555750601e20 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/1761437298 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb001a890 con 0x555750601e20 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/1761437298 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0bb0019070 con 0x555750601e20 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0bc55c3740 10 client.16485 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0bc55c3740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0bc55c3740 20 client.16485 populate_metadata read hostname 'smithi019' 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555750687350 0x555750689770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0bc55c3740 4 client.16485 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0bc55c3740 1 -- 172.21.15.19:0/1761437298 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55575056e200 con 0x555750687350 2024-04-23T18:32:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.202+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555750687350 0x555750689770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.203+0000 7f0b9b7fe640 10 client.16485 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.203+0000 7f0bbbfff640 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555750687350 0x555750689770 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e05ccbe65736d7c server_cookie=56b3ea15a0cc0a3f in_seq=0 out_seq=0 2024-04-23T18:32:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0b9b7fe640 1 -- 172.21.15.19:0/1761437298 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55575056e200 con 0x555750687350 2024-04-23T18:32:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0b9b7fe640 10 client.16485 handle_command_reply: tid=0 2024-04-23T18:32:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 client.16485 shutdown 2024-04-23T18:32:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 -- 172.21.15.19:0/1761437298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 msgr2=0x555750619160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750619160 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0006010 tx=0x7f0bb001a460 comp rx=0 tx=0).stop 2024-04-23T18:32:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 -- 172.21.15.19:0/1761437298 shutdown_connections 2024-04-23T18:32:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555750687350 0x555750689770 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555750610c10 0x555750609c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555750601e20 0x555750619160 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 --2- 172.21.15.19:0/1761437298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555750601500 0x555750618c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0b9b7fe640 0 client.16485 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 -- 172.21.15.19:0/1761437298 >> 172.21.15.19:0/1761437298 conn(0x5557505fdb30 msgr2=0x555750608f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 -- 172.21.15.19:0/1761437298 shutdown_connections 2024-04-23T18:32:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 1 -- 172.21.15.19:0/1761437298 wait complete. 2024-04-23T18:32:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc55c3740 20 client.16485 trim_cache size 0 max 16384 2024-04-23T18:32:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:32:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:32:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:32:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:32:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 -- 172.21.15.19:0/2073249543 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ba40a3dc0 msgr2=0x7f0ba40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ba40a3dc0 0x7f0ba40a6280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4005cc0 tx=0x7f0bb4009880 comp rx=0 tx=0).stop 2024-04-23T18:32:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 -- 172.21.15.19:0/2073249543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bbc100720 msgr2=0x7f0bbc1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bbc100720 0x7f0bbc1986c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00e9f0 tx=0x7f0bac00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 -- 172.21.15.19:0/2073249543 shutdown_connections 2024-04-23T18:32:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ba40a3dc0 0x7f0ba40a6280 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bbc19cf30 0x7f0bbc19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bbc107330 0x7f0bbc198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 --2- 172.21.15.19:0/2073249543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bbc100720 0x7f0bbc1986c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 -- 172.21.15.19:0/2073249543 >> 172.21.15.19:0/2073249543 conn(0x7f0bbc077390 msgr2=0x7f0bbc0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 -- 172.21.15.19:0/2073249543 shutdown_connections 2024-04-23T18:32:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:32.218+0000 7f0bc27cd640 1 -- 172.21.15.19:0/2073249543 wait complete. 2024-04-23T18:32:32.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:31 smithi019 ceph-mon[29215]: pgmap v194: 97 pgs: 97 active+clean; 3.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:32:34.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:33 smithi044 ceph-mon[32458]: pgmap v195: 97 pgs: 97 active+clean; 3.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-23T18:32:34.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:33 smithi155 ceph-mon[33468]: pgmap v195: 97 pgs: 97 active+clean; 3.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-23T18:32:34.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:33 smithi019 ceph-mon[29215]: pgmap v195: 97 pgs: 97 active+clean; 3.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-23T18:32:36.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:35 smithi155 ceph-mon[33468]: pgmap v196: 97 pgs: 97 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:36.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:35 smithi019 ceph-mon[29215]: pgmap v196: 97 pgs: 97 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:36.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:35 smithi044 ceph-mon[32458]: pgmap v196: 97 pgs: 97 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:38.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:37 smithi155 ceph-mon[33468]: pgmap v197: 97 pgs: 97 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:32:38.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:37 smithi019 ceph-mon[29215]: pgmap v197: 97 pgs: 97 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:32:38.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:37 smithi044 ceph-mon[32458]: pgmap v197: 97 pgs: 97 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:32:40.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:39 smithi155 ceph-mon[33468]: pgmap v198: 97 pgs: 97 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:32:40.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:39 smithi019 ceph-mon[29215]: pgmap v198: 97 pgs: 97 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:32:40.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:39 smithi044 ceph-mon[32458]: pgmap v198: 97 pgs: 97 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:32:42.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:41 smithi019 ceph-mon[29215]: pgmap v199: 97 pgs: 97 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:42.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:41 smithi044 ceph-mon[32458]: pgmap v199: 97 pgs: 97 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:42.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:41 smithi155 ceph-mon[33468]: pgmap v199: 97 pgs: 97 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:44.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:43 smithi019 ceph-mon[29215]: pgmap v200: 97 pgs: 97 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:32:44.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:43 smithi044 ceph-mon[32458]: pgmap v200: 97 pgs: 97 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:32:44.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:43 smithi155 ceph-mon[33468]: pgmap v200: 97 pgs: 97 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:32:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.316+0000 7f2afb87b640 1 -- 172.21.15.19:0/466428475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af410e930 msgr2=0x7f2af4110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.316+0000 7f2afb87b640 1 --2- 172.21.15.19:0/466428475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af410e930 0x7f2af4110d20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009ca0 tx=0x7f2aec02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.317+0000 7f2afb87b640 1 -- 172.21.15.19:0/466428475 shutdown_connections 2024-04-23T18:32:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.317+0000 7f2afb87b640 1 --2- 172.21.15.19:0/466428475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af41016a0 0x7f2af410e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.317+0000 7f2afb87b640 1 --2- 172.21.15.19:0/466428475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af410e930 0x7f2af4110d20 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.317+0000 7f2afb87b640 1 --2- 172.21.15.19:0/466428475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af4100d80 0x7f2af4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.317+0000 7f2afb87b640 1 -- 172.21.15.19:0/466428475 >> 172.21.15.19:0/466428475 conn(0x7f2af40771d0 msgr2=0x7f2af40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.317+0000 7f2afb87b640 1 -- 172.21.15.19:0/466428475 shutdown_connections 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.317+0000 7f2afb87b640 1 -- 172.21.15.19:0/466428475 wait complete. 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.317+0000 7f2afb87b640 1 Processor -- start 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.317+0000 7f2afb87b640 1 -- start start 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2afb87b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af4100d80 0x7f2af41a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2afb87b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af41016a0 0x7f2af41a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2afb87b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af410e930 0x7f2af419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2afb87b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2af41143f0 con 0x7f2af410e930 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2afb87b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2af4114270 con 0x7f2af41016a0 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2afb87b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2af4114570 con 0x7f2af4100d80 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9e18640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af410e930 0x7f2af419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9e18640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af410e930 0x7f2af419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40138/0 (socket says 172.21.15.19:40138) 2024-04-23T18:32:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9e18640 1 -- 172.21.15.19:0/1191506595 learned_addr learned my addr 172.21.15.19:0/1191506595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9617640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af4100d80 0x7f2af41a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af8e16640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af41016a0 0x7f2af41a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9e18640 1 -- 172.21.15.19:0/1191506595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af4100d80 msgr2=0x7f2af41a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9e18640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af4100d80 0x7f2af41a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9e18640 1 -- 172.21.15.19:0/1191506595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af41016a0 msgr2=0x7f2af41a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9e18640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af41016a0 0x7f2af41a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9e18640 1 -- 172.21.15.19:0/1191506595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af419af30 con 0x7f2af410e930 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9617640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af4100d80 0x7f2af41a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2af9e18640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af410e930 0x7f2af419a670 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2aec02ade0 tx=0x7f2aec0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.318+0000 7f2aea7fc640 1 -- 172.21.15.19:0/1191506595 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec003b70 con 0x7f2af410e930 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.319+0000 7f2aea7fc640 1 -- 172.21.15.19:0/1191506595 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec003d10 con 0x7f2af410e930 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.319+0000 7f2afb87b640 1 -- 172.21.15.19:0/1191506595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af419b1c0 con 0x7f2af410e930 2024-04-23T18:32:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.319+0000 7f2aea7fc640 1 -- 172.21.15.19:0/1191506595 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec02fa90 con 0x7f2af410e930 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.319+0000 7f2afb87b640 1 -- 172.21.15.19:0/1191506595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af41a7530 con 0x7f2af410e930 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.321+0000 7f2aea7fc640 1 -- 172.21.15.19:0/1191506595 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2aec02fc30 con 0x7f2af410e930 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.321+0000 7f2afe58a740 1 Processor -- start 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.321+0000 7f2afe58a740 1 -- start start 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96b620 0x560bcd96ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd96bf40 0x560bcd9762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd978bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2afe58a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560bcd8d0970 con 0x560bcd976800 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2afe58a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560bcd86d4a0 con 0x560bcd96b620 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2afe58a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560bcd8cc930 con 0x560bcd96bf40 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2aea7fc640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2ae00a3dc0 0x7f2ae00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af8e16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd978bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af8e16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd978bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40142/0 (socket says 172.21.15.19:40142) 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af8e16640 1 -- 172.21.15.19:0/2402082481 learned_addr learned my addr 172.21.15.19:0/2402082481 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af9e18640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2ae00a3dc0 0x7f2ae00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af8e16640 1 --2- 172.21.15.19:0/2402082481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96b620 0x560bcd96ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af9617640 1 --2- 172.21.15.19:0/2402082481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd96bf40 0x560bcd9762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2aea7fc640 1 -- 172.21.15.19:0/1191506595 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2aec0f1bb0 con 0x7f2af410e930 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af8e16640 1 -- 172.21.15.19:0/2402082481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd96bf40 msgr2=0x560bcd9762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af8e16640 1 --2- 172.21.15.19:0/2402082481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd96bf40 0x560bcd9762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af8e16640 1 -- 172.21.15.19:0/2402082481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96b620 msgr2=0x560bcd96ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af8e16640 1 --2- 172.21.15.19:0/2402082481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96b620 0x560bcd96ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af8e16640 1 -- 172.21.15.19:0/2402082481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcd5e3210 con 0x560bcd976800 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.322+0000 7f2af8e16640 1 --2- 172.21.15.19:0/2402082481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd978bf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4012710 tx=0x7f2ae4009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347fb5ac86e3cad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.323+0000 7f2af9e18640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2ae00a3dc0 0x7f2ae00a6280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0bd930 tx=0x7f2aec08f670 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.323+0000 7f2adf7fe640 1 -- 172.21.15.19:0/2402082481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4012d50 con 0x560bcd976800 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.323+0000 7f2adf7fe640 1 -- 172.21.15.19:0/2402082481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae40107d0 con 0x560bcd976800 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.323+0000 7f2adf7fe640 1 -- 172.21.15.19:0/2402082481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4003850 con 0x560bcd976800 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.323+0000 7f2afe58a740 1 -- 172.21.15.19:0/2402082481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 msgr2=0x560bcd978bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.323+0000 7f2afe58a740 1 --2- 172.21.15.19:0/2402082481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd978bf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4012710 tx=0x7f2ae4009660 comp rx=0 tx=0).stop 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 -- 172.21.15.19:0/2402082481 shutdown_connections 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 --2- 172.21.15.19:0/2402082481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd978bf0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 --2- 172.21.15.19:0/2402082481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd96bf40 0x560bcd9762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 --2- 172.21.15.19:0/2402082481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96b620 0x560bcd96ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 -- 172.21.15.19:0/2402082481 >> 172.21.15.19:0/2402082481 conn(0x560bcd967b30 msgr2=0x560bcd967f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 -- 172.21.15.19:0/2402082481 shutdown_connections 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 -- 172.21.15.19:0/2402082481 wait complete. 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 -- start start 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 Processor -- start 2024-04-23T18:32:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96b620 0x560bcd976150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 0x560bcd973440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 0x560bcd9739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.324+0000 7f2afe58a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560bcd979430 con 0x560bcd96b620 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2afe58a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560bcd86b0b0 con 0x560bcd96bf40 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2afe58a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560bcd9795b0 con 0x560bcd976800 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2afe58a740 10 client.0 _ll_register_callbacks cb 0x560bcd858d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af8e16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96b620 0x560bcd976150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af8e16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96b620 0x560bcd976150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40152/0 (socket says 172.21.15.19:40152) 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af8e16640 1 -- 172.21.15.19:0/3566905808 learned_addr learned my addr 172.21.15.19:0/3566905808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af9617640 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 0x560bcd973440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af8e16640 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 0x560bcd9739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af8e16640 1 -- 172.21.15.19:0/3566905808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 msgr2=0x560bcd9739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af8e16640 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 0x560bcd9739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af8e16640 1 -- 172.21.15.19:0/3566905808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 msgr2=0x560bcd973440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af8e16640 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 0x560bcd973440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af8e16640 1 -- 172.21.15.19:0/3566905808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcd6af620 con 0x560bcd96b620 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af9617640 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 0x560bcd973440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2af8e16640 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96b620 0x560bcd976150 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40092b0 tx=0x7f2ae401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b013c98a85dce356 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2adcff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2adcff9640 1 -- 172.21.15.19:0/3566905808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40052e0 con 0x560bcd96b620 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2adcff9640 1 -- 172.21.15.19:0/3566905808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2ae4005480 con 0x560bcd96b620 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.325+0000 7f2afe58a740 1 -- 172.21.15.19:0/3566905808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae40107d0 con 0x560bcd96b620 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2adcff9640 1 -- 172.21.15.19:0/3566905808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ae40035f0 con 0x560bcd96b620 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2afe58a740 10 client.16497 fetch_fsmap learned FSMap version 27 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2afe58a740 1 -- 172.21.15.19:0/3566905808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bcd9853a0 con 0x560bcd96b620 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2adcff9640 1 -- 172.21.15.19:0/3566905808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae40107d0 con 0x560bcd96b620 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2adcff9640 1 -- 172.21.15.19:0/3566905808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ae401f630 con 0x560bcd96b620 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2afe58a740 10 client.16497 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2afe58a740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2afe58a740 20 client.16497 populate_metadata read hostname 'smithi019' 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2afe58a740 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560bcd9867b0 0x560bcd988ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2afe58a740 4 client.16497 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2afe58a740 1 -- 172.21.15.19:0/3566905808 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bcd8de390 con 0x560bcd9867b0 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.326+0000 7f2af9617640 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560bcd9867b0 0x560bcd988ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.327+0000 7f2af9617640 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560bcd9867b0 0x560bcd988ba0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ab02e2c3eb772c server_cookie=481911c8ab5a0e02 in_seq=0 out_seq=0 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.327+0000 7f2adcff9640 10 client.16497 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2adcff9640 1 -- 172.21.15.19:0/3566905808 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bcd8de390 con 0x560bcd9867b0 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2adcff9640 10 client.16497 handle_command_reply: tid=0 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 client.16497 shutdown 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 -- 172.21.15.19:0/3566905808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96b620 msgr2=0x560bcd976150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96b620 0x560bcd976150 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40092b0 tx=0x7f2ae401b040 comp rx=0 tx=0).stop 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 -- 172.21.15.19:0/3566905808 shutdown_connections 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560bcd9867b0 0x560bcd988ba0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 0x560bcd973440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 0x560bcd9739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2adcff9640 0 client.16497 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 --2- 172.21.15.19:0/3566905808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96b620 0x560bcd976150 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 -- 172.21.15.19:0/3566905808 >> 172.21.15.19:0/3566905808 conn(0x560bcd967b30 msgr2=0x560bcd96d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 -- 172.21.15.19:0/3566905808 shutdown_connections 2024-04-23T18:32:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 1 -- 172.21.15.19:0/3566905808 wait complete. 2024-04-23T18:32:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.328+0000 7f2afe58a740 20 client.16497 trim_cache size 0 max 16384 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2afe58a740 1 Processor -- start 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2afe58a740 1 -- start start 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96b620 0x560bcd9822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96bf40 0x560bcd966b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 0x560bcd98e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2afe58a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2af41afca0 con 0x560bcd96bf40 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2afe58a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2af41b17e0 con 0x560bcd96b620 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2afe58a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2af41b0370 con 0x560bcd976800 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af9617640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96bf40 0x560bcd966b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af9617640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96bf40 0x560bcd966b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40164/0 (socket says 172.21.15.19:40164) 2024-04-23T18:32:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af9617640 1 -- 172.21.15.19:0/2680774168 learned_addr learned my addr 172.21.15.19:0/2680774168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af8e16640 1 --2- 172.21.15.19:0/2680774168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96b620 0x560bcd9822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af9617640 1 -- 172.21.15.19:0/2680774168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 msgr2=0x560bcd98e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af8e16640 1 --2- 172.21.15.19:0/2680774168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 0x560bcd98e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af9617640 1 --2- 172.21.15.19:0/2680774168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 0x560bcd98e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af9617640 1 -- 172.21.15.19:0/2680774168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96b620 msgr2=0x560bcd9822f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af9617640 1 --2- 172.21.15.19:0/2680774168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96b620 0x560bcd9822f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af9617640 1 -- 172.21.15.19:0/2680774168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcd971990 con 0x560bcd96bf40 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af8e16640 1 --2- 172.21.15.19:0/2680774168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 0x560bcd98e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2af9617640 1 --2- 172.21.15.19:0/2680774168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96bf40 0x560bcd966b30 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x560bcd96cbc0 tx=0x7f2af000b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1267fe05090c50fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2adeffd640 1 -- 172.21.15.19:0/2680774168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0003e80 con 0x560bcd96bf40 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2adeffd640 1 -- 172.21.15.19:0/2680774168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af001d920 con 0x560bcd96bf40 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.344+0000 7f2adeffd640 1 -- 172.21.15.19:0/2680774168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af001c3d0 con 0x560bcd96bf40 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 -- 172.21.15.19:0/2680774168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96bf40 msgr2=0x560bcd966b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 --2- 172.21.15.19:0/2680774168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96bf40 0x560bcd966b30 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x560bcd96cbc0 tx=0x7f2af000b500 comp rx=0 tx=0).stop 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 -- 172.21.15.19:0/2680774168 shutdown_connections 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 --2- 172.21.15.19:0/2680774168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd976800 0x560bcd98e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 --2- 172.21.15.19:0/2680774168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd96bf40 0x560bcd966b30 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 --2- 172.21.15.19:0/2680774168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96b620 0x560bcd9822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 -- 172.21.15.19:0/2680774168 >> 172.21.15.19:0/2680774168 conn(0x560bcd967b30 msgr2=0x560bcd966420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 -- 172.21.15.19:0/2680774168 shutdown_connections 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 -- 172.21.15.19:0/2680774168 wait complete. 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 -- start start 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 Processor -- start 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd96b620 0x560bcd97e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 0x560bcd97eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd97b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2af41b1110 con 0x560bcd976800 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2af41b0a40 con 0x560bcd96bf40 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.345+0000 7f2afe58a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2af41adca0 con 0x560bcd96b620 2024-04-23T18:32:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2afe58a740 10 client.0 _ll_register_callbacks cb 0x7f2ae40125e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af8e16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd97b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af8e16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd97b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40174/0 (socket says 172.21.15.19:40174) 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af8e16640 1 -- 172.21.15.19:0/1453730830 learned_addr learned my addr 172.21.15.19:0/1453730830 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af9617640 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 0x560bcd97eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af8e16640 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd96b620 0x560bcd97e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af8e16640 1 -- 172.21.15.19:0/1453730830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd96b620 msgr2=0x560bcd97e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af8e16640 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd96b620 0x560bcd97e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af8e16640 1 -- 172.21.15.19:0/1453730830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 msgr2=0x560bcd97eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af8e16640 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 0x560bcd97eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af8e16640 1 -- 172.21.15.19:0/1453730830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcd97c1e0 con 0x560bcd976800 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2af8e16640 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd97b9e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40059f0 tx=0x7f2ae4024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b1f11307cd1482 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2adcff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2adcff9640 1 -- 172.21.15.19:0/1453730830 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4010d00 con 0x560bcd976800 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2adcff9640 1 -- 172.21.15.19:0/1453730830 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2ae4012260 con 0x560bcd976800 2024-04-23T18:32:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2afe58a740 1 -- 172.21.15.19:0/1453730830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560bcd8d8200 con 0x560bcd976800 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2adcff9640 1 -- 172.21.15.19:0/1453730830 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ae4042050 con 0x560bcd976800 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2afe58a740 10 client.16503 fetch_fsmap learned FSMap version 27 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.346+0000 7f2afe58a740 1 -- 172.21.15.19:0/1453730830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bcd9ec560 con 0x560bcd976800 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.347+0000 7f2adcff9640 1 -- 172.21.15.19:0/1453730830 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560bcd8d8200 con 0x560bcd976800 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.347+0000 7f2adcff9640 1 -- 172.21.15.19:0/1453730830 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ae4003c50 con 0x560bcd976800 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.347+0000 7f2afe58a740 10 client.16503 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.347+0000 7f2afe58a740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.347+0000 7f2afe58a740 20 client.16503 populate_metadata read hostname 'smithi019' 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.347+0000 7f2afe58a740 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560bcd9ecc80 0x560bcd9ef070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.347+0000 7f2afe58a740 4 client.16503 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.347+0000 7f2afe58a740 1 -- 172.21.15.19:0/1453730830 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bcd8545f0 con 0x560bcd9ecc80 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.347+0000 7f2af9617640 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560bcd9ecc80 0x560bcd9ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.348+0000 7f2af9617640 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560bcd9ecc80 0x560bcd9ef070 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b7df30965d9f87 server_cookie=936fe52d60a0ea6e in_seq=0 out_seq=0 2024-04-23T18:32:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.348+0000 7f2adcff9640 10 client.16503 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2adcff9640 1 -- 172.21.15.19:0/1453730830 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bcd8545f0 con 0x560bcd9ecc80 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2adcff9640 10 client.16503 handle_command_reply: tid=0 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 client.16503 shutdown 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 -- 172.21.15.19:0/1453730830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 msgr2=0x560bcd97b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd97b9e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40059f0 tx=0x7f2ae4024040 comp rx=0 tx=0).stop 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 -- 172.21.15.19:0/1453730830 shutdown_connections 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560bcd9ecc80 0x560bcd9ef070 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560bcd96bf40 0x560bcd97eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bcd976800 0x560bcd97b9e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2adcff9640 0 client.16503 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 --2- 172.21.15.19:0/1453730830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560bcd96b620 0x560bcd97e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 -- 172.21.15.19:0/1453730830 >> 172.21.15.19:0/1453730830 conn(0x560bcd967b30 msgr2=0x560bcd966cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 -- 172.21.15.19:0/1453730830 shutdown_connections 2024-04-23T18:32:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 1 -- 172.21.15.19:0/1453730830 wait complete. 2024-04-23T18:32:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afe58a740 20 client.16503 trim_cache size 0 max 16384 2024-04-23T18:32:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:32:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:32:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:32:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:32:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 -- 172.21.15.19:0/1191506595 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2ae00a3dc0 msgr2=0x7f2ae00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2ae00a3dc0 0x7f2ae00a6280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0bd930 tx=0x7f2aec08f670 comp rx=0 tx=0).stop 2024-04-23T18:32:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 -- 172.21.15.19:0/1191506595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af410e930 msgr2=0x7f2af419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af410e930 0x7f2af419a670 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2aec02ade0 tx=0x7f2aec0316f0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 -- 172.21.15.19:0/1191506595 shutdown_connections 2024-04-23T18:32:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2ae00a3dc0 0x7f2ae00a6280 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af41016a0 0x7f2af41a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af410e930 0x7f2af419a670 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 --2- 172.21.15.19:0/1191506595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af4100d80 0x7f2af41a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 -- 172.21.15.19:0/1191506595 >> 172.21.15.19:0/1191506595 conn(0x7f2af40771d0 msgr2=0x7f2af410fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 -- 172.21.15.19:0/1191506595 shutdown_connections 2024-04-23T18:32:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:45.364+0000 7f2afb87b640 1 -- 172.21.15.19:0/1191506595 wait complete. 2024-04-23T18:32:46.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:45 smithi019 ceph-mon[29215]: pgmap v201: 97 pgs: 97 active+clean; 3.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:46.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:45 smithi044 ceph-mon[32458]: pgmap v201: 97 pgs: 97 active+clean; 3.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:46.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:45 smithi155 ceph-mon[33468]: pgmap v201: 97 pgs: 97 active+clean; 3.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:48.124 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:47 smithi155 ceph-mon[33468]: pgmap v202: 97 pgs: 97 active+clean; 3.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T18:32:48.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:47 smithi019 ceph-mon[29215]: pgmap v202: 97 pgs: 97 active+clean; 3.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T18:32:48.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:47 smithi044 ceph-mon[32458]: pgmap v202: 97 pgs: 97 active+clean; 3.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T18:32:50.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:49 smithi019 ceph-mon[29215]: pgmap v203: 97 pgs: 97 active+clean; 3.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T18:32:50.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:49 smithi044 ceph-mon[32458]: pgmap v203: 97 pgs: 97 active+clean; 3.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T18:32:50.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:49 smithi155 ceph-mon[33468]: pgmap v203: 97 pgs: 97 active+clean; 3.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T18:32:52.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:51 smithi019 ceph-mon[29215]: pgmap v204: 97 pgs: 97 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:52.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:51 smithi044 ceph-mon[32458]: pgmap v204: 97 pgs: 97 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:52.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:51 smithi155 ceph-mon[33468]: pgmap v204: 97 pgs: 97 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-23T18:32:54.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:53 smithi019 ceph-mon[29215]: pgmap v205: 97 pgs: 97 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:54.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:53 smithi044 ceph-mon[32458]: pgmap v205: 97 pgs: 97 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:53 smithi155 ceph-mon[33468]: pgmap v205: 97 pgs: 97 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:32:56.114 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:55 smithi044 ceph-mon[32458]: pgmap v206: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T18:32:56.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:55 smithi019 ceph-mon[29215]: pgmap v206: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T18:32:56.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:55 smithi155 ceph-mon[33468]: pgmap v206: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T18:32:58.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:57 smithi019 ceph-mon[29215]: pgmap v207: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:32:58.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:57 smithi044 ceph-mon[32458]: pgmap v207: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:32:58.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:57 smithi155 ceph-mon[33468]: pgmap v207: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:32:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.465+0000 7f32a355d640 1 -- 172.21.15.19:0/1017113297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f329c100d80 msgr2=0x7f329c101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.465+0000 7f32a355d640 1 --2- 172.21.15.19:0/1017113297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f329c100d80 0x7f329c101160 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3298009900 tx=0x7f329802a410 comp rx=0 tx=0).stop 2024-04-23T18:32:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.465+0000 7f32a355d640 1 -- 172.21.15.19:0/1017113297 shutdown_connections 2024-04-23T18:32:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.465+0000 7f32a355d640 1 --2- 172.21.15.19:0/1017113297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f329c1016a0 0x7f329c10e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.465+0000 7f32a355d640 1 --2- 172.21.15.19:0/1017113297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329c10e930 0x7f329c110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.465+0000 7f32a355d640 1 --2- 172.21.15.19:0/1017113297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f329c100d80 0x7f329c101160 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.465+0000 7f32a355d640 1 -- 172.21.15.19:0/1017113297 >> 172.21.15.19:0/1017113297 conn(0x7f329c0771d0 msgr2=0x7f329c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.465+0000 7f32a355d640 1 -- 172.21.15.19:0/1017113297 shutdown_connections 2024-04-23T18:32:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.465+0000 7f32a355d640 1 -- 172.21.15.19:0/1017113297 wait complete. 2024-04-23T18:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.465+0000 7f32a355d640 1 Processor -- start 2024-04-23T18:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a355d640 1 -- start start 2024-04-23T18:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a355d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f329c100d80 0x7f329c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a355d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f329c1016a0 0x7f329c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a355d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329c10e930 0x7f329c19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a355d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f329c114430 con 0x7f329c100d80 2024-04-23T18:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a355d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1142b0 con 0x7f329c1016a0 2024-04-23T18:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a355d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1145b0 con 0x7f329c10e930 2024-04-23T18:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a12f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f329c100d80 0x7f329c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a12f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f329c100d80 0x7f329c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34144/0 (socket says 172.21.15.19:34144) 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a12f9640 1 -- 172.21.15.19:0/1429511868 learned_addr learned my addr 172.21.15.19:0/1429511868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a0af8640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f329c1016a0 0x7f329c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a1afa640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329c10e930 0x7f329c19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a12f9640 1 -- 172.21.15.19:0/1429511868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329c10e930 msgr2=0x7f329c19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a12f9640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329c10e930 0x7f329c19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a12f9640 1 -- 172.21.15.19:0/1429511868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f329c1016a0 msgr2=0x7f329c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a12f9640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f329c1016a0 0x7f329c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a12f9640 1 -- 172.21.15.19:0/1429511868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c19afc0 con 0x7f329c100d80 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.466+0000 7f32a1afa640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329c10e930 0x7f329c19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.467+0000 7f32a12f9640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f329c100d80 0x7f329c1a0630 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3298006010 tx=0x7f32980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.467+0000 7f32927fc640 1 -- 172.21.15.19:0/1429511868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329803f030 con 0x7f329c100d80 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.467+0000 7f32a355d640 1 -- 172.21.15.19:0/1429511868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c19b250 con 0x7f329c100d80 2024-04-23T18:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.467+0000 7f32927fc640 1 -- 172.21.15.19:0/1429511868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3298003800 con 0x7f329c100d80 2024-04-23T18:32:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.467+0000 7f32927fc640 1 -- 172.21.15.19:0/1429511868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298035070 con 0x7f329c100d80 2024-04-23T18:32:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.467+0000 7f32a355d640 1 -- 172.21.15.19:0/1429511868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c1a7480 con 0x7f329c100d80 2024-04-23T18:32:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.469+0000 7f32927fc640 1 -- 172.21.15.19:0/1429511868 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3298003cb0 con 0x7f329c100d80 2024-04-23T18:32:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a627a740 1 Processor -- start 2024-04-23T18:32:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a627a740 1 -- start start 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a627a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 0x55b4c7693a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a627a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693f40 0x55b4c769e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a627a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c769e800 0x55b4c76a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a627a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b4c75f8970 con 0x55b4c7693f40 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a627a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b4c75954a0 con 0x55b4c7693620 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a627a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b4c75f4930 con 0x55b4c769e800 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a12f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693f40 0x55b4c769e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a12f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693f40 0x55b4c769e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34148/0 (socket says 172.21.15.19:34148) 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a12f9640 1 -- 172.21.15.19:0/4243297787 learned_addr learned my addr 172.21.15.19:0/4243297787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a1afa640 1 --2- 172.21.15.19:0/4243297787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 0x55b4c7693a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32a0af8640 1 --2- 172.21.15.19:0/4243297787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c769e800 0x55b4c76a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.470+0000 7f32927fc640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32880a3dc0 0x7f32880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32a12f9640 1 -- 172.21.15.19:0/4243297787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c769e800 msgr2=0x55b4c76a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32a12f9640 1 --2- 172.21.15.19:0/4243297787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c769e800 0x55b4c76a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32a12f9640 1 -- 172.21.15.19:0/4243297787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 msgr2=0x55b4c7693a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32a12f9640 1 --2- 172.21.15.19:0/4243297787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 0x55b4c7693a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32a12f9640 1 -- 172.21.15.19:0/4243297787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4c730b210 con 0x55b4c7693f40 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32a1afa640 1 --2- 172.21.15.19:0/4243297787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 0x55b4c7693a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32a1afa640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32880a3dc0 0x7f32880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32927fc640 1 -- 172.21.15.19:0/1429511868 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f32980f1130 con 0x7f329c100d80 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32a12f9640 1 --2- 172.21.15.19:0/4243297787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693f40 0x55b4c769e2c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f32980ee860 tx=0x7f329804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f14f7b416d20dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32877fe640 1 -- 172.21.15.19:0/4243297787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298049070 con 0x55b4c7693f40 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32877fe640 1 -- 172.21.15.19:0/4243297787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32980bd840 con 0x55b4c7693f40 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.471+0000 7f32877fe640 1 -- 172.21.15.19:0/4243297787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32980bd140 con 0x55b4c7693f40 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 -- 172.21.15.19:0/4243297787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693f40 msgr2=0x55b4c769e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a1afa640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32880a3dc0 0x7f32880a6280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f329c06a570 tx=0x7f3294004a40 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 --2- 172.21.15.19:0/4243297787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693f40 0x55b4c769e2c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f32980ee860 tx=0x7f329804b000 comp rx=0 tx=0).stop 2024-04-23T18:32:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 -- 172.21.15.19:0/4243297787 shutdown_connections 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 --2- 172.21.15.19:0/4243297787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c769e800 0x55b4c76a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 --2- 172.21.15.19:0/4243297787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693f40 0x55b4c769e2c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 --2- 172.21.15.19:0/4243297787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 0x55b4c7693a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 -- 172.21.15.19:0/4243297787 >> 172.21.15.19:0/4243297787 conn(0x55b4c768fb30 msgr2=0x55b4c768ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 -- 172.21.15.19:0/4243297787 shutdown_connections 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 -- 172.21.15.19:0/4243297787 wait complete. 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 -- start start 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.472+0000 7f32a627a740 1 Processor -- start 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a627a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693620 0x55b4c769e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a627a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 0x55b4c769b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a627a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c769e800 0x55b4c769bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a627a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b4c75f8970 con 0x55b4c7693620 2024-04-23T18:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a627a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b4c75930b0 con 0x55b4c769e800 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a627a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b4c76a11f0 con 0x55b4c7693f40 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a627a740 10 client.0 _ll_register_callbacks cb 0x55b4c7580d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a12f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693620 0x55b4c769e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a12f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693620 0x55b4c769e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34156/0 (socket says 172.21.15.19:34156) 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a12f9640 1 -- 172.21.15.19:0/534273563 learned_addr learned my addr 172.21.15.19:0/534273563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a1afa640 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c769e800 0x55b4c769bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a0af8640 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 0x55b4c769b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a12f9640 1 -- 172.21.15.19:0/534273563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 msgr2=0x55b4c769b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a12f9640 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 0x55b4c769b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a12f9640 1 -- 172.21.15.19:0/534273563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c769e800 msgr2=0x55b4c769bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a12f9640 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c769e800 0x55b4c769bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a12f9640 1 -- 172.21.15.19:0/534273563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4c73d7620 con 0x55b4c7693620 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a1afa640 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c769e800 0x55b4c769bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a0af8640 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 0x55b4c769b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f32a12f9640 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693620 0x55b4c769e1c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3298003c60 tx=0x7f32980044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae75483c71f6e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.473+0000 7f3284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f3284ff9640 1 -- 172.21.15.19:0/534273563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298049030 con 0x55b4c7693620 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f3284ff9640 1 -- 172.21.15.19:0/534273563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f32980c1170 con 0x55b4c7693620 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f32a627a740 1 -- 172.21.15.19:0/534273563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32980bd840 con 0x55b4c7693620 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f3284ff9640 1 -- 172.21.15.19:0/534273563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32980bcc80 con 0x55b4c7693620 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f32a627a740 10 client.16515 fetch_fsmap learned FSMap version 27 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f32a627a740 1 -- 172.21.15.19:0/534273563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b4c76ad000 con 0x55b4c7693620 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f3284ff9640 1 -- 172.21.15.19:0/534273563 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32980bd840 con 0x55b4c7693620 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f3284ff9640 1 -- 172.21.15.19:0/534273563 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f32980b97b0 con 0x55b4c7693620 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f32a627a740 10 client.16515 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f32a627a740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f32a627a740 20 client.16515 populate_metadata read hostname 'smithi019' 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f32a627a740 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b4c76ae3f0 0x55b4c76b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f32a627a740 4 client.16515 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.474+0000 7f32a627a740 1 -- 172.21.15.19:0/534273563 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4c7606390 con 0x55b4c76ae3f0 2024-04-23T18:32:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.475+0000 7f32a0af8640 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b4c76ae3f0 0x55b4c76b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.475+0000 7f32a0af8640 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b4c76ae3f0 0x55b4c76b07e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7fd19da337eaa4 server_cookie=8d44af7bfba98a4b in_seq=0 out_seq=0 2024-04-23T18:32:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.475+0000 7f3284ff9640 10 client.16515 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f3284ff9640 1 -- 172.21.15.19:0/534273563 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4c7606390 con 0x55b4c76ae3f0 2024-04-23T18:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f3284ff9640 10 client.16515 handle_command_reply: tid=0 2024-04-23T18:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 client.16515 shutdown 2024-04-23T18:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 -- 172.21.15.19:0/534273563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693620 msgr2=0x55b4c769e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693620 0x55b4c769e1c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3298003c60 tx=0x7f32980044e0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 -- 172.21.15.19:0/534273563 shutdown_connections 2024-04-23T18:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b4c76ae3f0 0x55b4c76b07e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c769e800 0x55b4c769bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 0x55b4c769b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 --2- 172.21.15.19:0/534273563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c7693620 0x55b4c769e1c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f3284ff9640 0 client.16515 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 -- 172.21.15.19:0/534273563 >> 172.21.15.19:0/534273563 conn(0x55b4c768fb30 msgr2=0x55b4c7695860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 -- 172.21.15.19:0/534273563 shutdown_connections 2024-04-23T18:32:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 1 -- 172.21.15.19:0/534273563 wait complete. 2024-04-23T18:32:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.476+0000 7f32a627a740 20 client.16515 trim_cache size 0 max 16384 2024-04-23T18:32:58.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.491+0000 7f32a627a740 1 Processor -- start 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.491+0000 7f32a627a740 1 -- start start 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.491+0000 7f32a627a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693620 0x55b4c76a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.491+0000 7f32a627a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693f40 0x55b4c769dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a627a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c76b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a627a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1afd40 con 0x55b4c769e800 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a627a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1b1880 con 0x55b4c7693f40 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a627a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1b0410 con 0x55b4c7693620 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a1afa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c76b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a1afa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c76b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34162/0 (socket says 172.21.15.19:34162) 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a1afa640 1 -- 172.21.15.19:0/1552899609 learned_addr learned my addr 172.21.15.19:0/1552899609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a12f9640 1 --2- 172.21.15.19:0/1552899609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693620 0x55b4c76a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a0af8640 1 --2- 172.21.15.19:0/1552899609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693f40 0x55b4c769dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a1afa640 1 -- 172.21.15.19:0/1552899609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693620 msgr2=0x55b4c76a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a1afa640 1 --2- 172.21.15.19:0/1552899609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693620 0x55b4c76a9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a1afa640 1 -- 172.21.15.19:0/1552899609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693f40 msgr2=0x55b4c769dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a1afa640 1 --2- 172.21.15.19:0/1552899609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693f40 0x55b4c769dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a1afa640 1 -- 172.21.15.19:0/1552899609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4c76b0b50 con 0x55b4c769e800 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a12f9640 1 --2- 172.21.15.19:0/1552899609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693620 0x55b4c76a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f32a1afa640 1 --2- 172.21.15.19:0/1552899609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c76b60b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3294012760 tx=0x7f329400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9a2cd207842233 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f3286ffd640 1 -- 172.21.15.19:0/1552899609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329401eb30 con 0x55b4c769e800 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f3286ffd640 1 -- 172.21.15.19:0/1552899609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3294016070 con 0x55b4c769e800 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.492+0000 7f3286ffd640 1 -- 172.21.15.19:0/1552899609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294026640 con 0x55b4c769e800 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 -- 172.21.15.19:0/1552899609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 msgr2=0x55b4c76b60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 --2- 172.21.15.19:0/1552899609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c76b60b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3294012760 tx=0x7f329400af90 comp rx=0 tx=0).stop 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 -- 172.21.15.19:0/1552899609 shutdown_connections 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 --2- 172.21.15.19:0/1552899609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693f40 0x55b4c769dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 --2- 172.21.15.19:0/1552899609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c76b60b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 --2- 172.21.15.19:0/1552899609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693620 0x55b4c76a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 -- 172.21.15.19:0/1552899609 >> 172.21.15.19:0/1552899609 conn(0x55b4c768fb30 msgr2=0x55b4c76969d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 -- 172.21.15.19:0/1552899609 shutdown_connections 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 -- 172.21.15.19:0/1552899609 wait complete. 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 -- start start 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.493+0000 7f32a627a740 1 Processor -- start 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a627a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 0x55b4c7698530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a627a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 0x55b4c7698a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a627a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c7713920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a627a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1b11b0 con 0x55b4c769e800 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a627a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1b0ae0 con 0x55b4c7693620 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a627a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1add20 con 0x55b4c7693f40 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a627a740 10 client.0 _ll_register_callbacks cb 0x7f32980b9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a1afa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c7713920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a1afa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c7713920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34166/0 (socket says 172.21.15.19:34166) 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a1afa640 1 -- 172.21.15.19:0/1228611764 learned_addr learned my addr 172.21.15.19:0/1228611764 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a0af8640 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 0x55b4c7698a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a12f9640 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 0x55b4c7698530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a1afa640 1 -- 172.21.15.19:0/1228611764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 msgr2=0x55b4c7698a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a1afa640 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 0x55b4c7698a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a1afa640 1 -- 172.21.15.19:0/1228611764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 msgr2=0x55b4c7698530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a1afa640 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 0x55b4c7698530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a1afa640 1 -- 172.21.15.19:0/1228611764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4c7713e60 con 0x55b4c769e800 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.494+0000 7f32a12f9640 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 0x55b4c7698530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f32a1afa640 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c7713920 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f329401bcd0 tx=0x7f3294011270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01d42b4c19b5db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f3284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f3284ff9640 1 -- 172.21.15.19:0/1228611764 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329401b820 con 0x55b4c769e800 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f3284ff9640 1 -- 172.21.15.19:0/1228611764 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f329401b9c0 con 0x55b4c769e800 2024-04-23T18:32:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f32a627a740 1 -- 172.21.15.19:0/1228611764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3294016070 con 0x55b4c769e800 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f3284ff9640 1 -- 172.21.15.19:0/1228611764 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f329401ce80 con 0x55b4c769e800 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f32a627a740 10 client.16521 fetch_fsmap learned FSMap version 27 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f32a627a740 1 -- 172.21.15.19:0/1228611764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b4c7714fb0 con 0x55b4c769e800 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f3284ff9640 1 -- 172.21.15.19:0/1228611764 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3294016070 con 0x55b4c769e800 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f3284ff9640 1 -- 172.21.15.19:0/1228611764 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f329402f400 con 0x55b4c769e800 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f32a627a740 10 client.16521 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f32a627a740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f32a627a740 20 client.16521 populate_metadata read hostname 'smithi019' 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f32a627a740 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b4c7715640 0x55b4c7717a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f32a627a740 4 client.16521 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.495+0000 7f32a627a740 1 -- 172.21.15.19:0/1228611764 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4c7600200 con 0x55b4c7715640 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.496+0000 7f32a12f9640 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b4c7715640 0x55b4c7717a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.496+0000 7f3284ff9640 10 client.16521 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.496+0000 7f32a12f9640 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b4c7715640 0x55b4c7717a60 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8771940dc59a0ae0 server_cookie=140527a9ecc9cb54 in_seq=0 out_seq=0 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f3284ff9640 1 -- 172.21.15.19:0/1228611764 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4c7600200 con 0x55b4c7715640 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f3284ff9640 10 client.16521 handle_command_reply: tid=0 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 client.16521 shutdown 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 -- 172.21.15.19:0/1228611764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 msgr2=0x55b4c7713920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c7713920 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f329401bcd0 tx=0x7f3294011270 comp rx=0 tx=0).stop 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 -- 172.21.15.19:0/1228611764 shutdown_connections 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b4c7715640 0x55b4c7717a60 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c769e800 0x55b4c7713920 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4c7693f40 0x55b4c7698a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f3284ff9640 0 client.16521 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 --2- 172.21.15.19:0/1228611764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b4c7693620 0x55b4c7698530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 -- 172.21.15.19:0/1228611764 >> 172.21.15.19:0/1228611764 conn(0x55b4c768fb30 msgr2=0x55b4c7694fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 -- 172.21.15.19:0/1228611764 shutdown_connections 2024-04-23T18:32:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 1 -- 172.21.15.19:0/1228611764 wait complete. 2024-04-23T18:32:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.511+0000 7f32a627a740 20 client.16521 trim_cache size 0 max 16384 2024-04-23T18:32:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:32:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:32:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:32:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 -- 172.21.15.19:0/1429511868 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32880a3dc0 msgr2=0x7f32880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32880a3dc0 0x7f32880a6280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f329c06a570 tx=0x7f3294004a40 comp rx=0 tx=0).stop 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 -- 172.21.15.19:0/1429511868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f329c100d80 msgr2=0x7f329c1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f329c100d80 0x7f329c1a0630 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3298006010 tx=0x7f32980042f0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 -- 172.21.15.19:0/1429511868 shutdown_connections 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f32880a3dc0 0x7f32880a6280 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f329c1016a0 0x7f329c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329c10e930 0x7f329c19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 --2- 172.21.15.19:0/1429511868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f329c100d80 0x7f329c1a0630 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 -- 172.21.15.19:0/1429511868 >> 172.21.15.19:0/1429511868 conn(0x7f329c0771d0 msgr2=0x7f329c10fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 -- 172.21.15.19:0/1429511868 shutdown_connections 2024-04-23T18:32:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:32:58.512+0000 7f32a355d640 1 -- 172.21.15.19:0/1429511868 wait complete. 2024-04-23T18:33:00.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:32:59 smithi019 ceph-mon[29215]: pgmap v208: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:33:00.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:32:59 smithi044 ceph-mon[32458]: pgmap v208: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:33:00.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:32:59 smithi155 ceph-mon[33468]: pgmap v208: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:33:01.230 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:189/600: open fd = 3 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:190/600: open fd = 3 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:191/600: open fd = 3 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.231 INFO:tasks.workunit.client.0.smithi019.stdout:192/600: open fd = 3 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:193/600: open fd = 3 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:194/600: open fd = 3 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:195/600: open fd = 3 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:196/600: open fd = 3 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.232 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:197/600: open fd = 3 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:198/600: open fd = 3 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:199/600: open fd = 3 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:200/600: open fd = 3 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.233 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:201/600: open fd = 3 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:202/600: open fd = 3 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:203/600: open fd = 3 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:204/600: open fd = 3 2024-04-23T18:33:01.234 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:205/600: open fd = 3 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:206/600: open fd = 3 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:207/600: open fd = 3 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.235 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:208/600: open fd = 3 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:209/600: open fd = 3 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:210/600: open fd = 3 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:211/600: open fd = 3 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.236 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:212/600: open fd = 3 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:213/600: open fd = 3 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:214/600: open fd = 3 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:215/600: open fd = 3 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:216/600: open fd = 3 2024-04-23T18:33:01.237 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:217/600: open fd = 3 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:218/600: open fd = 3 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:219/600: open fd = 3 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.238 INFO:tasks.workunit.client.0.smithi019.stdout:220/600: open fd = 3 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:221/600: open fd = 3 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:222/600: open fd = 3 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:223/600: open fd = 3 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.239 INFO:tasks.workunit.client.0.smithi019.stdout:224/600: open fd = 3 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:225/600: open fd = 3 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:226/600: open fd = 3 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:227/600: open fd = 3 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.240 INFO:tasks.workunit.client.0.smithi019.stdout:228/600: open fd = 3 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:229/600: open fd = 3 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:230/600: open fd = 3 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:231/600: open fd = 3 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.241 INFO:tasks.workunit.client.0.smithi019.stdout:232/600: open fd = 3 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:233/600: open fd = 3 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:234/600: open fd = 3 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.242 INFO:tasks.workunit.client.0.smithi019.stdout:235/600: open fd = 3 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:236/600: open fd = 3 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:237/600: open fd = 3 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:238/600: open fd = 3 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.243 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:239/600: open fd = 3 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:240/600: open fd = 3 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:241/600: open fd = 3 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:242/600: open fd = 3 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.244 INFO:tasks.workunit.client.0.smithi019.stdout:243/600: open fd = 3 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:244/600: open fd = 3 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:245/600: open fd = 3 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:246/600: open fd = 3 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:247/600: open fd = 3 2024-04-23T18:33:01.245 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:248/600: open fd = 3 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:249/600: open fd = 3 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:250/600: open fd = 3 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.246 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:33:01.354 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:33:01.354 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.0 2024-04-23T18:33:01.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:01.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.478+0000 7fdf01a13640 1 -- 172.21.15.19:0/4075649111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdefc102bf0 msgr2=0x7fdefc106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.478+0000 7fdf01a13640 1 --2- 172.21.15.19:0/4075649111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdefc102bf0 0x7fdefc106ba0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009900 tx=0x7fdee402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:01.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.478+0000 7fdf01a13640 1 -- 172.21.15.19:0/4075649111 shutdown_connections 2024-04-23T18:33:01.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.478+0000 7fdf01a13640 1 --2- 172.21.15.19:0/4075649111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdefc107280 0x7fdefc10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.478+0000 7fdf01a13640 1 --2- 172.21.15.19:0/4075649111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdefc102bf0 0x7fdefc106ba0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.478+0000 7fdf01a13640 1 --2- 172.21.15.19:0/4075649111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdefc102240 0x7fdefc102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.478+0000 7fdf01a13640 1 -- 172.21.15.19:0/4075649111 >> 172.21.15.19:0/4075649111 conn(0x7fdefc0790e0 msgr2=0x7fdefc0794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:01.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.478+0000 7fdf01a13640 1 -- 172.21.15.19:0/4075649111 shutdown_connections 2024-04-23T18:33:01.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.478+0000 7fdf01a13640 1 -- 172.21.15.19:0/4075649111 wait complete. 2024-04-23T18:33:01.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.479+0000 7fdf01a13640 1 Processor -- start 2024-04-23T18:33:01.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.479+0000 7fdf01a13640 1 -- start start 2024-04-23T18:33:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.479+0000 7fdf01a13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdefc102240 0x7fdefc19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.479+0000 7fdf01a13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdefc107280 0x7fdefc19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.479+0000 7fdf01a13640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdefc198620 0x7fdefc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdf01a13640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc1121c0 con 0x7fdefc107280 2024-04-23T18:33:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdf01a13640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc112040 con 0x7fdefc102240 2024-04-23T18:33:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdf01a13640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc112340 con 0x7fdefc198620 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdef27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdefc107280 0x7fdefc19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdef27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdefc107280 0x7fdefc19e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34072/0 (socket says 172.21.15.19:34072) 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdef27fc640 1 -- 172.21.15.19:0/1716833219 learned_addr learned my addr 172.21.15.19:0/1716833219 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdefaffd640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdefc102240 0x7fdefc19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdefb7fe640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdefc198620 0x7fdefc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdef27fc640 1 -- 172.21.15.19:0/1716833219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdefc198620 msgr2=0x7fdefc198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdef27fc640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdefc198620 0x7fdefc198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdef27fc640 1 -- 172.21.15.19:0/1716833219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdefc102240 msgr2=0x7fdefc19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdef27fc640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdefc102240 0x7fdefc19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdef27fc640 1 -- 172.21.15.19:0/1716833219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc199140 con 0x7fdefc107280 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdefb7fe640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdefc198620 0x7fdefc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdef27fc640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdefc107280 0x7fdefc19e9b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdee4006010 tx=0x7fdee4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdf00a11640 1 -- 172.21.15.19:0/1716833219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee403f030 con 0x7fdefc107280 2024-04-23T18:33:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdf01a13640 1 -- 172.21.15.19:0/1716833219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc1993d0 con 0x7fdefc107280 2024-04-23T18:33:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdf00a11640 1 -- 172.21.15.19:0/1716833219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdee4003aa0 con 0x7fdefc107280 2024-04-23T18:33:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdf00a11640 1 -- 172.21.15.19:0/1716833219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4035070 con 0x7fdefc107280 2024-04-23T18:33:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.480+0000 7fdf01a13640 1 -- 172.21.15.19:0/1716833219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc1a5620 con 0x7fdefc107280 2024-04-23T18:33:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.482+0000 7fdf01a13640 1 -- 172.21.15.19:0/1716833219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdefc0691c0 con 0x7fdefc107280 2024-04-23T18:33:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.483+0000 7fdf00a11640 1 -- 172.21.15.19:0/1716833219 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdee4042030 con 0x7fdefc107280 2024-04-23T18:33:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.484+0000 7fdf00a11640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdee80a3d20 0x7fdee80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.484+0000 7fdefaffd640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdee80a3d20 0x7fdee80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.484+0000 7fdf00a11640 1 -- 172.21.15.19:0/1716833219 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fdee40f0c30 con 0x7fdefc107280 2024-04-23T18:33:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.484+0000 7fdefaffd640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdee80a3d20 0x7fdee80a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009bd0 tx=0x7fdeec005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:01.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.489+0000 7fdf00a11640 1 -- 172.21.15.19:0/1716833219 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdee40bd300 con 0x7fdefc107280 2024-04-23T18:33:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.715+0000 7fdf01a13640 1 -- 172.21.15.19:0/1716833219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdefc10a4a0 con 0x7fdefc107280 2024-04-23T18:33:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.716+0000 7fdf00a11640 1 -- 172.21.15.19:0/1716833219 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7fdee40bcc00 con 0x7fdefc107280 2024-04-23T18:33:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:33:01.708 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:33:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:33:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.717+0000 7fdf01a13640 1 -- 172.21.15.19:0/1716833219 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdee80a3d20 msgr2=0x7fdee80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.717+0000 7fdf01a13640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdee80a3d20 0x7fdee80a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009bd0 tx=0x7fdeec005f90 comp rx=0 tx=0).stop 2024-04-23T18:33:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.717+0000 7fdf01a13640 1 -- 172.21.15.19:0/1716833219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdefc107280 msgr2=0x7fdefc19e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.717+0000 7fdf01a13640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdefc107280 0x7fdefc19e9b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdee4006010 tx=0x7fdee4004060 comp rx=0 tx=0).stop 2024-04-23T18:33:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.718+0000 7fdf01a13640 1 -- 172.21.15.19:0/1716833219 shutdown_connections 2024-04-23T18:33:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.718+0000 7fdf01a13640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdee80a3d20 0x7fdee80a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.718+0000 7fdf01a13640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdefc198620 0x7fdefc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.718+0000 7fdf01a13640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdefc107280 0x7fdefc19e9b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.718+0000 7fdf01a13640 1 --2- 172.21.15.19:0/1716833219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdefc102240 0x7fdefc19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.718+0000 7fdf01a13640 1 -- 172.21.15.19:0/1716833219 >> 172.21.15.19:0/1716833219 conn(0x7fdefc0790e0 msgr2=0x7fdefc100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.718+0000 7fdf01a13640 1 -- 172.21.15.19:0/1716833219 shutdown_connections 2024-04-23T18:33:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.718+0000 7fdf01a13640 1 -- 172.21.15.19:0/1716833219 wait complete. 2024-04-23T18:33:01.721 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 85.0 --await-for 42.0' 2024-04-23T18:33:01.721 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 85.0 --await-for 42.0 2024-04-23T18:33:01.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.840+0000 7f7659b9b640 1 -- 172.21.15.19:0/2375215452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7654103b50 msgr2=0x7f765410c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.840+0000 7f7659b9b640 1 --2- 172.21.15.19:0/2375215452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7654103b50 0x7f765410c030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7648009900 tx=0x7f764802a480 comp rx=0 tx=0).stop 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.840+0000 7f7659b9b640 1 -- 172.21.15.19:0/2375215452 shutdown_connections 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.840+0000 7f7659b9b640 1 --2- 172.21.15.19:0/2375215452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765410c570 0x7f765410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.840+0000 7f7659b9b640 1 --2- 172.21.15.19:0/2375215452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7654103b50 0x7f765410c030 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.840+0000 7f7659b9b640 1 --2- 172.21.15.19:0/2375215452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7654103230 0x7f7654103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.840+0000 7f7659b9b640 1 -- 172.21.15.19:0/2375215452 >> 172.21.15.19:0/2375215452 conn(0x7f7654077fb0 msgr2=0x7f76540783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.840+0000 7f7659b9b640 1 -- 172.21.15.19:0/2375215452 shutdown_connections 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.840+0000 7f7659b9b640 1 -- 172.21.15.19:0/2375215452 wait complete. 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f7659b9b640 1 Processor -- start 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f7659b9b640 1 -- start start 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f7659b9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7654103230 0x7f7654198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f7659b9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7654103b50 0x7f7654198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f7659b9b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765410c570 0x7f765419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f7659b9b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76541121d0 con 0x7f7654103230 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f7659b9b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7654112050 con 0x7f7654103b50 2024-04-23T18:33:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f7659b9b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7654112350 con 0x7f765410c570 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f76537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7654103230 0x7f7654198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f76537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7654103230 0x7f7654198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34094/0 (socket says 172.21.15.19:34094) 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f76537fe640 1 -- 172.21.15.19:0/2866951353 learned_addr learned my addr 172.21.15.19:0/2866951353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f7652ffd640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7654103b50 0x7f7654198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.841+0000 7f7653fff640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765410c570 0x7f765419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f76537fe640 1 -- 172.21.15.19:0/2866951353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765410c570 msgr2=0x7f765419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f76537fe640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765410c570 0x7f765419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f76537fe640 1 -- 172.21.15.19:0/2866951353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7654103b50 msgr2=0x7f7654198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f76537fe640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7654103b50 0x7f7654198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f76537fe640 1 -- 172.21.15.19:0/2866951353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765419d720 con 0x7f7654103230 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f7652ffd640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7654103b50 0x7f7654198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f7653fff640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765410c570 0x7f765419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f76537fe640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7654103230 0x7f7654198730 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f764400d950 tx=0x7f764400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f7650ff9640 1 -- 172.21.15.19:0/2866951353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644017070 con 0x7f7654103230 2024-04-23T18:33:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f7650ff9640 1 -- 172.21.15.19:0/2866951353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7644009c30 con 0x7f7654103230 2024-04-23T18:33:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f7659b9b640 1 -- 172.21.15.19:0/2866951353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765419da10 con 0x7f7654103230 2024-04-23T18:33:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f7650ff9640 1 -- 172.21.15.19:0/2866951353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764400f620 con 0x7f7654103230 2024-04-23T18:33:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.842+0000 7f7659b9b640 1 -- 172.21.15.19:0/2866951353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76541a52a0 con 0x7f7654103230 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.844+0000 7f7650ff9640 1 -- 172.21.15.19:0/2866951353 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7644010460 con 0x7f7654103230 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.844+0000 7f765c992740 1 Processor -- start 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.844+0000 7f765c992740 1 -- start start 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f765c992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a25ffa0 0x56374a260380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f765c992740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f765c992740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a26f4a0 0x56374a271890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f765c992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56374a1c5370 con 0x56374a25ffa0 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f765c992740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56374a1622e0 con 0x56374a2608c0 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f765c992740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56374a1c1330 con 0x56374a26f4a0 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7653fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a25ffa0 0x56374a260380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7653fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a25ffa0 0x56374a260380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34096/0 (socket says 172.21.15.19:34096) 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7653fff640 1 -- 172.21.15.19:0/3779499828 learned_addr learned my addr 172.21.15.19:0/3779499828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7650ff9640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f763c0a3d20 0x7f763c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7653fff640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f763c0a3d20 0x7f763c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f76537fe640 1 --2- 172.21.15.19:0/3779499828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7652ffd640 1 --2- 172.21.15.19:0/3779499828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a26f4a0 0x56374a271890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7650ff9640 1 -- 172.21.15.19:0/2866951353 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f76440d04e0 con 0x7f7654103230 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7653fff640 1 -- 172.21.15.19:0/3779499828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a26f4a0 msgr2=0x56374a271890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7653fff640 1 --2- 172.21.15.19:0/3779499828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a26f4a0 0x56374a271890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7653fff640 1 -- 172.21.15.19:0/3779499828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 msgr2=0x56374a26ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7653fff640 1 --2- 172.21.15.19:0/3779499828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7653fff640 1 -- 172.21.15.19:0/3779499828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563749fa3d80 con 0x56374a25ffa0 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f76537fe640 1 --2- 172.21.15.19:0/3779499828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.845+0000 7f7653fff640 1 --2- 172.21.15.19:0/3779499828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a25ffa0 0x56374a260380 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f764c013a80 tx=0x7f764c013f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d65ad6f873b21a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f7639ffb640 1 -- 172.21.15.19:0/3779499828 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c022020 con 0x56374a25ffa0 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f7639ffb640 1 -- 172.21.15.19:0/3779499828 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f764c0133d0 con 0x56374a25ffa0 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f7639ffb640 1 -- 172.21.15.19:0/3779499828 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c010a00 con 0x56374a25ffa0 2024-04-23T18:33:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f7653fff640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f763c0a3d20 0x7f763c0a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f764c023000 tx=0x7f764c021000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f765c992740 1 -- 172.21.15.19:0/3779499828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a25ffa0 msgr2=0x56374a260380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f765c992740 1 --2- 172.21.15.19:0/3779499828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a25ffa0 0x56374a260380 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f764c013a80 tx=0x7f764c013f40 comp rx=0 tx=0).stop 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f765c992740 1 -- 172.21.15.19:0/3779499828 shutdown_connections 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f765c992740 1 --2- 172.21.15.19:0/3779499828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f765c992740 1 --2- 172.21.15.19:0/3779499828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a26f4a0 0x56374a271890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f765c992740 1 --2- 172.21.15.19:0/3779499828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a25ffa0 0x56374a260380 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f765c992740 1 -- 172.21.15.19:0/3779499828 >> 172.21.15.19:0/3779499828 conn(0x56374a25c3c0 msgr2=0x56374a25c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f765c992740 1 -- 172.21.15.19:0/3779499828 shutdown_connections 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.846+0000 7f765c992740 1 -- 172.21.15.19:0/3779499828 wait complete. 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f765c992740 1 -- start start 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f765c992740 1 Processor -- start 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f765c992740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a26c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f765c992740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f765c992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a26cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f765c992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56374a272260 con 0x56374a26f4a0 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f765c992740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56374a15ffb0 con 0x56374a2608c0 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f765c992740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56374a2723e0 con 0x56374a25ffa0 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f765c992740 10 client.0 _ll_register_callbacks cb 0x56374a15f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f7653fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a26cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f7653fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a26cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34104/0 (socket says 172.21.15.19:34104) 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f7653fff640 1 -- 172.21.15.19:0/535993405 learned_addr learned my addr 172.21.15.19:0/535993405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.847+0000 7f76537fe640 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a26c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7652ffd640 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7653fff640 1 -- 172.21.15.19:0/535993405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 msgr2=0x56374a26c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7653fff640 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a26c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7653fff640 1 -- 172.21.15.19:0/535993405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 msgr2=0x56374a26c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7653fff640 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7653fff640 1 -- 172.21.15.19:0/535993405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374a004510 con 0x56374a26f4a0 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7652ffd640 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7653fff640 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a26cea0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f764c009700 tx=0x7f764c005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4867da7792d19de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7632ffd640 1 -- 172.21.15.19:0/535993405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c022070 con 0x56374a26f4a0 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7632ffd640 1 -- 172.21.15.19:0/535993405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f764c01e9d0 con 0x56374a26f4a0 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f765c992740 1 -- 172.21.15.19:0/535993405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f764c0133d0 con 0x56374a26f4a0 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7632ffd640 1 -- 172.21.15.19:0/535993405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f764c01ebd0 con 0x56374a26f4a0 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f765c992740 10 client.16539 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f765c992740 1 -- 172.21.15.19:0/535993405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563749e97670 con 0x56374a26f4a0 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.848+0000 7f7632ffd640 1 -- 172.21.15.19:0/535993405 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f764c0133d0 con 0x56374a26f4a0 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.849+0000 7f7632ffd640 1 -- 172.21.15.19:0/535993405 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f764c02e030 con 0x56374a26f4a0 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.849+0000 7f765c992740 10 client.16539 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.849+0000 7f765c992740 10 client.16539 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.849+0000 7f765c992740 20 client.16539 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.849+0000 7f765c992740 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56374a27f610 0x56374a281a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.849+0000 7f765c992740 4 client.16539 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.849+0000 7f765c992740 1 -- 172.21.15.19:0/535993405 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56374a1d2d90 con 0x56374a27f610 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.849+0000 7f76537fe640 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56374a27f610 0x56374a281a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.849+0000 7f76537fe640 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56374a27f610 0x56374a281a00 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0c80da5f8e76c1 server_cookie=490745abd4c23794 in_seq=0 out_seq=0 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.849+0000 7f7632ffd640 10 client.16539 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f7632ffd640 1 -- 172.21.15.19:0/535993405 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56374a1d2d90 con 0x56374a27f610 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f7632ffd640 10 client.16539 handle_command_reply: tid=0 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 client.16539 shutdown 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 -- 172.21.15.19:0/535993405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 msgr2=0x56374a26cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a26cea0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f764c009700 tx=0x7f764c005ac0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 -- 172.21.15.19:0/535993405 shutdown_connections 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56374a27f610 0x56374a281a00 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a26cea0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 --2- 172.21.15.19:0/535993405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a26c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 -- 172.21.15.19:0/535993405 >> 172.21.15.19:0/535993405 conn(0x56374a25c3c0 msgr2=0x56374a262120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f7632ffd640 0 client.16539 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 -- 172.21.15.19:0/535993405 shutdown_connections 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 1 -- 172.21.15.19:0/535993405 wait complete. 2024-04-23T18:33:01.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.850+0000 7f765c992740 20 client.16539 trim_cache size 0 max 16384 2024-04-23T18:33:01.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f765c992740 1 Processor -- start 2024-04-23T18:33:01.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f765c992740 1 -- start start 2024-04-23T18:33:01.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f765c992740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a27b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f765c992740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f765c992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a2872b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f765c992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76541ab980 con 0x56374a26f4a0 2024-04-23T18:33:01.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f765c992740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76541ad4c0 con 0x56374a2608c0 2024-04-23T18:33:01.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f765c992740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76541ac050 con 0x56374a25ffa0 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f7653fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a2872b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f7653fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a2872b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34120/0 (socket says 172.21.15.19:34120) 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f7653fff640 1 -- 172.21.15.19:0/3655894033 learned_addr learned my addr 172.21.15.19:0/3655894033 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f76537fe640 1 --2- 172.21.15.19:0/3655894033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a27b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.868+0000 7f7652ffd640 1 --2- 172.21.15.19:0/3655894033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.869+0000 7f7653fff640 1 -- 172.21.15.19:0/3655894033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 msgr2=0x56374a27b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.869+0000 7f7653fff640 1 --2- 172.21.15.19:0/3655894033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a27b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.869+0000 7f7653fff640 1 -- 172.21.15.19:0/3655894033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 msgr2=0x56374a26b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.869+0000 7f7653fff640 1 --2- 172.21.15.19:0/3655894033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.869+0000 7f7653fff640 1 -- 172.21.15.19:0/3655894033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374a2606f0 con 0x56374a26f4a0 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.869+0000 7f7652ffd640 1 --2- 172.21.15.19:0/3655894033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.869+0000 7f7653fff640 1 --2- 172.21.15.19:0/3655894033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a2872b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f764c010bf0 tx=0x7f764c010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8fe047955d24d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.869+0000 7f76397fa640 1 -- 172.21.15.19:0/3655894033 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c03c020 con 0x56374a26f4a0 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.869+0000 7f76397fa640 1 -- 172.21.15.19:0/3655894033 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f764c0105d0 con 0x56374a26f4a0 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.869+0000 7f76397fa640 1 -- 172.21.15.19:0/3655894033 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c022040 con 0x56374a26f4a0 2024-04-23T18:33:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 -- 172.21.15.19:0/3655894033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 msgr2=0x56374a2872b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 --2- 172.21.15.19:0/3655894033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a2872b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f764c010bf0 tx=0x7f764c010ad0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 -- 172.21.15.19:0/3655894033 shutdown_connections 2024-04-23T18:33:01.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 --2- 172.21.15.19:0/3655894033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a2608c0 0x56374a26b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 --2- 172.21.15.19:0/3655894033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a26f4a0 0x56374a2872b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 --2- 172.21.15.19:0/3655894033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a27b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 -- 172.21.15.19:0/3655894033 >> 172.21.15.19:0/3655894033 conn(0x56374a25c3c0 msgr2=0x56374a26e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:01.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 -- 172.21.15.19:0/3655894033 shutdown_connections 2024-04-23T18:33:01.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 -- 172.21.15.19:0/3655894033 wait complete. 2024-04-23T18:33:01.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 -- start start 2024-04-23T18:33:01.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 Processor -- start 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a2690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a2608c0 0x56374a2695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.870+0000 7f765c992740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a26f4a0 0x56374a269b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f765c992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76541acdf0 con 0x56374a2608c0 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f765c992740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76541ac720 con 0x56374a26f4a0 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f765c992740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76541a9960 con 0x56374a25ffa0 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f765c992740 10 client.0 _ll_register_callbacks cb 0x7f764c00f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7652ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a2608c0 0x56374a2695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7652ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a2608c0 0x56374a2695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34134/0 (socket says 172.21.15.19:34134) 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7652ffd640 1 -- 172.21.15.19:0/2967359846 learned_addr learned my addr 172.21.15.19:0/2967359846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f76537fe640 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a2690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7653fff640 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a26f4a0 0x56374a269b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7652ffd640 1 -- 172.21.15.19:0/2967359846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 msgr2=0x56374a2690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7652ffd640 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a2690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7652ffd640 1 -- 172.21.15.19:0/2967359846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a26f4a0 msgr2=0x56374a269b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7652ffd640 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a26f4a0 0x56374a269b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7652ffd640 1 -- 172.21.15.19:0/2967359846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374a2e4b20 con 0x56374a2608c0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7652ffd640 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a2608c0 0x56374a2695f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7648005940 tx=0x7f76480316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b542c3e590fa3f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7632ffd640 1 -- 172.21.15.19:0/2967359846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764803f070 con 0x56374a2608c0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7632ffd640 1 -- 172.21.15.19:0/2967359846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7648004250 con 0x56374a2608c0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f765c992740 1 -- 172.21.15.19:0/2967359846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f764c0105d0 con 0x56374a2608c0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f7632ffd640 1 -- 172.21.15.19:0/2967359846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7648035070 con 0x56374a2608c0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f765c992740 10 client.16545 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.871+0000 7f765c992740 1 -- 172.21.15.19:0/2967359846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56374a2e5c50 con 0x56374a2608c0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.872+0000 7f7632ffd640 1 -- 172.21.15.19:0/2967359846 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f764c0105d0 con 0x56374a2608c0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.872+0000 7f7632ffd640 1 -- 172.21.15.19:0/2967359846 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7648004a60 con 0x56374a2608c0 2024-04-23T18:33:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.872+0000 7f765c992740 10 client.16545 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:01.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.872+0000 7f765c992740 10 client.16545 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:33:01.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.872+0000 7f765c992740 20 client.16545 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:01.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.872+0000 7f765c992740 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56374a2e6470 0x56374a2e8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:01.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.872+0000 7f765c992740 4 client.16545 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.0, "await_for": 42.0}] 2024-04-23T18:33:01.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.872+0000 7f765c992740 1 -- 172.21.15.19:0/2967359846 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.0, "await_for": 42.0}) -- 0x56374a1ccc00 con 0x56374a2e6470 2024-04-23T18:33:01.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.872+0000 7f76537fe640 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56374a2e6470 0x56374a2e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:01.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.873+0000 7f76537fe640 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56374a2e6470 0x56374a2e8890 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50ccb09c998c7c37 server_cookie=202f61560c81c92c in_seq=0 out_seq=0 2024-04-23T18:33:01.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.873+0000 7f7632ffd640 10 client.16545 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.888+0000 7f7632ffd640 1 -- 172.21.15.19:0/2967359846 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56374a1ccc00 con 0x56374a2e6470 2024-04-23T18:33:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.888+0000 7f7632ffd640 10 client.16545 handle_command_reply: tid=0 2024-04-23T18:33:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.888+0000 7f765c992740 1 client.16545 shutdown 2024-04-23T18:33:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 1 -- 172.21.15.19:0/2967359846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a2608c0 msgr2=0x56374a2695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a2608c0 0x56374a2695f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7648005940 tx=0x7f76480316d0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 1 -- 172.21.15.19:0/2967359846 shutdown_connections 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56374a2e6470 0x56374a2e8890 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56374a26f4a0 0x56374a269b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7632ffd640 0 client.16545 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56374a2608c0 0x56374a2695f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 1 --2- 172.21.15.19:0/2967359846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374a25ffa0 0x56374a2690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 1 -- 172.21.15.19:0/2967359846 >> 172.21.15.19:0/2967359846 conn(0x56374a25c3c0 msgr2=0x56374a26dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 1 -- 172.21.15.19:0/2967359846 shutdown_connections 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 1 -- 172.21.15.19:0/2967359846 wait complete. 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f765c992740 20 client.16545 trim_cache size 0 max 16384 2024-04-23T18:33:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 30, 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "e34d7e48": { 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 30, 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:33:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 85.0, 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:01.884 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:33:01.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7659b9b640 1 -- 172.21.15.19:0/2866951353 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f763c0a3d20 msgr2=0x7f763c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7659b9b640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f763c0a3d20 0x7f763c0a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f764c023000 tx=0x7f764c021000 comp rx=0 tx=0).stop 2024-04-23T18:33:01.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7659b9b640 1 -- 172.21.15.19:0/2866951353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7654103230 msgr2=0x7f7654198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:01.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7659b9b640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7654103230 0x7f7654198730 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f764400d950 tx=0x7f764400de10 comp rx=0 tx=0).stop 2024-04-23T18:33:01.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7659b9b640 1 -- 172.21.15.19:0/2866951353 shutdown_connections 2024-04-23T18:33:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7659b9b640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f763c0a3d20 0x7f763c0a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7659b9b640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7654103b50 0x7f7654198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7659b9b640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765410c570 0x7f765419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7659b9b640 1 --2- 172.21.15.19:0/2866951353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7654103230 0x7f7654198730 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.889+0000 7f7659b9b640 1 -- 172.21.15.19:0/2866951353 >> 172.21.15.19:0/2866951353 conn(0x7f7654077fb0 msgr2=0x7f76540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.890+0000 7f7659b9b640 1 -- 172.21.15.19:0/2866951353 shutdown_connections 2024-04-23T18:33:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:01.890+0000 7f7659b9b640 1 -- 172.21.15.19:0/2866951353 wait complete. 2024-04-23T18:33:01.892 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e34d7e48', quiesce took 0.5 seconds. Will release after: 25.0 2024-04-23T18:33:02.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:01 smithi019 ceph-mon[29215]: pgmap v209: 97 pgs: 97 active+clean; 4.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-04-23T18:33:02.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:01 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1716833219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:02.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:01 smithi044 ceph-mon[32458]: pgmap v209: 97 pgs: 97 active+clean; 4.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-04-23T18:33:02.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:01 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1716833219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:02.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:01 smithi155 ceph-mon[33468]: pgmap v209: 97 pgs: 97 active+clean; 4.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-04-23T18:33:02.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:01 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1716833219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:04.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:03 smithi019 ceph-mon[29215]: pgmap v210: 97 pgs: 97 active+clean; 4.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T18:33:04.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:03 smithi044 ceph-mon[32458]: pgmap v210: 97 pgs: 97 active+clean; 4.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T18:33:04.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:03 smithi155 ceph-mon[33468]: pgmap v210: 97 pgs: 97 active+clean; 4.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T18:33:06.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:05 smithi019 ceph-mon[29215]: pgmap v211: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-04-23T18:33:06.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:05 smithi044 ceph-mon[32458]: pgmap v211: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-04-23T18:33:06.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:05 smithi155 ceph-mon[33468]: pgmap v211: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-04-23T18:33:07.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:06 smithi019 ceph-mon[29215]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:33:07.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:06 smithi044 ceph-mon[32458]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:33:07.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:06 smithi155 ceph-mon[33468]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:33:08.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:07 smithi019 ceph-mon[29215]: pgmap v213: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-04-23T18:33:08.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:07 smithi044 ceph-mon[32458]: pgmap v213: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-04-23T18:33:08.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:07 smithi155 ceph-mon[33468]: pgmap v213: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-04-23T18:33:09.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:08 smithi019 ceph-mon[29215]: pgmap v214: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-04-23T18:33:09.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:08 smithi044 ceph-mon[32458]: pgmap v214: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-04-23T18:33:09.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:08 smithi155 ceph-mon[33468]: pgmap v214: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-04-23T18:33:11.561 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:11 smithi044 ceph-mon[32458]: pgmap v215: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-04-23T18:33:11.571 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:11 smithi155 ceph-mon[33468]: pgmap v215: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-04-23T18:33:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.612+0000 7f5c6017d640 1 -- 172.21.15.19:0/611774083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c58106170 msgr2=0x7f5c5810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.612+0000 7f5c6017d640 1 --2- 172.21.15.19:0/611774083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c58106170 0x7f5c5810dd40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009ca0 tx=0x7f5c5002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.612+0000 7f5c6017d640 1 -- 172.21.15.19:0/611774083 shutdown_connections 2024-04-23T18:33:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.612+0000 7f5c6017d640 1 --2- 172.21.15.19:0/611774083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c58106170 0x7f5c5810dd40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.612+0000 7f5c6017d640 1 --2- 172.21.15.19:0/611774083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c58101aa0 0x7f5c58105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.612+0000 7f5c6017d640 1 --2- 172.21.15.19:0/611774083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c581010f0 0x7f5c581014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.612+0000 7f5c6017d640 1 -- 172.21.15.19:0/611774083 >> 172.21.15.19:0/611774083 conn(0x7f5c58077fd0 msgr2=0x7f5c580783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.612+0000 7f5c6017d640 1 -- 172.21.15.19:0/611774083 shutdown_connections 2024-04-23T18:33:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.612+0000 7f5c6017d640 1 -- 172.21.15.19:0/611774083 wait complete. 2024-04-23T18:33:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c6017d640 1 Processor -- start 2024-04-23T18:33:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c6017d640 1 -- start start 2024-04-23T18:33:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c6017d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c581010f0 0x7f5c581972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c6017d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c58106170 0x7f5c58197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c6017d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c58197d70 0x7f5c581a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c6017d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581110e0 con 0x7f5c581010f0 2024-04-23T18:33:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c6017d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58110f60 con 0x7f5c58106170 2024-04-23T18:33:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c6017d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58111260 con 0x7f5c58197d70 2024-04-23T18:33:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c5df19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c581010f0 0x7f5c581972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c5df19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c581010f0 0x7f5c581972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34058/0 (socket says 172.21.15.19:34058) 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c5df19640 1 -- 172.21.15.19:0/1702097143 learned_addr learned my addr 172.21.15.19:0/1702097143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c5d718640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c58106170 0x7f5c58197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c58197d70 0x7f5c581a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c5df19640 1 -- 172.21.15.19:0/1702097143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c58197d70 msgr2=0x7f5c581a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c5df19640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c58197d70 0x7f5c581a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c5df19640 1 -- 172.21.15.19:0/1702097143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c58106170 msgr2=0x7f5c58197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c5df19640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c58106170 0x7f5c58197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.613+0000 7f5c5df19640 1 -- 172.21.15.19:0/1702097143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c581a6f00 con 0x7f5c581010f0 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.614+0000 7f5c5df19640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c581010f0 0x7f5c581972f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400d950 tx=0x7f5c5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.614+0000 7f5c4effd640 1 -- 172.21.15.19:0/1702097143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54017070 con 0x7f5c581010f0 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.614+0000 7f5c4effd640 1 -- 172.21.15.19:0/1702097143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c54009c30 con 0x7f5c581010f0 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.614+0000 7f5c6017d640 1 -- 172.21.15.19:0/1702097143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c581a71f0 con 0x7f5c581010f0 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.614+0000 7f5c4effd640 1 -- 172.21.15.19:0/1702097143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5400f620 con 0x7f5c581010f0 2024-04-23T18:33:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.614+0000 7f5c6017d640 1 -- 172.21.15.19:0/1702097143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c581a7710 con 0x7f5c581010f0 2024-04-23T18:33:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.616+0000 7f5c4effd640 1 -- 172.21.15.19:0/1702097143 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5c54010460 con 0x7f5c581010f0 2024-04-23T18:33:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.616+0000 7f5c62f73740 1 Processor -- start 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c62f73740 1 -- start start 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8f40 0x5640684d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684d3800 0x5640684d5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c62f73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56406842d970 con 0x5640684c8620 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c62f73740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5640683ca4a0 con 0x5640684c8f40 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c62f73740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564068429930 con 0x5640684d3800 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5e71a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5e71a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34066/0 (socket says 172.21.15.19:34066) 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5e71a640 1 -- 172.21.15.19:0/428579443 learned_addr learned my addr 172.21.15.19:0/428579443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c4effd640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5c440a3dc0 0x7f5c440a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5df19640 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8f40 0x5640684d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5d718640 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684d3800 0x5640684d5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5e71a640 1 -- 172.21.15.19:0/428579443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684d3800 msgr2=0x5640684d5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684d3800 0x5640684d5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5e71a640 1 -- 172.21.15.19:0/428579443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8f40 msgr2=0x5640684d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8f40 0x5640684d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5e71a640 1 -- 172.21.15.19:0/428579443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564068140210 con 0x5640684c8620 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5d718640 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684d3800 0x5640684d5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.617+0000 7f5c5df19640 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8f40 0x5640684d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.618+0000 7f5c4effd640 1 -- 172.21.15.19:0/1702097143 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f5c540d04e0 con 0x7f5c581010f0 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.618+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5c440a3dc0 0x7f5c440a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.618+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684c8a00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5c50005cc0 tx=0x7f5c50009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc1f52a2eb27f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.618+0000 7f5c43fff640 1 -- 172.21.15.19:0/428579443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5004d070 con 0x5640684c8620 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.618+0000 7f5c43fff640 1 -- 172.21.15.19:0/428579443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50036040 con 0x5640684c8620 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.618+0000 7f5c43fff640 1 -- 172.21.15.19:0/428579443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5002f660 con 0x5640684c8620 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 -- 172.21.15.19:0/428579443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 msgr2=0x5640684c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5c440a3dc0 0x7f5c440a6280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c5004a000 tx=0x7f5c50047000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684c8a00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5c50005cc0 tx=0x7f5c50009880 comp rx=0 tx=0).stop 2024-04-23T18:33:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 -- 172.21.15.19:0/428579443 shutdown_connections 2024-04-23T18:33:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8f40 0x5640684d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684d3800 0x5640684d5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 --2- 172.21.15.19:0/428579443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684c8a00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 -- 172.21.15.19:0/428579443 >> 172.21.15.19:0/428579443 conn(0x5640684c4b30 msgr2=0x5640684c4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 -- 172.21.15.19:0/428579443 shutdown_connections 2024-04-23T18:33:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 -- 172.21.15.19:0/428579443 wait complete. 2024-04-23T18:33:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 -- start start 2024-04-23T18:33:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.619+0000 7f5c62f73740 1 Processor -- start 2024-04-23T18:33:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 0x5640684d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c62f73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5640684d62b0 con 0x5640684d3800 2024-04-23T18:33:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c62f73740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5640683c80b0 con 0x5640684c8620 2024-04-23T18:33:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c62f73740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5640684d6430 con 0x5640684c8f40 2024-04-23T18:33:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c62f73740 10 client.0 _ll_register_callbacks cb 0x5640683b5d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5e71a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5e71a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34082/0 (socket says 172.21.15.19:34082) 2024-04-23T18:33:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5e71a640 1 -- 172.21.15.19:0/1246723675 learned_addr learned my addr 172.21.15.19:0/1246723675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5df19640 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 0x5640684d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5d718640 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5e71a640 1 -- 172.21.15.19:0/1246723675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 msgr2=0x5640684d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5e71a640 1 -- 172.21.15.19:0/1246723675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 msgr2=0x5640684d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 0x5640684d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5e71a640 1 -- 172.21.15.19:0/1246723675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56406820c620 con 0x5640684d3800 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.620+0000 7f5c5d718640 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d0ac0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009430 tx=0x7f5c50004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f396e41ca45d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c417fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c417fa640 1 -- 172.21.15.19:0/1246723675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50059060 con 0x5640684d3800 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c417fa640 1 -- 172.21.15.19:0/1246723675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f5c50004250 con 0x5640684d3800 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c62f73740 1 -- 172.21.15.19:0/1246723675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c50036040 con 0x5640684d3800 2024-04-23T18:33:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c417fa640 1 -- 172.21.15.19:0/1246723675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c5004d040 con 0x5640684d3800 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c62f73740 10 client.16557 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c62f73740 1 -- 172.21.15.19:0/1246723675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5640684d62b0 con 0x5640684d3800 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c417fa640 1 -- 172.21.15.19:0/1246723675 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50036040 con 0x5640684d3800 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c417fa640 1 -- 172.21.15.19:0/1246723675 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5c500305c0 con 0x5640684d3800 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c62f73740 10 client.16557 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c62f73740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c62f73740 20 client.16557 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c62f73740 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5640684e3400 0x5640684e57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c62f73740 4 client.16557 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.621+0000 7f5c62f73740 1 -- 172.21.15.19:0/1246723675 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56406843b390 con 0x5640684e3400 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.622+0000 7f5c5df19640 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5640684e3400 0x5640684e57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.622+0000 7f5c417fa640 10 client.16557 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.622+0000 7f5c5df19640 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5640684e3400 0x5640684e57f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a9145f3c4ed728 server_cookie=ab12aeee82ed591c in_seq=0 out_seq=0 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c417fa640 1 -- 172.21.15.19:0/1246723675 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56406843b390 con 0x5640684e3400 2024-04-23T18:33:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c417fa640 10 client.16557 handle_command_reply: tid=0 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 client.16557 shutdown 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 -- 172.21.15.19:0/1246723675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 msgr2=0x5640684d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d0ac0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009430 tx=0x7f5c50004630 comp rx=0 tx=0).stop 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 -- 172.21.15.19:0/1246723675 shutdown_connections 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5640684e3400 0x5640684e57f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d0ac0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c417fa640 0 client.16557 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 --2- 172.21.15.19:0/1246723675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 0x5640684d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 -- 172.21.15.19:0/1246723675 >> 172.21.15.19:0/1246723675 conn(0x5640684c4b30 msgr2=0x5640684ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 -- 172.21.15.19:0/1246723675 shutdown_connections 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.623+0000 7f5c62f73740 1 -- 172.21.15.19:0/1246723675 wait complete. 2024-04-23T18:33:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.624+0000 7f5c62f73740 20 client.16557 trim_cache size 0 max 16384 2024-04-23T18:33:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.638+0000 7f5c62f73740 1 Processor -- start 2024-04-23T18:33:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.638+0000 7f5c62f73740 1 -- start start 2024-04-23T18:33:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 0x5640684df170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c62f73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581afe30 con 0x5640684d3800 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c62f73740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581b1970 con 0x5640684c8620 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c62f73740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581b0500 con 0x5640684c8f40 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5e71a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5e71a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34092/0 (socket says 172.21.15.19:34092) 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5e71a640 1 -- 172.21.15.19:0/3870777130 learned_addr learned my addr 172.21.15.19:0/3870777130 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5d718640 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5df19640 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 0x5640684df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5e71a640 1 -- 172.21.15.19:0/3870777130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 msgr2=0x5640684c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5e71a640 1 -- 172.21.15.19:0/3870777130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 msgr2=0x5640684df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 0x5640684df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5e71a640 1 -- 172.21.15.19:0/3870777130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640684c8d70 con 0x5640684d3800 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5df19640 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 0x5640684df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5d718640 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d2cf0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5c5002ade0 tx=0x7f5c5002bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965116c8f21bd6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c437fe640 1 -- 172.21.15.19:0/3870777130 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50059040 con 0x5640684d3800 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c437fe640 1 -- 172.21.15.19:0/3870777130 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50009800 con 0x5640684d3800 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.639+0000 7f5c437fe640 1 -- 172.21.15.19:0/3870777130 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50004250 con 0x5640684d3800 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.640+0000 7f5c62f73740 1 -- 172.21.15.19:0/3870777130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 msgr2=0x5640684d2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.640+0000 7f5c62f73740 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d2cf0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5c5002ade0 tx=0x7f5c5002bfd0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.640+0000 7f5c62f73740 1 -- 172.21.15.19:0/3870777130 shutdown_connections 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.640+0000 7f5c62f73740 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684d3800 0x5640684d2cf0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.640+0000 7f5c62f73740 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.640+0000 7f5c62f73740 1 --2- 172.21.15.19:0/3870777130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684c8620 0x5640684df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.640+0000 7f5c62f73740 1 -- 172.21.15.19:0/3870777130 >> 172.21.15.19:0/3870777130 conn(0x5640684c4b30 msgr2=0x5640684ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.640+0000 7f5c62f73740 1 -- 172.21.15.19:0/3870777130 shutdown_connections 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.640+0000 7f5c62f73740 1 -- 172.21.15.19:0/3870777130 wait complete. 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.640+0000 7f5c62f73740 1 -- start start 2024-04-23T18:33:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c62f73740 1 Processor -- start 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684db8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684d6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c62f73740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684d3800 0x5640684d71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c62f73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581b12a0 con 0x5640684c8620 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c62f73740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581b0bd0 con 0x5640684d3800 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c62f73740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581ade30 con 0x5640684c8f40 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c62f73740 10 client.0 _ll_register_callbacks cb 0x7f5c5002f3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5df19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684db8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5df19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684db8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34098/0 (socket says 172.21.15.19:34098) 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5df19640 1 -- 172.21.15.19:0/2511651273 learned_addr learned my addr 172.21.15.19:0/2511651273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5d718640 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684d6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5df19640 1 -- 172.21.15.19:0/2511651273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 msgr2=0x5640684d6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684d3800 0x5640684d71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5df19640 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684d6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5df19640 1 -- 172.21.15.19:0/2511651273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684d3800 msgr2=0x5640684d71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5df19640 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684d3800 0x5640684d71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5df19640 1 -- 172.21.15.19:0/2511651273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640684d7840 con 0x5640684c8620 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.641+0000 7f5c5e71a640 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684d3800 0x5640684d71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c5df19640 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684db8f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5c5409c390 tx=0x7f5c54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95770e3220b0154f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c417fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c417fa640 1 -- 172.21.15.19:0/2511651273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54020040 con 0x5640684c8620 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c417fa640 1 -- 172.21.15.19:0/2511651273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f5c540cde30 con 0x5640684c8620 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c62f73740 1 -- 172.21.15.19:0/2511651273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c50009800 con 0x5640684c8620 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c417fa640 1 -- 172.21.15.19:0/2511651273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c540ce030 con 0x5640684c8620 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c62f73740 10 client.16563 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c62f73740 1 -- 172.21.15.19:0/2511651273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564068549790 con 0x5640684c8620 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c417fa640 1 -- 172.21.15.19:0/2511651273 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50009800 con 0x5640684c8620 2024-04-23T18:33:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c417fa640 1 -- 172.21.15.19:0/2511651273 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5c54010750 con 0x5640684c8620 2024-04-23T18:33:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c62f73740 10 client.16563 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c62f73740 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c62f73740 20 client.16563 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c62f73740 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564068549e90 0x56406854c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c62f73740 4 client.16563 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.642+0000 7f5c62f73740 1 -- 172.21.15.19:0/2511651273 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564068435200 con 0x564068549e90 2024-04-23T18:33:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.643+0000 7f5c5d718640 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564068549e90 0x56406854c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.643+0000 7f5c5d718640 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564068549e90 0x56406854c2b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dec8827b00e4119 server_cookie=10fe387e00652181 in_seq=0 out_seq=0 2024-04-23T18:33:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.643+0000 7f5c417fa640 10 client.16563 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.658+0000 7f5c417fa640 1 -- 172.21.15.19:0/2511651273 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564068435200 con 0x564068549e90 2024-04-23T18:33:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c417fa640 10 client.16563 handle_command_reply: tid=0 2024-04-23T18:33:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 client.16563 shutdown 2024-04-23T18:33:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 -- 172.21.15.19:0/2511651273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 msgr2=0x5640684db8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684db8f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5c5409c390 tx=0x7f5c54005880 comp rx=0 tx=0).stop 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 -- 172.21.15.19:0/2511651273 shutdown_connections 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564068549e90 0x56406854c2b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640684d3800 0x5640684d71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640684c8f40 0x5640684d6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c417fa640 0 client.16563 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 --2- 172.21.15.19:0/2511651273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640684c8620 0x5640684db8f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 -- 172.21.15.19:0/2511651273 >> 172.21.15.19:0/2511651273 conn(0x5640684c4b30 msgr2=0x5640684ca020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 -- 172.21.15.19:0/2511651273 shutdown_connections 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 1 -- 172.21.15.19:0/2511651273 wait complete. 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.659+0000 7f5c62f73740 20 client.16563 trim_cache size 0 max 16384 2024-04-23T18:33:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:33:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:33:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:33:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 -- 172.21.15.19:0/1702097143 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5c440a3dc0 msgr2=0x7f5c440a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5c440a3dc0 0x7f5c440a6280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c5004a000 tx=0x7f5c50047000 comp rx=0 tx=0).stop 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 -- 172.21.15.19:0/1702097143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c581010f0 msgr2=0x7f5c581972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c581010f0 0x7f5c581972f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400d950 tx=0x7f5c5400de10 comp rx=0 tx=0).stop 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 -- 172.21.15.19:0/1702097143 shutdown_connections 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5c440a3dc0 0x7f5c440a6280 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c58106170 0x7f5c58197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c58197d70 0x7f5c581a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 --2- 172.21.15.19:0/1702097143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c581010f0 0x7f5c581972f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 -- 172.21.15.19:0/1702097143 >> 172.21.15.19:0/1702097143 conn(0x7f5c58077fd0 msgr2=0x7f5c580fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 -- 172.21.15.19:0/1702097143 shutdown_connections 2024-04-23T18:33:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:11.660+0000 7f5c6017d640 1 -- 172.21.15.19:0/1702097143 wait complete. 2024-04-23T18:33:11.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:11 smithi019 ceph-mon[29215]: pgmap v215: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-04-23T18:33:14.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:13 smithi019 ceph-mon[29215]: pgmap v216: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-04-23T18:33:14.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:13 smithi019 ceph-mon[29215]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:33:14.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:13 smithi044 ceph-mon[32458]: pgmap v216: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-04-23T18:33:14.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:13 smithi044 ceph-mon[32458]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:33:14.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:13 smithi155 ceph-mon[33468]: pgmap v216: 97 pgs: 97 active+clean; 4.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-04-23T18:33:14.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:13 smithi155 ceph-mon[33468]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:33:16.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:15 smithi019 ceph-mon[29215]: pgmap v218: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-23T18:33:16.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:15 smithi044 ceph-mon[32458]: pgmap v218: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-23T18:33:16.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:15 smithi155 ceph-mon[33468]: pgmap v218: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-23T18:33:18.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:17 smithi019 ceph-mon[29215]: pgmap v219: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-23T18:33:18.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:17 smithi044 ceph-mon[32458]: pgmap v219: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-23T18:33:18.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:17 smithi155 ceph-mon[33468]: pgmap v219: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-23T18:33:20.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:19 smithi019 ceph-mon[29215]: pgmap v220: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-23T18:33:20.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:19 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:20.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:19 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:20.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:19 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:20.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:19 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:33:20.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:19 smithi044 ceph-mon[32458]: pgmap v220: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-23T18:33:20.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:19 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:20.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:19 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:20.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:19 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:20.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:19 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:33:20.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:19 smithi155 ceph-mon[33468]: pgmap v220: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-23T18:33:20.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:19 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:20.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:19 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:20.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:19 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:20.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:19 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:33:22.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:21 smithi019 ceph-mon[29215]: pgmap v221: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-04-23T18:33:22.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:21 smithi044 ceph-mon[32458]: pgmap v221: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-04-23T18:33:22.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:21 smithi155 ceph-mon[33468]: pgmap v221: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-04-23T18:33:24.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:23 smithi019 ceph-mon[29215]: pgmap v222: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-04-23T18:33:24.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:23 smithi044 ceph-mon[32458]: pgmap v222: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-04-23T18:33:24.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:23 smithi155 ceph-mon[33468]: pgmap v222: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-04-23T18:33:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.759+0000 7f1c17679640 1 -- 172.21.15.19:0/945520371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c101021c0 msgr2=0x7f1c101025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.759+0000 7f1c17679640 1 --2- 172.21.15.19:0/945520371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c101021c0 0x7f1c101025a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009900 tx=0x7f1c0c02a410 comp rx=0 tx=0).stop 2024-04-23T18:33:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.759+0000 7f1c17679640 1 -- 172.21.15.19:0/945520371 shutdown_connections 2024-04-23T18:33:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.759+0000 7f1c17679640 1 --2- 172.21.15.19:0/945520371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c10107290 0x7f1c1010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.760+0000 7f1c17679640 1 --2- 172.21.15.19:0/945520371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10102b70 0x7f1c10106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.760+0000 7f1c17679640 1 --2- 172.21.15.19:0/945520371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c101021c0 0x7f1c101025a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.760+0000 7f1c17679640 1 -- 172.21.15.19:0/945520371 >> 172.21.15.19:0/945520371 conn(0x7f1c100fda20 msgr2=0x7f1c100ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.760+0000 7f1c17679640 1 -- 172.21.15.19:0/945520371 shutdown_connections 2024-04-23T18:33:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.760+0000 7f1c17679640 1 -- 172.21.15.19:0/945520371 wait complete. 2024-04-23T18:33:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.760+0000 7f1c17679640 1 Processor -- start 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.760+0000 7f1c17679640 1 -- start start 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.760+0000 7f1c17679640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c101021c0 0x7f1c101986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.760+0000 7f1c17679640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c10107290 0x7f1c10198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.760+0000 7f1c17679640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1019cfb0 0x7f1c1019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c17679640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101121f0 con 0x7f1c10107290 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c17679640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10112070 con 0x7f1c1019cfb0 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c17679640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10112370 con 0x7f1c101021c0 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c14c14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c10107290 0x7f1c10198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c14c14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c10107290 0x7f1c10198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48452/0 (socket says 172.21.15.19:48452) 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c14c14640 1 -- 172.21.15.19:0/509689183 learned_addr learned my addr 172.21.15.19:0/509689183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c15c16640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1019cfb0 0x7f1c1019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c15415640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c101021c0 0x7f1c101986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c14c14640 1 -- 172.21.15.19:0/509689183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c101021c0 msgr2=0x7f1c101986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c14c14640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c101021c0 0x7f1c101986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c14c14640 1 -- 172.21.15.19:0/509689183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1019cfb0 msgr2=0x7f1c1019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c14c14640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1019cfb0 0x7f1c1019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c14c14640 1 -- 172.21.15.19:0/509689183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1019daa0 con 0x7f1c10107290 2024-04-23T18:33:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c14c14640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c10107290 0x7f1c10198c30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e9f0 tx=0x7f1c0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c067fc640 1 -- 172.21.15.19:0/509689183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000ccc0 con 0x7f1c10107290 2024-04-23T18:33:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c17679640 1 -- 172.21.15.19:0/509689183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1019dd90 con 0x7f1c10107290 2024-04-23T18:33:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c067fc640 1 -- 172.21.15.19:0/509689183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c000054d0 con 0x7f1c10107290 2024-04-23T18:33:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.761+0000 7f1c067fc640 1 -- 172.21.15.19:0/509689183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00010430 con 0x7f1c10107290 2024-04-23T18:33:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.762+0000 7f1c17679640 1 -- 172.21.15.19:0/509689183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c101a5530 con 0x7f1c10107290 2024-04-23T18:33:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.764+0000 7f1c1a38d740 1 Processor -- start 2024-04-23T18:33:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.764+0000 7f1c1a38d740 1 -- start start 2024-04-23T18:33:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388136a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388136f40 0x5623881412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388141800 0x562388143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56238809b970 con 0x562388136620 2024-04-23T18:33:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5623880384a0 con 0x562388141800 2024-04-23T18:33:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562388097930 con 0x562388136f40 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c15c16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388136a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c15c16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388136a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48460/0 (socket says 172.21.15.19:48460) 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c15c16640 1 -- 172.21.15.19:0/2543194325 learned_addr learned my addr 172.21.15.19:0/2543194325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c14c14640 1 --2- 172.21.15.19:0/2543194325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388141800 0x562388143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c15415640 1 --2- 172.21.15.19:0/2543194325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388136f40 0x5623881412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c15c16640 1 -- 172.21.15.19:0/2543194325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388136f40 msgr2=0x5623881412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c15c16640 1 --2- 172.21.15.19:0/2543194325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388136f40 0x5623881412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c15c16640 1 -- 172.21.15.19:0/2543194325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388141800 msgr2=0x562388143bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c15c16640 1 --2- 172.21.15.19:0/2543194325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388141800 0x562388143bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c15c16640 1 -- 172.21.15.19:0/2543194325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562387dae210 con 0x562388136620 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.765+0000 7f1c15c16640 1 --2- 172.21.15.19:0/2543194325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388136a00 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009f70 tx=0x7f1c08010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5c3c5f3991228f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.766+0000 7f1bfb7fe640 1 -- 172.21.15.19:0/2543194325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08016070 con 0x562388136620 2024-04-23T18:33:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.766+0000 7f1bfb7fe640 1 -- 172.21.15.19:0/2543194325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c08004d20 con 0x562388136620 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.766+0000 7f1bfb7fe640 1 -- 172.21.15.19:0/2543194325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c080127d0 con 0x562388136620 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.766+0000 7f1c1a38d740 1 -- 172.21.15.19:0/2543194325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 msgr2=0x562388136a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.766+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/2543194325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388136a00 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009f70 tx=0x7f1c08010800 comp rx=0 tx=0).stop 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.766+0000 7f1c1a38d740 1 -- 172.21.15.19:0/2543194325 shutdown_connections 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.766+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/2543194325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388141800 0x562388143bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.766+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/2543194325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388136f40 0x5623881412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.766+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/2543194325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388136a00 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.766+0000 7f1c1a38d740 1 -- 172.21.15.19:0/2543194325 >> 172.21.15.19:0/2543194325 conn(0x562388132b30 msgr2=0x562388132f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.767+0000 7f1c067fc640 1 -- 172.21.15.19:0/509689183 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c000046c0 con 0x7f1c10107290 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.767+0000 7f1c1a38d740 1 -- 172.21.15.19:0/2543194325 shutdown_connections 2024-04-23T18:33:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.767+0000 7f1c1a38d740 1 -- 172.21.15.19:0/2543194325 wait complete. 2024-04-23T18:33:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.767+0000 7f1c1a38d740 1 -- start start 2024-04-23T18:33:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.767+0000 7f1c1a38d740 1 Processor -- start 2024-04-23T18:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c067fc640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bfc0a3cd0 0x7f1bfc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136620 0x562388141150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136f40 0x56238813e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388141800 0x56238813e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c067fc640 1 -- 172.21.15.19:0/509689183 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1c000d0ef0 con 0x7f1c10107290 2024-04-23T18:33:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562388144430 con 0x562388136f40 2024-04-23T18:33:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c15c16640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bfc0a3cd0 0x7f1bfc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5623880360b0 con 0x562388136620 2024-04-23T18:33:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5623881445b0 con 0x562388141800 2024-04-23T18:33:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c1a38d740 10 client.0 _ll_register_callbacks cb 0x562388023d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c14c14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136f40 0x56238813e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c14c14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136f40 0x56238813e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48476/0 (socket says 172.21.15.19:48476) 2024-04-23T18:33:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c14c14640 1 -- 172.21.15.19:0/1591388224 learned_addr learned my addr 172.21.15.19:0/1591388224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c15c16640 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388141800 0x56238813e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c15415640 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136620 0x562388141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c14c14640 1 -- 172.21.15.19:0/1591388224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388141800 msgr2=0x56238813e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c14c14640 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388141800 0x56238813e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c14c14640 1 -- 172.21.15.19:0/1591388224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136620 msgr2=0x562388141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c14c14640 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136620 0x562388141150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c14c14640 1 -- 172.21.15.19:0/1591388224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562387e7a620 con 0x562388136f40 2024-04-23T18:33:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c15c16640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bfc0a3cd0 0x7f1bfc0a6190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009d30 tx=0x7f1c08004ed0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.768+0000 7f1c14c14640 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136f40 0x56238813e440 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1c000a5610 tx=0x7f1c000163f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4d62b491e28da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1bf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/1591388224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c000b6060 con 0x562388136f40 2024-04-23T18:33:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/1591388224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1c000a56b0 con 0x562388136f40 2024-04-23T18:33:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1c1a38d740 1 -- 172.21.15.19:0/1591388224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c08004d20 con 0x562388136f40 2024-04-23T18:33:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/1591388224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c000a1280 con 0x562388136f40 2024-04-23T18:33:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1c1a38d740 10 client.16575 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1c1a38d740 1 -- 172.21.15.19:0/1591388224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623881503a0 con 0x562388136f40 2024-04-23T18:33:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/1591388224 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c08004d20 con 0x562388136f40 2024-04-23T18:33:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/1591388224 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1c0009c320 con 0x562388136f40 2024-04-23T18:33:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1c1a38d740 10 client.16575 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1c1a38d740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1c1a38d740 20 client.16575 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562388151860 0x562388153c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1c1a38d740 4 client.16575 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.769+0000 7f1c1a38d740 1 -- 172.21.15.19:0/1591388224 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623880a9390 con 0x562388151860 2024-04-23T18:33:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.770+0000 7f1c15415640 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562388151860 0x562388153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.770+0000 7f1c15415640 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562388151860 0x562388153c50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5569014f6fe484 server_cookie=4ebf456c23d1ac97 in_seq=0 out_seq=0 2024-04-23T18:33:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.770+0000 7f1bf8ff9640 10 client.16575 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/1591388224 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623880a9390 con 0x562388151860 2024-04-23T18:33:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1bf8ff9640 10 client.16575 handle_command_reply: tid=0 2024-04-23T18:33:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 client.16575 shutdown 2024-04-23T18:33:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 -- 172.21.15.19:0/1591388224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136f40 msgr2=0x56238813e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136f40 0x56238813e440 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1c000a5610 tx=0x7f1c000163f0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 -- 172.21.15.19:0/1591388224 shutdown_connections 2024-04-23T18:33:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562388151860 0x562388153c50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1bf8ff9640 0 client.16575 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388141800 0x56238813e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136f40 0x56238813e440 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/1591388224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136620 0x562388141150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 -- 172.21.15.19:0/1591388224 >> 172.21.15.19:0/1591388224 conn(0x562388132b30 msgr2=0x562388138860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 -- 172.21.15.19:0/1591388224 shutdown_connections 2024-04-23T18:33:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 1 -- 172.21.15.19:0/1591388224 wait complete. 2024-04-23T18:33:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.771+0000 7f1c1a38d740 20 client.16575 trim_cache size 0 max 16384 2024-04-23T18:33:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.786+0000 7f1c1a38d740 1 Processor -- start 2024-04-23T18:33:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.786+0000 7f1c1a38d740 1 -- start start 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.786+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388132b30 0x56238814d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.786+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388136620 0x562388159400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.786+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136f40 0x562388159940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.786+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101adba0 con 0x562388132b30 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.786+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101af6e0 con 0x562388136f40 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.786+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101ae270 con 0x562388136620 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c15415640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388132b30 0x56238814d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c15415640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388132b30 0x56238814d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48478/0 (socket says 172.21.15.19:48478) 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c15415640 1 -- 172.21.15.19:0/3337479314 learned_addr learned my addr 172.21.15.19:0/3337479314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c14c14640 1 --2- 172.21.15.19:0/3337479314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388136620 0x562388159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c15c16640 1 --2- 172.21.15.19:0/3337479314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136f40 0x562388159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c15415640 1 -- 172.21.15.19:0/3337479314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388136620 msgr2=0x562388159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c15415640 1 --2- 172.21.15.19:0/3337479314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388136620 0x562388159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c15415640 1 -- 172.21.15.19:0/3337479314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136f40 msgr2=0x562388159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c15415640 1 --2- 172.21.15.19:0/3337479314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136f40 0x562388159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c15415640 1 -- 172.21.15.19:0/3337479314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238813f2c0 con 0x562388132b30 2024-04-23T18:33:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1c15415640 1 --2- 172.21.15.19:0/3337479314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388132b30 0x56238814d2f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c031440 tx=0x7f1c0c009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144b681a7411fda1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1bfaffd640 1 -- 172.21.15.19:0/3337479314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c04a070 con 0x562388132b30 2024-04-23T18:33:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1bfaffd640 1 -- 172.21.15.19:0/3337479314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c045dc0 con 0x562388132b30 2024-04-23T18:33:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.787+0000 7f1bfaffd640 1 -- 172.21.15.19:0/3337479314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c035040 con 0x562388132b30 2024-04-23T18:33:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 -- 172.21.15.19:0/3337479314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388132b30 msgr2=0x56238814d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/3337479314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388132b30 0x56238814d2f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c031440 tx=0x7f1c0c009660 comp rx=0 tx=0).stop 2024-04-23T18:33:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 -- 172.21.15.19:0/3337479314 shutdown_connections 2024-04-23T18:33:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/3337479314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136f40 0x562388159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/3337479314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388136620 0x562388159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/3337479314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388132b30 0x56238814d2f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 -- 172.21.15.19:0/3337479314 >> 172.21.15.19:0/3337479314 conn(0x562388153fc0 msgr2=0x562388131cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 -- 172.21.15.19:0/3337479314 shutdown_connections 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 -- 172.21.15.19:0/3337479314 wait complete. 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 -- start start 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.788+0000 7f1c1a38d740 1 Processor -- start 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388132b30 0x5623881474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388147a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c1a38d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136f40 0x562388147f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101af010 con 0x562388136620 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101ae940 con 0x562388136f40 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c1a38d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101abba0 con 0x562388132b30 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c1a38d740 10 client.0 _ll_register_callbacks cb 0x7f1c0009caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c14c14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c14c14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388147a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48484/0 (socket says 172.21.15.19:48484) 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c14c14640 1 -- 172.21.15.19:0/86218550 learned_addr learned my addr 172.21.15.19:0/86218550 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c15c16640 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136f40 0x562388147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c15415640 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388132b30 0x5623881474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c14c14640 1 -- 172.21.15.19:0/86218550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388132b30 msgr2=0x5623881474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c14c14640 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388132b30 0x5623881474c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c14c14640 1 -- 172.21.15.19:0/86218550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136f40 msgr2=0x562388147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c14c14640 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136f40 0x562388147f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c14c14640 1 -- 172.21.15.19:0/86218550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623881b6e80 con 0x562388136620 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1c14c14640 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388147a00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1c000998a0 tx=0x7f1c00004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d76b092a0ab57e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.789+0000 7f1bf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/86218550 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c000ba020 con 0x562388136620 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/86218550 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f1c00005000 con 0x562388136620 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1c1a38d740 1 -- 172.21.15.19:0/86218550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c045dc0 con 0x562388136620 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/86218550 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c00099170 con 0x562388136620 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1c1a38d740 10 client.16581 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1c1a38d740 1 -- 172.21.15.19:0/86218550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623881b8170 con 0x562388136620 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/86218550 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c045dc0 con 0x562388136620 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/86218550 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1c000a2410 con 0x562388136620 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1c1a38d740 10 client.16581 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1c1a38d740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1c1a38d740 20 client.16581 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562388141800 0x5623881ba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1c1a38d740 4 client.16581 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1c1a38d740 1 -- 172.21.15.19:0/86218550 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623880a3200 con 0x562388141800 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.790+0000 7f1c15415640 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562388141800 0x5623881ba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.791+0000 7f1bf8ff9640 10 client.16581 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.791+0000 7f1c15415640 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562388141800 0x5623881ba980 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa3e8115324d632 server_cookie=dfea361593645a88 in_seq=0 out_seq=0 2024-04-23T18:33:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1bf8ff9640 1 -- 172.21.15.19:0/86218550 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623880a3200 con 0x562388141800 2024-04-23T18:33:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1bf8ff9640 10 client.16581 handle_command_reply: tid=0 2024-04-23T18:33:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1c1a38d740 1 client.16581 shutdown 2024-04-23T18:33:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1c1a38d740 1 -- 172.21.15.19:0/86218550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 msgr2=0x562388147a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388147a00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1c000998a0 tx=0x7f1c00004a60 comp rx=0 tx=0).stop 2024-04-23T18:33:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1c1a38d740 1 -- 172.21.15.19:0/86218550 shutdown_connections 2024-04-23T18:33:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562388141800 0x5623881ba980 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562388136f40 0x562388147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562388136620 0x562388147a00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1bf8ff9640 0 client.16581 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1c1a38d740 1 --2- 172.21.15.19:0/86218550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562388132b30 0x5623881474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.802+0000 7f1c1a38d740 1 -- 172.21.15.19:0/86218550 >> 172.21.15.19:0/86218550 conn(0x562388153fc0 msgr2=0x56238814c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c1a38d740 1 -- 172.21.15.19:0/86218550 shutdown_connections 2024-04-23T18:33:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c1a38d740 1 -- 172.21.15.19:0/86218550 wait complete. 2024-04-23T18:33:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c1a38d740 20 client.16581 trim_cache size 0 max 16384 2024-04-23T18:33:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:33:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:33:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:33:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:33:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 -- 172.21.15.19:0/509689183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bfc0a3cd0 msgr2=0x7f1bfc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bfc0a3cd0 0x7f1bfc0a6190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009d30 tx=0x7f1c08004ed0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 -- 172.21.15.19:0/509689183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c10107290 msgr2=0x7f1c10198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c10107290 0x7f1c10198c30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e9f0 tx=0x7f1c0000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 -- 172.21.15.19:0/509689183 shutdown_connections 2024-04-23T18:33:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1bfc0a3cd0 0x7f1bfc0a6190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1019cfb0 0x7f1c1019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c10107290 0x7f1c10198c30 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 --2- 172.21.15.19:0/509689183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c101021c0 0x7f1c101986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 -- 172.21.15.19:0/509689183 >> 172.21.15.19:0/509689183 conn(0x7f1c100fda20 msgr2=0x7f1c100fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 -- 172.21.15.19:0/509689183 shutdown_connections 2024-04-23T18:33:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:24.803+0000 7f1c17679640 1 -- 172.21.15.19:0/509689183 wait complete. 2024-04-23T18:33:26.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:25 smithi019 ceph-mon[29215]: pgmap v223: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 16 op/s 2024-04-23T18:33:26.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:25 smithi044 ceph-mon[32458]: pgmap v223: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 16 op/s 2024-04-23T18:33:26.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:25 smithi155 ceph-mon[33468]: pgmap v223: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 16 op/s 2024-04-23T18:33:26.893 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.016+0000 7f5850259640 1 -- 172.21.15.19:0/1347465244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848074380 msgr2=0x7f5848074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.016+0000 7f5850259640 1 --2- 172.21.15.19:0/1347465244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848074380 0x7f5848074760 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5844009900 tx=0x7f584402a480 comp rx=0 tx=0).stop 2024-04-23T18:33:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.016+0000 7f5850259640 1 -- 172.21.15.19:0/1347465244 shutdown_connections 2024-04-23T18:33:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.016+0000 7f5850259640 1 --2- 172.21.15.19:0/1347465244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f584810e760 0x7f5848110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.016+0000 7f5850259640 1 --2- 172.21.15.19:0/1347465244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5848074ca0 0x7f584810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.016+0000 7f5850259640 1 --2- 172.21.15.19:0/1347465244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848074380 0x7f5848074760 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.016+0000 7f5850259640 1 -- 172.21.15.19:0/1347465244 >> 172.21.15.19:0/1347465244 conn(0x7f5848079310 msgr2=0x7f5848079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.016+0000 7f5850259640 1 -- 172.21.15.19:0/1347465244 shutdown_connections 2024-04-23T18:33:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.016+0000 7f5850259640 1 -- 172.21.15.19:0/1347465244 wait complete. 2024-04-23T18:33:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f5850259640 1 Processor -- start 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f5850259640 1 -- start start 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f5850259640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848074380 0x7f58481a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f5850259640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848074ca0 0x7f58481a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f5850259640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584810e760 0x7f584819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f5850259640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5848114410 con 0x7f5848074ca0 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f5850259640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5848114290 con 0x7f5848074380 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f5850259640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5848114590 con 0x7f584810e760 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f584d7f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848074ca0 0x7f58481a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f584d7f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848074ca0 0x7f58481a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48508/0 (socket says 172.21.15.19:48508) 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f584d7f4640 1 -- 172.21.15.19:0/3376731422 learned_addr learned my addr 172.21.15.19:0/3376731422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.017+0000 7f584e7f6640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584810e760 0x7f584819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f584dff5640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848074380 0x7f58481a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f584d7f4640 1 -- 172.21.15.19:0/3376731422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584810e760 msgr2=0x7f584819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f584d7f4640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584810e760 0x7f584819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f584d7f4640 1 -- 172.21.15.19:0/3376731422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848074380 msgr2=0x7f58481a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f584d7f4640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848074380 0x7f58481a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f584d7f4640 1 -- 172.21.15.19:0/3376731422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584819afd0 con 0x7f5848074ca0 2024-04-23T18:33:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f584dff5640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848074380 0x7f58481a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f584d7f4640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848074ca0 0x7f58481a0b80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f583800d770 tx=0x7f583800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f583effd640 1 -- 172.21.15.19:0/3376731422 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838017070 con 0x7f5848074ca0 2024-04-23T18:33:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f583effd640 1 -- 172.21.15.19:0/3376731422 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58380054d0 con 0x7f5848074ca0 2024-04-23T18:33:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f5850259640 1 -- 172.21.15.19:0/3376731422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584819b2c0 con 0x7f5848074ca0 2024-04-23T18:33:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f583effd640 1 -- 172.21.15.19:0/3376731422 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838004c30 con 0x7f5848074ca0 2024-04-23T18:33:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.018+0000 7f5850259640 1 -- 172.21.15.19:0/3376731422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58481a7490 con 0x7f5848074ca0 2024-04-23T18:33:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.019+0000 7f5850259640 1 -- 172.21.15.19:0/3376731422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5848075a30 con 0x7f5848074ca0 2024-04-23T18:33:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.020+0000 7f583effd640 1 -- 172.21.15.19:0/3376731422 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5838004dd0 con 0x7f5848074ca0 2024-04-23T18:33:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.021+0000 7f583effd640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3d20 0x7f58340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.022+0000 7f584dff5640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3d20 0x7f58340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.022+0000 7f583effd640 1 -- 172.21.15.19:0/3376731422 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f58380d0710 con 0x7f5848074ca0 2024-04-23T18:33:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.022+0000 7f584dff5640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3d20 0x7f58340a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5844031440 tx=0x7f5844005a20 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.026+0000 7f583effd640 1 -- 172.21.15.19:0/3376731422 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f583809cdb0 con 0x7f5848074ca0 2024-04-23T18:33:27.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.255+0000 7f5850259640 1 -- 172.21.15.19:0/3376731422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58480691c0 con 0x7f5848074ca0 2024-04-23T18:33:27.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.255+0000 7f583effd640 1 -- 172.21.15.19:0/3376731422 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f58480691c0 con 0x7f5848074ca0 2024-04-23T18:33:27.248 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:33:27.248 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:33:27.248 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:33:27.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 -- 172.21.15.19:0/3376731422 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3d20 msgr2=0x7f58340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3d20 0x7f58340a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5844031440 tx=0x7f5844005a20 comp rx=0 tx=0).stop 2024-04-23T18:33:27.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 -- 172.21.15.19:0/3376731422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848074ca0 msgr2=0x7f58481a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848074ca0 0x7f58481a0b80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f583800d770 tx=0x7f583800dc30 comp rx=0 tx=0).stop 2024-04-23T18:33:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 -- 172.21.15.19:0/3376731422 shutdown_connections 2024-04-23T18:33:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3d20 0x7f58340a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584810e760 0x7f584819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848074ca0 0x7f58481a0b80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 --2- 172.21.15.19:0/3376731422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848074380 0x7f58481a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 -- 172.21.15.19:0/3376731422 >> 172.21.15.19:0/3376731422 conn(0x7f5848079310 msgr2=0x7f58481003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 -- 172.21.15.19:0/3376731422 shutdown_connections 2024-04-23T18:33:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.258+0000 7f5850259640 1 -- 172.21.15.19:0/3376731422 wait complete. 2024-04-23T18:33:27.261 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id e34d7e48 --release --await' 2024-04-23T18:33:27.261 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id e34d7e48 --release --await 2024-04-23T18:33:27.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.377+0000 7fb226e59640 1 -- 172.21.15.19:0/3828306298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb220102190 msgr2=0x7fb220102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.377+0000 7fb226e59640 1 --2- 172.21.15.19:0/3828306298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb220102190 0x7fb220102570 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb210009900 tx=0x7fb21002a410 comp rx=0 tx=0).stop 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.377+0000 7fb226e59640 1 -- 172.21.15.19:0/3828306298 shutdown_connections 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.377+0000 7fb226e59640 1 --2- 172.21.15.19:0/3828306298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb220102b40 0x7fb220106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.377+0000 7fb226e59640 1 --2- 172.21.15.19:0/3828306298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb220107260 0x7fb22010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.377+0000 7fb226e59640 1 --2- 172.21.15.19:0/3828306298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb220102190 0x7fb220102570 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.377+0000 7fb226e59640 1 -- 172.21.15.19:0/3828306298 >> 172.21.15.19:0/3828306298 conn(0x7fb2200793a0 msgr2=0x7fb2200797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.377+0000 7fb226e59640 1 -- 172.21.15.19:0/3828306298 shutdown_connections 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.377+0000 7fb226e59640 1 -- 172.21.15.19:0/3828306298 wait complete. 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb226e59640 1 Processor -- start 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb226e59640 1 -- start start 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb226e59640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb220102190 0x7fb220198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb226e59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb220107260 0x7fb220198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb226e59640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb22019d020 0x7fb22019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb226e59640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201121a0 con 0x7fb220107260 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb226e59640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb220112020 con 0x7fb22019d020 2024-04-23T18:33:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb226e59640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb220112320 con 0x7fb220102190 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb220107260 0x7fb220198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb220107260 0x7fb220198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48526/0 (socket says 172.21.15.19:48526) 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.378+0000 7fb21ffff640 1 -- 172.21.15.19:0/3638483753 learned_addr learned my addr 172.21.15.19:0/3638483753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb2253f6640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb22019d020 0x7fb22019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb224bf5640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb220102190 0x7fb220198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb21ffff640 1 -- 172.21.15.19:0/3638483753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb220102190 msgr2=0x7fb220198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb21ffff640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb220102190 0x7fb220198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb21ffff640 1 -- 172.21.15.19:0/3638483753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb22019d020 msgr2=0x7fb22019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb21ffff640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb22019d020 0x7fb22019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb21ffff640 1 -- 172.21.15.19:0/3638483753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22019db10 con 0x7fb220107260 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb21ffff640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb220107260 0x7fb220198ca0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb21400d770 tx=0x7fb21400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb21dffb640 1 -- 172.21.15.19:0/3638483753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214017070 con 0x7fb220107260 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb21dffb640 1 -- 172.21.15.19:0/3638483753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb214009c30 con 0x7fb220107260 2024-04-23T18:33:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb226e59640 1 -- 172.21.15.19:0/3638483753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22019de00 con 0x7fb220107260 2024-04-23T18:33:27.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb21dffb640 1 -- 172.21.15.19:0/3638483753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21400f620 con 0x7fb220107260 2024-04-23T18:33:27.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.379+0000 7fb226e59640 1 -- 172.21.15.19:0/3638483753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2201a55f0 con 0x7fb220107260 2024-04-23T18:33:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.381+0000 7fb21dffb640 1 -- 172.21.15.19:0/3638483753 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb21401a070 con 0x7fb220107260 2024-04-23T18:33:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb229b6c740 1 Processor -- start 2024-04-23T18:33:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb229b6c740 1 -- start start 2024-04-23T18:33:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705490 0x55e695705870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695705db0 0x55e69570ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695710480 0x55e695712870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb229b6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e69566a6d0 con 0x55e695705490 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb229b6c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e6956071a0 con 0x55e695705db0 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb229b6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e695666690 con 0x55e695710480 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb2253f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705490 0x55e695705870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb2253f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705490 0x55e695705870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48534/0 (socket says 172.21.15.19:48534) 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb2253f6640 1 -- 172.21.15.19:0/3427503860 learned_addr learned my addr 172.21.15.19:0/3427503860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb21dffb640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb2080a3d20 0x7fb2080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb21ffff640 1 --2- 172.21.15.19:0/3427503860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695710480 0x55e695712870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb2253f6640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb2080a3d20 0x7fb2080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb224bf5640 1 --2- 172.21.15.19:0/3427503860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695705db0 0x55e69570ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb21dffb640 1 -- 172.21.15.19:0/3638483753 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fb2140d0370 con 0x7fb220107260 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb2253f6640 1 -- 172.21.15.19:0/3427503860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695710480 msgr2=0x55e695712870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb2253f6640 1 --2- 172.21.15.19:0/3427503860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695710480 0x55e695712870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb2253f6640 1 -- 172.21.15.19:0/3427503860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695705db0 msgr2=0x55e69570ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb2253f6640 1 --2- 172.21.15.19:0/3427503860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695705db0 0x55e69570ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.382+0000 7fb2253f6640 1 -- 172.21.15.19:0/3427503860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6956115b0 con 0x55e695705490 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.383+0000 7fb224bf5640 1 --2- 172.21.15.19:0/3427503860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695705db0 0x55e69570ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.383+0000 7fb2253f6640 1 --2- 172.21.15.19:0/3427503860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705490 0x55e695705870 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb218014a60 tx=0x7fb218014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36fbcc48be8ea24 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.383+0000 7fb206ffd640 1 -- 172.21.15.19:0/3427503860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21801ec50 con 0x55e695705490 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.383+0000 7fb206ffd640 1 -- 172.21.15.19:0/3427503860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb218023070 con 0x55e695705490 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.383+0000 7fb206ffd640 1 -- 172.21.15.19:0/3427503860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2180054f0 con 0x55e695705490 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 -- 172.21.15.19:0/3427503860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705490 msgr2=0x55e695705870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb2253f6640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb2080a3d20 0x7fb2080a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb218022000 tx=0x7fb21801e000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3427503860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705490 0x55e695705870 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb218014a60 tx=0x7fb218014f20 comp rx=0 tx=0).stop 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 -- 172.21.15.19:0/3427503860 shutdown_connections 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3427503860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695705db0 0x55e69570ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3427503860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695710480 0x55e695712870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3427503860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705490 0x55e695705870 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 -- 172.21.15.19:0/3427503860 >> 172.21.15.19:0/3427503860 conn(0x55e6957017b0 msgr2=0x55e695701b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 -- 172.21.15.19:0/3427503860 shutdown_connections 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 -- 172.21.15.19:0/3427503860 wait complete. 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 -- start start 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 Processor -- start 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69570fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e69570d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e695713310 con 0x55e695705db0 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e695604b20 con 0x55e695710480 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.384+0000 7fb229b6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e695713490 con 0x55e695705490 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb229b6c740 10 client.0 _ll_register_callbacks cb 0x55e695604530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48544/0 (socket says 172.21.15.19:48544) 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb21ffff640 1 -- 172.21.15.19:0/3039070502 learned_addr learned my addr 172.21.15.19:0/3039070502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb224bf5640 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69570fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb2253f6640 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e69570d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb21ffff640 1 -- 172.21.15.19:0/3039070502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 msgr2=0x55e69570fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb21ffff640 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69570fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb21ffff640 1 -- 172.21.15.19:0/3039070502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 msgr2=0x55e69570d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb21ffff640 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e69570d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb21ffff640 1 -- 172.21.15.19:0/3039070502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e69537c920 con 0x55e695705db0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb21ffff640 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570d130 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb2140ce0c0 tx=0x7fb21406acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd5c40d031eb99b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb1fffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb1fffff640 1 -- 172.21.15.19:0/3039070502 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214021040 con 0x55e695705db0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb1fffff640 1 -- 172.21.15.19:0/3039070502 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fb21409b760 con 0x55e695705db0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb229b6c740 1 -- 172.21.15.19:0/3039070502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb218023070 con 0x55e695705db0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb1fffff640 1 -- 172.21.15.19:0/3039070502 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb214009de0 con 0x55e695705db0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb229b6c740 10 client.16599 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.385+0000 7fb229b6c740 1 -- 172.21.15.19:0/3039070502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e695448d30 con 0x55e695705db0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb1fffff640 1 -- 172.21.15.19:0/3039070502 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb218023070 con 0x55e695705db0 2024-04-23T18:33:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb1fffff640 1 -- 172.21.15.19:0/3039070502 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb214098060 con 0x55e695705db0 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb229b6c740 10 client.16599 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb229b6c740 10 client.16599 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb229b6c740 20 client.16599 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e695720590 0x55e695722980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb229b6c740 4 client.16599 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb229b6c740 1 -- 172.21.15.19:0/3039070502 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e695628500 con 0x55e695720590 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb224bf5640 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e695720590 0x55e695722980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb1fffff640 10 client.16599 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.386+0000 7fb224bf5640 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e695720590 0x55e695722980 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d6271579e7a6d1 server_cookie=a05ff8ab7800141f in_seq=0 out_seq=0 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb1fffff640 1 -- 172.21.15.19:0/3039070502 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e695628500 con 0x55e695720590 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb1fffff640 10 client.16599 handle_command_reply: tid=0 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb229b6c740 1 client.16599 shutdown 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb229b6c740 1 -- 172.21.15.19:0/3039070502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 msgr2=0x55e69570d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570d130 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb2140ce0c0 tx=0x7fb21406acc0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb229b6c740 1 -- 172.21.15.19:0/3039070502 shutdown_connections 2024-04-23T18:33:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e695720590 0x55e695722980 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e69570d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570d130 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3039070502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69570fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb229b6c740 1 -- 172.21.15.19:0/3039070502 >> 172.21.15.19:0/3039070502 conn(0x55e6957017b0 msgr2=0x55e6957075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.387+0000 7fb1fffff640 0 client.16599 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.388+0000 7fb229b6c740 1 -- 172.21.15.19:0/3039070502 shutdown_connections 2024-04-23T18:33:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.388+0000 7fb229b6c740 1 -- 172.21.15.19:0/3039070502 wait complete. 2024-04-23T18:33:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.388+0000 7fb229b6c740 20 client.16599 trim_cache size 0 max 16384 2024-04-23T18:33:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb229b6c740 1 Processor -- start 2024-04-23T18:33:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb229b6c740 1 -- start start 2024-04-23T18:33:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69571c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e6957281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb229b6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201adb70 con 0x55e695705db0 2024-04-23T18:33:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb229b6c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201af6b0 con 0x55e695710480 2024-04-23T18:33:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb229b6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201ae240 con 0x55e695705490 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48548/0 (socket says 172.21.15.19:48548) 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb21ffff640 1 -- 172.21.15.19:0/2736370570 learned_addr learned my addr 172.21.15.19:0/2736370570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb224bf5640 1 --2- 172.21.15.19:0/2736370570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69571c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb21ffff640 1 -- 172.21.15.19:0/2736370570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 msgr2=0x55e69571c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb21ffff640 1 --2- 172.21.15.19:0/2736370570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69571c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb21ffff640 1 -- 172.21.15.19:0/2736370570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 msgr2=0x55e6957281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb21ffff640 1 --2- 172.21.15.19:0/2736370570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e6957281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.405+0000 7fb21ffff640 1 -- 172.21.15.19:0/2736370570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e695704aa0 con 0x55e695705db0 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb21ffff640 1 --2- 172.21.15.19:0/2736370570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570f970 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb2140cdb70 tx=0x7fb214010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dbbafc4036c1d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb2067fc640 1 -- 172.21.15.19:0/2736370570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214021070 con 0x55e695705db0 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb2067fc640 1 -- 172.21.15.19:0/2736370570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2140a4f80 con 0x55e695705db0 2024-04-23T18:33:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb2067fc640 1 -- 172.21.15.19:0/2736370570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21409b1d0 con 0x55e695705db0 2024-04-23T18:33:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb229b6c740 1 -- 172.21.15.19:0/2736370570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 msgr2=0x55e69570f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb229b6c740 1 --2- 172.21.15.19:0/2736370570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570f970 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb2140cdb70 tx=0x7fb214010c00 comp rx=0 tx=0).stop 2024-04-23T18:33:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb229b6c740 1 -- 172.21.15.19:0/2736370570 shutdown_connections 2024-04-23T18:33:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb229b6c740 1 --2- 172.21.15.19:0/2736370570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e6957281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb229b6c740 1 --2- 172.21.15.19:0/2736370570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e69570f970 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb229b6c740 1 --2- 172.21.15.19:0/2736370570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69571c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb229b6c740 1 -- 172.21.15.19:0/2736370570 >> 172.21.15.19:0/2736370570 conn(0x55e6957017b0 msgr2=0x55e695706cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb229b6c740 1 -- 172.21.15.19:0/2736370570 shutdown_connections 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.406+0000 7fb229b6c740 1 -- 172.21.15.19:0/2736370570 wait complete. 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb229b6c740 1 -- start start 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb229b6c740 1 Processor -- start 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69570add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e6957186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb229b6c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e695711610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb229b6c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201aefe0 con 0x55e695705db0 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb229b6c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201ae910 con 0x55e695710480 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb229b6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201abb70 con 0x55e695705490 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb229b6c740 10 client.0 _ll_register_callbacks cb 0x7fb21409c0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e6957186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e6957186e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48554/0 (socket says 172.21.15.19:48554) 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb21ffff640 1 -- 172.21.15.19:0/3604865014 learned_addr learned my addr 172.21.15.19:0/3604865014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb224bf5640 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69570add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.407+0000 7fb2253f6640 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e695711610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb21ffff640 1 -- 172.21.15.19:0/3604865014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 msgr2=0x55e69570add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb21ffff640 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69570add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb21ffff640 1 -- 172.21.15.19:0/3604865014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 msgr2=0x55e695711610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb21ffff640 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e695711610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb21ffff640 1 -- 172.21.15.19:0/3604865014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e695718fe0 con 0x55e695705db0 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb2253f6640 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e695711610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb21ffff640 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e6957186e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb214000c00 tx=0x7fb214098bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0265ea28d9e9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb1fffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb1fffff640 1 -- 172.21.15.19:0/3604865014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21406a050 con 0x55e695705db0 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb1fffff640 1 -- 172.21.15.19:0/3604865014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fb2140a0150 con 0x55e695705db0 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb229b6c740 1 -- 172.21.15.19:0/3604865014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2140a4f80 con 0x55e695705db0 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb1fffff640 1 -- 172.21.15.19:0/3604865014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb214021070 con 0x55e695705db0 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb229b6c740 10 client.16605 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:27.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb229b6c740 1 -- 172.21.15.19:0/3604865014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e69578b500 con 0x55e695705db0 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb1fffff640 1 -- 172.21.15.19:0/3604865014 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2140a4f80 con 0x55e695705db0 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb1fffff640 1 -- 172.21.15.19:0/3604865014 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb2140b5cc0 con 0x55e695705db0 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb229b6c740 10 client.16605 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb229b6c740 10 client.16605 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb229b6c740 20 client.16605 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e69578b990 0x55e69578ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb229b6c740 4 client.16605 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e34d7e48", "release": true, "await": true}] 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.408+0000 7fb229b6c740 1 -- 172.21.15.19:0/3604865014 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e34d7e48", "release": true, "await": true}) -- 0x55e6956780f0 con 0x55e69578b990 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.409+0000 7fb224bf5640 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e69578b990 0x55e69578ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.409+0000 7fb224bf5640 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e69578b990 0x55e69578ddb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=545364820c8af0ac server_cookie=8ab67c9cfa86b636 in_seq=0 out_seq=0 2024-04-23T18:33:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.409+0000 7fb1fffff640 10 client.16605 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb1fffff640 1 -- 172.21.15.19:0/3604865014 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55e6956780f0 con 0x55e69578b990 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb1fffff640 10 client.16605 handle_command_reply: tid=0 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 client.16605 shutdown 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 -- 172.21.15.19:0/3604865014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 msgr2=0x55e6957186e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e6957186e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb214000c00 tx=0x7fb214098bb0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 -- 172.21.15.19:0/3604865014 shutdown_connections 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e69578b990 0x55e69578ddb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e695710480 0x55e695711610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e695705db0 0x55e6957186e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb1fffff640 0 client.16605 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 --2- 172.21.15.19:0/3604865014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e695705490 0x55e69570add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 -- 172.21.15.19:0/3604865014 >> 172.21.15.19:0/3604865014 conn(0x55e6957017b0 msgr2=0x55e69571ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 -- 172.21.15.19:0/3604865014 shutdown_connections 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 1 -- 172.21.15.19:0/3604865014 wait complete. 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.413+0000 7fb229b6c740 20 client.16605 trim_cache size 0 max 16384 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 32, 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stdout: "e34d7e48": { 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 32, 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:33:27.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 85.0, 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:27.410 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:27.411 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:33:27.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 -- 172.21.15.19:0/3638483753 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb2080a3d20 msgr2=0x7fb2080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb2080a3d20 0x7fb2080a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb218022000 tx=0x7fb21801e000 comp rx=0 tx=0).stop 2024-04-23T18:33:27.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 -- 172.21.15.19:0/3638483753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb220107260 msgr2=0x7fb220198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb220107260 0x7fb220198ca0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb21400d770 tx=0x7fb21400dc30 comp rx=0 tx=0).stop 2024-04-23T18:33:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 -- 172.21.15.19:0/3638483753 shutdown_connections 2024-04-23T18:33:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb2080a3d20 0x7fb2080a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb22019d020 0x7fb22019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb220107260 0x7fb220198ca0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 --2- 172.21.15.19:0/3638483753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb220102190 0x7fb220198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 -- 172.21.15.19:0/3638483753 >> 172.21.15.19:0/3638483753 conn(0x7fb2200793a0 msgr2=0x7fb2201000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 -- 172.21.15.19:0/3638483753 shutdown_connections 2024-04-23T18:33:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.414+0000 7fb226e59640 1 -- 172.21.15.19:0/3638483753 wait complete. 2024-04-23T18:33:27.417 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e34d7e48', seconds elapsed: 26.1 2024-04-23T18:33:27.417 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.0 2024-04-23T18:33:27.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.527+0000 7fbebc79b640 1 -- 172.21.15.19:0/663260721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb410c570 msgr2=0x7fbeb410e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.527+0000 7fbebc79b640 1 --2- 172.21.15.19:0/663260721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb410c570 0x7fbeb410e9e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009ca0 tx=0x7fbeac02a950 comp rx=0 tx=0).stop 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.527+0000 7fbebc79b640 1 -- 172.21.15.19:0/663260721 shutdown_connections 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.527+0000 7fbebc79b640 1 --2- 172.21.15.19:0/663260721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbeb4103b50 0x7fbeb410c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.527+0000 7fbebc79b640 1 --2- 172.21.15.19:0/663260721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb410c570 0x7fbeb410e9e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.527+0000 7fbebc79b640 1 --2- 172.21.15.19:0/663260721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb4103230 0x7fbeb4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.527+0000 7fbebc79b640 1 -- 172.21.15.19:0/663260721 >> 172.21.15.19:0/663260721 conn(0x7fbeb4077fb0 msgr2=0x7fbeb40783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.527+0000 7fbebc79b640 1 -- 172.21.15.19:0/663260721 shutdown_connections 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.527+0000 7fbebc79b640 1 -- 172.21.15.19:0/663260721 wait complete. 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbebc79b640 1 Processor -- start 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbebc79b640 1 -- start start 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbebc79b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbeb4103230 0x7fbeb4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbebc79b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb4103b50 0x7fbeb4198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbebc79b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb410c570 0x7fbeb419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbebc79b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb41121b0 con 0x7fbeb4103b50 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbebc79b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4112030 con 0x7fbeb4103230 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbebc79b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4112330 con 0x7fbeb410c570 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbeb9d36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb4103b50 0x7fbeb4198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbeb9d36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb4103b50 0x7fbeb4198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48580/0 (socket says 172.21.15.19:48580) 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbeb9d36640 1 -- 172.21.15.19:0/980187147 learned_addr learned my addr 172.21.15.19:0/980187147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbeba537640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbeb4103230 0x7fbeb4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.528+0000 7fbebad38640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb410c570 0x7fbeb419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbeb9d36640 1 -- 172.21.15.19:0/980187147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb410c570 msgr2=0x7fbeb419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbeb9d36640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb410c570 0x7fbeb419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbeb9d36640 1 -- 172.21.15.19:0/980187147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbeb4103230 msgr2=0x7fbeb4198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbeb9d36640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbeb4103230 0x7fbeb4198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbeb9d36640 1 -- 172.21.15.19:0/980187147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb419d600 con 0x7fbeb4103b50 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbeb9d36640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb4103b50 0x7fbeb4198b50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbea800c920 tx=0x7fbea800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbea77fe640 1 -- 172.21.15.19:0/980187147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8007c70 con 0x7fbeb4103b50 2024-04-23T18:33:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbea77fe640 1 -- 172.21.15.19:0/980187147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea8007e10 con 0x7fbeb4103b50 2024-04-23T18:33:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbebc79b640 1 -- 172.21.15.19:0/980187147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb419d8f0 con 0x7fbeb4103b50 2024-04-23T18:33:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbea77fe640 1 -- 172.21.15.19:0/980187147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8018820 con 0x7fbeb4103b50 2024-04-23T18:33:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.529+0000 7fbebc79b640 1 -- 172.21.15.19:0/980187147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb41a5180 con 0x7fbeb4103b50 2024-04-23T18:33:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.530+0000 7fbebc79b640 1 -- 172.21.15.19:0/980187147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb40691c0 con 0x7fbeb4103b50 2024-04-23T18:33:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.531+0000 7fbea77fe640 1 -- 172.21.15.19:0/980187147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbea800f450 con 0x7fbeb4103b50 2024-04-23T18:33:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.532+0000 7fbea77fe640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbea00a3d20 0x7fbea00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.532+0000 7fbeba537640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbea00a3d20 0x7fbea00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.532+0000 7fbea77fe640 1 -- 172.21.15.19:0/980187147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fbea80d0be0 con 0x7fbeb4103b50 2024-04-23T18:33:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.532+0000 7fbeba537640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbea00a3d20 0x7fbea00a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005e10 tx=0x7fbeb0005da0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.535+0000 7fbea77fe640 1 -- 172.21.15.19:0/980187147 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbea8014030 con 0x7fbeb4103b50 2024-04-23T18:33:27.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.761+0000 7fbebc79b640 1 -- 172.21.15.19:0/980187147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeb4107750 con 0x7fbeb4103b50 2024-04-23T18:33:27.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.761+0000 7fbea77fe640 1 -- 172.21.15.19:0/980187147 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7fbea809d310 con 0x7fbeb4103b50 2024-04-23T18:33:27.754 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:33:27.754 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:33:27.754 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:33:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.763+0000 7fbebc79b640 1 -- 172.21.15.19:0/980187147 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbea00a3d20 msgr2=0x7fbea00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.763+0000 7fbebc79b640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbea00a3d20 0x7fbea00a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005e10 tx=0x7fbeb0005da0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.763+0000 7fbebc79b640 1 -- 172.21.15.19:0/980187147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb4103b50 msgr2=0x7fbeb4198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.763+0000 7fbebc79b640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb4103b50 0x7fbeb4198b50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbea800c920 tx=0x7fbea800cde0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.764+0000 7fbebc79b640 1 -- 172.21.15.19:0/980187147 shutdown_connections 2024-04-23T18:33:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.764+0000 7fbebc79b640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbea00a3d20 0x7fbea00a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.764+0000 7fbebc79b640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb410c570 0x7fbeb419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.764+0000 7fbebc79b640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb4103b50 0x7fbeb4198b50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.764+0000 7fbebc79b640 1 --2- 172.21.15.19:0/980187147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbeb4103230 0x7fbeb4198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.764+0000 7fbebc79b640 1 -- 172.21.15.19:0/980187147 >> 172.21.15.19:0/980187147 conn(0x7fbeb4077fb0 msgr2=0x7fbeb40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.764+0000 7fbebc79b640 1 -- 172.21.15.19:0/980187147 shutdown_connections 2024-04-23T18:33:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.764+0000 7fbebc79b640 1 -- 172.21.15.19:0/980187147 wait complete. 2024-04-23T18:33:27.767 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 85.0 --await-for 42.0' 2024-04-23T18:33:27.767 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 85.0 --await-for 42.0 2024-04-23T18:33:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 -- 172.21.15.19:0/93638063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b60103b50 msgr2=0x7f4b6010c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 --2- 172.21.15.19:0/93638063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b60103b50 0x7f4b6010c030 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009900 tx=0x7f4b5002a480 comp rx=0 tx=0).stop 2024-04-23T18:33:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 -- 172.21.15.19:0/93638063 shutdown_connections 2024-04-23T18:33:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 --2- 172.21.15.19:0/93638063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b6010c570 0x7f4b6010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 --2- 172.21.15.19:0/93638063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b60103b50 0x7f4b6010c030 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 --2- 172.21.15.19:0/93638063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60103230 0x7f4b60103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 -- 172.21.15.19:0/93638063 >> 172.21.15.19:0/93638063 conn(0x7f4b60077fb0 msgr2=0x7f4b600783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 -- 172.21.15.19:0/93638063 shutdown_connections 2024-04-23T18:33:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 -- 172.21.15.19:0/93638063 wait complete. 2024-04-23T18:33:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 Processor -- start 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.888+0000 7f4b6576f640 1 -- start start 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b6576f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60103230 0x7f4b6019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b6576f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b60103b50 0x7f4b6019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b6576f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b6010c570 0x7f4b60198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b6576f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601121b0 con 0x7f4b60103b50 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b6576f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60112030 con 0x7f4b6010c570 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b6576f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60112330 con 0x7f4b60103230 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b60103b50 0x7f4b6019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b60103b50 0x7f4b6019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48600/0 (socket says 172.21.15.19:48600) 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5e7fc640 1 -- 172.21.15.19:0/444057962 learned_addr learned my addr 172.21.15.19:0/444057962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5effd640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60103230 0x7f4b6019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b6010c570 0x7f4b60198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5e7fc640 1 -- 172.21.15.19:0/444057962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60103230 msgr2=0x7f4b6019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5e7fc640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60103230 0x7f4b6019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5e7fc640 1 -- 172.21.15.19:0/444057962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b6010c570 msgr2=0x7f4b60198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5e7fc640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b6010c570 0x7f4b60198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5e7fc640 1 -- 172.21.15.19:0/444057962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b60198d10 con 0x7f4b60103b50 2024-04-23T18:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b5e7fc640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b60103b50 0x7f4b6019e920 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4b50005940 tx=0x7f4b500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b47fff640 1 -- 172.21.15.19:0/444057962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5003f030 con 0x7f4b60103b50 2024-04-23T18:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b47fff640 1 -- 172.21.15.19:0/444057962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b50003cb0 con 0x7f4b60103b50 2024-04-23T18:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b6576f640 1 -- 172.21.15.19:0/444057962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b60198fa0 con 0x7f4b60103b50 2024-04-23T18:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b47fff640 1 -- 172.21.15.19:0/444057962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50035070 con 0x7f4b60103b50 2024-04-23T18:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.889+0000 7f4b6576f640 1 -- 172.21.15.19:0/444057962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b601a52a0 con 0x7f4b60103b50 2024-04-23T18:33:27.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b47fff640 1 -- 172.21.15.19:0/444057962 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b50042030 con 0x7f4b60103b50 2024-04-23T18:33:27.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b68567740 1 Processor -- start 2024-04-23T18:33:27.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b68567740 1 -- start start 2024-04-23T18:33:27.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b68567740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fa6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b68567740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fa68c0 0x5607a4fb4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b68567740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fb54a0 0x5607a4fb7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b68567740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5607a4f0b370 con 0x5607a4fa68c0 2024-04-23T18:33:27.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b68567740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5607a4ea82e0 con 0x5607a4fa5fa0 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b68567740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5607a4f07330 con 0x5607a4fb54a0 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b47fff640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b480a3d20 0x7f4b480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b5effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fa68c0 0x5607a4fb4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b5effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fa68c0 0x5607a4fb4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48608/0 (socket says 172.21.15.19:48608) 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b5effd640 1 -- 172.21.15.19:0/483750934 learned_addr learned my addr 172.21.15.19:0/483750934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b480a3d20 0x7f4b480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b47fff640 1 -- 172.21.15.19:0/444057962 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f4b500f10a0 con 0x7f4b60103b50 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/483750934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fa6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.892+0000 7f4b5e7fc640 1 --2- 172.21.15.19:0/483750934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fb54a0 0x5607a4fb7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b5effd640 1 -- 172.21.15.19:0/483750934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fb54a0 msgr2=0x5607a4fb7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b5effd640 1 --2- 172.21.15.19:0/483750934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fb54a0 0x5607a4fb7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b5effd640 1 -- 172.21.15.19:0/483750934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 msgr2=0x5607a4fa6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b5effd640 1 --2- 172.21.15.19:0/483750934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fa6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b5effd640 1 -- 172.21.15.19:0/483750934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607a4ce9d80 con 0x5607a4fa68c0 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b5e7fc640 1 --2- 172.21.15.19:0/483750934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fb54a0 0x5607a4fb7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:27.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b480a3d20 0x7f4b480a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b58005cc0 tx=0x7f4b5800acb0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b5effd640 1 --2- 172.21.15.19:0/483750934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fa68c0 0x5607a4fb4f60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400bcd0 tx=0x7f4b540107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d548bc0bdf695b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b457fa640 1 -- 172.21.15.19:0/483750934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54005220 con 0x5607a4fa68c0 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b457fa640 1 -- 172.21.15.19:0/483750934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b54004c50 con 0x5607a4fa68c0 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.893+0000 7f4b457fa640 1 -- 172.21.15.19:0/483750934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54011640 con 0x5607a4fa68c0 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 -- 172.21.15.19:0/483750934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fa68c0 msgr2=0x5607a4fb4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 --2- 172.21.15.19:0/483750934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fa68c0 0x5607a4fb4f60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400bcd0 tx=0x7f4b540107a0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 -- 172.21.15.19:0/483750934 shutdown_connections 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 --2- 172.21.15.19:0/483750934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fb54a0 0x5607a4fb7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 --2- 172.21.15.19:0/483750934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fa68c0 0x5607a4fb4f60 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 --2- 172.21.15.19:0/483750934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fa6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 -- 172.21.15.19:0/483750934 >> 172.21.15.19:0/483750934 conn(0x5607a4fa23c0 msgr2=0x5607a4fa27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 -- 172.21.15.19:0/483750934 shutdown_connections 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 -- 172.21.15.19:0/483750934 wait complete. 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 -- start start 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 Processor -- start 2024-04-23T18:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fb23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fb28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5607a4fb8260 con 0x5607a4fb54a0 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5607a4ea5fb0 con 0x5607a4fa5fa0 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5607a4fb83e0 con 0x5607a4fa68c0 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.894+0000 7f4b68567740 10 client.0 _ll_register_callbacks cb 0x5607a4ea59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fb23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fb23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:35284/0 (socket says 172.21.15.19:35284) 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48622/0 (socket says 172.21.15.19:48622) 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5effd640 1 -- 172.21.15.19:0/1174180407 learned_addr learned my addr 172.21.15.19:0/1174180407 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5e7fc640 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fb28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1174180407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 msgr2=0x5607a4fb28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fb28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1174180407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 msgr2=0x5607a4fb23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fb23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1174180407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607a4d4a510 con 0x5607a4fb54a0 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb2e20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4b580107a0 tx=0x7f4b58005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4764a0e1233f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1174180407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58026070 con 0x5607a4fb54a0 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1174180407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f4b58004d20 con 0x5607a4fb54a0 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b68567740 1 -- 172.21.15.19:0/1174180407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b54004c50 con 0x5607a4fb54a0 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1174180407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b58031050 con 0x5607a4fb54a0 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b68567740 10 client.16623 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.895+0000 7f4b68567740 1 -- 172.21.15.19:0/1174180407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5607a4bdd670 con 0x5607a4fb54a0 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1174180407 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b54004c50 con 0x5607a4fb54a0 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1174180407 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4b58020480 con 0x5607a4fb54a0 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b68567740 10 client.16623 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b68567740 10 client.16623 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b68567740 20 client.16623 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:27.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b68567740 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5607a4fc56d0 0x5607a4fc7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b68567740 4 client.16623 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b68567740 1 -- 172.21.15.19:0/1174180407 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607a4f18d90 con 0x5607a4fc56d0 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b5effd640 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5607a4fc56d0 0x5607a4fc7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b42ff5640 10 client.16623 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.896+0000 7f4b5effd640 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5607a4fc56d0 0x5607a4fc7ae0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76168e97ad81938 server_cookie=2a6157ef102366db in_seq=0 out_seq=0 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1174180407 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607a4f18d90 con 0x5607a4fc56d0 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b42ff5640 10 client.16623 handle_command_reply: tid=0 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 client.16623 shutdown 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 -- 172.21.15.19:0/1174180407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 msgr2=0x5607a4fb2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb2e20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4b580107a0 tx=0x7f4b58005bd0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 -- 172.21.15.19:0/1174180407 shutdown_connections 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5607a4fc56d0 0x5607a4fc7ae0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb2e20 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fb28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 --2- 172.21.15.19:0/1174180407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fb23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 -- 172.21.15.19:0/1174180407 >> 172.21.15.19:0/1174180407 conn(0x5607a4fa23c0 msgr2=0x5607a4fa8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b42ff5640 0 client.16623 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:27.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 -- 172.21.15.19:0/1174180407 shutdown_connections 2024-04-23T18:33:27.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.897+0000 7f4b68567740 1 -- 172.21.15.19:0/1174180407 wait complete. 2024-04-23T18:33:27.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.898+0000 7f4b68567740 20 client.16623 trim_cache size 0 max 16384 2024-04-23T18:33:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b68567740 1 Processor -- start 2024-04-23T18:33:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b68567740 1 -- start start 2024-04-23T18:33:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b68567740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fc1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b68567740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fa1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b68567740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fcd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b68567740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601ab9e0 con 0x5607a4fb54a0 2024-04-23T18:33:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b68567740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601ad520 con 0x5607a4fa5fa0 2024-04-23T18:33:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b68567740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601ac0b0 con 0x5607a4fa68c0 2024-04-23T18:33:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fcd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fcd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48636/0 (socket says 172.21.15.19:48636) 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1983117530 learned_addr learned my addr 172.21.15.19:0/1983117530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5e7fc640 1 --2- 172.21.15.19:0/1983117530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fa1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5effd640 1 --2- 172.21.15.19:0/1983117530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fc1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1983117530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 msgr2=0x5607a4fa1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/1983117530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fa1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1983117530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 msgr2=0x5607a4fc1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/1983117530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fc1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1983117530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607a4fb08f0 con 0x5607a4fb54a0 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.915+0000 7f4b5e7fc640 1 --2- 172.21.15.19:0/1983117530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fa1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/1983117530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fcd3b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4b58014010 tx=0x7f4b58004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c150515ffaf7178 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b44ff9640 1 -- 172.21.15.19:0/1983117530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58026040 con 0x5607a4fb54a0 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b44ff9640 1 -- 172.21.15.19:0/1983117530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b58020480 con 0x5607a4fb54a0 2024-04-23T18:33:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b44ff9640 1 -- 172.21.15.19:0/1983117530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58003d00 con 0x5607a4fb54a0 2024-04-23T18:33:27.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b68567740 1 -- 172.21.15.19:0/1983117530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 msgr2=0x5607a4fcd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b68567740 1 --2- 172.21.15.19:0/1983117530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fcd3b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4b58014010 tx=0x7f4b58004a00 comp rx=0 tx=0).stop 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b68567740 1 -- 172.21.15.19:0/1983117530 shutdown_connections 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b68567740 1 --2- 172.21.15.19:0/1983117530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fcd3b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b68567740 1 --2- 172.21.15.19:0/1983117530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fa1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b68567740 1 --2- 172.21.15.19:0/1983117530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fc1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b68567740 1 -- 172.21.15.19:0/1983117530 >> 172.21.15.19:0/1983117530 conn(0x5607a4fa23c0 msgr2=0x5607a4fa0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b68567740 1 -- 172.21.15.19:0/1983117530 shutdown_connections 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.916+0000 7f4b68567740 1 -- 172.21.15.19:0/1983117530 wait complete. 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b68567740 1 -- start start 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b68567740 1 Processor -- start 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b68567740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fbd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b68567740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fbdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b68567740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b68567740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601ace50 con 0x5607a4fb54a0 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b68567740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601ac780 con 0x5607a4fa5fa0 2024-04-23T18:33:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b68567740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601a99e0 con 0x5607a4fa68c0 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b68567740 10 client.0 _ll_register_callbacks cb 0x7f4b5800aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48642/0 (socket says 172.21.15.19:48642) 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1771248772 learned_addr learned my addr 172.21.15.19:0/1771248772 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5e7fc640 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fbdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5effd640 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fbd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1771248772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 msgr2=0x5607a4fbdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fbdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1771248772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 msgr2=0x5607a4fbd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fbd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5f7fe640 1 -- 172.21.15.19:0/1771248772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607a4fb9570 con 0x5607a4fb54a0 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.917+0000 7f4b5effd640 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fbd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b5f7fe640 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb8da0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4b58020850 tx=0x7f4b580136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb003af4ac3f36d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1771248772 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58039030 con 0x5607a4fb54a0 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1771248772 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f4b58020a40 con 0x5607a4fb54a0 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b68567740 1 -- 172.21.15.19:0/1771248772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b58020480 con 0x5607a4fb54a0 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1771248772 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b58026040 con 0x5607a4fb54a0 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b68567740 10 client.16629 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b68567740 1 -- 172.21.15.19:0/1771248772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5607a502b860 con 0x5607a4fb54a0 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1771248772 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b58020480 con 0x5607a4fb54a0 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1771248772 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4b58005490 con 0x5607a4fb54a0 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b68567740 10 client.16629 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b68567740 10 client.16629 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b68567740 20 client.16629 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b68567740 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5607a502bf30 0x5607a502e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b68567740 4 client.16629 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.0, "await_for": 42.0}] 2024-04-23T18:33:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.918+0000 7f4b68567740 1 -- 172.21.15.19:0/1771248772 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.0, "await_for": 42.0}) -- 0x5607a4f12c00 con 0x5607a502bf30 2024-04-23T18:33:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.919+0000 7f4b5effd640 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5607a502bf30 0x5607a502e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.919+0000 7f4b5effd640 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5607a502bf30 0x5607a502e350 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e8cc4193d253b3 server_cookie=86cf6499186fb50d in_seq=0 out_seq=0 2024-04-23T18:33:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.919+0000 7f4b42ff5640 10 client.16629 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b42ff5640 1 -- 172.21.15.19:0/1771248772 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5607a4f12c00 con 0x5607a502bf30 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b42ff5640 10 client.16629 handle_command_reply: tid=0 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 client.16629 shutdown 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 -- 172.21.15.19:0/1771248772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 msgr2=0x5607a4fb8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb8da0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4b58020850 tx=0x7f4b580136b0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 -- 172.21.15.19:0/1771248772 shutdown_connections 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5607a502bf30 0x5607a502e350 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607a4fb54a0 0x5607a4fb8da0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607a4fa68c0 0x5607a4fbdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b42ff5640 0 client.16629 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 --2- 172.21.15.19:0/1771248772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5607a4fa5fa0 0x5607a4fbd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 -- 172.21.15.19:0/1771248772 >> 172.21.15.19:0/1771248772 conn(0x5607a4fa23c0 msgr2=0x5607a4fa1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 -- 172.21.15.19:0/1771248772 shutdown_connections 2024-04-23T18:33:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 1 -- 172.21.15.19:0/1771248772 wait complete. 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.934+0000 7f4b68567740 20 client.16629 trim_cache size 0 max 16384 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 34, 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout: "a8698f7": { 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 34, 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:33:27.928 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 85.0, 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:27.929 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:33:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 -- 172.21.15.19:0/444057962 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b480a3d20 msgr2=0x7f4b480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b480a3d20 0x7f4b480a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b58005cc0 tx=0x7f4b5800acb0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 -- 172.21.15.19:0/444057962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b60103b50 msgr2=0x7f4b6019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:27.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b60103b50 0x7f4b6019e920 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4b50005940 tx=0x7f4b500042f0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 -- 172.21.15.19:0/444057962 shutdown_connections 2024-04-23T18:33:27.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4b480a3d20 0x7f4b480a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b6010c570 0x7f4b60198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b60103b50 0x7f4b6019e920 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 --2- 172.21.15.19:0/444057962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60103230 0x7f4b6019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:27.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 -- 172.21.15.19:0/444057962 >> 172.21.15.19:0/444057962 conn(0x7f4b60077fb0 msgr2=0x7f4b600fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:27.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 -- 172.21.15.19:0/444057962 shutdown_connections 2024-04-23T18:33:27.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:27.935+0000 7f4b6576f640 1 -- 172.21.15.19:0/444057962 wait complete. 2024-04-23T18:33:27.938 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a8698f7', quiesce took 0.5 seconds. Will release after: 25.0 2024-04-23T18:33:28.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:27 smithi019 ceph-mon[29215]: pgmap v224: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T18:33:28.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:27 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3376731422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:28.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:27 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/980187147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:28.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:27 smithi044 ceph-mon[32458]: pgmap v224: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T18:33:28.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:27 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3376731422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:28.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:27 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/980187147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:28.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:27 smithi155 ceph-mon[33468]: pgmap v224: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T18:33:28.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:27 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3376731422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:28.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:27 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/980187147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:30.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:29 smithi044 ceph-mon[32458]: pgmap v225: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T18:33:30.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:29 smithi155 ceph-mon[33468]: pgmap v225: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T18:33:30.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:29 smithi019 ceph-mon[29215]: pgmap v225: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T18:33:32.061 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:31 smithi019 ceph-mon[29215]: pgmap v226: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-04-23T18:33:32.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:31 smithi044 ceph-mon[32458]: pgmap v226: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-04-23T18:33:32.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:31 smithi155 ceph-mon[33468]: pgmap v226: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-04-23T18:33:34.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:33 smithi044 ceph-mon[32458]: pgmap v227: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-04-23T18:33:34.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:33 smithi155 ceph-mon[33468]: pgmap v227: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-04-23T18:33:34.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:33 smithi019 ceph-mon[29215]: pgmap v227: 97 pgs: 97 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-04-23T18:33:36.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:35 smithi044 ceph-mon[32458]: pgmap v228: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-23T18:33:36.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:35 smithi155 ceph-mon[33468]: pgmap v228: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-23T18:33:36.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:35 smithi019 ceph-mon[29215]: pgmap v228: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-23T18:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 -- 172.21.15.19:0/3960645928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6098103060 msgr2=0x7f6098103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 --2- 172.21.15.19:0/3960645928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6098103060 0x7f6098103440 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6088009900 tx=0x7f608802a480 comp rx=0 tx=0).stop 2024-04-23T18:33:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 -- 172.21.15.19:0/3960645928 shutdown_connections 2024-04-23T18:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 --2- 172.21.15.19:0/3960645928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6098103980 0x7f609810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 --2- 172.21.15.19:0/3960645928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f609810c580 0x7f609810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 --2- 172.21.15.19:0/3960645928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6098103060 0x7f6098103440 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 -- 172.21.15.19:0/3960645928 >> 172.21.15.19:0/3960645928 conn(0x7f6098077fd0 msgr2=0x7f60980783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 -- 172.21.15.19:0/3960645928 shutdown_connections 2024-04-23T18:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 -- 172.21.15.19:0/3960645928 wait complete. 2024-04-23T18:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 Processor -- start 2024-04-23T18:33:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.904+0000 7f609e80b640 1 -- start start 2024-04-23T18:33:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f609e80b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6098103060 0x7f60981986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f609e80b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6098103980 0x7f6098198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f609e80b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f609810c580 0x7f609819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f609e80b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60981121c0 con 0x7f6098103060 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f609e80b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6098112040 con 0x7f609810c580 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f609e80b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6098112340 con 0x7f6098103980 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f6097fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6098103060 0x7f60981986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f6097fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6098103060 0x7f60981986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48772/0 (socket says 172.21.15.19:48772) 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f6097fff640 1 -- 172.21.15.19:0/2561532541 learned_addr learned my addr 172.21.15.19:0/2561532541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f60977fe640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6098103980 0x7f6098198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f609cda8640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f609810c580 0x7f609819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f6097fff640 1 -- 172.21.15.19:0/2561532541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6098103980 msgr2=0x7f6098198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f6097fff640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6098103980 0x7f6098198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f6097fff640 1 -- 172.21.15.19:0/2561532541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f609810c580 msgr2=0x7f609819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f6097fff640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f609810c580 0x7f609819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f6097fff640 1 -- 172.21.15.19:0/2561532541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609819d660 con 0x7f6098103060 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f609cda8640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f609810c580 0x7f609819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.905+0000 7f6097fff640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6098103060 0x7f60981986c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6088005940 tx=0x7f60880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.906+0000 7f60957fa640 1 -- 172.21.15.19:0/2561532541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608803f030 con 0x7f6098103060 2024-04-23T18:33:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.906+0000 7f60957fa640 1 -- 172.21.15.19:0/2561532541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6088003cb0 con 0x7f6098103060 2024-04-23T18:33:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.906+0000 7f609e80b640 1 -- 172.21.15.19:0/2561532541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609819d8f0 con 0x7f6098103060 2024-04-23T18:33:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.906+0000 7f60957fa640 1 -- 172.21.15.19:0/2561532541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088035070 con 0x7f6098103060 2024-04-23T18:33:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.906+0000 7f609e80b640 1 -- 172.21.15.19:0/2561532541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60981a51e0 con 0x7f6098103060 2024-04-23T18:33:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.908+0000 7f60957fa640 1 -- 172.21.15.19:0/2561532541 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6088042030 con 0x7f6098103060 2024-04-23T18:33:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60a15f4740 1 Processor -- start 2024-04-23T18:33:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60a15f4740 1 -- start start 2024-04-23T18:33:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55807286d620 0x55807286da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286df40 0x5580728782c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558072878800 0x55807287abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60a15f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5580727d2970 con 0x558072878800 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60a15f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55807276f4a0 con 0x55807286df40 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60a15f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5580727ce930 con 0x55807286d620 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60957fa640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60840a3dc0 0x7f60840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60977fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558072878800 0x55807287abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60977fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558072878800 0x55807287abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48776/0 (socket says 172.21.15.19:48776) 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f60977fe640 1 -- 172.21.15.19:0/523474817 learned_addr learned my addr 172.21.15.19:0/523474817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f609cda8640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60840a3dc0 0x7f60840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f609cda8640 1 --2- 172.21.15.19:0/523474817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55807286d620 0x55807286da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.909+0000 7f6097fff640 1 --2- 172.21.15.19:0/523474817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286df40 0x5580728782c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f60957fa640 1 -- 172.21.15.19:0/2561532541 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f60880f1170 con 0x7f6098103060 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f60977fe640 1 -- 172.21.15.19:0/523474817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55807286d620 msgr2=0x55807286da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f60977fe640 1 --2- 172.21.15.19:0/523474817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55807286d620 0x55807286da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f60977fe640 1 -- 172.21.15.19:0/523474817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286df40 msgr2=0x5580728782c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f60977fe640 1 --2- 172.21.15.19:0/523474817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286df40 0x5580728782c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f60977fe640 1 -- 172.21.15.19:0/523474817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580724e5210 con 0x558072878800 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f6097fff640 1 --2- 172.21.15.19:0/523474817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286df40 0x5580728782c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f609cda8640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60840a3dc0 0x7f60840a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6090005c40 tx=0x7f6090009cd0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f60977fe640 1 --2- 172.21.15.19:0/523474817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558072878800 0x55807287abf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f608000cc40 tx=0x7f608000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d296a933f83bfb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f608e7fc640 1 -- 172.21.15.19:0/523474817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080014070 con 0x558072878800 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f608e7fc640 1 -- 172.21.15.19:0/523474817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60800047a0 con 0x558072878800 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.910+0000 7f608e7fc640 1 -- 172.21.15.19:0/523474817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080010690 con 0x558072878800 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.911+0000 7f60a15f4740 1 -- 172.21.15.19:0/523474817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558072878800 msgr2=0x55807287abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.911+0000 7f60a15f4740 1 --2- 172.21.15.19:0/523474817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558072878800 0x55807287abf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f608000cc40 tx=0x7f608000e720 comp rx=0 tx=0).stop 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.911+0000 7f60a15f4740 1 -- 172.21.15.19:0/523474817 shutdown_connections 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.911+0000 7f60a15f4740 1 --2- 172.21.15.19:0/523474817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286df40 0x5580728782c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.911+0000 7f60a15f4740 1 --2- 172.21.15.19:0/523474817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558072878800 0x55807287abf0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.911+0000 7f60a15f4740 1 --2- 172.21.15.19:0/523474817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55807286d620 0x55807286da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.911+0000 7f60a15f4740 1 -- 172.21.15.19:0/523474817 >> 172.21.15.19:0/523474817 conn(0x558072869b30 msgr2=0x558072869f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.911+0000 7f60a15f4740 1 -- 172.21.15.19:0/523474817 shutdown_connections 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.911+0000 7f60a15f4740 1 -- 172.21.15.19:0/523474817 wait complete. 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.911+0000 7f60a15f4740 1 -- start start 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60a15f4740 1 Processor -- start 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x558072878150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072875440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072878800 0x5580728759a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60a15f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55807287b430 con 0x55807286df40 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60a15f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55807276d0b0 con 0x55807286d620 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60a15f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55807287b5b0 con 0x558072878800 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60a15f4740 10 client.0 _ll_register_callbacks cb 0x55807275ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60977fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072875440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60977fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072875440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48782/0 (socket says 172.21.15.19:48782) 2024-04-23T18:33:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60977fe640 1 -- 172.21.15.19:0/2537888768 learned_addr learned my addr 172.21.15.19:0/2537888768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f6097fff640 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x558072878150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f609cda8640 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072878800 0x5580728759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60977fe640 1 -- 172.21.15.19:0/2537888768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072878800 msgr2=0x5580728759a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60977fe640 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072878800 0x5580728759a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60977fe640 1 -- 172.21.15.19:0/2537888768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 msgr2=0x558072878150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60977fe640 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x558072878150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f60977fe640 1 -- 172.21.15.19:0/2537888768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580725b1620 con 0x55807286df40 2024-04-23T18:33:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.912+0000 7f6097fff640 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x558072878150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60977fe640 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072875440 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f608000f010 tx=0x7f6080010a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be867b9d642424a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60777fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60777fe640 1 -- 172.21.15.19:0/2537888768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080014030 con 0x55807286df40 2024-04-23T18:33:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60777fe640 1 -- 172.21.15.19:0/2537888768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f6080004050 con 0x55807286df40 2024-04-23T18:33:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60a15f4740 1 -- 172.21.15.19:0/2537888768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60800047a0 con 0x55807286df40 2024-04-23T18:33:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60777fe640 1 -- 172.21.15.19:0/2537888768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6080010e50 con 0x55807286df40 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60a15f4740 10 client.16641 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60a15f4740 1 -- 172.21.15.19:0/2537888768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558072887360 con 0x55807286df40 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60777fe640 1 -- 172.21.15.19:0/2537888768 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60800047a0 con 0x55807286df40 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60777fe640 1 -- 172.21.15.19:0/2537888768 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6080023420 con 0x55807286df40 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60a15f4740 10 client.16641 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60a15f4740 10 client.16641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60a15f4740 20 client.16641 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60a15f4740 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558072888760 0x55807288ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60a15f4740 4 client.16641 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.913+0000 7f60a15f4740 1 -- 172.21.15.19:0/2537888768 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580727e0390 con 0x558072888760 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.914+0000 7f6097fff640 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558072888760 0x55807288ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.914+0000 7f6097fff640 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558072888760 0x55807288ab50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6982eff080a3217e server_cookie=9f21eca6dbea4c0c in_seq=0 out_seq=0 2024-04-23T18:33:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.914+0000 7f60777fe640 10 client.16641 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60777fe640 1 -- 172.21.15.19:0/2537888768 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580727e0390 con 0x558072888760 2024-04-23T18:33:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60777fe640 10 client.16641 handle_command_reply: tid=0 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 client.16641 shutdown 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 -- 172.21.15.19:0/2537888768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 msgr2=0x558072875440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072875440 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f608000f010 tx=0x7f6080010a20 comp rx=0 tx=0).stop 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 -- 172.21.15.19:0/2537888768 shutdown_connections 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558072888760 0x55807288ab50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072878800 0x5580728759a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072875440 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60777fe640 0 client.16641 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 --2- 172.21.15.19:0/2537888768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x558072878150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 -- 172.21.15.19:0/2537888768 >> 172.21.15.19:0/2537888768 conn(0x558072869b30 msgr2=0x55807286f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 -- 172.21.15.19:0/2537888768 shutdown_connections 2024-04-23T18:33:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 1 -- 172.21.15.19:0/2537888768 wait complete. 2024-04-23T18:33:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.915+0000 7f60a15f4740 20 client.16641 trim_cache size 0 max 16384 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f60a15f4740 1 Processor -- start 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f60a15f4740 1 -- start start 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072869b30 0x5580728842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x558072868cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072890300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f60a15f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ab8e0 con 0x55807286df40 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f60a15f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ad420 con 0x55807286d620 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f60a15f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60981abfb0 con 0x558072869b30 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f609cda8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072890300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f609cda8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072890300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48784/0 (socket says 172.21.15.19:48784) 2024-04-23T18:33:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f609cda8640 1 -- 172.21.15.19:0/1993159875 learned_addr learned my addr 172.21.15.19:0/1993159875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f609cda8640 1 -- 172.21.15.19:0/1993159875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072869b30 msgr2=0x5580728842f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f609cda8640 1 --2- 172.21.15.19:0/1993159875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072869b30 0x5580728842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f609cda8640 1 -- 172.21.15.19:0/1993159875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 msgr2=0x558072868cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f609cda8640 1 --2- 172.21.15.19:0/1993159875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x558072868cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f609cda8640 1 -- 172.21.15.19:0/1993159875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580728743f0 con 0x55807286df40 2024-04-23T18:33:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.931+0000 7f609cda8640 1 --2- 172.21.15.19:0/1993159875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072890300 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f60900188b0 tx=0x7f6090004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212cc2ac7e8f30b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f608dffb640 1 -- 172.21.15.19:0/1993159875 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090029070 con 0x55807286df40 2024-04-23T18:33:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f608dffb640 1 -- 172.21.15.19:0/1993159875 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6090009690 con 0x55807286df40 2024-04-23T18:33:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f608dffb640 1 -- 172.21.15.19:0/1993159875 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090005720 con 0x55807286df40 2024-04-23T18:33:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f60a15f4740 1 -- 172.21.15.19:0/1993159875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 msgr2=0x558072890300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f60a15f4740 1 --2- 172.21.15.19:0/1993159875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072890300 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f60900188b0 tx=0x7f6090004ab0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f60a15f4740 1 -- 172.21.15.19:0/1993159875 shutdown_connections 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f60a15f4740 1 --2- 172.21.15.19:0/1993159875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x558072868cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f60a15f4740 1 --2- 172.21.15.19:0/1993159875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x558072890300 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f60a15f4740 1 --2- 172.21.15.19:0/1993159875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072869b30 0x5580728842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f60a15f4740 1 -- 172.21.15.19:0/1993159875 >> 172.21.15.19:0/1993159875 conn(0x55807288aec0 msgr2=0x558072869f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f60a15f4740 1 -- 172.21.15.19:0/1993159875 shutdown_connections 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.932+0000 7f60a15f4740 1 -- 172.21.15.19:0/1993159875 wait complete. 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f60a15f4740 1 -- start start 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f60a15f4740 1 Processor -- start 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072869b30 0x558072880ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x55807287e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f60a15f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x55807287e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f60a15f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60981acd50 con 0x55807286df40 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f60a15f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ac680 con 0x55807286d620 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f60a15f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60981a98e0 con 0x558072869b30 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f60a15f4740 10 client.0 _ll_register_callbacks cb 0x7f6080005740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f609cda8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x55807287e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f609cda8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x55807287e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48798/0 (socket says 172.21.15.19:48798) 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f609cda8640 1 -- 172.21.15.19:0/480883031 learned_addr learned my addr 172.21.15.19:0/480883031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f60977fe640 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x55807287e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.933+0000 7f6097fff640 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072869b30 0x558072880ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f609cda8640 1 -- 172.21.15.19:0/480883031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072869b30 msgr2=0x558072880ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f609cda8640 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072869b30 0x558072880ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f609cda8640 1 -- 172.21.15.19:0/480883031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 msgr2=0x55807287e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f609cda8640 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x55807287e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f609cda8640 1 -- 172.21.15.19:0/480883031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55807287ee10 con 0x55807286df40 2024-04-23T18:33:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f609cda8640 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x55807287e640 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6090005210 tx=0x7f6090018540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fa311699740b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f60777fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f60777fe640 1 -- 172.21.15.19:0/480883031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609003b060 con 0x55807286df40 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f60777fe640 1 -- 172.21.15.19:0/480883031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f6090005690 con 0x55807286df40 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f60a15f4740 1 -- 172.21.15.19:0/480883031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580727da200 con 0x55807286df40 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f60777fe640 1 -- 172.21.15.19:0/480883031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6090029040 con 0x55807286df40 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f60a15f4740 10 client.16647 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f60a15f4740 1 -- 172.21.15.19:0/480883031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5580728eec60 con 0x55807286df40 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.934+0000 7f60777fe640 1 -- 172.21.15.19:0/480883031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5580727da200 con 0x55807286df40 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.935+0000 7f60777fe640 1 -- 172.21.15.19:0/480883031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6090035030 con 0x55807286df40 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.935+0000 7f60a15f4740 10 client.16647 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.935+0000 7f60a15f4740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.935+0000 7f60a15f4740 20 client.16647 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.935+0000 7f60a15f4740 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558072878800 0x5580728f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.935+0000 7f60a15f4740 4 client.16647 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.935+0000 7f60a15f4740 1 -- 172.21.15.19:0/480883031 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580727565f0 con 0x558072878800 2024-04-23T18:33:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.935+0000 7f6097fff640 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558072878800 0x5580728f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.935+0000 7f60777fe640 10 client.16647 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.935+0000 7f6097fff640 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558072878800 0x5580728f13c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa3449cb14f9515 server_cookie=44e37568ced5b6f7 in_seq=0 out_seq=0 2024-04-23T18:33:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60777fe640 1 -- 172.21.15.19:0/480883031 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580727565f0 con 0x558072878800 2024-04-23T18:33:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60777fe640 10 client.16647 handle_command_reply: tid=0 2024-04-23T18:33:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 client.16647 shutdown 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 -- 172.21.15.19:0/480883031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 msgr2=0x55807287e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x55807287e640 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6090005210 tx=0x7f6090018540 comp rx=0 tx=0).stop 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 -- 172.21.15.19:0/480883031 shutdown_connections 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558072878800 0x5580728f13c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55807286d620 0x55807287e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55807286df40 0x55807287e640 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 --2- 172.21.15.19:0/480883031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558072869b30 0x558072880ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60777fe640 0 client.16647 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 -- 172.21.15.19:0/480883031 >> 172.21.15.19:0/480883031 conn(0x55807288aec0 msgr2=0x558072868cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 -- 172.21.15.19:0/480883031 shutdown_connections 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 1 -- 172.21.15.19:0/480883031 wait complete. 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.946+0000 7f60a15f4740 20 client.16647 trim_cache size 0 max 16384 2024-04-23T18:33:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:33:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:33:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:33:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:33:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 -- 172.21.15.19:0/2561532541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60840a3dc0 msgr2=0x7f60840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60840a3dc0 0x7f60840a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6090005c40 tx=0x7f6090009cd0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 -- 172.21.15.19:0/2561532541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6098103060 msgr2=0x7f60981986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6098103060 0x7f60981986c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6088005940 tx=0x7f60880042f0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 -- 172.21.15.19:0/2561532541 shutdown_connections 2024-04-23T18:33:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60840a3dc0 0x7f60840a6280 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f609810c580 0x7f609819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6098103980 0x7f6098198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 --2- 172.21.15.19:0/2561532541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6098103060 0x7f60981986c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 -- 172.21.15.19:0/2561532541 >> 172.21.15.19:0/2561532541 conn(0x7f6098077fd0 msgr2=0x7f60980fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 -- 172.21.15.19:0/2561532541 shutdown_connections 2024-04-23T18:33:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:37.947+0000 7f609e80b640 1 -- 172.21.15.19:0/2561532541 wait complete. 2024-04-23T18:33:38.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:37 smithi044 ceph-mon[32458]: pgmap v229: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:33:38.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:37 smithi155 ceph-mon[33468]: pgmap v229: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:33:38.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:37 smithi019 ceph-mon[29215]: pgmap v229: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:33:40.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:39 smithi044 ceph-mon[32458]: pgmap v230: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:33:40.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:39 smithi155 ceph-mon[33468]: pgmap v230: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:33:40.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:39 smithi019 ceph-mon[29215]: pgmap v230: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:33:42.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:41 smithi044 ceph-mon[32458]: pgmap v231: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-04-23T18:33:42.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:41 smithi155 ceph-mon[33468]: pgmap v231: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-04-23T18:33:42.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:41 smithi019 ceph-mon[29215]: pgmap v231: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-04-23T18:33:44.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:43 smithi044 ceph-mon[32458]: pgmap v232: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-23T18:33:44.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:43 smithi155 ceph-mon[33468]: pgmap v232: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-23T18:33:44.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:43 smithi019 ceph-mon[29215]: pgmap v232: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-23T18:33:46.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:45 smithi044 ceph-mon[32458]: pgmap v233: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-23T18:33:46.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:45 smithi155 ceph-mon[33468]: pgmap v233: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-23T18:33:46.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:45 smithi019 ceph-mon[29215]: pgmap v233: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-23T18:33:48.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:47 smithi044 ceph-mon[32458]: pgmap v234: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T18:33:48.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:47 smithi155 ceph-mon[33468]: pgmap v234: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T18:33:48.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:47 smithi019 ceph-mon[29215]: pgmap v234: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T18:33:50.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:49 smithi044 ceph-mon[32458]: pgmap v235: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T18:33:50.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:49 smithi155 ceph-mon[33468]: pgmap v235: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T18:33:50.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:49 smithi019 ceph-mon[29215]: pgmap v235: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T18:33:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.046+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/2026608517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3a4102b70 msgr2=0x7fe3a4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.046+0000 7fe3ab8d0640 1 --2- 172.21.15.19:0/2026608517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3a4102b70 0x7fe3a4106bb0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe394009900 tx=0x7fe39402a3f0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.046+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/2026608517 shutdown_connections 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.046+0000 7fe3ab8d0640 1 --2- 172.21.15.19:0/2026608517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a4107290 0x7fe3a410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.046+0000 7fe3ab8d0640 1 --2- 172.21.15.19:0/2026608517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3a4102b70 0x7fe3a4106bb0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.046+0000 7fe3ab8d0640 1 --2- 172.21.15.19:0/2026608517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3a41021c0 0x7fe3a41025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.046+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/2026608517 >> 172.21.15.19:0/2026608517 conn(0x7fe3a40fda40 msgr2=0x7fe3a40ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.046+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/2026608517 shutdown_connections 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.046+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/2026608517 wait complete. 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3ab8d0640 1 Processor -- start 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3ab8d0640 1 -- start start 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3ab8d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3a41021c0 0x7fe3a41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3ab8d0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a4107290 0x7fe3a4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3ab8d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3a419cf20 0x7fe3a419d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3ab8d0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41121b0 con 0x7fe3a419cf20 2024-04-23T18:33:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3ab8d0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4112030 con 0x7fe3a41021c0 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3ab8d0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4112330 con 0x7fe3a4107290 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3a9e6d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3a419cf20 0x7fe3a419d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3a9e6d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3a419cf20 0x7fe3a419d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34210/0 (socket says 172.21.15.19:34210) 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3a9e6d640 1 -- 172.21.15.19:0/1921848086 learned_addr learned my addr 172.21.15.19:0/1921848086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a4107290 0x7fe3a4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.047+0000 7fe3a966c640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3a41021c0 0x7fe3a41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe3a9e6d640 1 -- 172.21.15.19:0/1921848086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a4107290 msgr2=0x7fe3a4198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe3a9e6d640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a4107290 0x7fe3a4198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe3a9e6d640 1 -- 172.21.15.19:0/1921848086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3a41021c0 msgr2=0x7fe3a41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe3a9e6d640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3a41021c0 0x7fe3a41986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe3a9e6d640 1 -- 172.21.15.19:0/1921848086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a419da10 con 0x7fe3a419cf20 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe3a966c640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3a41021c0 0x7fe3a41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe3a9e6d640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3a419cf20 0x7fe3a419d3d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00cce0 tx=0x7fe39c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe39a7fc640 1 -- 172.21.15.19:0/1921848086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00f450 con 0x7fe3a419cf20 2024-04-23T18:33:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe39a7fc640 1 -- 172.21.15.19:0/1921848086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe39c00faa0 con 0x7fe3a419cf20 2024-04-23T18:33:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe39a7fc640 1 -- 172.21.15.19:0/1921848086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c018b70 con 0x7fe3a419cf20 2024-04-23T18:33:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/1921848086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a419dd00 con 0x7fe3a419cf20 2024-04-23T18:33:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.048+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/1921848086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a41a5470 con 0x7fe3a419cf20 2024-04-23T18:33:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.050+0000 7fe39a7fc640 1 -- 172.21.15.19:0/1921848086 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe39c018d10 con 0x7fe3a419cf20 2024-04-23T18:33:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3ae5dd740 1 Processor -- start 2024-04-23T18:33:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3ae5dd740 1 -- start start 2024-04-23T18:33:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b520ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe39a7fc640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3900a3dc0 0x7fe3900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520cf40 0x55c1b52172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b5219bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c1b5171970 con 0x55c1b520c620 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c1b510e4a0 con 0x55c1b5217800 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c1b516d930 con 0x55c1b520cf40 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3a8e6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b520ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3a8e6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b520ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34224/0 (socket says 172.21.15.19:34224) 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe39a7fc640 1 -- 172.21.15.19:0/1921848086 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fe39c0d0c10 con 0x7fe3a419cf20 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3a8e6b640 1 -- 172.21.15.19:0/3647955801 learned_addr learned my addr 172.21.15.19:0/3647955801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.051+0000 7fe3a966c640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3900a3dc0 0x7fe3900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe3a9e6d640 1 --2- 172.21.15.19:0/3647955801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b5219bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/3647955801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520cf40 0x55c1b52172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe3a8e6b640 1 -- 172.21.15.19:0/3647955801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520cf40 msgr2=0x55c1b52172c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/3647955801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520cf40 0x55c1b52172c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe3a8e6b640 1 -- 172.21.15.19:0/3647955801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 msgr2=0x55c1b5219bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/3647955801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b5219bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe3a8e6b640 1 -- 172.21.15.19:0/3647955801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1b4e84210 con 0x55c1b520c620 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/3647955801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b520ca00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe394032fd0 tx=0x7fe3940095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cbc6f3c9b9c94c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe3a966c640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3900a3dc0 0x7fe3900a6280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009c20 tx=0x7fe3a0005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe38f7fe640 1 -- 172.21.15.19:0/3647955801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394044020 con 0x55c1b520c620 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe38f7fe640 1 -- 172.21.15.19:0/3647955801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3940317a0 con 0x55c1b520c620 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.052+0000 7fe38f7fe640 1 -- 172.21.15.19:0/3647955801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394035040 con 0x55c1b520c620 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.053+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3647955801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 msgr2=0x55c1b520ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.053+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/3647955801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b520ca00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe394032fd0 tx=0x7fe3940095a0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.053+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3647955801 shutdown_connections 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.053+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/3647955801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b5219bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.053+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/3647955801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520cf40 0x55c1b52172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.053+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/3647955801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b520ca00 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.053+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3647955801 >> 172.21.15.19:0/3647955801 conn(0x55c1b5208b30 msgr2=0x55c1b5208f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.053+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3647955801 shutdown_connections 2024-04-23T18:33:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.053+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3647955801 wait complete. 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.053+0000 7fe3ae5dd740 1 -- start start 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3ae5dd740 1 Processor -- start 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 0x55c1b5217160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520cf40 0x55c1b5214450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b52149b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c1b521a3d0 con 0x55c1b520cf40 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c1b510c0b0 con 0x55c1b5217800 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c1b521a550 con 0x55c1b520c620 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3ae5dd740 10 client.0 _ll_register_callbacks cb 0x55c1b50f9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3a8e6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520cf40 0x55c1b5214450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3a8e6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520cf40 0x55c1b5214450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34232/0 (socket says 172.21.15.19:34232) 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3a8e6b640 1 -- 172.21.15.19:0/645494175 learned_addr learned my addr 172.21.15.19:0/645494175 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3a966c640 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 0x55c1b5217160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3a9e6d640 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b52149b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.054+0000 7fe3a8e6b640 1 -- 172.21.15.19:0/645494175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 msgr2=0x55c1b5217160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 0x55c1b5217160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe3a8e6b640 1 -- 172.21.15.19:0/645494175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 msgr2=0x55c1b52149b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b52149b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe3a8e6b640 1 -- 172.21.15.19:0/645494175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1b4f50620 con 0x55c1b520cf40 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520cf40 0x55c1b5214450 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe394009a20 tx=0x7fe394005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f60dae962068ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe38cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe38cff9640 1 -- 172.21.15.19:0/645494175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39404d070 con 0x55c1b520cf40 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe38cff9640 1 -- 172.21.15.19:0/645494175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fe394004260 con 0x55c1b520cf40 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/645494175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3940317a0 con 0x55c1b520cf40 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe38cff9640 1 -- 172.21.15.19:0/645494175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe394059050 con 0x55c1b520cf40 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe3ae5dd740 10 client.16659 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/645494175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c1b5226300 con 0x55c1b520cf40 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.055+0000 7fe38cff9640 1 -- 172.21.15.19:0/645494175 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3940317a0 con 0x55c1b520cf40 2024-04-23T18:33:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.056+0000 7fe38cff9640 1 -- 172.21.15.19:0/645494175 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe394049510 con 0x55c1b520cf40 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.056+0000 7fe3ae5dd740 10 client.16659 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.056+0000 7fe3ae5dd740 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.056+0000 7fe3ae5dd740 20 client.16659 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.056+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c1b5227650 0x55c1b5229a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.056+0000 7fe3ae5dd740 4 client.16659 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.056+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/645494175 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1b517f390 con 0x55c1b5227650 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.056+0000 7fe3a966c640 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c1b5227650 0x55c1b5229a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.056+0000 7fe3a966c640 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c1b5227650 0x55c1b5229a40 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc644e95c96c982 server_cookie=73f97c8030d57dda in_seq=0 out_seq=0 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.056+0000 7fe38cff9640 10 client.16659 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe38cff9640 1 -- 172.21.15.19:0/645494175 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1b517f390 con 0x55c1b5227650 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe38cff9640 10 client.16659 handle_command_reply: tid=0 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 client.16659 shutdown 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/645494175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520cf40 msgr2=0x55c1b5214450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520cf40 0x55c1b5214450 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe394009a20 tx=0x7fe394005780 comp rx=0 tx=0).stop 2024-04-23T18:33:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/645494175 shutdown_connections 2024-04-23T18:33:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c1b5227650 0x55c1b5229a40 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b52149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520cf40 0x55c1b5214450 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe38cff9640 0 client.16659 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/645494175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 0x55c1b5217160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/645494175 >> 172.21.15.19:0/645494175 conn(0x55c1b5208b30 msgr2=0x55c1b520e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/645494175 shutdown_connections 2024-04-23T18:33:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.057+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/645494175 wait complete. 2024-04-23T18:33:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.058+0000 7fe3ae5dd740 20 client.16659 trim_cache size 0 max 16384 2024-04-23T18:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.072+0000 7fe3ae5dd740 1 Processor -- start 2024-04-23T18:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.072+0000 7fe3ae5dd740 1 -- start start 2024-04-23T18:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b52135d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520cf40 0x55c1b5222eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b522ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41ad940 con 0x55c1b520c620 2024-04-23T18:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41af480 con 0x55c1b5217800 2024-04-23T18:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41ae010 con 0x55c1b520cf40 2024-04-23T18:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a966c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b52135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a966c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b52135d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34234/0 (socket says 172.21.15.19:34234) 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a966c640 1 -- 172.21.15.19:0/687810764 learned_addr learned my addr 172.21.15.19:0/687810764 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/687810764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520cf40 0x55c1b5222eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a966c640 1 -- 172.21.15.19:0/687810764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520cf40 msgr2=0x55c1b5222eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a9e6d640 1 --2- 172.21.15.19:0/687810764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b522ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a966c640 1 --2- 172.21.15.19:0/687810764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520cf40 0x55c1b5222eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a966c640 1 -- 172.21.15.19:0/687810764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 msgr2=0x55c1b522ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a966c640 1 --2- 172.21.15.19:0/687810764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b522ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a966c640 1 -- 172.21.15.19:0/687810764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1b520cd70 con 0x55c1b520c620 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe3a966c640 1 --2- 172.21.15.19:0/687810764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b52135d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009ae0 tx=0x7fe3a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ce3bf89e51414b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe38effd640 1 -- 172.21.15.19:0/687810764 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a002f070 con 0x55c1b520c620 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe38effd640 1 -- 172.21.15.19:0/687810764 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a0016070 con 0x55c1b520c620 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.073+0000 7fe38effd640 1 -- 172.21.15.19:0/687810764 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0003bb0 con 0x55c1b520c620 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.074+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/687810764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 msgr2=0x55c1b52135d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.074+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/687810764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b52135d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009ae0 tx=0x7fe3a0014000 comp rx=0 tx=0).stop 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.074+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/687810764 shutdown_connections 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.074+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/687810764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b5217800 0x55c1b522ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.074+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/687810764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520cf40 0x55c1b5222eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.074+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/687810764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b520c620 0x55c1b52135d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.074+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/687810764 >> 172.21.15.19:0/687810764 conn(0x55c1b5208b30 msgr2=0x55c1b52227a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.074+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/687810764 shutdown_connections 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.074+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/687810764 wait complete. 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.074+0000 7fe3ae5dd740 1 -- start start 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3ae5dd740 1 Processor -- start 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 0x55c1b522be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b520cf40 0x55c1b522c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3ae5dd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b5217800 0x55c1b5285950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41aedb0 con 0x55c1b5217800 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41ae6e0 con 0x55c1b520cf40 2024-04-23T18:33:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3ae5dd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41ab940 con 0x55c1b520c620 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3ae5dd740 10 client.0 _ll_register_callbacks cb 0x7fe394005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a9e6d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b5217800 0x55c1b5285950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a966c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 0x55c1b522be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a9e6d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b5217800 0x55c1b5285950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34246/0 (socket says 172.21.15.19:34246) 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a966c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 0x55c1b522be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:50340/0 (socket says 172.21.15.19:50340) 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a9e6d640 1 -- 172.21.15.19:0/3592008738 learned_addr learned my addr 172.21.15.19:0/3592008738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b520cf40 0x55c1b522c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a9e6d640 1 -- 172.21.15.19:0/3592008738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 msgr2=0x55c1b522be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a9e6d640 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 0x55c1b522be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a9e6d640 1 -- 172.21.15.19:0/3592008738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b520cf40 msgr2=0x55c1b522c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a9e6d640 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b520cf40 0x55c1b522c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a9e6d640 1 -- 172.21.15.19:0/3592008738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1b5285ff0 con 0x55c1b5217800 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.075+0000 7fe3a8e6b640 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b520cf40 0x55c1b522c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe3a9e6d640 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b5217800 0x55c1b5285950 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0ce3c0 tx=0x7fe39c09c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3fc1fca0b3febe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe38cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe38cff9640 1 -- 172.21.15.19:0/3592008738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c09d2b0 con 0x55c1b5217800 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe38cff9640 1 -- 172.21.15.19:0/3592008738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fe39c0bd0b0 con 0x55c1b5217800 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3592008738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3a0016070 con 0x55c1b5217800 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe38cff9640 1 -- 172.21.15.19:0/3592008738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe39c0b6050 con 0x55c1b5217800 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe3ae5dd740 10 client.16665 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3592008738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c1b528d390 con 0x55c1b5217800 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe38cff9640 1 -- 172.21.15.19:0/3592008738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a0016070 con 0x55c1b5217800 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe38cff9640 1 -- 172.21.15.19:0/3592008738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe39c0ce5e0 con 0x55c1b5217800 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe3ae5dd740 10 client.16665 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe3ae5dd740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe3ae5dd740 20 client.16665 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c1b528dab0 0x55c1b528fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe3ae5dd740 4 client.16665 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.076+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3592008738 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1b5179200 con 0x55c1b528dab0 2024-04-23T18:33:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.077+0000 7fe3a966c640 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c1b528dab0 0x55c1b528fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.077+0000 7fe38cff9640 10 client.16665 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.077+0000 7fe3a966c640 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c1b528dab0 0x55c1b528fed0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57aa6189072aad1f server_cookie=276917b8ab84f08e in_seq=0 out_seq=0 2024-04-23T18:33:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.084+0000 7fe38cff9640 1 -- 172.21.15.19:0/3592008738 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1b5179200 con 0x55c1b528dab0 2024-04-23T18:33:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.084+0000 7fe38cff9640 10 client.16665 handle_command_reply: tid=0 2024-04-23T18:33:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.084+0000 7fe3ae5dd740 1 client.16665 shutdown 2024-04-23T18:33:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3592008738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b5217800 msgr2=0x55c1b5285950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b5217800 0x55c1b5285950 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0ce3c0 tx=0x7fe39c09c510 comp rx=0 tx=0).stop 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3592008738 shutdown_connections 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c1b528dab0 0x55c1b528fed0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c1b520cf40 0x55c1b522c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1b5217800 0x55c1b5285950 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 1 --2- 172.21.15.19:0/3592008738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1b520c620 0x55c1b522be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe38cff9640 0 client.16665 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3592008738 >> 172.21.15.19:0/3592008738 conn(0x55c1b5208b30 msgr2=0x55c1b5223030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3592008738 shutdown_connections 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 1 -- 172.21.15.19:0/3592008738 wait complete. 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.085+0000 7fe3ae5dd740 20 client.16665 trim_cache size 0 max 16384 2024-04-23T18:33:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:33:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:33:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:33:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:33:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/1921848086 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3900a3dc0 msgr2=0x7fe3900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3900a3dc0 0x7fe3900a6280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009c20 tx=0x7fe3a0005f90 comp rx=0 tx=0).stop 2024-04-23T18:33:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/1921848086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3a419cf20 msgr2=0x7fe3a419d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3a419cf20 0x7fe3a419d3d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00cce0 tx=0x7fe39c00a620 comp rx=0 tx=0).stop 2024-04-23T18:33:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/1921848086 shutdown_connections 2024-04-23T18:33:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3900a3dc0 0x7fe3900a6280 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3a419cf20 0x7fe3a419d3d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a4107290 0x7fe3a4198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 --2- 172.21.15.19:0/1921848086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3a41021c0 0x7fe3a41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/1921848086 >> 172.21.15.19:0/1921848086 conn(0x7fe3a40fda40 msgr2=0x7fe3a40ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/1921848086 shutdown_connections 2024-04-23T18:33:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:33:51.086+0000 7fe3ab8d0640 1 -- 172.21.15.19:0/1921848086 wait complete. 2024-04-23T18:33:52.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:51 smithi044 ceph-mon[32458]: pgmap v236: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 15 op/s 2024-04-23T18:33:52.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:51 smithi155 ceph-mon[33468]: pgmap v236: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 15 op/s 2024-04-23T18:33:52.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:51 smithi019 ceph-mon[29215]: pgmap v236: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 15 op/s 2024-04-23T18:33:52.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.062+0000 7f1827fe5640 1 -- 172.21.15.19:0/2530767363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f182010c520 msgr2=0x7f182010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.062+0000 7f1827fe5640 1 --2- 172.21.15.19:0/2530767363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f182010c520 0x7f182010e9c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1810009ca0 tx=0x7f181002a9e0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.062+0000 7f1827fe5640 1 -- 172.21.15.19:0/2530767363 shutdown_connections 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.062+0000 7f1827fe5640 1 --2- 172.21.15.19:0/2530767363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f182010c520 0x7f182010e9c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.062+0000 7f1827fe5640 1 --2- 172.21.15.19:0/2530767363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1820103b00 0x7f182010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.062+0000 7f1827fe5640 1 --2- 172.21.15.19:0/2530767363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18201031e0 0x7f18201035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.062+0000 7f1827fe5640 1 -- 172.21.15.19:0/2530767363 >> 172.21.15.19:0/2530767363 conn(0x7f1820077f50 msgr2=0x7f1820078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.062+0000 7f1827fe5640 1 -- 172.21.15.19:0/2530767363 shutdown_connections 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.062+0000 7f1827fe5640 1 -- 172.21.15.19:0/2530767363 wait complete. 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1827fe5640 1 Processor -- start 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1827fe5640 1 -- start start 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1827fe5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18201031e0 0x7f1820100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1827fe5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1820103b00 0x7f1820101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1827fe5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f182010c520 0x7f18201018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1827fe5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f18201121b0 con 0x7f182010c520 2024-04-23T18:33:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1827fe5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1820112030 con 0x7f18201031e0 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1827fe5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1820112330 con 0x7f1820103b00 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1826582640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f182010c520 0x7f18201018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1826582640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f182010c520 0x7f18201018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34276/0 (socket says 172.21.15.19:34276) 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1826582640 1 -- 172.21.15.19:0/2500178623 learned_addr learned my addr 172.21.15.19:0/2500178623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1825580640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1820103b00 0x7f1820101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1825d81640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18201031e0 0x7f1820100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1826582640 1 -- 172.21.15.19:0/2500178623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1820103b00 msgr2=0x7f1820101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1826582640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1820103b00 0x7f1820101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1826582640 1 -- 172.21.15.19:0/2500178623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18201031e0 msgr2=0x7f1820100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1826582640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18201031e0 0x7f1820100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1826582640 1 -- 172.21.15.19:0/2500178623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18201a6bb0 con 0x7f182010c520 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.063+0000 7f1825d81640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18201031e0 0x7f1820100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.064+0000 7f1826582640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f182010c520 0x7f18201018b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1810006010 tx=0x7f181002fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.064+0000 7f181effd640 1 -- 172.21.15.19:0/2500178623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810033040 con 0x7f182010c520 2024-04-23T18:33:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.064+0000 7f181effd640 1 -- 172.21.15.19:0/2500178623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181002e7a0 con 0x7f182010c520 2024-04-23T18:33:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.064+0000 7f1827fe5640 1 -- 172.21.15.19:0/2500178623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18201a6e40 con 0x7f182010c520 2024-04-23T18:33:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.064+0000 7f181effd640 1 -- 172.21.15.19:0/2500178623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810043720 con 0x7f182010c520 2024-04-23T18:33:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.064+0000 7f1827fe5640 1 -- 172.21.15.19:0/2500178623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18201a72a0 con 0x7f182010c520 2024-04-23T18:33:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.065+0000 7f1827fe5640 1 -- 172.21.15.19:0/2500178623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1820068d70 con 0x7f182010c520 2024-04-23T18:33:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.066+0000 7f181effd640 1 -- 172.21.15.19:0/2500178623 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1810036070 con 0x7f182010c520 2024-04-23T18:33:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.067+0000 7f181effd640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f18080a3d20 0x7f18080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.067+0000 7f181effd640 1 -- 172.21.15.19:0/2500178623 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f18100f1ce0 con 0x7f182010c520 2024-04-23T18:33:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.067+0000 7f1825d81640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f18080a3d20 0x7f18080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.067+0000 7f1825d81640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f18080a3d20 0x7f18080a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1818009b50 tx=0x7f1818005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:53.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.071+0000 7f181effd640 1 -- 172.21.15.19:0/2500178623 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f181003e070 con 0x7f182010c520 2024-04-23T18:33:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.296+0000 7f1827fe5640 1 -- 172.21.15.19:0/2500178623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f182002d5c0 con 0x7f182010c520 2024-04-23T18:33:53.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.297+0000 7f181effd640 1 -- 172.21.15.19:0/2500178623 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f18100098b0 con 0x7f182010c520 2024-04-23T18:33:53.289 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:33:53.290 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:33:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:33:53.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 -- 172.21.15.19:0/2500178623 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f18080a3d20 msgr2=0x7f18080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f18080a3d20 0x7f18080a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1818009b50 tx=0x7f1818005f90 comp rx=0 tx=0).stop 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 -- 172.21.15.19:0/2500178623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f182010c520 msgr2=0x7f18201018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f182010c520 0x7f18201018b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1810006010 tx=0x7f181002fbe0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 -- 172.21.15.19:0/2500178623 shutdown_connections 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f18080a3d20 0x7f18080a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f182010c520 0x7f18201018b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1820103b00 0x7f1820101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 --2- 172.21.15.19:0/2500178623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18201031e0 0x7f1820100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 -- 172.21.15.19:0/2500178623 >> 172.21.15.19:0/2500178623 conn(0x7f1820077f50 msgr2=0x7f18200fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 -- 172.21.15.19:0/2500178623 shutdown_connections 2024-04-23T18:33:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.299+0000 7f1827fe5640 1 -- 172.21.15.19:0/2500178623 wait complete. 2024-04-23T18:33:53.302 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id a8698f7 --release --await' 2024-04-23T18:33:53.302 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id a8698f7 --release --await 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.419+0000 7fe345edc640 1 -- 172.21.15.19:0/1163807676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe34006b1f0 msgr2=0x7fe340106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.419+0000 7fe345edc640 1 --2- 172.21.15.19:0/1163807676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe34006b1f0 0x7fe340106b90 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe334009900 tx=0x7fe33402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.419+0000 7fe345edc640 1 -- 172.21.15.19:0/1163807676 shutdown_connections 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.419+0000 7fe345edc640 1 --2- 172.21.15.19:0/1163807676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe340107270 0x7fe34010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.419+0000 7fe345edc640 1 --2- 172.21.15.19:0/1163807676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe34006b1f0 0x7fe340106b90 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.419+0000 7fe345edc640 1 --2- 172.21.15.19:0/1163807676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe34006a840 0x7fe34006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.419+0000 7fe345edc640 1 -- 172.21.15.19:0/1163807676 >> 172.21.15.19:0/1163807676 conn(0x7fe340077eb0 msgr2=0x7fe3400782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.419+0000 7fe345edc640 1 -- 172.21.15.19:0/1163807676 shutdown_connections 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.419+0000 7fe345edc640 1 -- 172.21.15.19:0/1163807676 wait complete. 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe345edc640 1 Processor -- start 2024-04-23T18:33:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe345edc640 1 -- start start 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe345edc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe34006a840 0x7fe340198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe345edc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe340107270 0x7fe340198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe345edc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe34019d010 0x7fe34019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe345edc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401121b0 con 0x7fe34006a840 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe345edc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe340112030 con 0x7fe340107270 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe345edc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe340112330 con 0x7fe34019d010 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe33f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe34006a840 0x7fe340198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe33f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe34006a840 0x7fe340198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34284/0 (socket says 172.21.15.19:34284) 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe33f7fe640 1 -- 172.21.15.19:0/3142629518 learned_addr learned my addr 172.21.15.19:0/3142629518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe33effd640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe340107270 0x7fe340198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.420+0000 7fe33ffff640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe34019d010 0x7fe34019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe33f7fe640 1 -- 172.21.15.19:0/3142629518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe34019d010 msgr2=0x7fe34019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe34019d010 0x7fe34019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe33f7fe640 1 -- 172.21.15.19:0/3142629518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe340107270 msgr2=0x7fe340198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe340107270 0x7fe340198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe33f7fe640 1 -- 172.21.15.19:0/3142629518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34019db00 con 0x7fe34006a840 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe34006a840 0x7fe340198750 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe33000e9f0 tx=0x7fe33000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe33cff9640 1 -- 172.21.15.19:0/3142629518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33000cd80 con 0x7fe34006a840 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe33cff9640 1 -- 172.21.15.19:0/3142629518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3300054d0 con 0x7fe34006a840 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe345edc640 1 -- 172.21.15.19:0/3142629518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34019ddf0 con 0x7fe34006a840 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe33cff9640 1 -- 172.21.15.19:0/3142629518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330010690 con 0x7fe34006a840 2024-04-23T18:33:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.421+0000 7fe345edc640 1 -- 172.21.15.19:0/3142629518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3401a5560 con 0x7fe34006a840 2024-04-23T18:33:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.423+0000 7fe33cff9640 1 -- 172.21.15.19:0/3142629518 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe330004e90 con 0x7fe34006a840 2024-04-23T18:33:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.423+0000 7fe348bea740 1 Processor -- start 2024-04-23T18:33:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe348bea740 1 -- start start 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe348bea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb01490 0x55c96eb01870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe348bea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 0x55c96eb0bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe348bea740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb0c480 0x55c96eb0e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe348bea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c96ea666d0 con 0x55c96eb01490 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe348bea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c96ea031a0 con 0x55c96eb01db0 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe348bea740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c96ea62690 con 0x55c96eb0c480 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb01490 0x55c96eb01870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33cff9640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3280a3d20 0x7fe3280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb01490 0x55c96eb01870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34288/0 (socket says 172.21.15.19:34288) 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33ffff640 1 -- 172.21.15.19:0/3198144390 learned_addr learned my addr 172.21.15.19:0/3198144390 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 0x55c96eb0bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33ffff640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3280a3d20 0x7fe3280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33cff9640 1 -- 172.21.15.19:0/3142629518 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fe3300d0c60 con 0x7fe34006a840 2024-04-23T18:33:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33ffff640 1 -- 172.21.15.19:0/3198144390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb0c480 msgr2=0x55c96eb0e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33effd640 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb0c480 0x55c96eb0e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33ffff640 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb0c480 0x55c96eb0e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33ffff640 1 -- 172.21.15.19:0/3198144390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 msgr2=0x55c96eb0bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33ffff640 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 0x55c96eb0bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33ffff640 1 -- 172.21.15.19:0/3198144390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96ea0d5b0 con 0x55c96eb01490 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 0x55c96eb0bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33effd640 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb0c480 0x55c96eb0e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.424+0000 7fe33ffff640 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb01490 0x55c96eb01870 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe338005c40 tx=0x7fe338013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f795e512c590c097 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe325ffb640 1 -- 172.21.15.19:0/3198144390 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338022020 con 0x55c96eb01490 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe325ffb640 1 -- 172.21.15.19:0/3198144390 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3380134e0 con 0x55c96eb01490 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe33ffff640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3280a3d20 0x7fe3280a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe338023000 tx=0x7fe338021000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe325ffb640 1 -- 172.21.15.19:0/3198144390 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338010910 con 0x55c96eb01490 2024-04-23T18:33:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe348bea740 1 -- 172.21.15.19:0/3198144390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb01490 msgr2=0x55c96eb01870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe348bea740 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb01490 0x55c96eb01870 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe338005c40 tx=0x7fe338013f50 comp rx=0 tx=0).stop 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe348bea740 1 -- 172.21.15.19:0/3198144390 shutdown_connections 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe348bea740 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 0x55c96eb0bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe348bea740 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb0c480 0x55c96eb0e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe348bea740 1 --2- 172.21.15.19:0/3198144390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb01490 0x55c96eb01870 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe348bea740 1 -- 172.21.15.19:0/3198144390 >> 172.21.15.19:0/3198144390 conn(0x55c96eafd7b0 msgr2=0x55c96eafdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe348bea740 1 -- 172.21.15.19:0/3198144390 shutdown_connections 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.425+0000 7fe348bea740 1 -- 172.21.15.19:0/3198144390 wait complete. 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe348bea740 1 -- start start 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe348bea740 1 Processor -- start 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe348bea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01490 0x55c96eb0be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe348bea740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01db0 0x55c96eb09130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe348bea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb09740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe348bea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c96eb0f310 con 0x55c96eb0c480 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe348bea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c96ea00b20 con 0x55c96eb01490 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe348bea740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c96eb0f490 con 0x55c96eb01db0 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe348bea740 10 client.0 _ll_register_callbacks cb 0x55c96ea00530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe33ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb09740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe33ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb09740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34304/0 (socket says 172.21.15.19:34304) 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe33ffff640 1 -- 172.21.15.19:0/4124305610 learned_addr learned my addr 172.21.15.19:0/4124305610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe33effd640 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01db0 0x55c96eb09130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.426+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01490 0x55c96eb0be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe33ffff640 1 -- 172.21.15.19:0/4124305610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01db0 msgr2=0x55c96eb09130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe33ffff640 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01db0 0x55c96eb09130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe33ffff640 1 -- 172.21.15.19:0/4124305610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01490 msgr2=0x55c96eb0be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe33ffff640 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01490 0x55c96eb0be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe33ffff640 1 -- 172.21.15.19:0/4124305610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96e778920 con 0x55c96eb0c480 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe33ffff640 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb09740 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe338005c10 tx=0x7fe338012270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc56636de6080bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe31effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe31effd640 1 -- 172.21.15.19:0/4124305610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338022050 con 0x55c96eb0c480 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe348bea740 1 -- 172.21.15.19:0/4124305610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c96ea24500 con 0x55c96eb0c480 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe31effd640 1 -- 172.21.15.19:0/4124305610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fe338010860 con 0x55c96eb0c480 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe31effd640 1 -- 172.21.15.19:0/4124305610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe338012820 con 0x55c96eb0c480 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe348bea740 10 client.16683 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe348bea740 1 -- 172.21.15.19:0/4124305610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c96e844d30 con 0x55c96eb0c480 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.427+0000 7fe31effd640 1 -- 172.21.15.19:0/4124305610 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c96ea24500 con 0x55c96eb0c480 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.428+0000 7fe31effd640 1 -- 172.21.15.19:0/4124305610 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe33803a9e0 con 0x55c96eb0c480 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.428+0000 7fe348bea740 10 client.16683 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.428+0000 7fe348bea740 10 client.16683 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.428+0000 7fe348bea740 20 client.16683 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.428+0000 7fe348bea740 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c96eb1c3d0 0x55c96eb1e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.428+0000 7fe348bea740 4 client.16683 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.428+0000 7fe348bea740 1 -- 172.21.15.19:0/4124305610 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96ea740f0 con 0x55c96eb1c3d0 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.428+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c96eb1c3d0 0x55c96eb1e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.428+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c96eb1c3d0 0x55c96eb1e7c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b08c1ea2bc30c4c1 server_cookie=a7679d7b698c711b in_seq=0 out_seq=0 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.428+0000 7fe31effd640 10 client.16683 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe31effd640 1 -- 172.21.15.19:0/4124305610 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c96ea740f0 con 0x55c96eb1c3d0 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe31effd640 10 client.16683 handle_command_reply: tid=0 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 client.16683 shutdown 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 -- 172.21.15.19:0/4124305610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 msgr2=0x55c96eb09740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb09740 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe338005c10 tx=0x7fe338012270 comp rx=0 tx=0).stop 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 -- 172.21.15.19:0/4124305610 shutdown_connections 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c96eb1c3d0 0x55c96eb1e7c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb09740 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01db0 0x55c96eb09130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 --2- 172.21.15.19:0/4124305610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01490 0x55c96eb0be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe31effd640 0 client.16683 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 -- 172.21.15.19:0/4124305610 >> 172.21.15.19:0/4124305610 conn(0x55c96eafd7b0 msgr2=0x55c96eb035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 -- 172.21.15.19:0/4124305610 shutdown_connections 2024-04-23T18:33:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.429+0000 7fe348bea740 1 -- 172.21.15.19:0/4124305610 wait complete. 2024-04-23T18:33:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.430+0000 7fe348bea740 20 client.16683 trim_cache size 0 max 16384 2024-04-23T18:33:53.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe348bea740 1 Processor -- start 2024-04-23T18:33:53.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe348bea740 1 -- start start 2024-04-23T18:33:53.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe348bea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01490 0x55c96eb03c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe348bea740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01db0 0x55c96eb181d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe348bea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe348bea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401afdd0 con 0x55c96eb0c480 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe348bea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401b1910 con 0x55c96eb01490 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe348bea740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401b04a0 con 0x55c96eb01db0 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb086b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34308/0 (socket says 172.21.15.19:34308) 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33ffff640 1 -- 172.21.15.19:0/4264034521 learned_addr learned my addr 172.21.15.19:0/4264034521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/4264034521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01490 0x55c96eb03c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33effd640 1 --2- 172.21.15.19:0/4264034521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01db0 0x55c96eb181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33ffff640 1 -- 172.21.15.19:0/4264034521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01db0 msgr2=0x55c96eb181d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33ffff640 1 --2- 172.21.15.19:0/4264034521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01db0 0x55c96eb181d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33ffff640 1 -- 172.21.15.19:0/4264034521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01490 msgr2=0x55c96eb03c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33ffff640 1 --2- 172.21.15.19:0/4264034521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01490 0x55c96eb03c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33ffff640 1 -- 172.21.15.19:0/4264034521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96eb1edf0 con 0x55c96eb0c480 2024-04-23T18:33:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe33ffff640 1 --2- 172.21.15.19:0/4264034521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb086b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe33803aeb0 tx=0x7fe3380105e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e35740d95d923e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.447+0000 7fe3257fa640 1 -- 172.21.15.19:0/4264034521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33803f030 con 0x55c96eb0c480 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe3257fa640 1 -- 172.21.15.19:0/4264034521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33800fde0 con 0x55c96eb0c480 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe3257fa640 1 -- 172.21.15.19:0/4264034521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338022040 con 0x55c96eb0c480 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe348bea740 1 -- 172.21.15.19:0/4264034521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 msgr2=0x55c96eb086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe348bea740 1 --2- 172.21.15.19:0/4264034521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb086b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe33803aeb0 tx=0x7fe3380105e0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe348bea740 1 -- 172.21.15.19:0/4264034521 shutdown_connections 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe348bea740 1 --2- 172.21.15.19:0/4264034521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb086b0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe348bea740 1 --2- 172.21.15.19:0/4264034521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01db0 0x55c96eb181d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe348bea740 1 --2- 172.21.15.19:0/4264034521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01490 0x55c96eb03c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe348bea740 1 -- 172.21.15.19:0/4264034521 >> 172.21.15.19:0/4264034521 conn(0x55c96eafd7b0 msgr2=0x55c96eb03580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe348bea740 1 -- 172.21.15.19:0/4264034521 shutdown_connections 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe348bea740 1 -- 172.21.15.19:0/4264034521 wait complete. 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.448+0000 7fe348bea740 1 -- start start 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe348bea740 1 Processor -- start 2024-04-23T18:33:53.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe348bea740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01490 0x55c96eb14c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe348bea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 0x55c96eb06820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe348bea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb0d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe348bea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401b1240 con 0x55c96eb0c480 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe348bea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401b0b70 con 0x55c96eb01db0 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe348bea740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401addf0 con 0x55c96eb01490 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe348bea740 10 client.0 _ll_register_callbacks cb 0x7fe338013730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb0d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 0x55c96eb06820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb0d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34316/0 (socket says 172.21.15.19:34316) 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33ffff640 1 -- 172.21.15.19:0/985730351 learned_addr learned my addr 172.21.15.19:0/985730351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01490 0x55c96eb14c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33ffff640 1 -- 172.21.15.19:0/985730351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01490 msgr2=0x55c96eb14c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33ffff640 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01490 0x55c96eb14c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33ffff640 1 -- 172.21.15.19:0/985730351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 msgr2=0x55c96eb06820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33ffff640 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 0x55c96eb06820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33ffff640 1 -- 172.21.15.19:0/985730351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96eb071d0 con 0x55c96eb0c480 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33effd640 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 0x55c96eb06820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.449+0000 7fe33ffff640 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb0d530 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe3380108d0 tx=0x7fe338004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2c48d17a12069f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe31effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe31effd640 1 -- 172.21.15.19:0/985730351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338022070 con 0x55c96eb0c480 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe31effd640 1 -- 172.21.15.19:0/985730351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fe33803a480 con 0x55c96eb0c480 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe348bea740 1 -- 172.21.15.19:0/985730351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe33800fde0 con 0x55c96eb0c480 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe31effd640 1 -- 172.21.15.19:0/985730351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe338010430 con 0x55c96eb0c480 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe348bea740 10 client.16689 fetch_fsmap learned FSMap version 27 2024-04-23T18:33:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe348bea740 1 -- 172.21.15.19:0/985730351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c96eb82e40 con 0x55c96eb0c480 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe31effd640 1 -- 172.21.15.19:0/985730351 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33800fde0 con 0x55c96eb0c480 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe31effd640 1 -- 172.21.15.19:0/985730351 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe338005040 con 0x55c96eb0c480 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe348bea740 10 client.16689 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe348bea740 10 client.16689 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe348bea740 20 client.16689 populate_metadata read hostname 'smithi019' 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe348bea740 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c96eb834f0 0x55c96eb858e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe348bea740 4 client.16689 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a8698f7", "release": true, "await": true}] 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.450+0000 7fe348bea740 1 -- 172.21.15.19:0/985730351 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a8698f7", "release": true, "await": true}) -- 0x55c96ea6df60 con 0x55c96eb834f0 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.451+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c96eb834f0 0x55c96eb858e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.451+0000 7fe33f7fe640 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c96eb834f0 0x55c96eb858e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed45a6d73a8604aa server_cookie=29489965157bb3d7 in_seq=0 out_seq=0 2024-04-23T18:33:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.451+0000 7fe31effd640 10 client.16689 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe31effd640 1 -- 172.21.15.19:0/985730351 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55c96ea6df60 con 0x55c96eb834f0 2024-04-23T18:33:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe31effd640 10 client.16689 handle_command_reply: tid=0 2024-04-23T18:33:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 client.16689 shutdown 2024-04-23T18:33:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 -- 172.21.15.19:0/985730351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 msgr2=0x55c96eb0d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb0d530 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe3380108d0 tx=0x7fe338004c00 comp rx=0 tx=0).stop 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 -- 172.21.15.19:0/985730351 shutdown_connections 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55c96eb834f0 0x55c96eb858e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c96eb01db0 0x55c96eb06820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c96eb0c480 0x55c96eb0d530 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe31effd640 0 client.16689 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 --2- 172.21.15.19:0/985730351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96eb01490 0x55c96eb14c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 -- 172.21.15.19:0/985730351 >> 172.21.15.19:0/985730351 conn(0x55c96eafd7b0 msgr2=0x55c96eb171c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 -- 172.21.15.19:0/985730351 shutdown_connections 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 1 -- 172.21.15.19:0/985730351 wait complete. 2024-04-23T18:33:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.456+0000 7fe348bea740 20 client.16689 trim_cache size 0 max 16384 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 36, 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "a8698f7": { 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 36, 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:33:53.452 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 85.0, 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:33:53.453 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:33:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 -- 172.21.15.19:0/3142629518 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3280a3d20 msgr2=0x7fe3280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3280a3d20 0x7fe3280a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe338023000 tx=0x7fe338021000 comp rx=0 tx=0).stop 2024-04-23T18:33:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 -- 172.21.15.19:0/3142629518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe34006a840 msgr2=0x7fe340198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe34006a840 0x7fe340198750 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe33000e9f0 tx=0x7fe33000edf0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 -- 172.21.15.19:0/3142629518 shutdown_connections 2024-04-23T18:33:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3280a3d20 0x7fe3280a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe340107270 0x7fe340198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe34019d010 0x7fe34019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 --2- 172.21.15.19:0/3142629518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe34006a840 0x7fe340198750 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 -- 172.21.15.19:0/3142629518 >> 172.21.15.19:0/3142629518 conn(0x7fe340077eb0 msgr2=0x7fe340108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 -- 172.21.15.19:0/3142629518 shutdown_connections 2024-04-23T18:33:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:33:53.457+0000 7fe345edc640 1 -- 172.21.15.19:0/3142629518 wait complete. 2024-04-23T18:33:53.460 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a8698f7', seconds elapsed: 26.0 2024-04-23T18:33:54.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:53 smithi044 ceph-mon[32458]: pgmap v237: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:33:54.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:53 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2500178623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:54.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:53 smithi155 ceph-mon[33468]: pgmap v237: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:33:54.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:53 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2500178623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:54.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:53 smithi019 ceph-mon[29215]: pgmap v237: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:33:54.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:53 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2500178623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:56.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:55 smithi044 ceph-mon[32458]: pgmap v238: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-04-23T18:33:56.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:55 smithi155 ceph-mon[33468]: pgmap v238: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-04-23T18:33:56.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:55 smithi019 ceph-mon[29215]: pgmap v238: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-04-23T18:33:58.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:57 smithi044 ceph-mon[32458]: pgmap v239: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-23T18:33:58.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:57 smithi155 ceph-mon[33468]: pgmap v239: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-23T18:33:58.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:57 smithi019 ceph-mon[29215]: pgmap v239: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-23T18:34:00.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:33:59 smithi044 ceph-mon[32458]: pgmap v240: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-23T18:34:00.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:33:59 smithi155 ceph-mon[33468]: pgmap v240: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-23T18:34:00.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:33:59 smithi019 ceph-mon[29215]: pgmap v240: 97 pgs: 97 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-23T18:34:02.061 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:01 smithi019 ceph-mon[29215]: pgmap v241: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-23T18:34:02.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:01 smithi044 ceph-mon[32458]: pgmap v241: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-23T18:34:02.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:01 smithi155 ceph-mon[33468]: pgmap v241: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-23T18:34:04.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:03 smithi155 ceph-mon[33468]: pgmap v242: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.188+0000 7fda16259640 1 -- 172.21.15.19:0/4292148260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda10106170 msgr2=0x7fda1010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.188+0000 7fda16259640 1 --2- 172.21.15.19:0/4292148260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda10106170 0x7fda1010dd40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fda08009ca0 tx=0x7fda0802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.188+0000 7fda16259640 1 -- 172.21.15.19:0/4292148260 shutdown_connections 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.188+0000 7fda16259640 1 --2- 172.21.15.19:0/4292148260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda10101aa0 0x7fda10105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.188+0000 7fda16259640 1 --2- 172.21.15.19:0/4292148260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda10106170 0x7fda1010dd40 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.188+0000 7fda16259640 1 --2- 172.21.15.19:0/4292148260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda101010f0 0x7fda101014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.188+0000 7fda16259640 1 -- 172.21.15.19:0/4292148260 >> 172.21.15.19:0/4292148260 conn(0x7fda10077fd0 msgr2=0x7fda100783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.188+0000 7fda16259640 1 -- 172.21.15.19:0/4292148260 shutdown_connections 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.188+0000 7fda16259640 1 -- 172.21.15.19:0/4292148260 wait complete. 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda16259640 1 Processor -- start 2024-04-23T18:34:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda16259640 1 -- start start 2024-04-23T18:34:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda16259640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda101010f0 0x7fda10102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda16259640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda10106170 0x7fda10103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda16259640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda101035b0 0x7fda101a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda16259640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda101110e0 con 0x7fda101035b0 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda16259640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda10110f60 con 0x7fda10106170 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda16259640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda10111260 con 0x7fda101010f0 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda101035b0 0x7fda101a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda101035b0 0x7fda101a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33820/0 (socket says 172.21.15.19:33820) 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda0ffff640 1 -- 172.21.15.19:0/393439220 learned_addr learned my addr 172.21.15.19:0/393439220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda0effd640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda10106170 0x7fda10103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.189+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda101010f0 0x7fda10102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda0ffff640 1 -- 172.21.15.19:0/393439220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda101010f0 msgr2=0x7fda10102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda0ffff640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda101010f0 0x7fda10102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda0ffff640 1 -- 172.21.15.19:0/393439220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda10106170 msgr2=0x7fda10103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda0ffff640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda10106170 0x7fda10103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda0ffff640 1 -- 172.21.15.19:0/393439220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda101a6f00 con 0x7fda101035b0 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda0ffff640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda101035b0 0x7fda101a6890 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fda08005cc0 tx=0x7fda0802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda0cff9640 1 -- 172.21.15.19:0/393439220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08003bf0 con 0x7fda101035b0 2024-04-23T18:34:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda16259640 1 -- 172.21.15.19:0/393439220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda101a7190 con 0x7fda101035b0 2024-04-23T18:34:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda0cff9640 1 -- 172.21.15.19:0/393439220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda08003d90 con 0x7fda101035b0 2024-04-23T18:34:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda0cff9640 1 -- 172.21.15.19:0/393439220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0802fbc0 con 0x7fda101035b0 2024-04-23T18:34:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.190+0000 7fda16259640 1 -- 172.21.15.19:0/393439220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda101a76e0 con 0x7fda101035b0 2024-04-23T18:34:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda0cff9640 1 -- 172.21.15.19:0/393439220 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fda0802fd60 con 0x7fda101035b0 2024-04-23T18:34:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda18f6e740 1 Processor -- start 2024-04-23T18:34:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda0cff9640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9f80a3dc0 0x7fd9f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda18f6e740 1 -- start start 2024-04-23T18:34:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460620 0x55ee92460a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460f40 0x55ee9246b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda0effd640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9f80a3dc0 0x7fd9f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee9246b800 0x55ee9246dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda18f6e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ee923c5970 con 0x55ee92460620 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda18f6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ee923624a0 con 0x55ee92460f40 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda18f6e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ee923c1930 con 0x55ee9246b800 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda0cff9640 1 -- 172.21.15.19:0/393439220 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fda080f1d60 con 0x7fda101035b0 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.193+0000 7fda0f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460620 0x55ee92460a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460620 0x55ee92460a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33836/0 (socket says 172.21.15.19:33836) 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0f7fe640 1 -- 172.21.15.19:0/827331545 learned_addr learned my addr 172.21.15.19:0/827331545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0effd640 1 --2- 172.21.15.19:0/827331545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460f40 0x55ee9246b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0ffff640 1 --2- 172.21.15.19:0/827331545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee9246b800 0x55ee9246dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0f7fe640 1 -- 172.21.15.19:0/827331545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee9246b800 msgr2=0x55ee9246dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/827331545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee9246b800 0x55ee9246dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0f7fe640 1 -- 172.21.15.19:0/827331545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460f40 msgr2=0x55ee9246b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/827331545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460f40 0x55ee9246b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0f7fe640 1 -- 172.21.15.19:0/827331545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee920d8210 con 0x55ee92460620 2024-04-23T18:34:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0effd640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9f80a3dc0 0x7fd9f80a6280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda10104520 tx=0x7fda04014890 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/827331545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460620 0x55ee92460a00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fda0000b760 tx=0x7fda0000bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cbd508a8f2549e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fd9f5ffb640 1 -- 172.21.15.19:0/827331545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00003a70 con 0x55ee92460620 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fd9f5ffb640 1 -- 172.21.15.19:0/827331545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda00015070 con 0x55ee92460620 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.194+0000 7fd9f5ffb640 1 -- 172.21.15.19:0/827331545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00004b60 con 0x55ee92460620 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.195+0000 7fda18f6e740 1 -- 172.21.15.19:0/827331545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460620 msgr2=0x55ee92460a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.195+0000 7fda18f6e740 1 --2- 172.21.15.19:0/827331545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460620 0x55ee92460a00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fda0000b760 tx=0x7fda0000bb60 comp rx=0 tx=0).stop 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.195+0000 7fda18f6e740 1 -- 172.21.15.19:0/827331545 shutdown_connections 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.195+0000 7fda18f6e740 1 --2- 172.21.15.19:0/827331545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460f40 0x55ee9246b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.195+0000 7fda18f6e740 1 --2- 172.21.15.19:0/827331545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee9246b800 0x55ee9246dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.195+0000 7fda18f6e740 1 --2- 172.21.15.19:0/827331545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460620 0x55ee92460a00 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.195+0000 7fda18f6e740 1 -- 172.21.15.19:0/827331545 >> 172.21.15.19:0/827331545 conn(0x55ee9245cb30 msgr2=0x55ee9245cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.195+0000 7fda18f6e740 1 -- 172.21.15.19:0/827331545 shutdown_connections 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.195+0000 7fda18f6e740 1 -- 172.21.15.19:0/827331545 wait complete. 2024-04-23T18:34:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.195+0000 7fda18f6e740 1 -- start start 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda18f6e740 1 Processor -- start 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 0x55ee9246b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee92460f40 0x55ee924683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee9246b800 0x55ee924688f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda18f6e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ee9246e430 con 0x55ee9246b800 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda18f6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ee923600b0 con 0x55ee92460620 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda18f6e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ee9246e5b0 con 0x55ee92460f40 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda18f6e740 10 client.0 _ll_register_callbacks cb 0x55ee9234dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee9246b800 0x55ee924688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee9246b800 0x55ee924688f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33844/0 (socket says 172.21.15.19:33844) 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda0ffff640 1 -- 172.21.15.19:0/3825612799 learned_addr learned my addr 172.21.15.19:0/3825612799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda0effd640 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee92460f40 0x55ee924683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.196+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 0x55ee9246b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fda0ffff640 1 -- 172.21.15.19:0/3825612799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee92460f40 msgr2=0x55ee924683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fda0ffff640 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee92460f40 0x55ee924683b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fda0ffff640 1 -- 172.21.15.19:0/3825612799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 msgr2=0x55ee9246b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fda0ffff640 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 0x55ee9246b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fda0ffff640 1 -- 172.21.15.19:0/3825612799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee921a4620 con 0x55ee9246b800 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 0x55ee9246b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fda0ffff640 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee9246b800 0x55ee924688f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fda080b9af0 tx=0x7fda08004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09e3708a3e89806 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fd9eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fd9eeffd640 1 -- 172.21.15.19:0/3825612799 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda080d6020 con 0x55ee9246b800 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fd9eeffd640 1 -- 172.21.15.19:0/3825612799 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fda080ba480 con 0x55ee9246b800 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fda18f6e740 1 -- 172.21.15.19:0/3825612799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda00015070 con 0x55ee9246b800 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fd9eeffd640 1 -- 172.21.15.19:0/3825612799 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda080bde10 con 0x55ee9246b800 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fda18f6e740 10 client.16701 fetch_fsmap learned FSMap version 27 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fda18f6e740 1 -- 172.21.15.19:0/3825612799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee9246e430 con 0x55ee9246b800 2024-04-23T18:34:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.197+0000 7fd9eeffd640 1 -- 172.21.15.19:0/3825612799 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda00015070 con 0x55ee9246b800 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.198+0000 7fd9eeffd640 1 -- 172.21.15.19:0/3825612799 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fda080c6680 con 0x55ee9246b800 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.198+0000 7fda18f6e740 10 client.16701 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.198+0000 7fda18f6e740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.198+0000 7fda18f6e740 20 client.16701 populate_metadata read hostname 'smithi019' 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.198+0000 7fda18f6e740 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ee9247b5a0 0x55ee9247da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.198+0000 7fda18f6e740 4 client.16701 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.198+0000 7fda18f6e740 1 -- 172.21.15.19:0/3825612799 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee923d3390 con 0x55ee9247b5a0 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.198+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ee9247b5a0 0x55ee9247da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.199+0000 7fd9eeffd640 10 client.16701 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.198+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ee9247b5a0 0x55ee9247da00 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a57a79a181aef3f server_cookie=ccb64006ef75797f in_seq=0 out_seq=0 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.199+0000 7fd9eeffd640 1 -- 172.21.15.19:0/3825612799 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee923d3390 con 0x55ee9247b5a0 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.199+0000 7fd9eeffd640 10 client.16701 handle_command_reply: tid=0 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.199+0000 7fda18f6e740 1 client.16701 shutdown 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.199+0000 7fda18f6e740 1 -- 172.21.15.19:0/3825612799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee9246b800 msgr2=0x55ee924688f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.199+0000 7fda18f6e740 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee9246b800 0x55ee924688f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fda080b9af0 tx=0x7fda08004610 comp rx=0 tx=0).stop 2024-04-23T18:34:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.199+0000 7fda18f6e740 1 -- 172.21.15.19:0/3825612799 shutdown_connections 2024-04-23T18:34:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.199+0000 7fda18f6e740 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ee9247b5a0 0x55ee9247da00 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.200+0000 7fd9eeffd640 0 client.16701 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.199+0000 7fda18f6e740 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee9246b800 0x55ee924688f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.200+0000 7fda18f6e740 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee92460f40 0x55ee924683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.200+0000 7fda18f6e740 1 --2- 172.21.15.19:0/3825612799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 0x55ee9246b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.200+0000 7fda18f6e740 1 -- 172.21.15.19:0/3825612799 >> 172.21.15.19:0/3825612799 conn(0x55ee9245cb30 msgr2=0x55ee924628b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.200+0000 7fda18f6e740 1 -- 172.21.15.19:0/3825612799 shutdown_connections 2024-04-23T18:34:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.200+0000 7fda18f6e740 1 -- 172.21.15.19:0/3825612799 wait complete. 2024-04-23T18:34:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.200+0000 7fda18f6e740 20 client.16701 trim_cache size 0 max 16384 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda18f6e740 1 Processor -- start 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda18f6e740 1 -- start start 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee9245cb30 0x55ee924772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 0x55ee92483310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92483850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda18f6e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda101afbc0 con 0x55ee92460f40 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda18f6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda101b1700 con 0x55ee92460620 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda18f6e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda101b0290 con 0x55ee9245cb30 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92483850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92483850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33856/0 (socket says 172.21.15.19:33856) 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda0ffff640 1 -- 172.21.15.19:0/2202855802 learned_addr learned my addr 172.21.15.19:0/2202855802 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/2202855802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee9245cb30 0x55ee924772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda0effd640 1 --2- 172.21.15.19:0/2202855802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 0x55ee92483310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda0ffff640 1 -- 172.21.15.19:0/2202855802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee9245cb30 msgr2=0x55ee924772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda0ffff640 1 --2- 172.21.15.19:0/2202855802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee9245cb30 0x55ee924772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda0ffff640 1 -- 172.21.15.19:0/2202855802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 msgr2=0x55ee92483310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda0ffff640 1 --2- 172.21.15.19:0/2202855802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 0x55ee92483310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.215+0000 7fda0ffff640 1 -- 172.21.15.19:0/2202855802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee92460d70 con 0x55ee92460f40 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fda0ffff640 1 --2- 172.21.15.19:0/2202855802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92483850 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fda08031a90 tx=0x7fda0802f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fa6e3556ccb754 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fd9f57fa640 1 -- 172.21.15.19:0/2202855802 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda080d6070 con 0x55ee92460f40 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fd9f57fa640 1 -- 172.21.15.19:0/2202855802 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda08036040 con 0x55ee92460f40 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fd9f57fa640 1 -- 172.21.15.19:0/2202855802 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda080c65f0 con 0x55ee92460f40 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fda18f6e740 1 -- 172.21.15.19:0/2202855802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 msgr2=0x55ee92483850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fda18f6e740 1 --2- 172.21.15.19:0/2202855802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92483850 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fda08031a90 tx=0x7fda0802f660 comp rx=0 tx=0).stop 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fda18f6e740 1 -- 172.21.15.19:0/2202855802 shutdown_connections 2024-04-23T18:34:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fda18f6e740 1 --2- 172.21.15.19:0/2202855802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee92460620 0x55ee92483310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fda18f6e740 1 --2- 172.21.15.19:0/2202855802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92483850 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fda18f6e740 1 --2- 172.21.15.19:0/2202855802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee9245cb30 0x55ee924772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fda18f6e740 1 -- 172.21.15.19:0/2202855802 >> 172.21.15.19:0/2202855802 conn(0x55ee92468a50 msgr2=0x55ee9245b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fda18f6e740 1 -- 172.21.15.19:0/2202855802 shutdown_connections 2024-04-23T18:34:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.216+0000 7fda18f6e740 1 -- 172.21.15.19:0/2202855802 wait complete. 2024-04-23T18:34:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda18f6e740 1 -- start start 2024-04-23T18:34:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda18f6e740 1 Processor -- start 2024-04-23T18:34:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee9245cb30 0x55ee92473cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee92460620 0x55ee92471080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda18f6e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92471690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda18f6e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda101b1030 con 0x55ee92460f40 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda18f6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda101b0960 con 0x55ee9245cb30 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda18f6e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda101adbc0 con 0x55ee92460620 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda18f6e740 10 client.0 _ll_register_callbacks cb 0x7fda080bda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92471690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92471690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33868/0 (socket says 172.21.15.19:33868) 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda0ffff640 1 -- 172.21.15.19:0/1830311071 learned_addr learned my addr 172.21.15.19:0/1830311071 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda0effd640 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee92460620 0x55ee92471080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.217+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee9245cb30 0x55ee92473cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fda0ffff640 1 -- 172.21.15.19:0/1830311071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee92460620 msgr2=0x55ee92471080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fda0ffff640 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee92460620 0x55ee92471080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fda0ffff640 1 -- 172.21.15.19:0/1830311071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee9245cb30 msgr2=0x55ee92473cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fda0ffff640 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee9245cb30 0x55ee92473cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fda0ffff640 1 -- 172.21.15.19:0/1830311071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee92471ec0 con 0x55ee92460f40 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee9245cb30 0x55ee92473cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fda0ffff640 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92471690 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fda080efab0 tx=0x7fda080ef9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1fe87dab351fae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fd9eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fd9eeffd640 1 -- 172.21.15.19:0/1830311071 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda080d6030 con 0x55ee92460f40 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fd9eeffd640 1 -- 172.21.15.19:0/1830311071 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7fda080a70b0 con 0x55ee92460f40 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fda18f6e740 1 -- 172.21.15.19:0/1830311071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda08036040 con 0x55ee92460f40 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fd9eeffd640 1 -- 172.21.15.19:0/1830311071 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda080c6800 con 0x55ee92460f40 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fda18f6e740 10 client.16707 fetch_fsmap learned FSMap version 27 2024-04-23T18:34:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fda18f6e740 1 -- 172.21.15.19:0/1830311071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee92471ec0 con 0x55ee92460f40 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.218+0000 7fd9eeffd640 1 -- 172.21.15.19:0/1830311071 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda08036040 con 0x55ee92460f40 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.219+0000 7fd9eeffd640 1 -- 172.21.15.19:0/1830311071 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fda080a3070 con 0x55ee92460f40 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.219+0000 7fda18f6e740 10 client.16707 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.219+0000 7fda18f6e740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.219+0000 7fda18f6e740 20 client.16707 populate_metadata read hostname 'smithi019' 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.219+0000 7fda18f6e740 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ee9246b800 0x55ee924e3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.219+0000 7fda18f6e740 4 client.16707 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.219+0000 7fda18f6e740 1 -- 172.21.15.19:0/1830311071 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee923cd200 con 0x55ee9246b800 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.219+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ee9246b800 0x55ee924e3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.219+0000 7fd9eeffd640 10 client.16707 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.219+0000 7fda0f7fe640 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ee9246b800 0x55ee924e3f90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21e93b5e2703ca1 server_cookie=dcc6cb2f0b733b89 in_seq=0 out_seq=0 2024-04-23T18:34:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.231+0000 7fd9eeffd640 1 -- 172.21.15.19:0/1830311071 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee923cd200 con 0x55ee9246b800 2024-04-23T18:34:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.231+0000 7fd9eeffd640 10 client.16707 handle_command_reply: tid=0 2024-04-23T18:34:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 client.16707 shutdown 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 -- 172.21.15.19:0/1830311071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 msgr2=0x55ee92471690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92471690 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fda080efab0 tx=0x7fda080ef9b0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 -- 172.21.15.19:0/1830311071 shutdown_connections 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ee9246b800 0x55ee924e3f90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee92460f40 0x55ee92471690 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee92460620 0x55ee92471080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fd9eeffd640 0 client.16707 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 --2- 172.21.15.19:0/1830311071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee9245cb30 0x55ee92473cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 -- 172.21.15.19:0/1830311071 >> 172.21.15.19:0/1830311071 conn(0x55ee92468a50 msgr2=0x55ee9245bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 -- 172.21.15.19:0/1830311071 shutdown_connections 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 1 -- 172.21.15.19:0/1830311071 wait complete. 2024-04-23T18:34:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.232+0000 7fda18f6e740 20 client.16707 trim_cache size 0 max 16384 2024-04-23T18:34:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:34:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:34:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:34:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 -- 172.21.15.19:0/393439220 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9f80a3dc0 msgr2=0x7fd9f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9f80a3dc0 0x7fd9f80a6280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda10104520 tx=0x7fda04014890 comp rx=0 tx=0).stop 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 -- 172.21.15.19:0/393439220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda101035b0 msgr2=0x7fda101a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda101035b0 0x7fda101a6890 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fda08005cc0 tx=0x7fda0802f380 comp rx=0 tx=0).stop 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 -- 172.21.15.19:0/393439220 shutdown_connections 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9f80a3dc0 0x7fd9f80a6280 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda10106170 0x7fda10103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda101035b0 0x7fda101a6890 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 --2- 172.21.15.19:0/393439220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda101010f0 0x7fda10102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 -- 172.21.15.19:0/393439220 >> 172.21.15.19:0/393439220 conn(0x7fda10077fd0 msgr2=0x7fda100698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 -- 172.21.15.19:0/393439220 shutdown_connections 2024-04-23T18:34:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:04.233+0000 7fda16259640 1 -- 172.21.15.19:0/393439220 wait complete. 2024-04-23T18:34:04.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:03 smithi019 ceph-mon[29215]: pgmap v242: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:34:04.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:03 smithi044 ceph-mon[32458]: pgmap v242: 97 pgs: 97 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:34:06.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:05 smithi155 ceph-mon[33468]: pgmap v243: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 35 op/s 2024-04-23T18:34:06.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:05 smithi019 ceph-mon[29215]: pgmap v243: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 35 op/s 2024-04-23T18:34:06.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:05 smithi044 ceph-mon[32458]: pgmap v243: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 35 op/s 2024-04-23T18:34:07.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:06 smithi019 ceph-mon[29215]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:34:07.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:06 smithi044 ceph-mon[32458]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:34:07.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:06 smithi155 ceph-mon[33468]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:34:08.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:07 smithi019 ceph-mon[29215]: pgmap v245: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 96 KiB/s wr, 36 op/s 2024-04-23T18:34:08.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:07 smithi044 ceph-mon[32458]: pgmap v245: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 96 KiB/s wr, 36 op/s 2024-04-23T18:34:08.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:07 smithi155 ceph-mon[33468]: pgmap v245: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 96 KiB/s wr, 36 op/s 2024-04-23T18:34:10.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:09 smithi019 ceph-mon[29215]: pgmap v246: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 96 KiB/s wr, 36 op/s 2024-04-23T18:34:10.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:09 smithi044 ceph-mon[32458]: pgmap v246: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 96 KiB/s wr, 36 op/s 2024-04-23T18:34:10.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:09 smithi155 ceph-mon[33468]: pgmap v246: 97 pgs: 97 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 96 KiB/s wr, 36 op/s 2024-04-23T18:34:12.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:11 smithi019 ceph-mon[29215]: pgmap v247: 97 pgs: 97 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 39 op/s 2024-04-23T18:34:12.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:11 smithi044 ceph-mon[32458]: pgmap v247: 97 pgs: 97 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 39 op/s 2024-04-23T18:34:12.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:11 smithi155 ceph-mon[33468]: pgmap v247: 97 pgs: 97 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 39 op/s 2024-04-23T18:34:14.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:13 smithi019 ceph-mon[29215]: pgmap v248: 97 pgs: 97 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 39 op/s 2024-04-23T18:34:14.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:13 smithi019 ceph-mon[29215]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:34:14.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:13 smithi044 ceph-mon[32458]: pgmap v248: 97 pgs: 97 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 39 op/s 2024-04-23T18:34:14.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:13 smithi044 ceph-mon[32458]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:34:14.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:13 smithi155 ceph-mon[33468]: pgmap v248: 97 pgs: 97 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 39 op/s 2024-04-23T18:34:14.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:13 smithi155 ceph-mon[33468]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:34:16.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:15 smithi019 ceph-mon[29215]: pgmap v250: 97 pgs: 97 active+clean; 1.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:34:16.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:15 smithi044 ceph-mon[32458]: pgmap v250: 97 pgs: 97 active+clean; 1.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:34:16.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:15 smithi155 ceph-mon[33468]: pgmap v250: 97 pgs: 97 active+clean; 1.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:34:17.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.331+0000 7f2ec6c05640 1 -- 172.21.15.19:0/1596933076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ec0101a90 msgr2=0x7f2ec0105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.331+0000 7f2ec6c05640 1 --2- 172.21.15.19:0/1596933076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ec0101a90 0x7f2ec0105ad0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009900 tx=0x7f2ea802a410 comp rx=0 tx=0).stop 2024-04-23T18:34:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.331+0000 7f2ec6c05640 1 -- 172.21.15.19:0/1596933076 shutdown_connections 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.331+0000 7f2ec6c05640 1 --2- 172.21.15.19:0/1596933076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ec01061b0 0x7f2ec010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.331+0000 7f2ec6c05640 1 --2- 172.21.15.19:0/1596933076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ec0101a90 0x7f2ec0105ad0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.331+0000 7f2ec6c05640 1 --2- 172.21.15.19:0/1596933076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ec01010e0 0x7f2ec01014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.331+0000 7f2ec6c05640 1 -- 172.21.15.19:0/1596933076 >> 172.21.15.19:0/1596933076 conn(0x7f2ec00771d0 msgr2=0x7f2ec00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.331+0000 7f2ec6c05640 1 -- 172.21.15.19:0/1596933076 shutdown_connections 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.331+0000 7f2ec6c05640 1 -- 172.21.15.19:0/1596933076 wait complete. 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec6c05640 1 Processor -- start 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec6c05640 1 -- start start 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec6c05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ec01010e0 0x7f2ec0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec6c05640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ec01061b0 0x7f2ec0198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec6c05640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ec019cec0 0x7f2ec019d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec6c05640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01110d0 con 0x7f2ec01010e0 2024-04-23T18:34:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec6c05640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0110f50 con 0x7f2ec01061b0 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec6c05640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0111250 con 0x7f2ec019cec0 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec49a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ec01010e0 0x7f2ec0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec49a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ec01010e0 0x7f2ec0198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33558/0 (socket says 172.21.15.19:33558) 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec49a1640 1 -- 172.21.15.19:0/221678173 learned_addr learned my addr 172.21.15.19:0/221678173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ec51a2640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ec019cec0 0x7f2ec019d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.332+0000 7f2ebffff640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ec01061b0 0x7f2ec0198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ec49a1640 1 -- 172.21.15.19:0/221678173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ec019cec0 msgr2=0x7f2ec019d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ec49a1640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ec019cec0 0x7f2ec019d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ec49a1640 1 -- 172.21.15.19:0/221678173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ec01061b0 msgr2=0x7f2ec0198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ec49a1640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ec01061b0 0x7f2ec0198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ec49a1640 1 -- 172.21.15.19:0/221678173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec019d9b0 con 0x7f2ec01010e0 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ebffff640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ec01061b0 0x7f2ec0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ec49a1640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ec01010e0 0x7f2ec0198600 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000c890 tx=0x7f2eb000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ebdffb640 1 -- 172.21.15.19:0/221678173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0004220 con 0x7f2ec01010e0 2024-04-23T18:34:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ebdffb640 1 -- 172.21.15.19:0/221678173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2eb00043c0 con 0x7f2ec01010e0 2024-04-23T18:34:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ec6c05640 1 -- 172.21.15.19:0/221678173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec019dca0 con 0x7f2ec01010e0 2024-04-23T18:34:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ebdffb640 1 -- 172.21.15.19:0/221678173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0004a00 con 0x7f2ec01010e0 2024-04-23T18:34:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.333+0000 7f2ec6c05640 1 -- 172.21.15.19:0/221678173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec01a5460 con 0x7f2ec01010e0 2024-04-23T18:34:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.336+0000 7f2ebdffb640 1 -- 172.21.15.19:0/221678173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2eb00079b0 con 0x7f2ec01010e0 2024-04-23T18:34:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.336+0000 7f2ec99fb740 1 Processor -- start 2024-04-23T18:34:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.336+0000 7f2ec99fb740 1 -- start start 2024-04-23T18:34:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.336+0000 7f2ebdffb640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2eac0a3dc0 0x7f2eac0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.336+0000 7f2ebdffb640 1 -- 172.21.15.19:0/221678173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f2eb00d0550 con 0x7f2ec01010e0 2024-04-23T18:34:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.336+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98ca30 0x55745d98ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.336+0000 7f2ec51a2640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2eac0a3dc0 0x7f2eac0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.336+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d9976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d99a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55745d8ed970 con 0x55745d98d350 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55745d88a4a0 con 0x55745d98ca30 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55745d8e9930 con 0x55745d997c10 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d9976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ec49a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98ca30 0x55745d98ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d9976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33562/0 (socket says 172.21.15.19:33562) 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ebffff640 1 -- 172.21.15.19:0/3624485906 learned_addr learned my addr 172.21.15.19:0/3624485906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ec51a2640 1 --2- 172.21.15.19:0/3624485906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d99a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ebffff640 1 -- 172.21.15.19:0/3624485906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 msgr2=0x55745d99a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3624485906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d99a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ebffff640 1 -- 172.21.15.19:0/3624485906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98ca30 msgr2=0x55745d98ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3624485906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98ca30 0x55745d98ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ebffff640 1 -- 172.21.15.19:0/3624485906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745d600210 con 0x55745d98d350 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ec51a2640 1 --2- 172.21.15.19:0/3624485906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d99a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ec51a2640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2eac0a3dc0 0x7f2eac0a6280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8007de0 tx=0x7f2eb800f790 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3624485906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d9976d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009900 tx=0x7f2ea8031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ff2bbb5b2700a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2eb6ffd640 1 -- 172.21.15.19:0/3624485906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea803f070 con 0x55745d98d350 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2eb6ffd640 1 -- 172.21.15.19:0/3624485906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ea80316a0 con 0x55745d98d350 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.337+0000 7f2eb6ffd640 1 -- 172.21.15.19:0/3624485906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8035070 con 0x55745d98d350 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.338+0000 7f2ec99fb740 1 -- 172.21.15.19:0/3624485906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 msgr2=0x55745d9976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.338+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/3624485906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d9976d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009900 tx=0x7f2ea8031e60 comp rx=0 tx=0).stop 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.338+0000 7f2ec99fb740 1 -- 172.21.15.19:0/3624485906 shutdown_connections 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.338+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/3624485906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d99a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.338+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/3624485906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d9976d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.338+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/3624485906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98ca30 0x55745d98ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.338+0000 7f2ec99fb740 1 -- 172.21.15.19:0/3624485906 >> 172.21.15.19:0/3624485906 conn(0x55745d988f40 msgr2=0x55745d989320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.338+0000 7f2ec99fb740 1 -- 172.21.15.19:0/3624485906 shutdown_connections 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.338+0000 7f2ec99fb740 1 -- 172.21.15.19:0/3624485906 wait complete. 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.338+0000 7f2ec99fb740 1 -- start start 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec99fb740 1 Processor -- start 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d997580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d98d350 0x55745d994870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d997c10 0x55745d994dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55745d99a7e0 con 0x55745d98ca30 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55745d8880b0 con 0x55745d997c10 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55745d99a960 con 0x55745d98d350 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec99fb740 10 client.0 _ll_register_callbacks cb 0x55745d875d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec49a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d997580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec49a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d997580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33578/0 (socket says 172.21.15.19:33578) 2024-04-23T18:34:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec49a1640 1 -- 172.21.15.19:0/560181813 learned_addr learned my addr 172.21.15.19:0/560181813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ebffff640 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d98d350 0x55745d994870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec51a2640 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d997c10 0x55745d994dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec49a1640 1 -- 172.21.15.19:0/560181813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d98d350 msgr2=0x55745d994870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.339+0000 7f2ec49a1640 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d98d350 0x55745d994870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec49a1640 1 -- 172.21.15.19:0/560181813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d997c10 msgr2=0x55745d994dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec49a1640 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d997c10 0x55745d994dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec49a1640 1 -- 172.21.15.19:0/560181813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745d6cc620 con 0x55745d98ca30 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec51a2640 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d997c10 0x55745d994dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec49a1640 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d997580 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2eb00cdd00 tx=0x7f2eb0005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a657f64198736b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2e9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2e9ffff640 1 -- 172.21.15.19:0/560181813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb001d040 con 0x55745d98ca30 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2e9ffff640 1 -- 172.21.15.19:0/560181813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f2eb0098b60 con 0x55745d98ca30 2024-04-23T18:34:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec99fb740 1 -- 172.21.15.19:0/560181813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ea80316a0 con 0x55745d98ca30 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2e9ffff640 1 -- 172.21.15.19:0/560181813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2eb00a08a0 con 0x55745d98ca30 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec99fb740 10 client.16719 fetch_fsmap learned FSMap version 27 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec99fb740 1 -- 172.21.15.19:0/560181813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55745d9a6710 con 0x55745d98ca30 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2e9ffff640 1 -- 172.21.15.19:0/560181813 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ea80316a0 con 0x55745d98ca30 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2e9ffff640 1 -- 172.21.15.19:0/560181813 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2eb009b940 con 0x55745d98ca30 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec99fb740 10 client.16719 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec99fb740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec99fb740 20 client.16719 populate_metadata read hostname 'smithi019' 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55745d9a7b80 0x55745d9a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec99fb740 4 client.16719 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.340+0000 7f2ec99fb740 1 -- 172.21.15.19:0/560181813 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55745d8fb390 con 0x55745d9a7b80 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.341+0000 7f2ebffff640 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55745d9a7b80 0x55745d9a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.341+0000 7f2ebffff640 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55745d9a7b80 0x55745d9a9f70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8a4744ec719af5 server_cookie=9e0715cf4e37ea1f in_seq=0 out_seq=0 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.341+0000 7f2e9ffff640 10 client.16719 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2e9ffff640 1 -- 172.21.15.19:0/560181813 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55745d8fb390 con 0x55745d9a7b80 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2e9ffff640 10 client.16719 handle_command_reply: tid=0 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 client.16719 shutdown 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 -- 172.21.15.19:0/560181813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 msgr2=0x55745d997580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d997580 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2eb00cdd00 tx=0x7f2eb0005290 comp rx=0 tx=0).stop 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 -- 172.21.15.19:0/560181813 shutdown_connections 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55745d9a7b80 0x55745d9a9f70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d997c10 0x55745d994dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d98d350 0x55745d994870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/560181813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d997580 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2e9ffff640 0 client.16719 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 -- 172.21.15.19:0/560181813 >> 172.21.15.19:0/560181813 conn(0x55745d988f40 msgr2=0x55745d98ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 -- 172.21.15.19:0/560181813 shutdown_connections 2024-04-23T18:34:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 1 -- 172.21.15.19:0/560181813 wait complete. 2024-04-23T18:34:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.342+0000 7f2ec99fb740 20 client.16719 trim_cache size 0 max 16384 2024-04-23T18:34:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.357+0000 7f2ec99fb740 1 Processor -- start 2024-04-23T18:34:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.357+0000 7f2ec99fb740 1 -- start start 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98ca30 0x55745d9a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d97fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d9af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01ab780 con 0x55745d98d350 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01ad2c0 con 0x55745d98ca30 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01abe50 con 0x55745d997c10 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d97fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d97fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33580/0 (socket says 172.21.15.19:33580) 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ebffff640 1 -- 172.21.15.19:0/3267688948 learned_addr learned my addr 172.21.15.19:0/3267688948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ec49a1640 1 --2- 172.21.15.19:0/3267688948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98ca30 0x55745d9a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ec51a2640 1 --2- 172.21.15.19:0/3267688948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d9af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ebffff640 1 -- 172.21.15.19:0/3267688948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 msgr2=0x55745d9af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3267688948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d9af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ebffff640 1 -- 172.21.15.19:0/3267688948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98ca30 msgr2=0x55745d9a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3267688948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98ca30 0x55745d9a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ebffff640 1 -- 172.21.15.19:0/3267688948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745d98d180 con 0x55745d98d350 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3267688948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d97fc30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8031440 tx=0x7f2ea8031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7255fa960c16eee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2eb67fc640 1 -- 172.21.15.19:0/3267688948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea803f040 con 0x55745d98d350 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2eb67fc640 1 -- 172.21.15.19:0/3267688948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ea804d900 con 0x55745d98d350 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.358+0000 7f2eb67fc640 1 -- 172.21.15.19:0/3267688948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8035040 con 0x55745d98d350 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 -- 172.21.15.19:0/3267688948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 msgr2=0x55745d97fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/3267688948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d97fc30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8031440 tx=0x7f2ea8031de0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 -- 172.21.15.19:0/3267688948 shutdown_connections 2024-04-23T18:34:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/3267688948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d9af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/3267688948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98d350 0x55745d97fc30 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/3267688948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98ca30 0x55745d9a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 -- 172.21.15.19:0/3267688948 >> 172.21.15.19:0/3267688948 conn(0x55745d988f40 msgr2=0x55745d983cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 -- 172.21.15.19:0/3267688948 shutdown_connections 2024-04-23T18:34:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 -- 172.21.15.19:0/3267688948 wait complete. 2024-04-23T18:34:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 -- start start 2024-04-23T18:34:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.359+0000 7f2ec99fb740 1 Processor -- start 2024-04-23T18:34:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d99ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98d350 0x55745d99d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec99fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d99d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01acbf0 con 0x55745d98ca30 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01ac520 con 0x55745d98d350 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec99fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01a97a0 con 0x55745d997c10 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec99fb740 10 client.0 _ll_register_callbacks cb 0x7f2eb009c300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec49a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d99ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec49a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d99ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33582/0 (socket says 172.21.15.19:33582) 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec49a1640 1 -- 172.21.15.19:0/4027559487 learned_addr learned my addr 172.21.15.19:0/4027559487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec51a2640 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d99d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ebffff640 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98d350 0x55745d99d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec49a1640 1 -- 172.21.15.19:0/4027559487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 msgr2=0x55745d99d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec49a1640 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d99d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec49a1640 1 -- 172.21.15.19:0/4027559487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98d350 msgr2=0x55745d99d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec49a1640 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98d350 0x55745d99d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ec49a1640 1 -- 172.21.15.19:0/4027559487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745d99e0c0 con 0x55745d98ca30 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.360+0000 7f2ebffff640 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98d350 0x55745d99d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2ec49a1640 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d99ffc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2eb00cdd50 tx=0x7f2eb00d02e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b786f3f11d9f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2e9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2e9ffff640 1 -- 172.21.15.19:0/4027559487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb001d070 con 0x55745d98ca30 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2e9ffff640 1 -- 172.21.15.19:0/4027559487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f2eb00983e0 con 0x55745d98ca30 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2ec99fb740 1 -- 172.21.15.19:0/4027559487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ea804d900 con 0x55745d98ca30 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2e9ffff640 1 -- 172.21.15.19:0/4027559487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2eb0098790 con 0x55745d98ca30 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2ec99fb740 10 client.16725 fetch_fsmap learned FSMap version 27 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2ec99fb740 1 -- 172.21.15.19:0/4027559487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55745da0e0b0 con 0x55745d98ca30 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2e9ffff640 1 -- 172.21.15.19:0/4027559487 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ea804d900 con 0x55745d98ca30 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2e9ffff640 1 -- 172.21.15.19:0/4027559487 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2eb009b440 con 0x55745d98ca30 2024-04-23T18:34:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2ec99fb740 10 client.16725 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:34:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2ec99fb740 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2ec99fb740 20 client.16725 populate_metadata read hostname 'smithi019' 2024-04-23T18:34:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55745da0e820 0x55745da10c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2ec99fb740 4 client.16725 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.361+0000 7f2ec99fb740 1 -- 172.21.15.19:0/4027559487 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55745d8f5200 con 0x55745da0e820 2024-04-23T18:34:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.362+0000 7f2ebffff640 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55745da0e820 0x55745da10c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.362+0000 7f2ebffff640 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55745da0e820 0x55745da10c30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a7ee828f9b27f2 server_cookie=a7e857bb6e11081f in_seq=0 out_seq=0 2024-04-23T18:34:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.362+0000 7f2e9ffff640 10 client.16725 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2e9ffff640 1 -- 172.21.15.19:0/4027559487 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55745d8f5200 con 0x55745da0e820 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2e9ffff640 10 client.16725 handle_command_reply: tid=0 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 client.16725 shutdown 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 -- 172.21.15.19:0/4027559487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 msgr2=0x55745d99ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d99ffc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2eb00cdd50 tx=0x7f2eb00d02e0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 -- 172.21.15.19:0/4027559487 shutdown_connections 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55745da0e820 0x55745da10c30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55745d98d350 0x55745d99d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745d997c10 0x55745d99d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2e9ffff640 0 client.16725 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 --2- 172.21.15.19:0/4027559487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55745d98ca30 0x55745d99ffc0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 -- 172.21.15.19:0/4027559487 >> 172.21.15.19:0/4027559487 conn(0x55745d988f40 msgr2=0x55745d983cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 -- 172.21.15.19:0/4027559487 shutdown_connections 2024-04-23T18:34:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 1 -- 172.21.15.19:0/4027559487 wait complete. 2024-04-23T18:34:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.377+0000 7f2ec99fb740 20 client.16725 trim_cache size 0 max 16384 2024-04-23T18:34:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:34:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:34:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:34:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:34:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 -- 172.21.15.19:0/221678173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2eac0a3dc0 msgr2=0x7f2eac0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2eac0a3dc0 0x7f2eac0a6280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8007de0 tx=0x7f2eb800f790 comp rx=0 tx=0).stop 2024-04-23T18:34:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 -- 172.21.15.19:0/221678173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ec01010e0 msgr2=0x7f2ec0198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ec01010e0 0x7f2ec0198600 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000c890 tx=0x7f2eb000cd50 comp rx=0 tx=0).stop 2024-04-23T18:34:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 -- 172.21.15.19:0/221678173 shutdown_connections 2024-04-23T18:34:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f2eac0a3dc0 0x7f2eac0a6280 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ec01061b0 0x7f2ec0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ec019cec0 0x7f2ec019d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 --2- 172.21.15.19:0/221678173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ec01010e0 0x7f2ec0198600 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 -- 172.21.15.19:0/221678173 >> 172.21.15.19:0/221678173 conn(0x7f2ec00771d0 msgr2=0x7f2ec006aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 -- 172.21.15.19:0/221678173 shutdown_connections 2024-04-23T18:34:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:17.378+0000 7f2ec6c05640 1 -- 172.21.15.19:0/221678173 wait complete. 2024-04-23T18:34:18.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:17 smithi044 ceph-mon[32458]: pgmap v251: 97 pgs: 97 active+clean; 1.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:34:18.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:17 smithi155 ceph-mon[33468]: pgmap v251: 97 pgs: 97 active+clean; 1.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:34:18.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:17 smithi019 ceph-mon[29215]: pgmap v251: 97 pgs: 97 active+clean; 1.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:34:20.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:19 smithi044 ceph-mon[32458]: pgmap v252: 97 pgs: 97 active+clean; 1.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:34:20.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:19 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:20.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:19 smithi155 ceph-mon[33468]: pgmap v252: 97 pgs: 97 active+clean; 1.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:34:20.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:19 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:20.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:19 smithi019 ceph-mon[29215]: pgmap v252: 97 pgs: 97 active+clean; 1.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:34:20.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:19 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:21.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:20 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:34:21.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:20 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:34:21.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:20 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:34:21.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:20 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:34:21.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:20 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:34:21.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:20 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:34:21.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:20 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:34:21.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:20 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:34:21.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:20 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:34:22.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:21 smithi044 ceph-mon[32458]: pgmap v253: 97 pgs: 97 active+clean; 2.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-23T18:34:22.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:21 smithi155 ceph-mon[33468]: pgmap v253: 97 pgs: 97 active+clean; 2.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-23T18:34:22.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:21 smithi019 ceph-mon[29215]: pgmap v253: 97 pgs: 97 active+clean; 2.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-23T18:34:24.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:23 smithi044 ceph-mon[32458]: pgmap v254: 97 pgs: 97 active+clean; 2.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-23T18:34:24.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:23 smithi155 ceph-mon[33468]: pgmap v254: 97 pgs: 97 active+clean; 2.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-23T18:34:24.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:23 smithi019 ceph-mon[29215]: pgmap v254: 97 pgs: 97 active+clean; 2.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-23T18:34:26.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:25 smithi044 ceph-mon[32458]: pgmap v255: 97 pgs: 97 active+clean; 2.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-23T18:34:26.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:25 smithi155 ceph-mon[33468]: pgmap v255: 97 pgs: 97 active+clean; 2.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-23T18:34:26.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:25 smithi019 ceph-mon[29215]: pgmap v255: 97 pgs: 97 active+clean; 2.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-23T18:34:28.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:27 smithi044 ceph-mon[32458]: pgmap v256: 97 pgs: 97 active+clean; 2.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:28.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:27 smithi155 ceph-mon[33468]: pgmap v256: 97 pgs: 97 active+clean; 2.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:28.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:27 smithi019 ceph-mon[29215]: pgmap v256: 97 pgs: 97 active+clean; 2.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:30.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:29 smithi044 ceph-mon[32458]: pgmap v257: 97 pgs: 97 active+clean; 2.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:30.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:29 smithi155 ceph-mon[33468]: pgmap v257: 97 pgs: 97 active+clean; 2.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:30.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:29 smithi019 ceph-mon[29215]: pgmap v257: 97 pgs: 97 active+clean; 2.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.479+0000 7fe414e69640 1 -- 172.21.15.19:0/3323834918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe410102f00 msgr2=0x7fe41006a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.479+0000 7fe414e69640 1 --2- 172.21.15.19:0/3323834918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe410102f00 0x7fe41006a780 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe400009900 tx=0x7fe40002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.479+0000 7fe414e69640 1 -- 172.21.15.19:0/3323834918 shutdown_connections 2024-04-23T18:34:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.479+0000 7fe414e69640 1 --2- 172.21.15.19:0/3323834918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe41010e8a0 0x7fe410110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.479+0000 7fe414e69640 1 --2- 172.21.15.19:0/3323834918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe41006acc0 0x7fe41006b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.479+0000 7fe414e69640 1 --2- 172.21.15.19:0/3323834918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe410102f00 0x7fe41006a780 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.479+0000 7fe414e69640 1 -- 172.21.15.19:0/3323834918 >> 172.21.15.19:0/3323834918 conn(0x7fe410076c50 msgr2=0x7fe410077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.479+0000 7fe414e69640 1 -- 172.21.15.19:0/3323834918 shutdown_connections 2024-04-23T18:34:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.479+0000 7fe414e69640 1 -- 172.21.15.19:0/3323834918 wait complete. 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe414e69640 1 Processor -- start 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe414e69640 1 -- start start 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe414e69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe41006acc0 0x7fe4101a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe414e69640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe410102f00 0x7fe4101a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe414e69640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe41010e8a0 0x7fe41019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe414e69640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe410114490 con 0x7fe41010e8a0 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe414e69640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe410114310 con 0x7fe41006acc0 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe414e69640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe410114610 con 0x7fe410102f00 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe40ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe41010e8a0 0x7fe41019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe40ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe41010e8a0 0x7fe41019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46756/0 (socket says 172.21.15.19:46756) 2024-04-23T18:34:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe40ed9d640 1 -- 172.21.15.19:0/249259640 learned_addr learned my addr 172.21.15.19:0/249259640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe410102f00 0x7fe4101a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe40e59c640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe41006acc0 0x7fe4101a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe40ed9d640 1 -- 172.21.15.19:0/249259640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe410102f00 msgr2=0x7fe4101a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe40ed9d640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe410102f00 0x7fe4101a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe40ed9d640 1 -- 172.21.15.19:0/249259640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe41006acc0 msgr2=0x7fe4101a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.480+0000 7fe40ed9d640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe41006acc0 0x7fe4101a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.481+0000 7fe40ed9d640 1 -- 172.21.15.19:0/249259640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41019b030 con 0x7fe41010e8a0 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.481+0000 7fe40e59c640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe41006acc0 0x7fe4101a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.481+0000 7fe40ed9d640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe41010e8a0 0x7fe41019a770 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe40800cad0 tx=0x7fe40800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.481+0000 7fe3ff7fe640 1 -- 172.21.15.19:0/249259640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40800ae20 con 0x7fe41010e8a0 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.481+0000 7fe3ff7fe640 1 -- 172.21.15.19:0/249259640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe408005020 con 0x7fe41010e8a0 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.481+0000 7fe414e69640 1 -- 172.21.15.19:0/249259640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41019b320 con 0x7fe41010e8a0 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.481+0000 7fe3ff7fe640 1 -- 172.21.15.19:0/249259640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40800f6b0 con 0x7fe41010e8a0 2024-04-23T18:34:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.481+0000 7fe414e69640 1 -- 172.21.15.19:0/249259640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4101a7630 con 0x7fe41010e8a0 2024-04-23T18:34:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.483+0000 7fe3ff7fe640 1 -- 172.21.15.19:0/249259640 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4080051d0 con 0x7fe41010e8a0 2024-04-23T18:34:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe417b76740 1 Processor -- start 2024-04-23T18:34:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe417b76740 1 -- start start 2024-04-23T18:34:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe417b76740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc84a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe417b76740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84f40 0x55e47bc8f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe417b76740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc91bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe3ff7fe640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3f80a3dc0 0x7fe3f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe417b76740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e47bbe9970 con 0x55e47bc84620 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe417b76740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e47bb864a0 con 0x55e47bc84f40 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe417b76740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e47bbe5930 con 0x55e47bc8f800 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe40dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc84a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe40dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc84a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46764/0 (socket says 172.21.15.19:46764) 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe40dd9b640 1 -- 172.21.15.19:0/2701786601 learned_addr learned my addr 172.21.15.19:0/2701786601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe40ed9d640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3f80a3dc0 0x7fe3f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.484+0000 7fe3ff7fe640 1 -- 172.21.15.19:0/249259640 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fe4080d0910 con 0x7fe41010e8a0 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/2701786601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc91bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe40e59c640 1 --2- 172.21.15.19:0/2701786601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84f40 0x55e47bc8f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe40dd9b640 1 -- 172.21.15.19:0/2701786601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 msgr2=0x55e47bc91bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/2701786601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc91bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe40dd9b640 1 -- 172.21.15.19:0/2701786601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84f40 msgr2=0x55e47bc8f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/2701786601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84f40 0x55e47bc8f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe40dd9b640 1 -- 172.21.15.19:0/2701786601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47b8fc210 con 0x55e47bc84620 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/2701786601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc84a00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe40400a500 tx=0x7fe40400a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a7a04eab7feac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe40ed9d640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3f80a3dc0 0x7fe3f80a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe408062260 tx=0x7fe40806b200 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe3fcff9640 1 -- 172.21.15.19:0/2701786601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404019030 con 0x55e47bc84620 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe3fcff9640 1 -- 172.21.15.19:0/2701786601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4040110d0 con 0x55e47bc84620 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.485+0000 7fe3fcff9640 1 -- 172.21.15.19:0/2701786601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404004db0 con 0x55e47bc84620 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 -- 172.21.15.19:0/2701786601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 msgr2=0x55e47bc84a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 --2- 172.21.15.19:0/2701786601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc84a00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe40400a500 tx=0x7fe40400a780 comp rx=0 tx=0).stop 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 -- 172.21.15.19:0/2701786601 shutdown_connections 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 --2- 172.21.15.19:0/2701786601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84f40 0x55e47bc8f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 --2- 172.21.15.19:0/2701786601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc91bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 --2- 172.21.15.19:0/2701786601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc84a00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 -- 172.21.15.19:0/2701786601 >> 172.21.15.19:0/2701786601 conn(0x55e47bc80b30 msgr2=0x55e47bc80f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 -- 172.21.15.19:0/2701786601 shutdown_connections 2024-04-23T18:34:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 -- 172.21.15.19:0/2701786601 wait complete. 2024-04-23T18:34:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 -- start start 2024-04-23T18:34:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.486+0000 7fe417b76740 1 Processor -- start 2024-04-23T18:34:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe417b76740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc8f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe417b76740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84f40 0x55e47bc8c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe417b76740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc8c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe417b76740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e47bc92430 con 0x55e47bc84620 2024-04-23T18:34:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe417b76740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e47bb840b0 con 0x55e47bc84f40 2024-04-23T18:34:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe417b76740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e47bc925b0 con 0x55e47bc8f800 2024-04-23T18:34:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe417b76740 10 client.0 _ll_register_callbacks cb 0x55e47bb71d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe40dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc8f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe40dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc8f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46776/0 (socket says 172.21.15.19:46776) 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe40dd9b640 1 -- 172.21.15.19:0/1850253510 learned_addr learned my addr 172.21.15.19:0/1850253510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe40e59c640 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84f40 0x55e47bc8c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe40dd9b640 1 -- 172.21.15.19:0/1850253510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 msgr2=0x55e47bc8c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc8c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe40dd9b640 1 -- 172.21.15.19:0/1850253510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84f40 msgr2=0x55e47bc8c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84f40 0x55e47bc8c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe40dd9b640 1 -- 172.21.15.19:0/1850253510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47b9c8620 con 0x55e47bc84620 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc8f100 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe404005af0 tx=0x7fe404005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3914f33a38af45b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.487+0000 7fe3f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/1850253510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404019040 con 0x55e47bc84620 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/1850253510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fe4040051e0 con 0x55e47bc84620 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe417b76740 1 -- 172.21.15.19:0/1850253510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4040110d0 con 0x55e47bc84620 2024-04-23T18:34:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/1850253510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4040053e0 con 0x55e47bc84620 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe417b76740 10 client.16737 fetch_fsmap learned FSMap version 27 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe417b76740 1 -- 172.21.15.19:0/1850253510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e47bc9e340 con 0x55e47bc84620 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/1850253510 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4040110d0 con 0x55e47bc84620 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/1850253510 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe40401e560 con 0x55e47bc84620 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe417b76740 10 client.16737 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe417b76740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe417b76740 20 client.16737 populate_metadata read hostname 'smithi019' 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe417b76740 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e47bc9f6f0 0x55e47bca1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe417b76740 4 client.16737 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.488+0000 7fe417b76740 1 -- 172.21.15.19:0/1850253510 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e47bbf7390 con 0x55e47bc9f6f0 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.489+0000 7fe40e59c640 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e47bc9f6f0 0x55e47bca1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.489+0000 7fe3f5ffb640 10 client.16737 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.489+0000 7fe40e59c640 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e47bc9f6f0 0x55e47bca1ae0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bfb0e280ed1f482 server_cookie=b7aabe08f69c5330 in_seq=0 out_seq=0 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/1850253510 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e47bbf7390 con 0x55e47bc9f6f0 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe3f5ffb640 10 client.16737 handle_command_reply: tid=0 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 client.16737 shutdown 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 -- 172.21.15.19:0/1850253510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 msgr2=0x55e47bc8f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc8f100 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe404005af0 tx=0x7fe404005da0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 -- 172.21.15.19:0/1850253510 shutdown_connections 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e47bc9f6f0 0x55e47bca1ae0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84f40 0x55e47bc8c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc8c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe3f5ffb640 0 client.16737 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 --2- 172.21.15.19:0/1850253510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84620 0x55e47bc8f100 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 -- 172.21.15.19:0/1850253510 >> 172.21.15.19:0/1850253510 conn(0x55e47bc80b30 msgr2=0x55e47bc868b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 -- 172.21.15.19:0/1850253510 shutdown_connections 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 1 -- 172.21.15.19:0/1850253510 wait complete. 2024-04-23T18:34:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.490+0000 7fe417b76740 20 client.16737 trim_cache size 0 max 16384 2024-04-23T18:34:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe417b76740 1 Processor -- start 2024-04-23T18:34:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe417b76740 1 -- start start 2024-04-23T18:34:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe417b76740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84620 0x55e47bc9b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe417b76740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc84f40 0x55e47bc85fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe417b76740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc8f800 0x55e47bca70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe417b76740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101afcd0 con 0x55e47bc8f800 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe417b76740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b1810 con 0x55e47bc84620 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe417b76740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b03a0 con 0x55e47bc84f40 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe40dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc8f800 0x55e47bca70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe40dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc8f800 0x55e47bca70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46790/0 (socket says 172.21.15.19:46790) 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.505+0000 7fe40dd9b640 1 -- 172.21.15.19:0/374585165 learned_addr learned my addr 172.21.15.19:0/374585165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe40e59c640 1 --2- 172.21.15.19:0/374585165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc84f40 0x55e47bc85fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/374585165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84620 0x55e47bc9b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe40dd9b640 1 -- 172.21.15.19:0/374585165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc84f40 msgr2=0x55e47bc85fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/374585165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc84f40 0x55e47bc85fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe40dd9b640 1 -- 172.21.15.19:0/374585165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84620 msgr2=0x55e47bc9b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/374585165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84620 0x55e47bc9b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe40dd9b640 1 -- 172.21.15.19:0/374585165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47bc85480 con 0x55e47bc8f800 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/374585165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc8f800 0x55e47bca70e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe40401e930 tx=0x7fe404005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d818b20c9a434b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe3f7fff640 1 -- 172.21.15.19:0/374585165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404036070 con 0x55e47bc8f800 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe3f7fff640 1 -- 172.21.15.19:0/374585165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe404024030 con 0x55e47bc8f800 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.506+0000 7fe3f7fff640 1 -- 172.21.15.19:0/374585165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404019030 con 0x55e47bc8f800 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 -- 172.21.15.19:0/374585165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc8f800 msgr2=0x55e47bca70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 --2- 172.21.15.19:0/374585165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc8f800 0x55e47bca70e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe40401e930 tx=0x7fe404005e30 comp rx=0 tx=0).stop 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 -- 172.21.15.19:0/374585165 shutdown_connections 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 --2- 172.21.15.19:0/374585165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc8f800 0x55e47bca70e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 --2- 172.21.15.19:0/374585165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc84f40 0x55e47bc85fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 --2- 172.21.15.19:0/374585165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84620 0x55e47bc9b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 -- 172.21.15.19:0/374585165 >> 172.21.15.19:0/374585165 conn(0x55e47bc80b30 msgr2=0x55e47bc7fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 -- 172.21.15.19:0/374585165 shutdown_connections 2024-04-23T18:34:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 -- 172.21.15.19:0/374585165 wait complete. 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 -- start start 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 Processor -- start 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84620 0x55e47bc97bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84f40 0x55e47bc94fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b1140 con 0x55e47bc84f40 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b0a70 con 0x55e47bc84620 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101adcd0 con 0x55e47bc8f800 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.507+0000 7fe417b76740 10 client.0 _ll_register_callbacks cb 0x7fe404004ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84f40 0x55e47bc94fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84f40 0x55e47bc94fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46802/0 (socket says 172.21.15.19:46802) 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40e59c640 1 -- 172.21.15.19:0/3722129185 learned_addr learned my addr 172.21.15.19:0/3722129185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84620 0x55e47bc97bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40e59c640 1 -- 172.21.15.19:0/3722129185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 msgr2=0x55e47bc954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40e59c640 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40e59c640 1 -- 172.21.15.19:0/3722129185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84620 msgr2=0x55e47bc97bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40e59c640 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84620 0x55e47bc97bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40e59c640 1 -- 172.21.15.19:0/3722129185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47bc95cc0 con 0x55e47bc84f40 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe40e59c640 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84f40 0x55e47bc94fa0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe400031440 tx=0x7fe4000040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125bb1293a29c6fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe3f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/3722129185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400048070 con 0x55e47bc84f40 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe417b76740 1 -- 172.21.15.19:0/3722129185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe404024030 con 0x55e47bc84f40 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/3722129185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fe4000043c0 con 0x55e47bc84f40 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/3722129185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe400035040 con 0x55e47bc84f40 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe417b76740 10 client.16743 fetch_fsmap learned FSMap version 27 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.508+0000 7fe417b76740 1 -- 172.21.15.19:0/3722129185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e47bd059e0 con 0x55e47bc84f40 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/3722129185 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe404024030 con 0x55e47bc84f40 2024-04-23T18:34:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/3722129185 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4000035f0 con 0x55e47bc84f40 2024-04-23T18:34:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe417b76740 10 client.16743 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:34:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe417b76740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe417b76740 20 client.16743 populate_metadata read hostname 'smithi019' 2024-04-23T18:34:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe417b76740 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e47bd06110 0x55e47bd08530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe417b76740 4 client.16743 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe417b76740 1 -- 172.21.15.19:0/3722129185 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e47bbf1200 con 0x55e47bd06110 2024-04-23T18:34:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e47bd06110 0x55e47bd08530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe40dd9b640 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e47bd06110 0x55e47bd08530 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118f7764fd9185dc server_cookie=ad7e71aadaba1b95 in_seq=0 out_seq=0 2024-04-23T18:34:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.509+0000 7fe3f5ffb640 10 client.16743 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.523+0000 7fe3f5ffb640 1 -- 172.21.15.19:0/3722129185 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e47bbf1200 con 0x55e47bd06110 2024-04-23T18:34:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.523+0000 7fe3f5ffb640 10 client.16743 handle_command_reply: tid=0 2024-04-23T18:34:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.523+0000 7fe417b76740 1 client.16743 shutdown 2024-04-23T18:34:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 1 -- 172.21.15.19:0/3722129185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84f40 msgr2=0x55e47bc94fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84f40 0x55e47bc94fa0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe400031440 tx=0x7fe4000040d0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 1 -- 172.21.15.19:0/3722129185 shutdown_connections 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e47bd06110 0x55e47bd08530 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e47bc8f800 0x55e47bc954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e47bc84f40 0x55e47bc94fa0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe3f5ffb640 0 client.16743 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 1 --2- 172.21.15.19:0/3722129185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47bc84620 0x55e47bc97bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 1 -- 172.21.15.19:0/3722129185 >> 172.21.15.19:0/3722129185 conn(0x55e47bc80b30 msgr2=0x55e47bc9a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 1 -- 172.21.15.19:0/3722129185 shutdown_connections 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 1 -- 172.21.15.19:0/3722129185 wait complete. 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe417b76740 20 client.16743 trim_cache size 0 max 16384 2024-04-23T18:34:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:34:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:34:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:34:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:34:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe414e69640 1 -- 172.21.15.19:0/249259640 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3f80a3dc0 msgr2=0x7fe3f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe414e69640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3f80a3dc0 0x7fe3f80a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe408062260 tx=0x7fe40806b200 comp rx=0 tx=0).stop 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe414e69640 1 -- 172.21.15.19:0/249259640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe41010e8a0 msgr2=0x7fe41019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.524+0000 7fe414e69640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe41010e8a0 0x7fe41019a770 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe40800cad0 tx=0x7fe40800cf90 comp rx=0 tx=0).stop 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.525+0000 7fe414e69640 1 -- 172.21.15.19:0/249259640 shutdown_connections 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.525+0000 7fe414e69640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3f80a3dc0 0x7fe3f80a6280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.525+0000 7fe414e69640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe41010e8a0 0x7fe41019a770 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.525+0000 7fe414e69640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe410102f00 0x7fe4101a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.525+0000 7fe414e69640 1 --2- 172.21.15.19:0/249259640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe41006acc0 0x7fe4101a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.525+0000 7fe414e69640 1 -- 172.21.15.19:0/249259640 >> 172.21.15.19:0/249259640 conn(0x7fe410076c50 msgr2=0x7fe41010f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.525+0000 7fe414e69640 1 -- 172.21.15.19:0/249259640 shutdown_connections 2024-04-23T18:34:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:30.525+0000 7fe414e69640 1 -- 172.21.15.19:0/249259640 wait complete. 2024-04-23T18:34:32.061 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:31 smithi019 ceph-mon[29215]: pgmap v258: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T18:34:32.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:31 smithi044 ceph-mon[32458]: pgmap v258: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T18:34:32.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:31 smithi155 ceph-mon[33468]: pgmap v258: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T18:34:34.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:33 smithi044 ceph-mon[32458]: pgmap v259: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:34:34.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:33 smithi155 ceph-mon[33468]: pgmap v259: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:34:34.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:33 smithi019 ceph-mon[29215]: pgmap v259: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:34:36.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:35 smithi044 ceph-mon[32458]: pgmap v260: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T18:34:36.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:35 smithi155 ceph-mon[33468]: pgmap v260: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T18:34:36.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:35 smithi019 ceph-mon[29215]: pgmap v260: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T18:34:38.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:37 smithi044 ceph-mon[32458]: pgmap v261: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:38.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:37 smithi155 ceph-mon[33468]: pgmap v261: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:38.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:37 smithi019 ceph-mon[29215]: pgmap v261: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:40.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:39 smithi044 ceph-mon[32458]: pgmap v262: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:40.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:39 smithi155 ceph-mon[33468]: pgmap v262: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:40.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:39 smithi019 ceph-mon[29215]: pgmap v262: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:34:42.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:41 smithi044 ceph-mon[32458]: pgmap v263: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T18:34:42.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:41 smithi155 ceph-mon[33468]: pgmap v263: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T18:34:42.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:41 smithi019 ceph-mon[29215]: pgmap v263: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T18:34:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.624+0000 7f5851843640 1 -- 172.21.15.19:0/888887838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584c100e30 msgr2=0x7f584c10ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.624+0000 7f5851843640 1 --2- 172.21.15.19:0/888887838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584c100e30 0x7f584c10ee00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5844009cd0 tx=0x7f584402a8e0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.624+0000 7f5851843640 1 -- 172.21.15.19:0/888887838 shutdown_connections 2024-04-23T18:34:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.624+0000 7f5851843640 1 --2- 172.21.15.19:0/888887838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584c100e30 0x7f584c10ee00 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.624+0000 7f5851843640 1 --2- 172.21.15.19:0/888887838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584c100470 0x7f584c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.624+0000 7f5851843640 1 --2- 172.21.15.19:0/888887838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f584c106f50 0x7f584c107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.624+0000 7f5851843640 1 -- 172.21.15.19:0/888887838 >> 172.21.15.19:0/888887838 conn(0x7f584c077390 msgr2=0x7f584c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.624+0000 7f5851843640 1 -- 172.21.15.19:0/888887838 shutdown_connections 2024-04-23T18:34:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.624+0000 7f5851843640 1 -- 172.21.15.19:0/888887838 wait complete. 2024-04-23T18:34:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.625+0000 7f5851843640 1 Processor -- start 2024-04-23T18:34:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.625+0000 7f5851843640 1 -- start start 2024-04-23T18:34:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f5851843640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f584c100470 0x7f584c101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f5851843640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584c100e30 0x7f584c101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f5851843640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584c106f50 0x7f584c1020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f5851843640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1122e0 con 0x7f584c106f50 2024-04-23T18:34:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f5851843640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f584c112160 con 0x7f584c100470 2024-04-23T18:34:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f5851843640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f584c112460 con 0x7f584c100e30 2024-04-23T18:34:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584c106f50 0x7f584c1020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584c106f50 0x7f584c1020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36082/0 (socket says 172.21.15.19:36082) 2024-04-23T18:34:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584b7fe640 1 -- 172.21.15.19:0/1966188693 learned_addr learned my addr 172.21.15.19:0/1966188693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584affd640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f584c100470 0x7f584c101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584a7fc640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584c100e30 0x7f584c101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584b7fe640 1 -- 172.21.15.19:0/1966188693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584c100e30 msgr2=0x7f584c101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584b7fe640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584c100e30 0x7f584c101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584b7fe640 1 -- 172.21.15.19:0/1966188693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f584c100470 msgr2=0x7f584c101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584b7fe640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f584c100470 0x7f584c101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584b7fe640 1 -- 172.21.15.19:0/1966188693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c1a6bc0 con 0x7f584c106f50 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584a7fc640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584c100e30 0x7f584c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f584b7fe640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584c106f50 0x7f584c1020d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5844005e30 tx=0x7f5844003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.626+0000 7f5850841640 1 -- 172.21.15.19:0/1966188693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844004240 con 0x7f584c106f50 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.627+0000 7f5850841640 1 -- 172.21.15.19:0/1966188693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5844031ce0 con 0x7f584c106f50 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.627+0000 7f5851843640 1 -- 172.21.15.19:0/1966188693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584c1a6e50 con 0x7f584c106f50 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.627+0000 7f5850841640 1 -- 172.21.15.19:0/1966188693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844046480 con 0x7f584c106f50 2024-04-23T18:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.627+0000 7f5851843640 1 -- 172.21.15.19:0/1966188693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584c1a7360 con 0x7f584c106f50 2024-04-23T18:34:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f5850841640 1 -- 172.21.15.19:0/1966188693 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f58440306a0 con 0x7f584c106f50 2024-04-23T18:34:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f5854636740 1 Processor -- start 2024-04-23T18:34:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f5854636740 1 -- start start 2024-04-23T18:34:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f5854636740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f86a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f5854636740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86f40 0x55f272f912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f5854636740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f91800 0x55f272f93bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f5854636740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f272eeb970 con 0x55f272f86620 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f5854636740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f272e884a0 con 0x55f272f86f40 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f5854636740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f272ee7930 con 0x55f272f91800 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f584a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f86a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f584a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f86a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36098/0 (socket says 172.21.15.19:36098) 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f584a7fc640 1 -- 172.21.15.19:0/648669008 learned_addr learned my addr 172.21.15.19:0/648669008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f5850841640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3df0 0x7f58340a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f584a7fc640 1 --2- 172.21.15.19:0/648669008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f91800 0x55f272f93bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.630+0000 7f584affd640 1 --2- 172.21.15.19:0/648669008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86f40 0x55f272f912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f584a7fc640 1 -- 172.21.15.19:0/648669008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f91800 msgr2=0x55f272f93bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f584a7fc640 1 --2- 172.21.15.19:0/648669008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f91800 0x55f272f93bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f584a7fc640 1 -- 172.21.15.19:0/648669008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86f40 msgr2=0x55f272f912c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f584a7fc640 1 --2- 172.21.15.19:0/648669008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86f40 0x55f272f912c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f584a7fc640 1 -- 172.21.15.19:0/648669008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f272bfe210 con 0x55f272f86620 2024-04-23T18:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f584b7fe640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3df0 0x7f58340a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f5850841640 1 -- 172.21.15.19:0/1966188693 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f584402e030 con 0x7f584c106f50 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f584a7fc640 1 --2- 172.21.15.19:0/648669008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f86a00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5840005fd0 tx=0x7f5840009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698c0912ab20cb5b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f5831ffb640 1 -- 172.21.15.19:0/648669008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58400134c0 con 0x55f272f86620 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f5831ffb640 1 -- 172.21.15.19:0/648669008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584001b070 con 0x55f272f86620 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.631+0000 7f5831ffb640 1 -- 172.21.15.19:0/648669008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58400126e0 con 0x55f272f86620 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 -- 172.21.15.19:0/648669008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 msgr2=0x55f272f86a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f584b7fe640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3df0 0x7f58340a62b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f58440ef150 tx=0x7f58440e8040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 --2- 172.21.15.19:0/648669008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f86a00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5840005fd0 tx=0x7f5840009630 comp rx=0 tx=0).stop 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 -- 172.21.15.19:0/648669008 shutdown_connections 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 --2- 172.21.15.19:0/648669008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86f40 0x55f272f912c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 --2- 172.21.15.19:0/648669008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f91800 0x55f272f93bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 --2- 172.21.15.19:0/648669008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f86a00 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 -- 172.21.15.19:0/648669008 >> 172.21.15.19:0/648669008 conn(0x55f272f82b30 msgr2=0x55f272f82f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 -- 172.21.15.19:0/648669008 shutdown_connections 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 -- 172.21.15.19:0/648669008 wait complete. 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 -- start start 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.632+0000 7f5854636740 1 Processor -- start 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f5854636740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86620 0x55f272f91190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f5854636740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f8e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f5854636740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f91800 0x55f272f8ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f5854636740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f272f94430 con 0x55f272f86f40 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f5854636740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f272e860b0 con 0x55f272f91800 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f5854636740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f272f945b0 con 0x55f272f86620 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f5854636740 10 client.0 _ll_register_callbacks cb 0x55f272e73d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f8e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f8e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36108/0 (socket says 172.21.15.19:36108) 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584affd640 1 -- 172.21.15.19:0/2596947852 learned_addr learned my addr 172.21.15.19:0/2596947852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584a7fc640 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86620 0x55f272f91190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584affd640 1 -- 172.21.15.19:0/2596947852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86620 msgr2=0x55f272f91190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584affd640 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86620 0x55f272f91190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584affd640 1 -- 172.21.15.19:0/2596947852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f91800 msgr2=0x55f272f8ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584affd640 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f91800 0x55f272f8ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584affd640 1 -- 172.21.15.19:0/2596947852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f272cca620 con 0x55f272f86f40 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584a7fc640 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86620 0x55f272f91190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f584affd640 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f8e480 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f583c00be90 tx=0x7f583c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4449a3c16c3ff08 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.633+0000 7f582affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f582affd640 1 -- 172.21.15.19:0/2596947852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c004220 con 0x55f272f86f40 2024-04-23T18:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f582affd640 1 -- 172.21.15.19:0/2596947852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f583c0043c0 con 0x55f272f86f40 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f5854636740 1 -- 172.21.15.19:0/2596947852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f584001b070 con 0x55f272f86f40 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f582affd640 1 -- 172.21.15.19:0/2596947852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f583c010650 con 0x55f272f86f40 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f5854636740 10 client.16755 fetch_fsmap learned FSMap version 27 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f5854636740 1 -- 172.21.15.19:0/2596947852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f272fa0420 con 0x55f272f86f40 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f582affd640 1 -- 172.21.15.19:0/2596947852 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584001b070 con 0x55f272f86f40 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f582affd640 1 -- 172.21.15.19:0/2596947852 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f583c003b00 con 0x55f272f86f40 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f5854636740 10 client.16755 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f5854636740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f5854636740 20 client.16755 populate_metadata read hostname 'smithi019' 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f5854636740 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f272fa19a0 0x55f272fa3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f5854636740 4 client.16755 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.634+0000 7f5854636740 1 -- 172.21.15.19:0/2596947852 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f272ef9390 con 0x55f272fa19a0 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.635+0000 7f584a7fc640 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f272fa19a0 0x55f272fa3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.635+0000 7f584a7fc640 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f272fa19a0 0x55f272fa3d90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b914a58110b45d6 server_cookie=2bc0bba8c5be2862 in_seq=0 out_seq=0 2024-04-23T18:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.635+0000 7f582affd640 10 client.16755 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f582affd640 1 -- 172.21.15.19:0/2596947852 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f272ef9390 con 0x55f272fa19a0 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f582affd640 10 client.16755 handle_command_reply: tid=0 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 client.16755 shutdown 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 -- 172.21.15.19:0/2596947852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 msgr2=0x55f272f8e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f8e480 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f583c00be90 tx=0x7f583c012820 comp rx=0 tx=0).stop 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 -- 172.21.15.19:0/2596947852 shutdown_connections 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f272fa19a0 0x55f272fa3d90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f91800 0x55f272f8ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f8e480 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 --2- 172.21.15.19:0/2596947852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86620 0x55f272f91190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 -- 172.21.15.19:0/2596947852 >> 172.21.15.19:0/2596947852 conn(0x55f272f82b30 msgr2=0x55f272f888b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f582affd640 0 client.16755 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 -- 172.21.15.19:0/2596947852 shutdown_connections 2024-04-23T18:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 1 -- 172.21.15.19:0/2596947852 wait complete. 2024-04-23T18:34:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.636+0000 7f5854636740 20 client.16755 trim_cache size 0 max 16384 2024-04-23T18:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f5854636740 1 Processor -- start 2024-04-23T18:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f5854636740 1 -- start start 2024-04-23T18:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f5854636740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f8e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f5854636740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86f40 0x55f272f9d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f5854636740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f91800 0x55f272f90cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f5854636740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1afd10 con 0x55f272f86620 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f5854636740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b1850 con 0x55f272f91800 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f5854636740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b03e0 con 0x55f272f86f40 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f584a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f8e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f584a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f8e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36124/0 (socket says 172.21.15.19:36124) 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.651+0000 7f584a7fc640 1 -- 172.21.15.19:0/263068733 learned_addr learned my addr 172.21.15.19:0/263068733 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f584affd640 1 --2- 172.21.15.19:0/263068733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86f40 0x55f272f9d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f584a7fc640 1 -- 172.21.15.19:0/263068733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86f40 msgr2=0x55f272f9d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f584a7fc640 1 --2- 172.21.15.19:0/263068733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86f40 0x55f272f9d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f584a7fc640 1 -- 172.21.15.19:0/263068733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f91800 msgr2=0x55f272f90cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f584a7fc640 1 --2- 172.21.15.19:0/263068733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f91800 0x55f272f90cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f584a7fc640 1 -- 172.21.15.19:0/263068733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f272f93bf0 con 0x55f272f86620 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f584affd640 1 --2- 172.21.15.19:0/263068733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86f40 0x55f272f9d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f584a7fc640 1 --2- 172.21.15.19:0/263068733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f8e420 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x55f272f82550 tx=0x7f5840013f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d219d331f4c4cd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f58317fa640 1 -- 172.21.15.19:0/263068733 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840028bc0 con 0x55f272f86620 2024-04-23T18:34:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f58317fa640 1 -- 172.21.15.19:0/263068733 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584001b070 con 0x55f272f86620 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.652+0000 7f58317fa640 1 -- 172.21.15.19:0/263068733 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58400055e0 con 0x55f272f86620 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 -- 172.21.15.19:0/263068733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 msgr2=0x55f272f8e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 --2- 172.21.15.19:0/263068733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f8e420 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x55f272f82550 tx=0x7f5840013f30 comp rx=0 tx=0).stop 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 -- 172.21.15.19:0/263068733 shutdown_connections 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 --2- 172.21.15.19:0/263068733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f91800 0x55f272f90cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 --2- 172.21.15.19:0/263068733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f86f40 0x55f272f9d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 --2- 172.21.15.19:0/263068733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86620 0x55f272f8e420 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 -- 172.21.15.19:0/263068733 >> 172.21.15.19:0/263068733 conn(0x55f272f82b30 msgr2=0x55f272f8dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 -- 172.21.15.19:0/263068733 shutdown_connections 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 -- 172.21.15.19:0/263068733 wait complete. 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 -- start start 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.653+0000 7f5854636740 1 Processor -- start 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f5854636740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86620 0x55f272f947c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f5854636740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f94d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f5854636740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f91800 0x55f272f95240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f5854636740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b1180 con 0x55f272f86f40 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f5854636740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b0ab0 con 0x55f272f86620 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f5854636740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1add10 con 0x55f272f91800 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f5854636740 10 client.0 _ll_register_callbacks cb 0x7f583c00b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f94d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f94d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36134/0 (socket says 172.21.15.19:36134) 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584affd640 1 -- 172.21.15.19:0/2580955264 learned_addr learned my addr 172.21.15.19:0/2580955264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584a7fc640 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86620 0x55f272f947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584affd640 1 -- 172.21.15.19:0/2580955264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f91800 msgr2=0x55f272f95240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584affd640 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f91800 0x55f272f95240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584affd640 1 -- 172.21.15.19:0/2580955264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86620 msgr2=0x55f272f947c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584affd640 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86620 0x55f272f947c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584affd640 1 -- 172.21.15.19:0/2580955264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f273005f70 con 0x55f272f86f40 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584a7fc640 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86620 0x55f272f947c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f584affd640 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f94d00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f583c000c00 tx=0x7f583c004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04152ac20334fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f582affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f582affd640 1 -- 172.21.15.19:0/2580955264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c005850 con 0x55f272f86f40 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f582affd640 1 -- 172.21.15.19:0/2580955264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f583c024aa0 con 0x55f272f86f40 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.654+0000 7f5854636740 1 -- 172.21.15.19:0/2580955264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f584001b070 con 0x55f272f86f40 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f582affd640 1 -- 172.21.15.19:0/2580955264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f583c024ca0 con 0x55f272f86f40 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f5854636740 10 client.16761 fetch_fsmap learned FSMap version 27 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f5854636740 1 -- 172.21.15.19:0/2580955264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2730071d0 con 0x55f272f86f40 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f582affd640 1 -- 172.21.15.19:0/2580955264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584001b070 con 0x55f272f86f40 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f582affd640 1 -- 172.21.15.19:0/2580955264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f583c003a40 con 0x55f272f86f40 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f5854636740 10 client.16761 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f5854636740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f5854636740 20 client.16761 populate_metadata read hostname 'smithi019' 2024-04-23T18:34:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f5854636740 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f273007900 0x55f273009d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f5854636740 4 client.16761 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f5854636740 1 -- 172.21.15.19:0/2580955264 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f272ef3200 con 0x55f273007900 2024-04-23T18:34:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.655+0000 7f584a7fc640 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f273007900 0x55f273009d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.656+0000 7f582affd640 10 client.16761 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.656+0000 7f584a7fc640 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f273007900 0x55f273009d40 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5e71f4339d21053 server_cookie=6baeca0b11ebbee1 in_seq=0 out_seq=0 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f582affd640 1 -- 172.21.15.19:0/2580955264 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f272ef3200 con 0x55f273007900 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f582affd640 10 client.16761 handle_command_reply: tid=0 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 client.16761 shutdown 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 -- 172.21.15.19:0/2580955264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 msgr2=0x55f272f94d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f94d00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f583c000c00 tx=0x7f583c004220 comp rx=0 tx=0).stop 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 -- 172.21.15.19:0/2580955264 shutdown_connections 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f273007900 0x55f273009d40 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f272f91800 0x55f272f95240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f272f86f40 0x55f272f94d00 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f582affd640 0 client.16761 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 --2- 172.21.15.19:0/2580955264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f272f86620 0x55f272f947c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 -- 172.21.15.19:0/2580955264 >> 172.21.15.19:0/2580955264 conn(0x55f272f82b30 msgr2=0x55f272f81420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 -- 172.21.15.19:0/2580955264 shutdown_connections 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.672+0000 7f5854636740 1 -- 172.21.15.19:0/2580955264 wait complete. 2024-04-23T18:34:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5854636740 20 client.16761 trim_cache size 0 max 16384 2024-04-23T18:34:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:34:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:34:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:34:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 -- 172.21.15.19:0/1966188693 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3df0 msgr2=0x7f58340a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3df0 0x7f58340a62b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f58440ef150 tx=0x7f58440e8040 comp rx=0 tx=0).stop 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 -- 172.21.15.19:0/1966188693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584c106f50 msgr2=0x7f584c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584c106f50 0x7f584c1020d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5844005e30 tx=0x7f5844003ba0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 -- 172.21.15.19:0/1966188693 shutdown_connections 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f58340a3df0 0x7f58340a62b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f584c106f50 0x7f584c1020d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f584c100e30 0x7f584c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 --2- 172.21.15.19:0/1966188693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f584c100470 0x7f584c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 -- 172.21.15.19:0/1966188693 >> 172.21.15.19:0/1966188693 conn(0x7f584c077390 msgr2=0x7f584c104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 -- 172.21.15.19:0/1966188693 shutdown_connections 2024-04-23T18:34:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:43.673+0000 7f5851843640 1 -- 172.21.15.19:0/1966188693 wait complete. 2024-04-23T18:34:44.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:43 smithi044 ceph-mon[32458]: pgmap v264: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:34:44.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:43 smithi155 ceph-mon[33468]: pgmap v264: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:34:44.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:43 smithi019 ceph-mon[29215]: pgmap v264: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:34:46.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:45 smithi044 ceph-mon[32458]: pgmap v265: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-23T18:34:46.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:45 smithi155 ceph-mon[33468]: pgmap v265: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-23T18:34:46.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:45 smithi019 ceph-mon[29215]: pgmap v265: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-23T18:34:48.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:47 smithi044 ceph-mon[32458]: pgmap v266: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:34:48.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:47 smithi155 ceph-mon[33468]: pgmap v266: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:34:48.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:47 smithi019 ceph-mon[29215]: pgmap v266: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:34:50.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:49 smithi044 ceph-mon[32458]: pgmap v267: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:34:50.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:49 smithi155 ceph-mon[33468]: pgmap v267: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:34:50.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:49 smithi019 ceph-mon[29215]: pgmap v267: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:34:52.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:51 smithi155 ceph-mon[33468]: pgmap v268: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:34:52.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:51 smithi019 ceph-mon[29215]: pgmap v268: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:34:52.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:51 smithi044 ceph-mon[32458]: pgmap v268: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:34:53.968 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:251/600: open fd = 3 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:252/600: open fd = 3 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:253/600: open fd = 3 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:254/600: open fd = 3 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.969 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:255/600: open fd = 3 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:256/600: open fd = 3 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:257/600: open fd = 3 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:258/600: open fd = 3 2024-04-23T18:34:53.970 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:259/600: open fd = 3 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:260/600: open fd = 3 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:261/600: open fd = 3 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.971 INFO:tasks.workunit.client.0.smithi019.stdout:262/600: open fd = 3 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:263/600: open fd = 3 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:264/600: open fd = 3 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:265/600: open fd = 3 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.972 INFO:tasks.workunit.client.0.smithi019.stdout:266/600: open fd = 3 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:267/600: open fd = 3 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:268/600: open fd = 3 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:269/600: open fd = 3 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.973 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:270/600: open fd = 3 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:271/600: open fd = 3 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:272/600: open fd = 3 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:273/600: open fd = 3 2024-04-23T18:34:53.974 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:274/600: open fd = 3 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:275/600: open fd = 3 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:276/600: open fd = 3 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:277/600: open fd = 3 2024-04-23T18:34:53.975 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:278/600: open fd = 3 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:279/600: open fd = 3 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:280/600: open fd = 3 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:281/600: open fd = 3 2024-04-23T18:34:53.976 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:282/600: open fd = 3 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:283/600: open fd = 3 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:284/600: open fd = 3 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.977 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:285/600: open fd = 3 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:286/600: open fd = 3 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:287/600: open fd = 3 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:288/600: open fd = 3 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.978 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:289/600: open fd = 3 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:290/600: open fd = 3 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:291/600: open fd = 3 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:292/600: open fd = 3 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.979 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:293/600: open fd = 3 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:294/600: open fd = 3 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:295/600: open fd = 3 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:296/600: open fd = 3 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:297/600: open fd = 3 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:298/600: open fd = 3 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:299/600: open fd = 3 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:300/600: open fd = 3 2024-04-23T18:34:53.981 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:301/600: open fd = 3 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:302/600: open fd = 3 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:303/600: open fd = 3 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:304/600: open fd = 3 2024-04-23T18:34:53.982 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:305/600: open fd = 3 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:306/600: open fd = 3 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:307/600: open fd = 3 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.983 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:308/600: open fd = 3 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:309/600: open fd = 3 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:310/600: open fd = 3 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:311/600: open fd = 3 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.984 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.985 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:34:53.985 INFO:tasks.workunit.client.0.smithi019.stdout:312/600: open fd = 3 2024-04-23T18:34:53.985 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:53.985 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:34:54.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:53 smithi155 ceph-mon[33468]: pgmap v269: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:34:54.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:53 smithi019 ceph-mon[29215]: pgmap v269: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:34:54.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:53 smithi044 ceph-mon[32458]: pgmap v269: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:34:56.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:55 smithi155 ceph-mon[33468]: pgmap v270: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:34:56.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:55 smithi019 ceph-mon[29215]: pgmap v270: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:34:56.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:55 smithi044 ceph-mon[32458]: pgmap v270: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.773+0000 7f37a570f640 1 -- 172.21.15.19:0/2367183524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37a0100d80 msgr2=0x7f37a0101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.773+0000 7f37a570f640 1 --2- 172.21.15.19:0/2367183524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37a0100d80 0x7f37a0101160 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3790009900 tx=0x7f379002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.773+0000 7f37a570f640 1 -- 172.21.15.19:0/2367183524 shutdown_connections 2024-04-23T18:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.773+0000 7f37a570f640 1 --2- 172.21.15.19:0/2367183524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a01016a0 0x7f37a010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.773+0000 7f37a570f640 1 --2- 172.21.15.19:0/2367183524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a010e930 0x7f37a0110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.773+0000 7f37a570f640 1 --2- 172.21.15.19:0/2367183524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37a0100d80 0x7f37a0101160 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.773+0000 7f37a570f640 1 -- 172.21.15.19:0/2367183524 >> 172.21.15.19:0/2367183524 conn(0x7f37a00771d0 msgr2=0x7f37a00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.773+0000 7f37a570f640 1 -- 172.21.15.19:0/2367183524 shutdown_connections 2024-04-23T18:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.773+0000 7f37a570f640 1 -- 172.21.15.19:0/2367183524 wait complete. 2024-04-23T18:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.774+0000 7f37a570f640 1 Processor -- start 2024-04-23T18:34:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.774+0000 7f37a570f640 1 -- start start 2024-04-23T18:34:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f37a570f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a0100d80 0x7f37a01a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f37a570f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a01016a0 0x7f37a01a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f37a570f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37a010e930 0x7f37a019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f37a570f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0114450 con 0x7f37a010e930 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f37a570f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01142d0 con 0x7f37a0100d80 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f37a570f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01145d0 con 0x7f37a01016a0 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37a010e930 0x7f37a019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a0100d80 0x7f37a01a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37a010e930 0x7f37a019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60714/0 (socket says 172.21.15.19:60714) 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379f7fe640 1 -- 172.21.15.19:0/3941885611 learned_addr learned my addr 172.21.15.19:0/3941885611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379e7fc640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a01016a0 0x7f37a01a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379f7fe640 1 -- 172.21.15.19:0/3941885611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a01016a0 msgr2=0x7f37a01a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379f7fe640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a01016a0 0x7f37a01a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379f7fe640 1 -- 172.21.15.19:0/3941885611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a0100d80 msgr2=0x7f37a01a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379f7fe640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a0100d80 0x7f37a01a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379f7fe640 1 -- 172.21.15.19:0/3941885611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a019af70 con 0x7f37a010e930 2024-04-23T18:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f379f7fe640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37a010e930 0x7f37a019a6b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3798009fd0 tx=0x7f379800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.775+0000 7f3797fff640 1 -- 172.21.15.19:0/3941885611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798004d20 con 0x7f37a010e930 2024-04-23T18:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.776+0000 7f3797fff640 1 -- 172.21.15.19:0/3941885611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3798005020 con 0x7f37a010e930 2024-04-23T18:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.776+0000 7f37a570f640 1 -- 172.21.15.19:0/3941885611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a019b260 con 0x7f37a010e930 2024-04-23T18:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.776+0000 7f3797fff640 1 -- 172.21.15.19:0/3941885611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37980116b0 con 0x7f37a010e930 2024-04-23T18:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.776+0000 7f37a570f640 1 -- 172.21.15.19:0/3941885611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a01a7570 con 0x7f37a010e930 2024-04-23T18:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.778+0000 7f3797fff640 1 -- 172.21.15.19:0/3941885611 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f37980051d0 con 0x7f37a010e930 2024-04-23T18:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.778+0000 7f37a8422740 1 Processor -- start 2024-04-23T18:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.778+0000 7f37a8422740 1 -- start start 2024-04-23T18:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f3797fff640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f378c0a3e40 0x7f378c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f37a8422740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0620 0x555a1b4c0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f37a8422740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0f40 0x555a1b4cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f37a8422740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 0x555a1b4cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f37a8422740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555a1b425970 con 0x555a1b4c0f40 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f37a8422740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555a1b3c24a0 con 0x555a1b4cb800 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f37a8422740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a1b421930 con 0x555a1b4c0620 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f3797fff640 1 -- 172.21.15.19:0/3941885611 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f37980d0c70 con 0x7f37a010e930 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f379e7fc640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f378c0a3e40 0x7f378c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f379e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0f40 0x555a1b4cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f379effd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0620 0x555a1b4c0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f379e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0f40 0x555a1b4cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60718/0 (socket says 172.21.15.19:60718) 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f379e7fc640 1 -- 172.21.15.19:0/289178383 learned_addr learned my addr 172.21.15.19:0/289178383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.779+0000 7f379f7fe640 1 --2- 172.21.15.19:0/289178383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 0x555a1b4cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f379effd640 1 -- 172.21.15.19:0/289178383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 msgr2=0x555a1b4cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f379effd640 1 --2- 172.21.15.19:0/289178383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 0x555a1b4cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f379effd640 1 -- 172.21.15.19:0/289178383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0f40 msgr2=0x555a1b4cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f379effd640 1 --2- 172.21.15.19:0/289178383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0f40 0x555a1b4cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f379effd640 1 -- 172.21.15.19:0/289178383 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1b138210 con 0x555a1b4c0620 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f379e7fc640 1 --2- 172.21.15.19:0/289178383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0f40 0x555a1b4cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f379e7fc640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f378c0a3e40 0x7f378c0a6300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37880149e0 tx=0x7f378800cbb0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f379effd640 1 --2- 172.21.15.19:0/289178383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0620 0x555a1b4c0a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3790009230 tx=0x7f3790032bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be3aedc924d07bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f37957fa640 1 -- 172.21.15.19:0/289178383 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790040070 con 0x555a1b4c0620 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f37957fa640 1 -- 172.21.15.19:0/289178383 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3790004030 con 0x555a1b4c0620 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.780+0000 7f37957fa640 1 -- 172.21.15.19:0/289178383 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790035040 con 0x555a1b4c0620 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 -- 172.21.15.19:0/289178383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0620 msgr2=0x555a1b4c0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 --2- 172.21.15.19:0/289178383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0620 0x555a1b4c0a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3790009230 tx=0x7f3790032bd0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 -- 172.21.15.19:0/289178383 shutdown_connections 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 --2- 172.21.15.19:0/289178383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 0x555a1b4cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 --2- 172.21.15.19:0/289178383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0f40 0x555a1b4cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 --2- 172.21.15.19:0/289178383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0620 0x555a1b4c0a00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 -- 172.21.15.19:0/289178383 >> 172.21.15.19:0/289178383 conn(0x555a1b4bcb30 msgr2=0x555a1b4bcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 -- 172.21.15.19:0/289178383 shutdown_connections 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 -- 172.21.15.19:0/289178383 wait complete. 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 -- start start 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.781+0000 7f37a8422740 1 Processor -- start 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f37a8422740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4cb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f37a8422740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 0x555a1b4c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f37a8422740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 0x555a1b4c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f37a8422740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555a1b4ce3d0 con 0x555a1b4c0620 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f37a8422740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555a1b3c00b0 con 0x555a1b4cb800 2024-04-23T18:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f37a8422740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a1b4ce550 con 0x555a1b4c0f40 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f37a8422740 10 client.0 _ll_register_callbacks cb 0x555a1b3add30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60734/0 (socket says 172.21.15.19:60734) 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379effd640 1 -- 172.21.15.19:0/983774769 learned_addr learned my addr 172.21.15.19:0/983774769 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379f7fe640 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 0x555a1b4c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379e7fc640 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 0x555a1b4c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379effd640 1 -- 172.21.15.19:0/983774769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 msgr2=0x555a1b4c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379effd640 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 0x555a1b4c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379effd640 1 -- 172.21.15.19:0/983774769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 msgr2=0x555a1b4c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379effd640 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 0x555a1b4c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379effd640 1 -- 172.21.15.19:0/983774769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1b204620 con 0x555a1b4c0620 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379f7fe640 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 0x555a1b4c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379e7fc640 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 0x555a1b4c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f379effd640 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4cb190 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3790009430 tx=0x7f3790009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e233e30ff04d6cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.782+0000 7f37867fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37867fc640 1 -- 172.21.15.19:0/983774769 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790040070 con 0x555a1b4c0620 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37a8422740 1 -- 172.21.15.19:0/983774769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3790004030 con 0x555a1b4c0620 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37867fc640 1 -- 172.21.15.19:0/983774769 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f3790005430 con 0x555a1b4c0620 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37867fc640 1 -- 172.21.15.19:0/983774769 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3790056050 con 0x555a1b4c0620 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37a8422740 10 client.16773 fetch_fsmap learned FSMap version 27 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37a8422740 1 -- 172.21.15.19:0/983774769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a1b4da320 con 0x555a1b4c0620 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37867fc640 1 -- 172.21.15.19:0/983774769 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3790004030 con 0x555a1b4c0620 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37867fc640 1 -- 172.21.15.19:0/983774769 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3790004d20 con 0x555a1b4c0620 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37a8422740 10 client.16773 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37a8422740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37a8422740 20 client.16773 populate_metadata read hostname 'smithi019' 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37a8422740 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555a1b4db670 0x555a1b4dda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37a8422740 4 client.16773 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.783+0000 7f37a8422740 1 -- 172.21.15.19:0/983774769 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a1b433390 con 0x555a1b4db670 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.784+0000 7f379e7fc640 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555a1b4db670 0x555a1b4dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.784+0000 7f379e7fc640 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555a1b4db670 0x555a1b4dda60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9520f7dafa5fdfd server_cookie=7ee744a506fc52d in_seq=0 out_seq=0 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.784+0000 7f37867fc640 10 client.16773 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37867fc640 1 -- 172.21.15.19:0/983774769 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a1b433390 con 0x555a1b4db670 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37867fc640 10 client.16773 handle_command_reply: tid=0 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 client.16773 shutdown 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 -- 172.21.15.19:0/983774769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 msgr2=0x555a1b4cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4cb190 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3790009430 tx=0x7f3790009640 comp rx=0 tx=0).stop 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 -- 172.21.15.19:0/983774769 shutdown_connections 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555a1b4db670 0x555a1b4dda60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4cb800 0x555a1b4c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 0x555a1b4c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37867fc640 0 client.16773 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 --2- 172.21.15.19:0/983774769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4cb190 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 -- 172.21.15.19:0/983774769 >> 172.21.15.19:0/983774769 conn(0x555a1b4bcb30 msgr2=0x555a1b4c28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 -- 172.21.15.19:0/983774769 shutdown_connections 2024-04-23T18:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 1 -- 172.21.15.19:0/983774769 wait complete. 2024-04-23T18:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.785+0000 7f37a8422740 20 client.16773 trim_cache size 0 max 16384 2024-04-23T18:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.800+0000 7f37a8422740 1 Processor -- start 2024-04-23T18:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.800+0000 7f37a8422740 1 -- start start 2024-04-23T18:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.800+0000 7f37a8422740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0620 0x555a1b4d7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.800+0000 7f37a8422740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 0x555a1b4bbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f37a8422740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4cb800 0x555a1b4cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f37a8422740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01afd40 con 0x555a1b4cb800 2024-04-23T18:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f37a8422740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01b1880 con 0x555a1b4c0620 2024-04-23T18:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f37a8422740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01b0410 con 0x555a1b4c0f40 2024-04-23T18:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4cb800 0x555a1b4cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4cb800 0x555a1b4cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60746/0 (socket says 172.21.15.19:60746) 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379f7fe640 1 -- 172.21.15.19:0/1019453871 learned_addr learned my addr 172.21.15.19:0/1019453871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379e7fc640 1 --2- 172.21.15.19:0/1019453871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 0x555a1b4bbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379effd640 1 --2- 172.21.15.19:0/1019453871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0620 0x555a1b4d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379f7fe640 1 -- 172.21.15.19:0/1019453871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 msgr2=0x555a1b4bbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379f7fe640 1 --2- 172.21.15.19:0/1019453871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 0x555a1b4bbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379f7fe640 1 -- 172.21.15.19:0/1019453871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0620 msgr2=0x555a1b4d7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379f7fe640 1 --2- 172.21.15.19:0/1019453871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0620 0x555a1b4d7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379f7fe640 1 -- 172.21.15.19:0/1019453871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1b4cdbf0 con 0x555a1b4cb800 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379effd640 1 --2- 172.21.15.19:0/1019453871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0620 0x555a1b4d7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f379f7fe640 1 --2- 172.21.15.19:0/1019453871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4cb800 0x555a1b4cacc0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3798099020 tx=0x7f37980808f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38aff1b06bb5429 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f3794ff9640 1 -- 172.21.15.19:0/1019453871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37980a0fc0 con 0x555a1b4cb800 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f3794ff9640 1 -- 172.21.15.19:0/1019453871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37980a5880 con 0x555a1b4cb800 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.801+0000 7f3794ff9640 1 -- 172.21.15.19:0/1019453871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798005510 con 0x555a1b4cb800 2024-04-23T18:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.802+0000 7f37a8422740 1 -- 172.21.15.19:0/1019453871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4cb800 msgr2=0x555a1b4cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.802+0000 7f37a8422740 1 --2- 172.21.15.19:0/1019453871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4cb800 0x555a1b4cacc0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3798099020 tx=0x7f37980808f0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.802+0000 7f37a8422740 1 -- 172.21.15.19:0/1019453871 shutdown_connections 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.802+0000 7f37a8422740 1 --2- 172.21.15.19:0/1019453871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4cb800 0x555a1b4cacc0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.802+0000 7f37a8422740 1 --2- 172.21.15.19:0/1019453871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4c0f40 0x555a1b4bbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.802+0000 7f37a8422740 1 --2- 172.21.15.19:0/1019453871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0620 0x555a1b4d7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.802+0000 7f37a8422740 1 -- 172.21.15.19:0/1019453871 >> 172.21.15.19:0/1019453871 conn(0x555a1b4bcb30 msgr2=0x555a1b4bb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.802+0000 7f37a8422740 1 -- 172.21.15.19:0/1019453871 shutdown_connections 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.802+0000 7f37a8422740 1 -- 172.21.15.19:0/1019453871 wait complete. 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.802+0000 7f37a8422740 1 -- start start 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f37a8422740 1 Processor -- start 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f37a8422740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f37a8422740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0f40 0x555a1b4cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f37a8422740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4cb800 0x555a1b4cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f37a8422740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01b11b0 con 0x555a1b4c0620 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f37a8422740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01b0ae0 con 0x555a1b4c0f40 2024-04-23T18:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f37a8422740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01add40 con 0x555a1b4cb800 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f37a8422740 10 client.0 _ll_register_callbacks cb 0x7f3790032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4d3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60762/0 (socket says 172.21.15.19:60762) 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379effd640 1 -- 172.21.15.19:0/2370554850 learned_addr learned my addr 172.21.15.19:0/2370554850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379f7fe640 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4cb800 0x555a1b4cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379effd640 1 -- 172.21.15.19:0/2370554850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4cb800 msgr2=0x555a1b4cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379effd640 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4cb800 0x555a1b4cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379effd640 1 -- 172.21.15.19:0/2370554850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0f40 msgr2=0x555a1b4cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379e7fc640 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0f40 0x555a1b4cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379effd640 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0f40 0x555a1b4cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379effd640 1 -- 172.21.15.19:0/2370554850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1b4cfd90 con 0x555a1b4c0620 2024-04-23T18:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f379effd640 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4d3c40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3790006010 tx=0x7f3790003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4153ea15530bb2ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.803+0000 7f37867fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:34:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37867fc640 1 -- 172.21.15.19:0/2370554850 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790040030 con 0x555a1b4c0620 2024-04-23T18:34:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37867fc640 1 -- 172.21.15.19:0/2370554850 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f37900054d0 con 0x555a1b4c0620 2024-04-23T18:34:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37a8422740 1 -- 172.21.15.19:0/2370554850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37980a5880 con 0x555a1b4c0620 2024-04-23T18:34:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37867fc640 1 -- 172.21.15.19:0/2370554850 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3790035040 con 0x555a1b4c0620 2024-04-23T18:34:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37a8422740 10 client.16779 fetch_fsmap learned FSMap version 27 2024-04-23T18:34:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37a8422740 1 -- 172.21.15.19:0/2370554850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a1b541950 con 0x555a1b4c0620 2024-04-23T18:34:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37867fc640 1 -- 172.21.15.19:0/2370554850 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37980a5880 con 0x555a1b4c0620 2024-04-23T18:34:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37867fc640 1 -- 172.21.15.19:0/2370554850 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3790004ba0 con 0x555a1b4c0620 2024-04-23T18:34:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37a8422740 10 client.16779 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:34:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37a8422740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37a8422740 20 client.16779 populate_metadata read hostname 'smithi019' 2024-04-23T18:34:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37a8422740 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555a1b5420f0 0x555a1b544510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37a8422740 4 client.16779 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f37a8422740 1 -- 172.21.15.19:0/2370554850 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a1b42d200 con 0x555a1b5420f0 2024-04-23T18:34:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.804+0000 7f379e7fc640 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555a1b5420f0 0x555a1b544510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.805+0000 7f379e7fc640 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555a1b5420f0 0x555a1b544510 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e314b54dbcc6cbf7 server_cookie=868259fa15bb45c8 in_seq=0 out_seq=0 2024-04-23T18:34:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.805+0000 7f37867fc640 10 client.16779 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37867fc640 1 -- 172.21.15.19:0/2370554850 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a1b42d200 con 0x555a1b5420f0 2024-04-23T18:34:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37867fc640 10 client.16779 handle_command_reply: tid=0 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 client.16779 shutdown 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 -- 172.21.15.19:0/2370554850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 msgr2=0x555a1b4d3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4d3c40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3790006010 tx=0x7f3790003a30 comp rx=0 tx=0).stop 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 -- 172.21.15.19:0/2370554850 shutdown_connections 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555a1b5420f0 0x555a1b544510 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a1b4c0f40 0x555a1b4cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a1b4cb800 0x555a1b4cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37867fc640 0 client.16779 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 --2- 172.21.15.19:0/2370554850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1b4c0620 0x555a1b4d3c40 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 -- 172.21.15.19:0/2370554850 >> 172.21.15.19:0/2370554850 conn(0x555a1b4bcb30 msgr2=0x555a1b4bbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 -- 172.21.15.19:0/2370554850 shutdown_connections 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 1 -- 172.21.15.19:0/2370554850 wait complete. 2024-04-23T18:34:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.821+0000 7f37a8422740 20 client.16779 trim_cache size 0 max 16384 2024-04-23T18:34:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:34:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:34:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:34:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 -- 172.21.15.19:0/3941885611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f378c0a3e40 msgr2=0x7f378c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f378c0a3e40 0x7f378c0a6300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37880149e0 tx=0x7f378800cbb0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 -- 172.21.15.19:0/3941885611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37a010e930 msgr2=0x7f37a019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37a010e930 0x7f37a019a6b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3798009fd0 tx=0x7f379800ee60 comp rx=0 tx=0).stop 2024-04-23T18:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 -- 172.21.15.19:0/3941885611 shutdown_connections 2024-04-23T18:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f378c0a3e40 0x7f378c0a6300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37a010e930 0x7f37a019a6b0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a01016a0 0x7f37a01a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 --2- 172.21.15.19:0/3941885611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a0100d80 0x7f37a01a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 -- 172.21.15.19:0/3941885611 >> 172.21.15.19:0/3941885611 conn(0x7f37a00771d0 msgr2=0x7f37a010fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 -- 172.21.15.19:0/3941885611 shutdown_connections 2024-04-23T18:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:34:56.822+0000 7f37a570f640 1 -- 172.21.15.19:0/3941885611 wait complete. 2024-04-23T18:34:58.114 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:57 smithi044 ceph-mon[32458]: pgmap v271: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:34:58.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:57 smithi019 ceph-mon[29215]: pgmap v271: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:34:58.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:57 smithi155 ceph-mon[33468]: pgmap v271: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:35:00.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:34:59 smithi019 ceph-mon[29215]: pgmap v272: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:35:00.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:34:59 smithi044 ceph-mon[32458]: pgmap v272: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:35:00.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:34:59 smithi155 ceph-mon[33468]: pgmap v272: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:35:02.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:01 smithi019 ceph-mon[29215]: pgmap v273: 97 pgs: 97 active+clean; 3.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:35:02.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:01 smithi044 ceph-mon[32458]: pgmap v273: 97 pgs: 97 active+clean; 3.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:35:02.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:01 smithi155 ceph-mon[33468]: pgmap v273: 97 pgs: 97 active+clean; 3.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:35:04.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:03 smithi019 ceph-mon[29215]: pgmap v274: 97 pgs: 97 active+clean; 3.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:35:04.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:03 smithi044 ceph-mon[32458]: pgmap v274: 97 pgs: 97 active+clean; 3.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:35:04.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:03 smithi155 ceph-mon[33468]: pgmap v274: 97 pgs: 97 active+clean; 3.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:35:06.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:05 smithi019 ceph-mon[29215]: pgmap v275: 97 pgs: 97 active+clean; 3.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 106 KiB/s wr, 39 op/s 2024-04-23T18:35:06.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:05 smithi044 ceph-mon[32458]: pgmap v275: 97 pgs: 97 active+clean; 3.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 106 KiB/s wr, 39 op/s 2024-04-23T18:35:06.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:05 smithi155 ceph-mon[33468]: pgmap v275: 97 pgs: 97 active+clean; 3.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 106 KiB/s wr, 39 op/s 2024-04-23T18:35:07.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:06 smithi019 ceph-mon[29215]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:35:07.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:06 smithi044 ceph-mon[32458]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:35:07.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:06 smithi155 ceph-mon[33468]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:35:08.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:07 smithi019 ceph-mon[29215]: pgmap v277: 97 pgs: 97 active+clean; 3.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-23T18:35:08.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:07 smithi044 ceph-mon[32458]: pgmap v277: 97 pgs: 97 active+clean; 3.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-23T18:35:08.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:07 smithi155 ceph-mon[33468]: pgmap v277: 97 pgs: 97 active+clean; 3.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-23T18:35:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.922+0000 7fe3c9519640 1 -- 172.21.15.19:0/912707419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c4103b20 msgr2=0x7fe3c410c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.922+0000 7fe3c9519640 1 --2- 172.21.15.19:0/912707419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c4103b20 0x7fe3c410c000 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009900 tx=0x7fe3b802a480 comp rx=0 tx=0).stop 2024-04-23T18:35:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.922+0000 7fe3c9519640 1 -- 172.21.15.19:0/912707419 shutdown_connections 2024-04-23T18:35:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.922+0000 7fe3c9519640 1 --2- 172.21.15.19:0/912707419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c410c5d0 0x7fe3c410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.922+0000 7fe3c9519640 1 --2- 172.21.15.19:0/912707419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c4103b20 0x7fe3c410c000 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.922+0000 7fe3c9519640 1 --2- 172.21.15.19:0/912707419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c4103200 0x7fe3c41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.922+0000 7fe3c9519640 1 -- 172.21.15.19:0/912707419 >> 172.21.15.19:0/912707419 conn(0x7fe3c40771d0 msgr2=0x7fe3c40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.923+0000 7fe3c9519640 1 -- 172.21.15.19:0/912707419 shutdown_connections 2024-04-23T18:35:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.923+0000 7fe3c9519640 1 -- 172.21.15.19:0/912707419 wait complete. 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.923+0000 7fe3c9519640 1 Processor -- start 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.923+0000 7fe3c9519640 1 -- start start 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c9519640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c4103200 0x7fe3c41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c9519640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c4103b20 0x7fe3c4198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c9519640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c410c5d0 0x7fe3c419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c9519640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4112210 con 0x7fe3c4103200 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c9519640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4112090 con 0x7fe3c410c5d0 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c9519640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4112390 con 0x7fe3c4103b20 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c4103200 0x7fe3c41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c4103200 0x7fe3c41986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54692/0 (socket says 172.21.15.19:54692) 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c2ffd640 1 -- 172.21.15.19:0/293080466 learned_addr learned my addr 172.21.15.19:0/293080466 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c37fe640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c410c5d0 0x7fe3c419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c4103b20 0x7fe3c4198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c2ffd640 1 -- 172.21.15.19:0/293080466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c4103b20 msgr2=0x7fe3c4198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c2ffd640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c4103b20 0x7fe3c4198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c2ffd640 1 -- 172.21.15.19:0/293080466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c410c5d0 msgr2=0x7fe3c419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c2ffd640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c410c5d0 0x7fe3c419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c2ffd640 1 -- 172.21.15.19:0/293080466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c419d690 con 0x7fe3c4103200 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c4103b20 0x7fe3c4198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3c2ffd640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c4103200 0x7fe3c41986f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400d950 tx=0x7fe3b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.924+0000 7fe3abfff640 1 -- 172.21.15.19:0/293080466 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4017030 con 0x7fe3c4103200 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.925+0000 7fe3c9519640 1 -- 172.21.15.19:0/293080466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c419d980 con 0x7fe3c4103200 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.925+0000 7fe3abfff640 1 -- 172.21.15.19:0/293080466 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b4004020 con 0x7fe3c4103200 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.925+0000 7fe3abfff640 1 -- 172.21.15.19:0/293080466 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40047a0 con 0x7fe3c4103200 2024-04-23T18:35:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.925+0000 7fe3c9519640 1 -- 172.21.15.19:0/293080466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c41a5210 con 0x7fe3c4103200 2024-04-23T18:35:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.927+0000 7fe3abfff640 1 -- 172.21.15.19:0/293080466 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe3b4009a30 con 0x7fe3c4103200 2024-04-23T18:35:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.927+0000 7fe3cc22c740 1 Processor -- start 2024-04-23T18:35:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.927+0000 7fe3cc22c740 1 -- start start 2024-04-23T18:35:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.927+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eeabee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.927+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.927+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeacc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.927+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5595eea1f970 con 0x5595eeabf350 2024-04-23T18:35:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.927+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5595ee9bc4a0 con 0x5595eeabea30 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.927+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5595eea1b930 con 0x5595eeac9c10 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54694/0 (socket says 172.21.15.19:54694) 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c2ffd640 1 -- 172.21.15.19:0/3218746421 learned_addr learned my addr 172.21.15.19:0/3218746421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c37fe640 1 --2- 172.21.15.19:0/3218746421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eeabee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3abfff640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3ac0a3dc0 0x7fe3ac0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/3218746421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeacc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c37fe640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3ac0a3dc0 0x7fe3ac0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c37fe640 1 -- 172.21.15.19:0/3218746421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 msgr2=0x5595eeacc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c37fe640 1 --2- 172.21.15.19:0/3218746421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeacc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c37fe640 1 -- 172.21.15.19:0/3218746421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 msgr2=0x5595eeac96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3abfff640 1 -- 172.21.15.19:0/293080466 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7fe3b40d02e0 con 0x7fe3c4103200 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c37fe640 1 --2- 172.21.15.19:0/3218746421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c37fe640 1 -- 172.21.15.19:0/3218746421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595ee732210 con 0x5595eeabea30 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c2ffd640 1 --2- 172.21.15.19:0/3218746421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c37fe640 1 --2- 172.21.15.19:0/3218746421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eeabee10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc005fd0 tx=0x7fe3bc014e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec49e0271b1284c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.928+0000 7fe3c37fe640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3ac0a3dc0 0x7fe3ac0a6280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc01c000 tx=0x7fe3bc01e000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3a97fa640 1 -- 172.21.15.19:0/3218746421 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc023070 con 0x5595eeabea30 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3a97fa640 1 -- 172.21.15.19:0/3218746421 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3bc014030 con 0x5595eeabea30 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3a97fa640 1 -- 172.21.15.19:0/3218746421 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0138b0 con 0x5595eeabea30 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3cc22c740 1 -- 172.21.15.19:0/3218746421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 msgr2=0x5595eeabee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/3218746421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eeabee10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc005fd0 tx=0x7fe3bc014e30 comp rx=0 tx=0).stop 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3cc22c740 1 -- 172.21.15.19:0/3218746421 shutdown_connections 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/3218746421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeacc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/3218746421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/3218746421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eeabee10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3cc22c740 1 -- 172.21.15.19:0/3218746421 >> 172.21.15.19:0/3218746421 conn(0x5595eeabaf40 msgr2=0x5595eeabb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3cc22c740 1 -- 172.21.15.19:0/3218746421 shutdown_connections 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.929+0000 7fe3cc22c740 1 -- 172.21.15.19:0/3218746421 wait complete. 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3cc22c740 1 -- start start 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3cc22c740 1 Processor -- start 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabea30 0x5595eeac95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeabf350 0x5595eeac68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeac9c10 0x5595eeac6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5595eeacc7e0 con 0x5595eeabea30 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5595ee9ba0b0 con 0x5595eeac9c10 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5595eeacc960 con 0x5595eeabf350 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3cc22c740 10 client.0 _ll_register_callbacks cb 0x5595ee9a7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabea30 0x5595eeac95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabea30 0x5595eeac95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54696/0 (socket says 172.21.15.19:54696) 2024-04-23T18:35:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c2ffd640 1 -- 172.21.15.19:0/2083452418 learned_addr learned my addr 172.21.15.19:0/2083452418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c37fe640 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeac9c10 0x5595eeac6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeabf350 0x5595eeac68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c2ffd640 1 -- 172.21.15.19:0/2083452418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeabf350 msgr2=0x5595eeac68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c2ffd640 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeabf350 0x5595eeac68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c2ffd640 1 -- 172.21.15.19:0/2083452418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeac9c10 msgr2=0x5595eeac6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c2ffd640 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeac9c10 0x5595eeac6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c2ffd640 1 -- 172.21.15.19:0/2083452418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595ee7fe620 con 0x5595eeabea30 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.930+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeabf350 0x5595eeac68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3c2ffd640 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabea30 0x5595eeac95d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40a4940 tx=0x7fe3b4005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83b6851e7918bfe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2083452418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b409bb60 con 0x5595eeabea30 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2083452418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7fe3b4098870 con 0x5595eeabea30 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2083452418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3bc014030 con 0x5595eeabea30 2024-04-23T18:35:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2083452418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe3b409c900 con 0x5595eeabea30 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3cc22c740 10 client.16791 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2083452418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595eead87d0 con 0x5595eeabea30 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2083452418 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3bc014030 con 0x5595eeabea30 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2083452418 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe3b40cdc30 con 0x5595eeabea30 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3cc22c740 10 client.16791 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3cc22c740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.931+0000 7fe3cc22c740 20 client.16791 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.932+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5595eead9c90 0x5595eeadc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.932+0000 7fe3cc22c740 4 client.16791 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.932+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2083452418 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595eea2d390 con 0x5595eead9c90 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.932+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5595eead9c90 0x5595eeadc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.932+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5595eead9c90 0x5595eeadc080 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd6b941f6df14c8 server_cookie=22cde15f69d76975 in_seq=0 out_seq=0 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.932+0000 7fe3a6ff5640 10 client.16791 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2083452418 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595eea2d390 con 0x5595eead9c90 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3a6ff5640 10 client.16791 handle_command_reply: tid=0 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 client.16791 shutdown 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2083452418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabea30 msgr2=0x5595eeac95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabea30 0x5595eeac95d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40a4940 tx=0x7fe3b4005cd0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2083452418 shutdown_connections 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5595eead9c90 0x5595eeadc080 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeac9c10 0x5595eeac6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeabf350 0x5595eeac68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3a6ff5640 0 client.16791 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2083452418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabea30 0x5595eeac95d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2083452418 >> 172.21.15.19:0/2083452418 conn(0x5595eeabaf40 msgr2=0x5595eeac0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2083452418 shutdown_connections 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2083452418 wait complete. 2024-04-23T18:35:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.933+0000 7fe3cc22c740 20 client.16791 trim_cache size 0 max 16384 2024-04-23T18:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3cc22c740 1 Processor -- start 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3cc22c740 1 -- start start 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eead56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeae18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ab9f0 con 0x5595eeabf350 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ad530 con 0x5595eeabea30 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ac0c0 con 0x5595eeac9c10 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3c27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3c27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54708/0 (socket says 172.21.15.19:54708) 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3c27fc640 1 -- 172.21.15.19:0/2200865421 learned_addr learned my addr 172.21.15.19:0/2200865421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3c27fc640 1 -- 172.21.15.19:0/2200865421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 msgr2=0x5595eeae18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/2200865421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeae18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3c27fc640 1 -- 172.21.15.19:0/2200865421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 msgr2=0x5595eead56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/2200865421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eead56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3c27fc640 1 -- 172.21.15.19:0/2200865421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595eeac7240 con 0x5595eeabf350 2024-04-23T18:35:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/2200865421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac9100 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe3b802a9a0 tx=0x7fe3b8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6804eb5c964b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.949+0000 7fe3a8ff9640 1 -- 172.21.15.19:0/2200865421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8049070 con 0x5595eeabf350 2024-04-23T18:35:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3a8ff9640 1 -- 172.21.15.19:0/2200865421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b80054f0 con 0x5595eeabf350 2024-04-23T18:35:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3a8ff9640 1 -- 172.21.15.19:0/2200865421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8035040 con 0x5595eeabf350 2024-04-23T18:35:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2200865421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 msgr2=0x5595eeac9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2200865421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac9100 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe3b802a9a0 tx=0x7fe3b8005b50 comp rx=0 tx=0).stop 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2200865421 shutdown_connections 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2200865421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeae18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2200865421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeac9100 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2200865421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eead56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2200865421 >> 172.21.15.19:0/2200865421 conn(0x5595eeabaf40 msgr2=0x5595eeab5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2200865421 shutdown_connections 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2200865421 wait complete. 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.950+0000 7fe3cc22c740 1 -- start start 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3cc22c740 1 Processor -- start 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eead2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeacd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3cc22c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeacdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ace60 con 0x5595eeabf350 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ac790 con 0x5595eeabea30 2024-04-23T18:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3cc22c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41a99f0 con 0x5595eeac9c10 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3cc22c740 10 client.0 _ll_register_callbacks cb 0x7fe3b40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeacd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeacd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54720/0 (socket says 172.21.15.19:54720) 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c27fc640 1 -- 172.21.15.19:0/2288224015 learned_addr learned my addr 172.21.15.19:0/2288224015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c2ffd640 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eead2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c37fe640 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeacdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c27fc640 1 -- 172.21.15.19:0/2288224015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 msgr2=0x5595eeacdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeacdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c27fc640 1 -- 172.21.15.19:0/2288224015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 msgr2=0x5595eead2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eead2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c27fc640 1 -- 172.21.15.19:0/2288224015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595eeb3f160 con 0x5595eeabf350 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3c27fc640 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeacd490 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009230 tx=0x7fe3b8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6a519a1e801226 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.951+0000 7fe3a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2288224015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8049040 con 0x5595eeabf350 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2288224015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7fe3b80035f0 con 0x5595eeabf350 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2288224015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b80054f0 con 0x5595eeabf350 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2288224015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe3b8035040 con 0x5595eeabf350 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3cc22c740 10 client.16797 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2288224015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595eeb40310 con 0x5595eeabf350 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2288224015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b80054f0 con 0x5595eeabf350 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2288224015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe3b804c070 con 0x5595eeabf350 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3cc22c740 10 client.16797 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3cc22c740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3cc22c740 20 client.16797 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5595eeb409a0 0x5595eeb42dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3cc22c740 4 client.16797 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.952+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2288224015 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595eea27200 con 0x5595eeb409a0 2024-04-23T18:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.953+0000 7fe3c2ffd640 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5595eeb409a0 0x5595eeb42dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.953+0000 7fe3c2ffd640 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5595eeb409a0 0x5595eeb42dc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa62d27dd263b34 server_cookie=49dce4bd62e869a4 in_seq=0 out_seq=0 2024-04-23T18:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.953+0000 7fe3a6ff5640 10 client.16797 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.968+0000 7fe3a6ff5640 1 -- 172.21.15.19:0/2288224015 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595eea27200 con 0x5595eeb409a0 2024-04-23T18:35:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.968+0000 7fe3a6ff5640 10 client.16797 handle_command_reply: tid=0 2024-04-23T18:35:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.968+0000 7fe3cc22c740 1 client.16797 shutdown 2024-04-23T18:35:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2288224015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 msgr2=0x5595eeacd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeacd490 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009230 tx=0x7fe3b8004040 comp rx=0 tx=0).stop 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2288224015 shutdown_connections 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5595eeb409a0 0x5595eeb42dc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595eeac9c10 0x5595eeacdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595eeabf350 0x5595eeacd490 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3a6ff5640 0 client.16797 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 1 --2- 172.21.15.19:0/2288224015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595eeabea30 0x5595eead2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2288224015 >> 172.21.15.19:0/2288224015 conn(0x5595eeabaf40 msgr2=0x5595eead4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2288224015 shutdown_connections 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 1 -- 172.21.15.19:0/2288224015 wait complete. 2024-04-23T18:35:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3cc22c740 20 client.16797 trim_cache size 0 max 16384 2024-04-23T18:35:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:35:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:35:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:35:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:35:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3c9519640 1 -- 172.21.15.19:0/293080466 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3ac0a3dc0 msgr2=0x7fe3ac0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3c9519640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3ac0a3dc0 0x7fe3ac0a6280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc01c000 tx=0x7fe3bc01e000 comp rx=0 tx=0).stop 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3c9519640 1 -- 172.21.15.19:0/293080466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c4103200 msgr2=0x7fe3c41986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3c9519640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c4103200 0x7fe3c41986f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400d950 tx=0x7fe3b400de10 comp rx=0 tx=0).stop 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3c9519640 1 -- 172.21.15.19:0/293080466 shutdown_connections 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3c9519640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3ac0a3dc0 0x7fe3ac0a6280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3c9519640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3c410c5d0 0x7fe3c419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3c9519640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c4103b20 0x7fe3c4198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3c9519640 1 --2- 172.21.15.19:0/293080466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c4103200 0x7fe3c41986f0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.969+0000 7fe3c9519640 1 -- 172.21.15.19:0/293080466 >> 172.21.15.19:0/293080466 conn(0x7fe3c40771d0 msgr2=0x7fe3c40fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.970+0000 7fe3c9519640 1 -- 172.21.15.19:0/293080466 shutdown_connections 2024-04-23T18:35:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:09.970+0000 7fe3c9519640 1 -- 172.21.15.19:0/293080466 wait complete. 2024-04-23T18:35:10.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:09 smithi019 ceph-mon[29215]: pgmap v278: 97 pgs: 97 active+clean; 3.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-23T18:35:10.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:09 smithi044 ceph-mon[32458]: pgmap v278: 97 pgs: 97 active+clean; 3.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-23T18:35:10.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:09 smithi155 ceph-mon[33468]: pgmap v278: 97 pgs: 97 active+clean; 3.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-23T18:35:12.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:11 smithi019 ceph-mon[29215]: pgmap v279: 97 pgs: 97 active+clean; 4.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-23T18:35:12.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:11 smithi044 ceph-mon[32458]: pgmap v279: 97 pgs: 97 active+clean; 4.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-23T18:35:12.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:11 smithi155 ceph-mon[33468]: pgmap v279: 97 pgs: 97 active+clean; 4.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-23T18:35:14.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:13 smithi044 ceph-mon[32458]: pgmap v280: 97 pgs: 97 active+clean; 4.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-23T18:35:14.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:13 smithi044 ceph-mon[32458]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:35:14.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:13 smithi155 ceph-mon[33468]: pgmap v280: 97 pgs: 97 active+clean; 4.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-23T18:35:14.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:13 smithi155 ceph-mon[33468]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:35:14.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:13 smithi019 ceph-mon[29215]: pgmap v280: 97 pgs: 97 active+clean; 4.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-23T18:35:14.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:13 smithi019 ceph-mon[29215]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:35:16.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:15 smithi044 ceph-mon[32458]: pgmap v282: 97 pgs: 97 active+clean; 4.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-04-23T18:35:16.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:15 smithi155 ceph-mon[33468]: pgmap v282: 97 pgs: 97 active+clean; 4.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-04-23T18:35:16.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:15 smithi019 ceph-mon[29215]: pgmap v282: 97 pgs: 97 active+clean; 4.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-04-23T18:35:18.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:17 smithi044 ceph-mon[32458]: pgmap v283: 97 pgs: 97 active+clean; 4.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:35:18.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:17 smithi155 ceph-mon[33468]: pgmap v283: 97 pgs: 97 active+clean; 4.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:35:18.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:17 smithi019 ceph-mon[29215]: pgmap v283: 97 pgs: 97 active+clean; 4.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:35:20.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:19 smithi044 ceph-mon[32458]: pgmap v284: 97 pgs: 97 active+clean; 4.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:35:20.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:19 smithi155 ceph-mon[33468]: pgmap v284: 97 pgs: 97 active+clean; 4.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:35:20.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:19 smithi019 ceph-mon[29215]: pgmap v284: 97 pgs: 97 active+clean; 4.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:35:21.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:20 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:35:21.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:20 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:35:21.258 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:20 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:35:22.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:21 smithi044 ceph-mon[32458]: pgmap v285: 97 pgs: 97 active+clean; 4.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-23T18:35:22.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:21 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:35:22.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:21 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:35:22.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:21 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:35:22.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:21 smithi155 ceph-mon[33468]: pgmap v285: 97 pgs: 97 active+clean; 4.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-23T18:35:22.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:21 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:35:22.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:21 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:35:22.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:21 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:35:22.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:21 smithi019 ceph-mon[29215]: pgmap v285: 97 pgs: 97 active+clean; 4.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-23T18:35:22.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:21 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:35:22.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:21 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:35:22.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:21 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:35:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.069+0000 7fbefca69640 1 -- 172.21.15.19:0/2637956480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbef8102b70 msgr2=0x7fbef8106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.069+0000 7fbefca69640 1 --2- 172.21.15.19:0/2637956480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbef8102b70 0x7fbef8106bb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009900 tx=0x7fbee002a410 comp rx=0 tx=0).stop 2024-04-23T18:35:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.069+0000 7fbefca69640 1 -- 172.21.15.19:0/2637956480 shutdown_connections 2024-04-23T18:35:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.069+0000 7fbefca69640 1 --2- 172.21.15.19:0/2637956480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef8107290 0x7fbef810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.069+0000 7fbefca69640 1 --2- 172.21.15.19:0/2637956480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbef8102b70 0x7fbef8106bb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.069+0000 7fbefca69640 1 --2- 172.21.15.19:0/2637956480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef81021c0 0x7fbef81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.069+0000 7fbefca69640 1 -- 172.21.15.19:0/2637956480 >> 172.21.15.19:0/2637956480 conn(0x7fbef80fda20 msgr2=0x7fbef80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.069+0000 7fbefca69640 1 -- 172.21.15.19:0/2637956480 shutdown_connections 2024-04-23T18:35:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.069+0000 7fbefca69640 1 -- 172.21.15.19:0/2637956480 wait complete. 2024-04-23T18:35:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.070+0000 7fbefca69640 1 Processor -- start 2024-04-23T18:35:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.070+0000 7fbefca69640 1 -- start start 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbefca69640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbef81021c0 0x7fbef81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbefca69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef8107290 0x7fbef8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbefca69640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef819cfa0 0x7fbef819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbefca69640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbef8112220 con 0x7fbef81021c0 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbefca69640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81120a0 con 0x7fbef8107290 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbefca69640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81123a0 con 0x7fbef819cfa0 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbef659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbef81021c0 0x7fbef81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbef659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbef81021c0 0x7fbef81986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60414/0 (socket says 172.21.15.19:60414) 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbef659c640 1 -- 172.21.15.19:0/1016283315 learned_addr learned my addr 172.21.15.19:0/1016283315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbeeffff640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef8107290 0x7fbef8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef819cfa0 0x7fbef819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbef659c640 1 -- 172.21.15.19:0/1016283315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef819cfa0 msgr2=0x7fbef819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbef659c640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef819cfa0 0x7fbef819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbef659c640 1 -- 172.21.15.19:0/1016283315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef8107290 msgr2=0x7fbef8198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbef659c640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef8107290 0x7fbef8198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbef659c640 1 -- 172.21.15.19:0/1016283315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef819da90 con 0x7fbef81021c0 2024-04-23T18:35:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbef659c640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbef81021c0 0x7fbef81986e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbee800e9f0 tx=0x7fbee800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbeef7fe640 1 -- 172.21.15.19:0/1016283315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee800cd80 con 0x7fbef81021c0 2024-04-23T18:35:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbefca69640 1 -- 172.21.15.19:0/1016283315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef819dd80 con 0x7fbef81021c0 2024-04-23T18:35:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbeef7fe640 1 -- 172.21.15.19:0/1016283315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee80054d0 con 0x7fbef81021c0 2024-04-23T18:35:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.071+0000 7fbeef7fe640 1 -- 172.21.15.19:0/1016283315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8010690 con 0x7fbef81021c0 2024-04-23T18:35:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.072+0000 7fbefca69640 1 -- 172.21.15.19:0/1016283315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef81a54f0 con 0x7fbef81021c0 2024-04-23T18:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.074+0000 7fbeef7fe640 1 -- 172.21.15.19:0/1016283315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbee8004e90 con 0x7fbef81021c0 2024-04-23T18:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.074+0000 7fbeff776740 1 Processor -- start 2024-04-23T18:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.074+0000 7fbeff776740 1 -- start start 2024-04-23T18:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeff776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3e520 0x55f6c0e3e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeff776740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e4d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeff776740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e50000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeff776740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f6c0da3970 con 0x55f6c0e4dc10 2024-04-23T18:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeff776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f6c0d404a0 con 0x55f6c0e3e520 2024-04-23T18:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeff776740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f6c0d9f930 con 0x55f6c0e3ee40 2024-04-23T18:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeef7fe640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbee40a3dc0 0x7fbee40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e50000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e50000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60416/0 (socket says 172.21.15.19:60416) 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeeffff640 1 -- 172.21.15.19:0/2035042282 learned_addr learned my addr 172.21.15.19:0/2035042282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbef659c640 1 --2- 172.21.15.19:0/2035042282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e4d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbee40a3dc0 0x7fbee40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeef7fe640 1 -- 172.21.15.19:0/1016283315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fbee80d0d10 con 0x7fbef81021c0 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/2035042282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3e520 0x55f6c0e3e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeeffff640 1 -- 172.21.15.19:0/2035042282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3ee40 msgr2=0x55f6c0e4d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeeffff640 1 --2- 172.21.15.19:0/2035042282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e4d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeeffff640 1 -- 172.21.15.19:0/2035042282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3e520 msgr2=0x55f6c0e3e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeeffff640 1 --2- 172.21.15.19:0/2035042282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3e520 0x55f6c0e3e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeeffff640 1 -- 172.21.15.19:0/2035042282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c0ab6210 con 0x55f6c0e4dc10 2024-04-23T18:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/2035042282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3e520 0x55f6c0e3e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.075+0000 7fbeeffff640 1 --2- 172.21.15.19:0/2035042282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e50000 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbee0006010 tx=0x7fbee0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f913f25934e2c06 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbee40a3dc0 0x7fbee40a6280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbef0005fd0 tx=0x7fbef0009330 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbeecff9640 1 -- 172.21.15.19:0/2035042282 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0035040 con 0x55f6c0e4dc10 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbeecff9640 1 -- 172.21.15.19:0/2035042282 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee00054f0 con 0x55f6c0e4dc10 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbeecff9640 1 -- 172.21.15.19:0/2035042282 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00037f0 con 0x55f6c0e4dc10 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbeff776740 1 -- 172.21.15.19:0/2035042282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e4dc10 msgr2=0x55f6c0e50000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbeff776740 1 --2- 172.21.15.19:0/2035042282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e50000 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbee0006010 tx=0x7fbee0004170 comp rx=0 tx=0).stop 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbeff776740 1 -- 172.21.15.19:0/2035042282 shutdown_connections 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbeff776740 1 --2- 172.21.15.19:0/2035042282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e50000 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbeff776740 1 --2- 172.21.15.19:0/2035042282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e4d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbeff776740 1 --2- 172.21.15.19:0/2035042282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3e520 0x55f6c0e3e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.076+0000 7fbeff776740 1 -- 172.21.15.19:0/2035042282 >> 172.21.15.19:0/2035042282 conn(0x55f6c0e3ab30 msgr2=0x55f6c0e3af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 1 -- 172.21.15.19:0/2035042282 shutdown_connections 2024-04-23T18:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 1 -- 172.21.15.19:0/2035042282 wait complete. 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 1 -- start start 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 1 Processor -- start 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e4d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e4a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e4acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f6c0e507e0 con 0x55f6c0e3e520 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f6c0d3e0b0 con 0x55f6c0e4dc10 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f6c0e50960 con 0x55f6c0e3ee40 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeff776740 10 client.0 _ll_register_callbacks cb 0x55f6c0d2bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbef659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e4d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbef659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e4d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60422/0 (socket says 172.21.15.19:60422) 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbef659c640 1 -- 172.21.15.19:0/1797542248 learned_addr learned my addr 172.21.15.19:0/1797542248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.077+0000 7fbeeffff640 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e4a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e4acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbef659c640 1 -- 172.21.15.19:0/1797542248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3ee40 msgr2=0x55f6c0e4a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbef659c640 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e4a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbef659c640 1 -- 172.21.15.19:0/1797542248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 msgr2=0x55f6c0e4acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbef659c640 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e4acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbef659c640 1 -- 172.21.15.19:0/1797542248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c0b82620 con 0x55f6c0e3e520 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e4acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbef659c640 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e4d4c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fbee80097d0 tx=0x7fbee801c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7310bf7c54d5cdc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbeddffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbeddffb640 1 -- 172.21.15.19:0/1797542248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee80b6070 con 0x55f6c0e3e520 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbeddffb640 1 -- 172.21.15.19:0/1797542248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fbee80bd0b0 con 0x55f6c0e3e520 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbeff776740 1 -- 172.21.15.19:0/1797542248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee00054f0 con 0x55f6c0e3e520 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbeddffb640 1 -- 172.21.15.19:0/1797542248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbee80ce880 con 0x55f6c0e3e520 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbeff776740 10 client.16809 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbeff776740 1 -- 172.21.15.19:0/1797542248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f6c0e5c730 con 0x55f6c0e3e520 2024-04-23T18:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.078+0000 7fbeddffb640 1 -- 172.21.15.19:0/1797542248 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee00054f0 con 0x55f6c0e3e520 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.079+0000 7fbeddffb640 1 -- 172.21.15.19:0/1797542248 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbee80973e0 con 0x55f6c0e3e520 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.079+0000 7fbeff776740 10 client.16809 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.079+0000 7fbeff776740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.079+0000 7fbeff776740 20 client.16809 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.079+0000 7fbeff776740 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f6c0e5dba0 0x55f6c0e5ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.079+0000 7fbeff776740 4 client.16809 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.079+0000 7fbeff776740 1 -- 172.21.15.19:0/1797542248 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6c0db1390 con 0x55f6c0e5dba0 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.079+0000 7fbeeffff640 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f6c0e5dba0 0x55f6c0e5ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.079+0000 7fbeddffb640 10 client.16809 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.079+0000 7fbeeffff640 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f6c0e5dba0 0x55f6c0e5ffd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc9f268b16bda78f server_cookie=52a2b96e27ee3f28 in_seq=0 out_seq=0 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.080+0000 7fbeddffb640 1 -- 172.21.15.19:0/1797542248 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6c0db1390 con 0x55f6c0e5dba0 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.080+0000 7fbeddffb640 10 client.16809 handle_command_reply: tid=0 2024-04-23T18:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.080+0000 7fbeff776740 1 client.16809 shutdown 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 1 -- 172.21.15.19:0/1797542248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 msgr2=0x55f6c0e4d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e4d4c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fbee80097d0 tx=0x7fbee801c590 comp rx=0 tx=0).stop 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 1 -- 172.21.15.19:0/1797542248 shutdown_connections 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f6c0e5dba0 0x55f6c0e5ffd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e4acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e4a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeddffb640 0 client.16809 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 1 --2- 172.21.15.19:0/1797542248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e4d4c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 1 -- 172.21.15.19:0/1797542248 >> 172.21.15.19:0/1797542248 conn(0x55f6c0e3ab30 msgr2=0x55f6c0e409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 1 -- 172.21.15.19:0/1797542248 shutdown_connections 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 1 -- 172.21.15.19:0/1797542248 wait complete. 2024-04-23T18:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.081+0000 7fbeff776740 20 client.16809 trim_cache size 0 max 16384 2024-04-23T18:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeff776740 1 Processor -- start 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeff776740 1 -- start start 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeff776740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3e520 0x55f6c0e596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeff776740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e45d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeff776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e4cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeff776740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81ab8f0 con 0x55f6c0e3ee40 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeff776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81ad430 con 0x55f6c0e4dc10 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeff776740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81abfc0 con 0x55f6c0e3e520 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e45d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e45d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60432/0 (socket says 172.21.15.19:60432) 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeeffff640 1 -- 172.21.15.19:0/4135178172 learned_addr learned my addr 172.21.15.19:0/4135178172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/4135178172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e4cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeeffff640 1 -- 172.21.15.19:0/4135178172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3e520 msgr2=0x55f6c0e596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeeffff640 1 --2- 172.21.15.19:0/4135178172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3e520 0x55f6c0e596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeeffff640 1 -- 172.21.15.19:0/4135178172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 msgr2=0x55f6c0e4cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeeffff640 1 --2- 172.21.15.19:0/4135178172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e4cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbeeffff640 1 -- 172.21.15.19:0/4135178172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c0e3a960 con 0x55f6c0e3ee40 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.096+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/4135178172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e4cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbeeffff640 1 --2- 172.21.15.19:0/4135178172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e45d80 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbee0031440 tx=0x7fbee0004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fc657b76e4d8a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbedffff640 1 -- 172.21.15.19:0/4135178172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0035040 con 0x55f6c0e3ee40 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbedffff640 1 -- 172.21.15.19:0/4135178172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee00048c0 con 0x55f6c0e3ee40 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbedffff640 1 -- 172.21.15.19:0/4135178172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0004c70 con 0x55f6c0e3ee40 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbeff776740 1 -- 172.21.15.19:0/4135178172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3ee40 msgr2=0x55f6c0e45d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbeff776740 1 --2- 172.21.15.19:0/4135178172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e45d80 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbee0031440 tx=0x7fbee0004c00 comp rx=0 tx=0).stop 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbeff776740 1 -- 172.21.15.19:0/4135178172 shutdown_connections 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbeff776740 1 --2- 172.21.15.19:0/4135178172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e4cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbeff776740 1 --2- 172.21.15.19:0/4135178172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e45d80 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbeff776740 1 --2- 172.21.15.19:0/4135178172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e3e520 0x55f6c0e596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbeff776740 1 -- 172.21.15.19:0/4135178172 >> 172.21.15.19:0/4135178172 conn(0x55f6c0e3ab30 msgr2=0x55f6c0e45670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbeff776740 1 -- 172.21.15.19:0/4135178172 shutdown_connections 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.097+0000 7fbeff776740 1 -- 172.21.15.19:0/4135178172 wait complete. 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbeff776740 1 -- start start 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbeff776740 1 Processor -- start 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbeff776740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e55f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbeff776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e47110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbeff776740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e47650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbeff776740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81acd60 con 0x55f6c0e3e520 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbeff776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81ac690 con 0x55f6c0e3ee40 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbeff776740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81a98f0 con 0x55f6c0e4dc10 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbeff776740 10 client.0 _ll_register_callbacks cb 0x7fbee809cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbef659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e55f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbef659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e55f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60434/0 (socket says 172.21.15.19:60434) 2024-04-23T18:35:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbef659c640 1 -- 172.21.15.19:0/3647911737 learned_addr learned my addr 172.21.15.19:0/3647911737 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbeeffff640 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e47110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e47650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbef659c640 1 -- 172.21.15.19:0/3647911737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e4dc10 msgr2=0x55f6c0e47650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbef659c640 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e47650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbef659c640 1 -- 172.21.15.19:0/3647911737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3ee40 msgr2=0x55f6c0e47110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.098+0000 7fbef659c640 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e47110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbef659c640 1 -- 172.21.15.19:0/3647911737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c0e47e80 con 0x55f6c0e3e520 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbef6d9d640 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e47650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbef659c640 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e55f50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c6a0 tx=0x7fbee80975f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22539490312f4822 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeddffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeddffb640 1 -- 172.21.15.19:0/3647911737 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee809c5b0 con 0x55f6c0e3e520 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeddffb640 1 -- 172.21.15.19:0/3647911737 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fbee80a5430 con 0x55f6c0e3e520 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeff776740 1 -- 172.21.15.19:0/3647911737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee00048c0 con 0x55f6c0e3e520 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeddffb640 1 -- 172.21.15.19:0/3647911737 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbee80a5910 con 0x55f6c0e3e520 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeff776740 10 client.16815 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeff776740 1 -- 172.21.15.19:0/3647911737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f6c0e47e80 con 0x55f6c0e3e520 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeddffb640 1 -- 172.21.15.19:0/3647911737 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee00048c0 con 0x55f6c0e3e520 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeddffb640 1 -- 172.21.15.19:0/3647911737 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbee809c0a0 con 0x55f6c0e3e520 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeff776740 10 client.16815 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeff776740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeff776740 20 client.16815 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeff776740 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f6c0ec4430 0x55f6c0ec6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeff776740 4 client.16815 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.099+0000 7fbeff776740 1 -- 172.21.15.19:0/3647911737 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6c0dab200 con 0x55f6c0ec4430 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.100+0000 7fbeeffff640 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f6c0ec4430 0x55f6c0ec6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.100+0000 7fbeeffff640 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f6c0ec4430 0x55f6c0ec6820 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96503a1a00ee7f3e server_cookie=73a7c623c8e57149 in_seq=0 out_seq=0 2024-04-23T18:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.100+0000 7fbeddffb640 10 client.16815 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.115+0000 7fbeddffb640 1 -- 172.21.15.19:0/3647911737 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6c0dab200 con 0x55f6c0ec4430 2024-04-23T18:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.115+0000 7fbeddffb640 10 client.16815 handle_command_reply: tid=0 2024-04-23T18:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.115+0000 7fbeff776740 1 client.16815 shutdown 2024-04-23T18:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.115+0000 7fbeff776740 1 -- 172.21.15.19:0/3647911737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 msgr2=0x55f6c0e55f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.115+0000 7fbeff776740 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e55f50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c6a0 tx=0x7fbee80975f0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbeff776740 1 -- 172.21.15.19:0/3647911737 shutdown_connections 2024-04-23T18:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbeff776740 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f6c0ec4430 0x55f6c0ec6820 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbeff776740 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6c0e3ee40 0x55f6c0e47110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbeff776740 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6c0e4dc10 0x55f6c0e47650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbeff776740 1 --2- 172.21.15.19:0/3647911737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c0e3e520 0x55f6c0e55f50 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbeddffb640 0 client.16815 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbeff776740 1 -- 172.21.15.19:0/3647911737 >> 172.21.15.19:0/3647911737 conn(0x55f6c0e3ab30 msgr2=0x55f6c0e39420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbeff776740 1 -- 172.21.15.19:0/3647911737 shutdown_connections 2024-04-23T18:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbeff776740 1 -- 172.21.15.19:0/3647911737 wait complete. 2024-04-23T18:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbeff776740 20 client.16815 trim_cache size 0 max 16384 2024-04-23T18:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbefca69640 1 -- 172.21.15.19:0/1016283315 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbee40a3dc0 msgr2=0x7fbee40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbefca69640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbee40a3dc0 0x7fbee40a6280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbef0005fd0 tx=0x7fbef0009330 comp rx=0 tx=0).stop 2024-04-23T18:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbefca69640 1 -- 172.21.15.19:0/1016283315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbef81021c0 msgr2=0x7fbef81986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbefca69640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbef81021c0 0x7fbef81986e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbee800e9f0 tx=0x7fbee800edf0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbefca69640 1 -- 172.21.15.19:0/1016283315 shutdown_connections 2024-04-23T18:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbefca69640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbee40a3dc0 0x7fbee40a6280 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbefca69640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef8107290 0x7fbef8198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbefca69640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef819cfa0 0x7fbef819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbefca69640 1 --2- 172.21.15.19:0/1016283315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbef81021c0 0x7fbef81986e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.116+0000 7fbefca69640 1 -- 172.21.15.19:0/1016283315 >> 172.21.15.19:0/1016283315 conn(0x7fbef80fda20 msgr2=0x7fbef80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.117+0000 7fbefca69640 1 -- 172.21.15.19:0/1016283315 shutdown_connections 2024-04-23T18:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:23.117+0000 7fbefca69640 1 -- 172.21.15.19:0/1016283315 wait complete. 2024-04-23T18:35:24.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:23 smithi044 ceph-mon[32458]: pgmap v286: 97 pgs: 97 active+clean; 4.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-23T18:35:24.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:23 smithi155 ceph-mon[33468]: pgmap v286: 97 pgs: 97 active+clean; 4.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-23T18:35:24.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:23 smithi019 ceph-mon[29215]: pgmap v286: 97 pgs: 97 active+clean; 4.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-23T18:35:26.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:25 smithi044 ceph-mon[32458]: pgmap v287: 97 pgs: 97 active+clean; 4.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-23T18:35:26.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:25 smithi155 ceph-mon[33468]: pgmap v287: 97 pgs: 97 active+clean; 4.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-23T18:35:26.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:25 smithi019 ceph-mon[29215]: pgmap v287: 97 pgs: 97 active+clean; 4.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-23T18:35:26.318 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:35:26.318 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.8 2024-04-23T18:35:26.318 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:35:26.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3283682085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb508074320 msgr2=0x7fb508074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 --2- 172.21.15.19:0/3283682085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb508074320 0x7fb508074700 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009900 tx=0x7fb4f802a480 comp rx=0 tx=0).stop 2024-04-23T18:35:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3283682085 shutdown_connections 2024-04-23T18:35:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 --2- 172.21.15.19:0/3283682085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508074c40 0x7fb50810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 --2- 172.21.15.19:0/3283682085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50810e7c0 0x7fb508110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 --2- 172.21.15.19:0/3283682085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb508074320 0x7fb508074700 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3283682085 >> 172.21.15.19:0/3283682085 conn(0x7fb508079310 msgr2=0x7fb508079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3283682085 shutdown_connections 2024-04-23T18:35:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3283682085 wait complete. 2024-04-23T18:35:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 Processor -- start 2024-04-23T18:35:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.442+0000 7fb50cd7c640 1 -- start start 2024-04-23T18:35:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb50cd7c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508074320 0x7fb5081a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb50cd7c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb508074c40 0x7fb5081a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb50cd7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb50810e7c0 0x7fb50819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb50cd7c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081143e0 con 0x7fb50810e7c0 2024-04-23T18:35:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb50cd7c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb508114260 con 0x7fb508074320 2024-04-23T18:35:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb50cd7c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb508114560 con 0x7fb508074c40 2024-04-23T18:35:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb506d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb50810e7c0 0x7fb50819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb506d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb50810e7c0 0x7fb50819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60458/0 (socket says 172.21.15.19:60458) 2024-04-23T18:35:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb506d9d640 1 -- 172.21.15.19:0/3865604135 learned_addr learned my addr 172.21.15.19:0/3865604135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb4fffff640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb508074c40 0x7fb5081a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb50659c640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508074320 0x7fb5081a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb506d9d640 1 -- 172.21.15.19:0/3865604135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb508074c40 msgr2=0x7fb5081a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb506d9d640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb508074c40 0x7fb5081a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb506d9d640 1 -- 172.21.15.19:0/3865604135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508074320 msgr2=0x7fb5081a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb506d9d640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508074320 0x7fb5081a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb506d9d640 1 -- 172.21.15.19:0/3865604135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50819b000 con 0x7fb50810e7c0 2024-04-23T18:35:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb50659c640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508074320 0x7fb5081a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.443+0000 7fb506d9d640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb50810e7c0 0x7fb50819a740 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb50000cce0 tx=0x7fb50000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.444+0000 7fb4ff7fe640 1 -- 172.21.15.19:0/3865604135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50000f450 con 0x7fb50810e7c0 2024-04-23T18:35:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.444+0000 7fb4ff7fe640 1 -- 172.21.15.19:0/3865604135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb50000faa0 con 0x7fb50810e7c0 2024-04-23T18:35:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.444+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3865604135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50819b2f0 con 0x7fb50810e7c0 2024-04-23T18:35:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.444+0000 7fb4ff7fe640 1 -- 172.21.15.19:0/3865604135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500018af0 con 0x7fb50810e7c0 2024-04-23T18:35:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.444+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3865604135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5081a7600 con 0x7fb50810e7c0 2024-04-23T18:35:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.445+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3865604135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5080759d0 con 0x7fb50810e7c0 2024-04-23T18:35:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.446+0000 7fb4ff7fe640 1 -- 172.21.15.19:0/3865604135 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb50000fc50 con 0x7fb50810e7c0 2024-04-23T18:35:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.446+0000 7fb4ff7fe640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb4f40a3d20 0x7fb4f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.447+0000 7fb50659c640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb4f40a3d20 0x7fb4f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.447+0000 7fb4ff7fe640 1 -- 172.21.15.19:0/3865604135 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb5000d0f90 con 0x7fb50810e7c0 2024-04-23T18:35:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.447+0000 7fb50659c640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb4f40a3d20 0x7fb4f40a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8031440 tx=0x7fb4f8005a20 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.450+0000 7fb4ff7fe640 1 -- 172.21.15.19:0/3865604135 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb500014030 con 0x7fb50810e7c0 2024-04-23T18:35:26.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.676+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3865604135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5080691c0 con 0x7fb50810e7c0 2024-04-23T18:35:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.677+0000 7fb4ff7fe640 1 -- 172.21.15.19:0/3865604135 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7fb50009d660 con 0x7fb50810e7c0 2024-04-23T18:35:26.669 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:35:26.670 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:35:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:35:26.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3865604135 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb4f40a3d20 msgr2=0x7fb4f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb4f40a3d20 0x7fb4f40a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8031440 tx=0x7fb4f8005a20 comp rx=0 tx=0).stop 2024-04-23T18:35:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3865604135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb50810e7c0 msgr2=0x7fb50819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb50810e7c0 0x7fb50819a740 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb50000cce0 tx=0x7fb50000a620 comp rx=0 tx=0).stop 2024-04-23T18:35:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3865604135 shutdown_connections 2024-04-23T18:35:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb4f40a3d20 0x7fb4f40a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb50810e7c0 0x7fb50819a740 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb508074c40 0x7fb5081a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 --2- 172.21.15.19:0/3865604135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508074320 0x7fb5081a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3865604135 >> 172.21.15.19:0/3865604135 conn(0x7fb508079310 msgr2=0x7fb508100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3865604135 shutdown_connections 2024-04-23T18:35:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.679+0000 7fb50cd7c640 1 -- 172.21.15.19:0/3865604135 wait complete. 2024-04-23T18:35:26.682 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 78.8 --await-for 42.0' 2024-04-23T18:35:26.683 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 78.8 --await-for 42.0 2024-04-23T18:35:26.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.805+0000 7fb77962c640 1 -- 172.21.15.19:0/2565403738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb774106190 msgr2=0x7fb77410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.805+0000 7fb77962c640 1 --2- 172.21.15.19:0/2565403738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb774106190 0x7fb77410dd60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009ca0 tx=0x7fb76c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.805+0000 7fb77962c640 1 -- 172.21.15.19:0/2565403738 shutdown_connections 2024-04-23T18:35:26.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.805+0000 7fb77962c640 1 --2- 172.21.15.19:0/2565403738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb774106190 0x7fb77410dd60 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.805+0000 7fb77962c640 1 --2- 172.21.15.19:0/2565403738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb774101ac0 0x7fb774105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.805+0000 7fb77962c640 1 --2- 172.21.15.19:0/2565403738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774101110 0x7fb7741014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.805+0000 7fb77962c640 1 -- 172.21.15.19:0/2565403738 >> 172.21.15.19:0/2565403738 conn(0x7fb774077fb0 msgr2=0x7fb7740783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.805+0000 7fb77962c640 1 -- 172.21.15.19:0/2565403738 shutdown_connections 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.805+0000 7fb77962c640 1 -- 172.21.15.19:0/2565403738 wait complete. 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb77962c640 1 Processor -- start 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb77962c640 1 -- start start 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb77962c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774101110 0x7fb774102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb77962c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb774106190 0x7fb774103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb77962c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7741035d0 0x7fb7741a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb77962c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb774111100 con 0x7fb774106190 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb77962c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb774110f80 con 0x7fb774101110 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb77962c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb774111280 con 0x7fb7741035d0 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb7727fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb774106190 0x7fb774103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb7727fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb774106190 0x7fb774103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60478/0 (socket says 172.21.15.19:60478) 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb7727fc640 1 -- 172.21.15.19:0/1551761706 learned_addr learned my addr 172.21.15.19:0/1551761706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb772ffd640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774101110 0x7fb774102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.806+0000 7fb7737fe640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7741035d0 0x7fb7741a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb7727fc640 1 -- 172.21.15.19:0/1551761706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7741035d0 msgr2=0x7fb7741a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb7727fc640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7741035d0 0x7fb7741a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb7727fc640 1 -- 172.21.15.19:0/1551761706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774101110 msgr2=0x7fb774102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb7727fc640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774101110 0x7fb774102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb7727fc640 1 -- 172.21.15.19:0/1551761706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7741a7010 con 0x7fb774106190 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb7727fc640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb774106190 0x7fb774103090 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e9f0 tx=0x7fb76800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb75bfff640 1 -- 172.21.15.19:0/1551761706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76800ccc0 con 0x7fb774106190 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb77962c640 1 -- 172.21.15.19:0/1551761706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7741a7300 con 0x7fb774106190 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb75bfff640 1 -- 172.21.15.19:0/1551761706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7680054d0 con 0x7fb774106190 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb75bfff640 1 -- 172.21.15.19:0/1551761706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768010430 con 0x7fb774106190 2024-04-23T18:35:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.807+0000 7fb77962c640 1 -- 172.21.15.19:0/1551761706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7741a7800 con 0x7fb774106190 2024-04-23T18:35:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb75bfff640 1 -- 172.21.15.19:0/1551761706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb7680046c0 con 0x7fb774106190 2024-04-23T18:35:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb77c416740 1 Processor -- start 2024-04-23T18:35:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb77c416740 1 -- start start 2024-04-23T18:35:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb77c416740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c0a0 0x564ca293c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb77c416740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c9c0 0x564ca2946b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb77c416740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 0x564ca2949480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb77c416740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ca28a1370 con 0x564ca293c9c0 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb77c416740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564ca283e2e0 con 0x564ca2947090 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb77c416740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564ca289d330 con 0x564ca293c0a0 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb772ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c9c0 0x564ca2946b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb772ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c9c0 0x564ca2946b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60484/0 (socket says 172.21.15.19:60484) 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb772ffd640 1 -- 172.21.15.19:0/1949005715 learned_addr learned my addr 172.21.15.19:0/1949005715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.810+0000 7fb7737fe640 1 --2- 172.21.15.19:0/1949005715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c0a0 0x564ca293c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb75bfff640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb75c0a3d50 0x7fb75c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb7727fc640 1 --2- 172.21.15.19:0/1949005715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 0x564ca2949480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb7737fe640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb75c0a3d50 0x7fb75c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb772ffd640 1 -- 172.21.15.19:0/1949005715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c0a0 msgr2=0x564ca293c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb772ffd640 1 --2- 172.21.15.19:0/1949005715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c0a0 0x564ca293c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb772ffd640 1 -- 172.21.15.19:0/1949005715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 msgr2=0x564ca2949480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb772ffd640 1 --2- 172.21.15.19:0/1949005715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 0x564ca2949480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb772ffd640 1 -- 172.21.15.19:0/1949005715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca267fd80 con 0x564ca293c9c0 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb7727fc640 1 --2- 172.21.15.19:0/1949005715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 0x564ca2949480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb75bfff640 1 -- 172.21.15.19:0/1551761706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb7680d0ad0 con 0x7fb774106190 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb772ffd640 1 --2- 172.21.15.19:0/1949005715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c9c0 0x564ca2946b50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb764009cf0 tx=0x7fb764011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7916b71620433475 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb7737fe640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb75c0a3d50 0x7fb75c0a6210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb76c006010 tx=0x7fb76c02a8c0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb7597fa640 1 -- 172.21.15.19:0/1949005715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764004230 con 0x564ca293c9c0 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb7597fa640 1 -- 172.21.15.19:0/1949005715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7640043d0 con 0x564ca293c9c0 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.811+0000 7fb7597fa640 1 -- 172.21.15.19:0/1949005715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764019880 con 0x564ca293c9c0 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 -- 172.21.15.19:0/1949005715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c9c0 msgr2=0x564ca2946b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 --2- 172.21.15.19:0/1949005715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c9c0 0x564ca2946b50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb764009cf0 tx=0x7fb764011810 comp rx=0 tx=0).stop 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 -- 172.21.15.19:0/1949005715 shutdown_connections 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 --2- 172.21.15.19:0/1949005715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 0x564ca2949480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 --2- 172.21.15.19:0/1949005715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c9c0 0x564ca2946b50 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 --2- 172.21.15.19:0/1949005715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c0a0 0x564ca293c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 -- 172.21.15.19:0/1949005715 >> 172.21.15.19:0/1949005715 conn(0x564ca29383c0 msgr2=0x564ca29387a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 -- 172.21.15.19:0/1949005715 shutdown_connections 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 -- 172.21.15.19:0/1949005715 wait complete. 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 -- start start 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.812+0000 7fb77c416740 1 Processor -- start 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb77c416740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2943fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb77c416740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c9c0 0x564ca29444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb77c416740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 0x564ca2944a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb77c416740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ca2949e80 con 0x564ca293c0a0 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb77c416740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564ca283bfb0 con 0x564ca2947090 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb77c416740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564ca294a000 con 0x564ca293c9c0 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb77c416740 10 client.0 _ll_register_callbacks cb 0x564ca283b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb772ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2943fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb772ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2943fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60490/0 (socket says 172.21.15.19:60490) 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb772ffd640 1 -- 172.21.15.19:0/2047193691 learned_addr learned my addr 172.21.15.19:0/2047193691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb7737fe640 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 0x564ca2944a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb772ffd640 1 -- 172.21.15.19:0/2047193691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c9c0 msgr2=0x564ca29444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb772ffd640 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c9c0 0x564ca29444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb772ffd640 1 -- 172.21.15.19:0/2047193691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 msgr2=0x564ca2944a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb772ffd640 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 0x564ca2944a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb772ffd640 1 -- 172.21.15.19:0/2047193691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ca26e0510 con 0x564ca293c0a0 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb772ffd640 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2943fb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb764009560 tx=0x7fb764003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c3c9979cc434f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb756ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb756ff5640 1 -- 172.21.15.19:0/2047193691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764027070 con 0x564ca293c0a0 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb756ff5640 1 -- 172.21.15.19:0/2047193691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb764005240 con 0x564ca293c0a0 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.813+0000 7fb77c416740 1 -- 172.21.15.19:0/2047193691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7640043d0 con 0x564ca293c0a0 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb756ff5640 1 -- 172.21.15.19:0/2047193691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb764033050 con 0x564ca293c0a0 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb77c416740 10 client.16833 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb77c416740 1 -- 172.21.15.19:0/2047193691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ca2573670 con 0x564ca293c0a0 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb756ff5640 1 -- 172.21.15.19:0/2047193691 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7640043d0 con 0x564ca293c0a0 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb756ff5640 1 -- 172.21.15.19:0/2047193691 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb764004b70 con 0x564ca293c0a0 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb77c416740 10 client.16833 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb77c416740 10 client.16833 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb77c416740 20 client.16833 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb77c416740 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ca29572a0 0x564ca2959690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb77c416740 4 client.16833 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb77c416740 1 -- 172.21.15.19:0/2047193691 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ca28aed90 con 0x564ca29572a0 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.814+0000 7fb7727fc640 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ca29572a0 0x564ca2959690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb7727fc640 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ca29572a0 0x564ca2959690 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef559335f94469 server_cookie=8ed4b256a0c02e8e in_seq=0 out_seq=0 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb756ff5640 10 client.16833 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb756ff5640 1 -- 172.21.15.19:0/2047193691 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ca28aed90 con 0x564ca29572a0 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb756ff5640 10 client.16833 handle_command_reply: tid=0 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb77c416740 1 client.16833 shutdown 2024-04-23T18:35:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb77c416740 1 -- 172.21.15.19:0/2047193691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 msgr2=0x564ca2943fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb77c416740 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2943fb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb764009560 tx=0x7fb764003820 comp rx=0 tx=0).stop 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb77c416740 1 -- 172.21.15.19:0/2047193691 shutdown_connections 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb77c416740 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ca29572a0 0x564ca2959690 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb77c416740 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca2947090 0x564ca2944a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb756ff5640 0 client.16833 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb77c416740 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c9c0 0x564ca29444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb77c416740 1 --2- 172.21.15.19:0/2047193691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2943fb0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.815+0000 7fb77c416740 1 -- 172.21.15.19:0/2047193691 >> 172.21.15.19:0/2047193691 conn(0x564ca29383c0 msgr2=0x564ca293e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.816+0000 7fb77c416740 1 -- 172.21.15.19:0/2047193691 shutdown_connections 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.816+0000 7fb77c416740 1 -- 172.21.15.19:0/2047193691 wait complete. 2024-04-23T18:35:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.816+0000 7fb77c416740 20 client.16833 trim_cache size 0 max 16384 2024-04-23T18:35:26.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb77c416740 1 Processor -- start 2024-04-23T18:35:26.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb77c416740 1 -- start start 2024-04-23T18:35:26.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb77c416740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2952d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb77c416740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c9c0 0x564ca293e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb77c416740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca2947090 0x564ca2937480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb77c416740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741afe60 con 0x564ca293c0a0 2024-04-23T18:35:26.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb77c416740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741b19a0 con 0x564ca293c9c0 2024-04-23T18:35:26.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb77c416740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741b0530 con 0x564ca2947090 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb772ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2952d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb772ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2952d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60502/0 (socket says 172.21.15.19:60502) 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb772ffd640 1 -- 172.21.15.19:0/2088392746 learned_addr learned my addr 172.21.15.19:0/2088392746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb7727fc640 1 --2- 172.21.15.19:0/2088392746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c9c0 0x564ca293e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.833+0000 7fb7737fe640 1 --2- 172.21.15.19:0/2088392746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca2947090 0x564ca2937480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb772ffd640 1 -- 172.21.15.19:0/2088392746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca2947090 msgr2=0x564ca2937480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb772ffd640 1 --2- 172.21.15.19:0/2088392746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca2947090 0x564ca2937480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb772ffd640 1 -- 172.21.15.19:0/2088392746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c9c0 msgr2=0x564ca293e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb772ffd640 1 --2- 172.21.15.19:0/2088392746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c9c0 0x564ca293e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb772ffd640 1 -- 172.21.15.19:0/2088392746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca293cf40 con 0x564ca293c0a0 2024-04-23T18:35:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb7727fc640 1 --2- 172.21.15.19:0/2088392746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c9c0 0x564ca293e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb772ffd640 1 --2- 172.21.15.19:0/2088392746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2952d40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb764004ee0 tx=0x7fb7640196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6875e4c78627eb90 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb758ff9640 1 -- 172.21.15.19:0/2088392746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764030070 con 0x564ca293c0a0 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb758ff9640 1 -- 172.21.15.19:0/2088392746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb764009b70 con 0x564ca293c0a0 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb758ff9640 1 -- 172.21.15.19:0/2088392746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764027070 con 0x564ca293c0a0 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb77c416740 1 -- 172.21.15.19:0/2088392746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 msgr2=0x564ca2952d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb77c416740 1 --2- 172.21.15.19:0/2088392746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2952d40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb764004ee0 tx=0x7fb7640196e0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb77c416740 1 -- 172.21.15.19:0/2088392746 shutdown_connections 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb77c416740 1 --2- 172.21.15.19:0/2088392746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c9c0 0x564ca293e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb77c416740 1 --2- 172.21.15.19:0/2088392746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca2947090 0x564ca2937480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb77c416740 1 --2- 172.21.15.19:0/2088392746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca293c0a0 0x564ca2952d40 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.834+0000 7fb77c416740 1 -- 172.21.15.19:0/2088392746 >> 172.21.15.19:0/2088392746 conn(0x564ca29383c0 msgr2=0x564ca2936d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 1 -- 172.21.15.19:0/2088392746 shutdown_connections 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 1 -- 172.21.15.19:0/2088392746 wait complete. 2024-04-23T18:35:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 1 -- start start 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 1 Processor -- start 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c0a0 0x564ca294f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c9c0 0x564ca2940b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca2947090 0x564ca2941250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741b12d0 con 0x564ca2947090 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741b0c00 con 0x564ca293c0a0 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741ade60 con 0x564ca293c9c0 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb77c416740 10 client.0 _ll_register_callbacks cb 0x7fb764005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb7737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca2947090 0x564ca2941250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb7737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca2947090 0x564ca2941250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60508/0 (socket says 172.21.15.19:60508) 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb7737fe640 1 -- 172.21.15.19:0/857794406 learned_addr learned my addr 172.21.15.19:0/857794406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.835+0000 7fb7727fc640 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c9c0 0x564ca2940b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb772ffd640 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c0a0 0x564ca294f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb7737fe640 1 -- 172.21.15.19:0/857794406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c9c0 msgr2=0x564ca2940b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb7737fe640 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c9c0 0x564ca2940b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb7737fe640 1 -- 172.21.15.19:0/857794406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c0a0 msgr2=0x564ca294f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb7737fe640 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c0a0 0x564ca294f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb7737fe640 1 -- 172.21.15.19:0/857794406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ca29bc730 con 0x564ca2947090 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb7727fc640 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c9c0 0x564ca2940b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb7737fe640 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca2947090 0x564ca2941250 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb76c005c20 tx=0x7fb76c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5807d61574d2855 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb756ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb756ff5640 1 -- 172.21.15.19:0/857794406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c004b40 con 0x564ca2947090 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb756ff5640 1 -- 172.21.15.19:0/857794406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb76c004ce0 con 0x564ca2947090 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb77c416740 1 -- 172.21.15.19:0/857794406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb764009b70 con 0x564ca2947090 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb756ff5640 1 -- 172.21.15.19:0/857794406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb76c004250 con 0x564ca2947090 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb77c416740 10 client.16839 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb77c416740 1 -- 172.21.15.19:0/857794406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ca29bd870 con 0x564ca2947090 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb756ff5640 1 -- 172.21.15.19:0/857794406 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb764009b70 con 0x564ca2947090 2024-04-23T18:35:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb756ff5640 1 -- 172.21.15.19:0/857794406 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb76c04a040 con 0x564ca2947090 2024-04-23T18:35:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb77c416740 10 client.16839 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb77c416740 10 client.16839 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:35:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb77c416740 20 client.16839 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.836+0000 7fb77c416740 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ca29be0e0 0x564ca29c0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.837+0000 7fb77c416740 4 client.16839 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.8, "await_for": 42.0}] 2024-04-23T18:35:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.837+0000 7fb77c416740 1 -- 172.21.15.19:0/857794406 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.8, "await_for": 42.0}) -- 0x564ca28a8c00 con 0x564ca29be0e0 2024-04-23T18:35:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.837+0000 7fb772ffd640 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ca29be0e0 0x564ca29c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.837+0000 7fb756ff5640 10 client.16839 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.837+0000 7fb772ffd640 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ca29be0e0 0x564ca29c0500 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcbbc8fe262523da server_cookie=e46220fe3b433fd2 in_seq=0 out_seq=0 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb756ff5640 1 -- 172.21.15.19:0/857794406 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x564ca28a8c00 con 0x564ca29be0e0 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb756ff5640 10 client.16839 handle_command_reply: tid=0 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 client.16839 shutdown 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 -- 172.21.15.19:0/857794406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca2947090 msgr2=0x564ca2941250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca2947090 0x564ca2941250 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb76c005c20 tx=0x7fb76c0095c0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 -- 172.21.15.19:0/857794406 shutdown_connections 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ca29be0e0 0x564ca29c0500 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ca2947090 0x564ca2941250 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ca293c9c0 0x564ca2940b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 --2- 172.21.15.19:0/857794406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ca293c0a0 0x564ca294f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 -- 172.21.15.19:0/857794406 >> 172.21.15.19:0/857794406 conn(0x564ca29383c0 msgr2=0x564ca29447e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb756ff5640 0 client.16839 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:26.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 -- 172.21.15.19:0/857794406 shutdown_connections 2024-04-23T18:35:26.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.852+0000 7fb77c416740 1 -- 172.21.15.19:0/857794406 wait complete. 2024-04-23T18:35:26.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77c416740 20 client.16839 trim_cache size 0 max 16384 2024-04-23T18:35:26.846 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:35:26.846 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:35:26.846 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:35:26.846 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 38, 2024-04-23T18:35:26.846 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:35:26.846 INFO:teuthology.orchestra.run.smithi019.stdout: "cbddd4d7": { 2024-04-23T18:35:26.846 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 38, 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 78.8, 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:26.847 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:26.848 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:26.848 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:26.848 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:35:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 -- 172.21.15.19:0/1551761706 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb75c0a3d50 msgr2=0x7fb75c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb75c0a3d50 0x7fb75c0a6210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb76c006010 tx=0x7fb76c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 -- 172.21.15.19:0/1551761706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb774106190 msgr2=0x7fb774103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb774106190 0x7fb774103090 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e9f0 tx=0x7fb76800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 -- 172.21.15.19:0/1551761706 shutdown_connections 2024-04-23T18:35:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fb75c0a3d50 0x7fb75c0a6210 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7741035d0 0x7fb7741a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb774106190 0x7fb774103090 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 --2- 172.21.15.19:0/1551761706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774101110 0x7fb774102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 -- 172.21.15.19:0/1551761706 >> 172.21.15.19:0/1551761706 conn(0x7fb774077fb0 msgr2=0x7fb77406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 -- 172.21.15.19:0/1551761706 shutdown_connections 2024-04-23T18:35:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:26.853+0000 7fb77962c640 1 -- 172.21.15.19:0/1551761706 wait complete. 2024-04-23T18:35:26.856 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cbddd4d7', quiesce took 0.5 seconds. Will release after: 18.8 2024-04-23T18:35:27.008 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:26 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3865604135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:27.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:26 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3865604135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:27.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:26 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3865604135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:28.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:27 smithi044 ceph-mon[32458]: pgmap v288: 97 pgs: 97 active+clean; 4.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:35:28.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:27 smithi155 ceph-mon[33468]: pgmap v288: 97 pgs: 97 active+clean; 4.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:35:28.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:27 smithi019 ceph-mon[29215]: pgmap v288: 97 pgs: 97 active+clean; 4.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:35:30.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:29 smithi044 ceph-mon[32458]: pgmap v289: 97 pgs: 97 active+clean; 4.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:35:30.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:29 smithi155 ceph-mon[33468]: pgmap v289: 97 pgs: 97 active+clean; 4.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:35:30.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:29 smithi019 ceph-mon[29215]: pgmap v289: 97 pgs: 97 active+clean; 4.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:35:32.061 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:31 smithi019 ceph-mon[29215]: pgmap v290: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-04-23T18:35:32.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:31 smithi044 ceph-mon[32458]: pgmap v290: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-04-23T18:35:32.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:31 smithi155 ceph-mon[33468]: pgmap v290: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-04-23T18:35:34.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:33 smithi044 ceph-mon[32458]: pgmap v291: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-23T18:35:34.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:33 smithi155 ceph-mon[33468]: pgmap v291: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-23T18:35:34.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:33 smithi019 ceph-mon[29215]: pgmap v291: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-23T18:35:36.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:35 smithi044 ceph-mon[32458]: pgmap v292: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-23T18:35:36.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:35 smithi155 ceph-mon[33468]: pgmap v292: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-23T18:35:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.216+0000 7f291cda2640 1 -- 172.21.15.19:0/3272494258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2918106170 msgr2=0x7f291810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.216+0000 7f291cda2640 1 --2- 172.21.15.19:0/3272494258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2918106170 0x7f291810dd40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2908009900 tx=0x7f290802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.216+0000 7f291cda2640 1 -- 172.21.15.19:0/3272494258 shutdown_connections 2024-04-23T18:35:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.216+0000 7f291cda2640 1 --2- 172.21.15.19:0/3272494258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2918101aa0 0x7f2918105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.216+0000 7f291cda2640 1 --2- 172.21.15.19:0/3272494258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2918106170 0x7f291810dd40 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.216+0000 7f291cda2640 1 --2- 172.21.15.19:0/3272494258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29181010f0 0x7f29181014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.216+0000 7f291cda2640 1 -- 172.21.15.19:0/3272494258 >> 172.21.15.19:0/3272494258 conn(0x7f2918077fd0 msgr2=0x7f29180783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.216+0000 7f291cda2640 1 -- 172.21.15.19:0/3272494258 shutdown_connections 2024-04-23T18:35:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.216+0000 7f291cda2640 1 -- 172.21.15.19:0/3272494258 wait complete. 2024-04-23T18:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291cda2640 1 Processor -- start 2024-04-23T18:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291cda2640 1 -- start start 2024-04-23T18:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291cda2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29181010f0 0x7f291810e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291cda2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918106170 0x7f291810ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291cda2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f291810f150 0x7f29181a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291cda2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29181121f0 con 0x7f29181010f0 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291cda2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2918112070 con 0x7f291810f150 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291cda2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2918112370 con 0x7f2918106170 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29181010f0 0x7f291810e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29181010f0 0x7f291810e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52600/0 (socket says 172.21.15.19:52600) 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.217+0000 7f291659c640 1 -- 172.21.15.19:0/375949020 learned_addr learned my addr 172.21.15.19:0/375949020 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f2915d9b640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918106170 0x7f291810ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f2916d9d640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f291810f150 0x7f29181a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f291659c640 1 -- 172.21.15.19:0/375949020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918106170 msgr2=0x7f291810ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f291659c640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918106170 0x7f291810ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f291659c640 1 -- 172.21.15.19:0/375949020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f291810f150 msgr2=0x7f29181a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f291659c640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f291810f150 0x7f29181a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f291659c640 1 -- 172.21.15.19:0/375949020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29181a6f80 con 0x7f29181010f0 2024-04-23T18:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f291659c640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29181010f0 0x7f291810e6d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f291000ea80 tx=0x7f291000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f29077fe640 1 -- 172.21.15.19:0/375949020 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291000ce40 con 0x7f29181010f0 2024-04-23T18:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f29077fe640 1 -- 172.21.15.19:0/375949020 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2910005020 con 0x7f29181010f0 2024-04-23T18:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f291cda2640 1 -- 172.21.15.19:0/375949020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29181a7270 con 0x7f29181010f0 2024-04-23T18:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f29077fe640 1 -- 172.21.15.19:0/375949020 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29100116d0 con 0x7f29181010f0 2024-04-23T18:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.218+0000 7f291cda2640 1 -- 172.21.15.19:0/375949020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29181a77c0 con 0x7f29181010f0 2024-04-23T18:35:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.221+0000 7f29077fe640 1 -- 172.21.15.19:0/375949020 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29100054d0 con 0x7f29181010f0 2024-04-23T18:35:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.221+0000 7f291fb97740 1 Processor -- start 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.221+0000 7f291fb97740 1 -- start start 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.221+0000 7f291fb97740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa404a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f291fb97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f291fb97740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa411bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f291fb97740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559daa369970 con 0x559daa404620 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f291fb97740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559daa3064a0 con 0x559daa404f40 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f291fb97740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559daa365930 con 0x559daa40f800 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f29077fe640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f29000a3e70 0x7f29000a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f2916d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa404a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f291659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f2916d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa404a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52610/0 (socket says 172.21.15.19:52610) 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f2916d9d640 1 -- 172.21.15.19:0/230727530 learned_addr learned my addr 172.21.15.19:0/230727530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f2916d9d640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f29000a3e70 0x7f29000a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f2915d9b640 1 --2- 172.21.15.19:0/230727530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa411bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f29077fe640 1 -- 172.21.15.19:0/375949020 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f29100d0a50 con 0x7f29181010f0 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f2916d9d640 1 -- 172.21.15.19:0/230727530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 msgr2=0x559daa411bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f2916d9d640 1 --2- 172.21.15.19:0/230727530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa411bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f2916d9d640 1 -- 172.21.15.19:0/230727530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 msgr2=0x559daa40f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f2916d9d640 1 --2- 172.21.15.19:0/230727530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f2916d9d640 1 -- 172.21.15.19:0/230727530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559daa07c210 con 0x559daa404620 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.222+0000 7f291659c640 1 --2- 172.21.15.19:0/230727530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.223+0000 7f2916d9d640 1 --2- 172.21.15.19:0/230727530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa404a00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f29080057c0 tx=0x7f2908031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98ec82d93b7e0c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.223+0000 7f2916d9d640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f29000a3e70 0x7f29000a6330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2908009900 tx=0x7f2908046000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.223+0000 7f2904ff9640 1 -- 172.21.15.19:0/230727530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908035040 con 0x559daa404620 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.223+0000 7f2904ff9640 1 -- 172.21.15.19:0/230727530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290804d070 con 0x559daa404620 2024-04-23T18:35:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.223+0000 7f2904ff9640 1 -- 172.21.15.19:0/230727530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908003910 con 0x559daa404620 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.223+0000 7f291fb97740 1 -- 172.21.15.19:0/230727530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 msgr2=0x559daa404a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.223+0000 7f291fb97740 1 --2- 172.21.15.19:0/230727530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa404a00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f29080057c0 tx=0x7f2908031970 comp rx=0 tx=0).stop 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 -- 172.21.15.19:0/230727530 shutdown_connections 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 --2- 172.21.15.19:0/230727530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 --2- 172.21.15.19:0/230727530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa411bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 --2- 172.21.15.19:0/230727530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa404a00 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 -- 172.21.15.19:0/230727530 >> 172.21.15.19:0/230727530 conn(0x559daa400b30 msgr2=0x559daa400f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 -- 172.21.15.19:0/230727530 shutdown_connections 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 -- 172.21.15.19:0/230727530 wait complete. 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 -- start start 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 Processor -- start 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa40f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa40ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559daa4123d0 con 0x559daa404620 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559daa3040b0 con 0x559daa404f40 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559daa412550 con 0x559daa40f800 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291fb97740 10 client.0 _ll_register_callbacks cb 0x559daa2f1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.224+0000 7f291659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa40f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa40f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52616/0 (socket says 172.21.15.19:52616) 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291659c640 1 -- 172.21.15.19:0/1194924272 learned_addr learned my addr 172.21.15.19:0/1194924272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f2916d9d640 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa40ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f2915d9b640 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291659c640 1 -- 172.21.15.19:0/1194924272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 msgr2=0x559daa40ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291659c640 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa40ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291659c640 1 -- 172.21.15.19:0/1194924272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 msgr2=0x559daa40c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291659c640 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291659c640 1 -- 172.21.15.19:0/1194924272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559daa148620 con 0x559daa404620 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f2916d9d640 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa40ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291659c640 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa40f1b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f291009c980 tx=0x7f291001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0efd80de0e9749 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f28fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f28fdffb640 1 -- 172.21.15.19:0/1194924272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291009c9f0 con 0x559daa404620 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f28fdffb640 1 -- 172.21.15.19:0/1194924272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f291009cc50 con 0x559daa404620 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291fb97740 1 -- 172.21.15.19:0/1194924272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290804d070 con 0x559daa404620 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f28fdffb640 1 -- 172.21.15.19:0/1194924272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f29100a0da0 con 0x559daa404620 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291fb97740 10 client.16851 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.225+0000 7f291fb97740 1 -- 172.21.15.19:0/1194924272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559daa41e3a0 con 0x559daa404620 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.226+0000 7f28fdffb640 1 -- 172.21.15.19:0/1194924272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290804d070 con 0x559daa404620 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.226+0000 7f28fdffb640 1 -- 172.21.15.19:0/1194924272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f291000f040 con 0x559daa404620 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.226+0000 7f291fb97740 10 client.16851 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.226+0000 7f291fb97740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.226+0000 7f291fb97740 20 client.16851 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.226+0000 7f291fb97740 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559daa41f810 0x559daa421c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.226+0000 7f291fb97740 4 client.16851 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.226+0000 7f291fb97740 1 -- 172.21.15.19:0/1194924272 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559daa377390 con 0x559daa41f810 2024-04-23T18:35:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.226+0000 7f2915d9b640 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559daa41f810 0x559daa421c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.227+0000 7f28fdffb640 10 client.16851 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.227+0000 7f2915d9b640 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559daa41f810 0x559daa421c00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=356c34a150369548 server_cookie=db6e2766cc048a58 in_seq=0 out_seq=0 2024-04-23T18:35:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.227+0000 7f28fdffb640 1 -- 172.21.15.19:0/1194924272 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559daa377390 con 0x559daa41f810 2024-04-23T18:35:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.227+0000 7f28fdffb640 10 client.16851 handle_command_reply: tid=0 2024-04-23T18:35:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.227+0000 7f291fb97740 1 client.16851 shutdown 2024-04-23T18:35:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 1 -- 172.21.15.19:0/1194924272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 msgr2=0x559daa40f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa40f1b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f291009c980 tx=0x7f291001f3f0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 1 -- 172.21.15.19:0/1194924272 shutdown_connections 2024-04-23T18:35:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559daa41f810 0x559daa421c00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa40ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f28fdffb640 0 client.16851 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 1 --2- 172.21.15.19:0/1194924272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404620 0x559daa40f1b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 1 -- 172.21.15.19:0/1194924272 >> 172.21.15.19:0/1194924272 conn(0x559daa400b30 msgr2=0x559daa406860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 1 -- 172.21.15.19:0/1194924272 shutdown_connections 2024-04-23T18:35:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 1 -- 172.21.15.19:0/1194924272 wait complete. 2024-04-23T18:35:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.228+0000 7f291fb97740 20 client.16851 trim_cache size 0 max 16384 2024-04-23T18:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f291fb97740 1 Processor -- start 2024-04-23T18:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f291fb97740 1 -- start start 2024-04-23T18:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f291fb97740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa404620 0x559daa41b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f291fb97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f291fb97740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa40f800 0x559daa427480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f291fb97740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29181afcc0 con 0x559daa40f800 2024-04-23T18:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f291fb97740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b1800 con 0x559daa404f40 2024-04-23T18:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f291fb97740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b0390 con 0x559daa404620 2024-04-23T18:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f2916d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa40f800 0x559daa427480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f2916d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa40f800 0x559daa427480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52626/0 (socket says 172.21.15.19:52626) 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f2916d9d640 1 -- 172.21.15.19:0/3394778256 learned_addr learned my addr 172.21.15.19:0/3394778256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.243+0000 7f291659c640 1 --2- 172.21.15.19:0/3394778256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa404620 0x559daa41b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f2915d9b640 1 --2- 172.21.15.19:0/3394778256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f2916d9d640 1 -- 172.21.15.19:0/3394778256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa404620 msgr2=0x559daa41b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f2916d9d640 1 --2- 172.21.15.19:0/3394778256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa404620 0x559daa41b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f2916d9d640 1 -- 172.21.15.19:0/3394778256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 msgr2=0x559daa40ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f2916d9d640 1 --2- 172.21.15.19:0/3394778256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f2916d9d640 1 -- 172.21.15.19:0/3394778256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559daa421f70 con 0x559daa40f800 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f291659c640 1 --2- 172.21.15.19:0/3394778256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa404620 0x559daa41b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f2916d9d640 1 --2- 172.21.15.19:0/3394778256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa40f800 0x559daa427480 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2908004620 tx=0x7f2908005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26bd8afa3112506 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f28fffff640 1 -- 172.21.15.19:0/3394778256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908055070 con 0x559daa40f800 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f28fffff640 1 -- 172.21.15.19:0/3394778256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290804d050 con 0x559daa40f800 2024-04-23T18:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.244+0000 7f28fffff640 1 -- 172.21.15.19:0/3394778256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908035040 con 0x559daa40f800 2024-04-23T18:35:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 -- 172.21.15.19:0/3394778256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa40f800 msgr2=0x559daa427480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 --2- 172.21.15.19:0/3394778256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa40f800 0x559daa427480 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2908004620 tx=0x7f2908005d50 comp rx=0 tx=0).stop 2024-04-23T18:35:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 -- 172.21.15.19:0/3394778256 shutdown_connections 2024-04-23T18:35:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 --2- 172.21.15.19:0/3394778256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404f40 0x559daa40ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 --2- 172.21.15.19:0/3394778256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa40f800 0x559daa427480 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 --2- 172.21.15.19:0/3394778256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa404620 0x559daa41b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 -- 172.21.15.19:0/3394778256 >> 172.21.15.19:0/3394778256 conn(0x559daa400b30 msgr2=0x559daa40c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 -- 172.21.15.19:0/3394778256 shutdown_connections 2024-04-23T18:35:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 -- 172.21.15.19:0/3394778256 wait complete. 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 -- start start 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 Processor -- start 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404620 0x559daa417990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404f40 0x559daa408d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa409250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b1130 con 0x559daa404f40 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b0a60 con 0x559daa404620 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29181adcc0 con 0x559daa40f800 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.245+0000 7f291fb97740 10 client.0 _ll_register_callbacks cb 0x7f291009c5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f2915d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404f40 0x559daa408d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f2915d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404f40 0x559daa408d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52632/0 (socket says 172.21.15.19:52632) 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f2915d9b640 1 -- 172.21.15.19:0/1800255143 learned_addr learned my addr 172.21.15.19:0/1800255143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f291659c640 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404620 0x559daa417990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f2916d9d640 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa409250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f2915d9b640 1 -- 172.21.15.19:0/1800255143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 msgr2=0x559daa409250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f2915d9b640 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa409250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f2915d9b640 1 -- 172.21.15.19:0/1800255143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404620 msgr2=0x559daa417990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f2915d9b640 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404620 0x559daa417990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f2915d9b640 1 -- 172.21.15.19:0/1800255143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559daa40cdb0 con 0x559daa404f40 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f2915d9b640 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404f40 0x559daa408d10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f290c000c00 tx=0x7f290c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e76c3d81497cde server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f28fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f28fdffb640 1 -- 172.21.15.19:0/1800255143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c004220 con 0x559daa404f40 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f28fdffb640 1 -- 172.21.15.19:0/1800255143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f290c0043c0 con 0x559daa404f40 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f291fb97740 1 -- 172.21.15.19:0/1800255143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290804d050 con 0x559daa404f40 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f28fdffb640 1 -- 172.21.15.19:0/1800255143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f290c02f050 con 0x559daa404f40 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f291fb97740 10 client.16857 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.246+0000 7f291fb97740 1 -- 172.21.15.19:0/1800255143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559daa485880 con 0x559daa404f40 2024-04-23T18:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.247+0000 7f28fdffb640 1 -- 172.21.15.19:0/1800255143 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290804d050 con 0x559daa404f40 2024-04-23T18:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.247+0000 7f28fdffb640 1 -- 172.21.15.19:0/1800255143 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f290c003b50 con 0x559daa404f40 2024-04-23T18:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.247+0000 7f291fb97740 10 client.16857 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.247+0000 7f291fb97740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.247+0000 7f291fb97740 20 client.16857 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.247+0000 7f291fb97740 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559daa486050 0x559daa488470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.247+0000 7f291fb97740 4 client.16857 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.247+0000 7f291fb97740 1 -- 172.21.15.19:0/1800255143 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559daa371200 con 0x559daa486050 2024-04-23T18:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.247+0000 7f291659c640 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559daa486050 0x559daa488470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.248+0000 7f291659c640 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559daa486050 0x559daa488470 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b333ab074053c server_cookie=56dd77e7f061c0c3 in_seq=0 out_seq=0 2024-04-23T18:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.248+0000 7f28fdffb640 10 client.16857 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f28fdffb640 1 -- 172.21.15.19:0/1800255143 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559daa371200 con 0x559daa486050 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f28fdffb640 10 client.16857 handle_command_reply: tid=0 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 client.16857 shutdown 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 -- 172.21.15.19:0/1800255143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404f40 msgr2=0x559daa408d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404f40 0x559daa408d10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f290c000c00 tx=0x7f290c0118d0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 -- 172.21.15.19:0/1800255143 shutdown_connections 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x559daa486050 0x559daa488470 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559daa40f800 0x559daa409250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559daa404f40 0x559daa408d10 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f28fdffb640 0 client.16857 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 --2- 172.21.15.19:0/1800255143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559daa404620 0x559daa417990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 -- 172.21.15.19:0/1800255143 >> 172.21.15.19:0/1800255143 conn(0x559daa400b30 msgr2=0x559daa407140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 -- 172.21.15.19:0/1800255143 shutdown_connections 2024-04-23T18:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 1 -- 172.21.15.19:0/1800255143 wait complete. 2024-04-23T18:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.262+0000 7f291fb97740 20 client.16857 trim_cache size 0 max 16384 2024-04-23T18:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 -- 172.21.15.19:0/375949020 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f29000a3e70 msgr2=0x7f29000a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f29000a3e70 0x7f29000a6330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2908009900 tx=0x7f2908046000 comp rx=0 tx=0).stop 2024-04-23T18:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 -- 172.21.15.19:0/375949020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29181010f0 msgr2=0x7f291810e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29181010f0 0x7f291810e6d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f291000ea80 tx=0x7f291000ef40 comp rx=0 tx=0).stop 2024-04-23T18:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 -- 172.21.15.19:0/375949020 shutdown_connections 2024-04-23T18:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f29000a3e70 0x7f29000a6330 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f291810f150 0x7f29181a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918106170 0x7f291810ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 --2- 172.21.15.19:0/375949020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29181010f0 0x7f291810e6d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 -- 172.21.15.19:0/375949020 >> 172.21.15.19:0/375949020 conn(0x7f2918077fd0 msgr2=0x7f29180fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 -- 172.21.15.19:0/375949020 shutdown_connections 2024-04-23T18:35:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:36.263+0000 7f291cda2640 1 -- 172.21.15.19:0/375949020 wait complete. 2024-04-23T18:35:36.258 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:35 smithi019 ceph-mon[29215]: pgmap v292: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-23T18:35:38.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:37 smithi155 ceph-mon[33468]: pgmap v293: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T18:35:38.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:37 smithi019 ceph-mon[29215]: pgmap v293: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T18:35:38.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:37 smithi044 ceph-mon[32458]: pgmap v293: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T18:35:40.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:39 smithi155 ceph-mon[33468]: pgmap v294: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T18:35:40.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:39 smithi019 ceph-mon[29215]: pgmap v294: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T18:35:40.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:39 smithi044 ceph-mon[32458]: pgmap v294: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T18:35:42.071 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:41 smithi155 ceph-mon[33468]: pgmap v295: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-23T18:35:42.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:41 smithi019 ceph-mon[29215]: pgmap v295: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-23T18:35:42.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:41 smithi044 ceph-mon[32458]: pgmap v295: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-23T18:35:44.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:43 smithi019 ceph-mon[29215]: pgmap v296: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-23T18:35:44.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:43 smithi044 ceph-mon[32458]: pgmap v296: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-23T18:35:44.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:43 smithi155 ceph-mon[33468]: pgmap v296: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-23T18:35:45.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:35:45.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.781+0000 7f4a6f66b640 1 -- 172.21.15.19:0/2321990254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a68103b00 msgr2=0x7f4a6810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.781+0000 7f4a6f66b640 1 --2- 172.21.15.19:0/2321990254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a68103b00 0x7f4a6810bfe0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009900 tx=0x7f4a5802a510 comp rx=0 tx=0).stop 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.781+0000 7f4a6f66b640 1 -- 172.21.15.19:0/2321990254 shutdown_connections 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.781+0000 7f4a6f66b640 1 --2- 172.21.15.19:0/2321990254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a6810c520 0x7f4a6810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.781+0000 7f4a6f66b640 1 --2- 172.21.15.19:0/2321990254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a68103b00 0x7f4a6810bfe0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.781+0000 7f4a6f66b640 1 --2- 172.21.15.19:0/2321990254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a681031e0 0x7f4a681035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.781+0000 7f4a6f66b640 1 -- 172.21.15.19:0/2321990254 >> 172.21.15.19:0/2321990254 conn(0x7f4a68077f50 msgr2=0x7f4a68078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.781+0000 7f4a6f66b640 1 -- 172.21.15.19:0/2321990254 shutdown_connections 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.781+0000 7f4a6f66b640 1 -- 172.21.15.19:0/2321990254 wait complete. 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6f66b640 1 Processor -- start 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6f66b640 1 -- start start 2024-04-23T18:35:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6f66b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a681031e0 0x7f4a68198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6f66b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a68103b00 0x7f4a68198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6f66b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a6810c520 0x7f4a6819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6f66b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68112190 con 0x7f4a681031e0 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6f66b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68112010 con 0x7f4a68103b00 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6f66b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68112310 con 0x7f4a6810c520 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6d407640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a681031e0 0x7f4a68198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6d407640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a681031e0 0x7f4a68198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37644/0 (socket says 172.21.15.19:37644) 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6d407640 1 -- 172.21.15.19:0/1445937703 learned_addr learned my addr 172.21.15.19:0/1445937703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6dc08640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a6810c520 0x7f4a6819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6cc06640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a68103b00 0x7f4a68198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6d407640 1 -- 172.21.15.19:0/1445937703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a6810c520 msgr2=0x7f4a6819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6d407640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a6810c520 0x7f4a6819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6d407640 1 -- 172.21.15.19:0/1445937703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a68103b00 msgr2=0x7f4a68198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6d407640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a68103b00 0x7f4a68198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6d407640 1 -- 172.21.15.19:0/1445937703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6819d740 con 0x7f4a681031e0 2024-04-23T18:35:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6dc08640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a6810c520 0x7f4a6819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.782+0000 7f4a6cc06640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a68103b00 0x7f4a68198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.783+0000 7f4a6d407640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a681031e0 0x7f4a68198750 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400d950 tx=0x7f4a6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.783+0000 7f4a5e7fc640 1 -- 172.21.15.19:0/1445937703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64017070 con 0x7f4a681031e0 2024-04-23T18:35:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.783+0000 7f4a5e7fc640 1 -- 172.21.15.19:0/1445937703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a64009c30 con 0x7f4a681031e0 2024-04-23T18:35:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.783+0000 7f4a6f66b640 1 -- 172.21.15.19:0/1445937703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6819da30 con 0x7f4a681031e0 2024-04-23T18:35:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.783+0000 7f4a5e7fc640 1 -- 172.21.15.19:0/1445937703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400f620 con 0x7f4a681031e0 2024-04-23T18:35:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.783+0000 7f4a6f66b640 1 -- 172.21.15.19:0/1445937703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a681a52c0 con 0x7f4a681031e0 2024-04-23T18:35:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.784+0000 7f4a6f66b640 1 -- 172.21.15.19:0/1445937703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a68068d70 con 0x7f4a681031e0 2024-04-23T18:35:45.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.786+0000 7f4a5e7fc640 1 -- 172.21.15.19:0/1445937703 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a64005030 con 0x7f4a681031e0 2024-04-23T18:35:45.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.786+0000 7f4a5e7fc640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4a540a3d20 0x7f4a540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:45.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.787+0000 7f4a5e7fc640 1 -- 172.21.15.19:0/1445937703 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f4a640d0550 con 0x7f4a681031e0 2024-04-23T18:35:45.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.787+0000 7f4a6cc06640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4a540a3d20 0x7f4a540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:45.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.787+0000 7f4a6cc06640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4a540a3d20 0x7f4a540a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a58000c00 tx=0x7f4a58031400 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:45.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:45.790+0000 7f4a5e7fc640 1 -- 172.21.15.19:0/1445937703 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a6409cb90 con 0x7f4a681031e0 2024-04-23T18:35:46.008 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:45 smithi019 ceph-mon[29215]: pgmap v297: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-04-23T18:35:46.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.019+0000 7f4a6f66b640 1 -- 172.21.15.19:0/1445937703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a68199aa0 con 0x7f4a681031e0 2024-04-23T18:35:46.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.020+0000 7f4a5e7fc640 1 -- 172.21.15.19:0/1445937703 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f4a64009de0 con 0x7f4a681031e0 2024-04-23T18:35:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:35:46.012 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:35:46.012 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:35:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 -- 172.21.15.19:0/1445937703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4a540a3d20 msgr2=0x7f4a540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4a540a3d20 0x7f4a540a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a58000c00 tx=0x7f4a58031400 comp rx=0 tx=0).stop 2024-04-23T18:35:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 -- 172.21.15.19:0/1445937703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a681031e0 msgr2=0x7f4a68198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a681031e0 0x7f4a68198750 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400d950 tx=0x7f4a6400de10 comp rx=0 tx=0).stop 2024-04-23T18:35:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 -- 172.21.15.19:0/1445937703 shutdown_connections 2024-04-23T18:35:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4a540a3d20 0x7f4a540a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a68103b00 0x7f4a68198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a6810c520 0x7f4a6819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 --2- 172.21.15.19:0/1445937703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a681031e0 0x7f4a68198750 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 -- 172.21.15.19:0/1445937703 >> 172.21.15.19:0/1445937703 conn(0x7f4a68077f50 msgr2=0x7f4a680fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 -- 172.21.15.19:0/1445937703 shutdown_connections 2024-04-23T18:35:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.022+0000 7f4a6f66b640 1 -- 172.21.15.19:0/1445937703 wait complete. 2024-04-23T18:35:46.025 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id cbddd4d7 --release --await' 2024-04-23T18:35:46.025 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id cbddd4d7 --release --await 2024-04-23T18:35:46.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.142+0000 7f33d8c6b640 1 -- 172.21.15.19:0/416743443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4101a40 msgr2=0x7f33d4105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.142+0000 7f33d8c6b640 1 --2- 172.21.15.19:0/416743443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4101a40 0x7f33d4105a80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009900 tx=0x7f33c802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:46.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.142+0000 7f33d8c6b640 1 -- 172.21.15.19:0/416743443 shutdown_connections 2024-04-23T18:35:46.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.142+0000 7f33d8c6b640 1 --2- 172.21.15.19:0/416743443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d4106160 0x7f33d410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.142+0000 7f33d8c6b640 1 --2- 172.21.15.19:0/416743443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4101a40 0x7f33d4105a80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.142+0000 7f33d8c6b640 1 --2- 172.21.15.19:0/416743443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33d4101090 0x7f33d4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.142+0000 7f33d8c6b640 1 -- 172.21.15.19:0/416743443 >> 172.21.15.19:0/416743443 conn(0x7f33d4077f50 msgr2=0x7f33d4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.142+0000 7f33d8c6b640 1 -- 172.21.15.19:0/416743443 shutdown_connections 2024-04-23T18:35:46.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.142+0000 7f33d8c6b640 1 -- 172.21.15.19:0/416743443 wait complete. 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d8c6b640 1 Processor -- start 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d8c6b640 1 -- start start 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d8c6b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d4101090 0x7f33d4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d8c6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4106160 0x7f33d4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d8c6b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33d419cfe0 0x7f33d419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d8c6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41121e0 con 0x7f33d4106160 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d8c6b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33d4112060 con 0x7f33d4101090 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d8c6b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f33d4112360 con 0x7f33d419cfe0 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4106160 0x7f33d4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4106160 0x7f33d4198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37668/0 (socket says 172.21.15.19:37668) 2024-04-23T18:35:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d1d9b640 1 -- 172.21.15.19:0/89852165 learned_addr learned my addr 172.21.15.19:0/89852165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d2d9d640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33d419cfe0 0x7f33d419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d259c640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d4101090 0x7f33d4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d1d9b640 1 -- 172.21.15.19:0/89852165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33d419cfe0 msgr2=0x7f33d419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33d419cfe0 0x7f33d419d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d1d9b640 1 -- 172.21.15.19:0/89852165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d4101090 msgr2=0x7f33d4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d4101090 0x7f33d4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d1d9b640 1 -- 172.21.15.19:0/89852165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d419dad0 con 0x7f33d4106160 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.143+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4106160 0x7f33d4198c60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f33c8032fd0 tx=0x7f33c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.144+0000 7f33c37fe640 1 -- 172.21.15.19:0/89852165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c803f030 con 0x7f33d4106160 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.144+0000 7f33d8c6b640 1 -- 172.21.15.19:0/89852165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d419dd60 con 0x7f33d4106160 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.144+0000 7f33c37fe640 1 -- 172.21.15.19:0/89852165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c8004490 con 0x7f33d4106160 2024-04-23T18:35:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.144+0000 7f33c37fe640 1 -- 172.21.15.19:0/89852165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8035070 con 0x7f33d4106160 2024-04-23T18:35:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.144+0000 7f33d8c6b640 1 -- 172.21.15.19:0/89852165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d41a5530 con 0x7f33d4106160 2024-04-23T18:35:46.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.146+0000 7f33c37fe640 1 -- 172.21.15.19:0/89852165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f33c8042030 con 0x7f33d4106160 2024-04-23T18:35:46.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.146+0000 7f33db97d740 1 Processor -- start 2024-04-23T18:35:46.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.146+0000 7f33db97d740 1 -- start start 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33c37fe640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f33bc0a3d20 0x7f33bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33db97d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201f1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33db97d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1db0 0x5581201fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33db97d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201fc480 0x5581201fe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33db97d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5581201566d0 con 0x5581201f1490 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33db97d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5581200f31a0 con 0x5581201f1db0 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33db97d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558120152690 con 0x5581201fc480 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d2d9d640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f33bc0a3d20 0x7f33bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33c37fe640 1 -- 172.21.15.19:0/89852165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f33c80f0dc0 con 0x7f33d4106160 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201f1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37674/0 (socket says 172.21.15.19:37674) 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d259c640 1 -- 172.21.15.19:0/2734071507 learned_addr learned my addr 172.21.15.19:0/2734071507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/2734071507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1db0 0x5581201fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d259c640 1 -- 172.21.15.19:0/2734071507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201fc480 msgr2=0x5581201fe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d259c640 1 --2- 172.21.15.19:0/2734071507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201fc480 0x5581201fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d259c640 1 -- 172.21.15.19:0/2734071507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1db0 msgr2=0x5581201fbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d259c640 1 --2- 172.21.15.19:0/2734071507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1db0 0x5581201fbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d259c640 1 -- 172.21.15.19:0/2734071507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581200fd5b0 con 0x5581201f1490 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d2d9d640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f33bc0a3d20 0x7f33bc0a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0092f0 tx=0x7f33cc0096d0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33d259c640 1 --2- 172.21.15.19:0/2734071507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201f1870 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009bc0 tx=0x7f33c400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22e21c87ca21101 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33c0ff9640 1 -- 172.21.15.19:0/2734071507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4016070 con 0x5581201f1490 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33c0ff9640 1 -- 172.21.15.19:0/2734071507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c400ee70 con 0x5581201f1490 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.147+0000 7f33c0ff9640 1 -- 172.21.15.19:0/2734071507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4010730 con 0x5581201f1490 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.148+0000 7f33db97d740 1 -- 172.21.15.19:0/2734071507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 msgr2=0x5581201f1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.148+0000 7f33db97d740 1 --2- 172.21.15.19:0/2734071507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201f1870 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009bc0 tx=0x7f33c400e6c0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.148+0000 7f33db97d740 1 -- 172.21.15.19:0/2734071507 shutdown_connections 2024-04-23T18:35:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.148+0000 7f33db97d740 1 --2- 172.21.15.19:0/2734071507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1db0 0x5581201fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.148+0000 7f33db97d740 1 --2- 172.21.15.19:0/2734071507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201fc480 0x5581201fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.148+0000 7f33db97d740 1 --2- 172.21.15.19:0/2734071507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201f1870 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.148+0000 7f33db97d740 1 -- 172.21.15.19:0/2734071507 >> 172.21.15.19:0/2734071507 conn(0x5581201ed7b0 msgr2=0x5581201edb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.148+0000 7f33db97d740 1 -- 172.21.15.19:0/2734071507 shutdown_connections 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.148+0000 7f33db97d740 1 -- 172.21.15.19:0/2734071507 wait complete. 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33db97d740 1 -- start start 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33db97d740 1 Processor -- start 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33db97d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33db97d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201f1db0 0x5581201f9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33db97d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201fc480 0x5581201f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33db97d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5581201ff310 con 0x5581201f1490 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33db97d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5581200f0b20 con 0x5581201fc480 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33db97d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5581201ff490 con 0x5581201f1db0 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33db97d740 10 client.0 _ll_register_callbacks cb 0x5581200f0530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37682/0 (socket says 172.21.15.19:37682) 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33d259c640 1 -- 172.21.15.19:0/2745578110 learned_addr learned my addr 172.21.15.19:0/2745578110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33d2d9d640 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201fc480 0x5581201f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201f1db0 0x5581201f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33d259c640 1 -- 172.21.15.19:0/2745578110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201f1db0 msgr2=0x5581201f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33d259c640 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201f1db0 0x5581201f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33d259c640 1 -- 172.21.15.19:0/2745578110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201fc480 msgr2=0x5581201f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33d259c640 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201fc480 0x5581201f9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.149+0000 7f33d259c640 1 -- 172.21.15.19:0/2745578110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55811fe68920 con 0x5581201f1490 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33d259c640 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fbe70 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009980 tx=0x7f33c40052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fd6b47a92a5737 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33b9ffb640 1 -- 172.21.15.19:0/2745578110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4016050 con 0x5581201f1490 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33b9ffb640 1 -- 172.21.15.19:0/2745578110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f33c40048c0 con 0x5581201f1490 2024-04-23T18:35:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33db97d740 1 -- 172.21.15.19:0/2745578110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33c400ee70 con 0x5581201f1490 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33b9ffb640 1 -- 172.21.15.19:0/2745578110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f33c4030050 con 0x5581201f1490 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33db97d740 10 client.16875 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33db97d740 1 -- 172.21.15.19:0/2745578110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55811ff34d30 con 0x5581201f1490 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33b9ffb640 1 -- 172.21.15.19:0/2745578110 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c400ee70 con 0x5581201f1490 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33b9ffb640 1 -- 172.21.15.19:0/2745578110 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f33c402b030 con 0x5581201f1490 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33db97d740 10 client.16875 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33db97d740 10 client.16875 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33db97d740 20 client.16875 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33db97d740 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55812020c5c0 0x55812020e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33db97d740 4 client.16875 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.150+0000 7f33db97d740 1 -- 172.21.15.19:0/2745578110 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558120114500 con 0x55812020c5c0 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.151+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55812020c5c0 0x55812020e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.151+0000 7f33b9ffb640 10 client.16875 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.151+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55812020c5c0 0x55812020e9b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c663f0847cfaafca server_cookie=a8349d330ba94378 in_seq=0 out_seq=0 2024-04-23T18:35:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33b9ffb640 1 -- 172.21.15.19:0/2745578110 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558120114500 con 0x55812020c5c0 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33b9ffb640 10 client.16875 handle_command_reply: tid=0 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 client.16875 shutdown 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 -- 172.21.15.19:0/2745578110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 msgr2=0x5581201fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fbe70 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009980 tx=0x7f33c40052f0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 -- 172.21.15.19:0/2745578110 shutdown_connections 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55812020c5c0 0x55812020e9b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201fc480 0x5581201f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201f1db0 0x5581201f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 --2- 172.21.15.19:0/2745578110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fbe70 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33b9ffb640 0 client.16875 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 -- 172.21.15.19:0/2745578110 >> 172.21.15.19:0/2745578110 conn(0x5581201ed7b0 msgr2=0x5581201f34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 -- 172.21.15.19:0/2745578110 shutdown_connections 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 1 -- 172.21.15.19:0/2745578110 wait complete. 2024-04-23T18:35:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.152+0000 7f33db97d740 20 client.16875 trim_cache size 0 max 16384 2024-04-23T18:35:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.169+0000 7f33db97d740 1 Processor -- start 2024-04-23T18:35:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.169+0000 7f33db97d740 1 -- start start 2024-04-23T18:35:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.169+0000 7f33db97d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1490 0x558120208200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.169+0000 7f33db97d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201f1db0 0x558120204af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.169+0000 7f33db97d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201fc480 0x5581201fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.169+0000 7f33db97d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41adbe0 con 0x5581201fc480 2024-04-23T18:35:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.169+0000 7f33db97d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41af720 con 0x5581201f1490 2024-04-23T18:35:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.169+0000 7f33db97d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41ae2b0 con 0x5581201f1db0 2024-04-23T18:35:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201fc480 0x5581201fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201fc480 0x5581201fb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37692/0 (socket says 172.21.15.19:37692) 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d2d9d640 1 -- 172.21.15.19:0/2901071122 learned_addr learned my addr 172.21.15.19:0/2901071122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/2901071122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201f1db0 0x558120204af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d259c640 1 --2- 172.21.15.19:0/2901071122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1490 0x558120208200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d2d9d640 1 -- 172.21.15.19:0/2901071122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201f1db0 msgr2=0x558120204af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d2d9d640 1 --2- 172.21.15.19:0/2901071122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201f1db0 0x558120204af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d2d9d640 1 -- 172.21.15.19:0/2901071122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1490 msgr2=0x558120208200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d2d9d640 1 --2- 172.21.15.19:0/2901071122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1490 0x558120208200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d2d9d640 1 -- 172.21.15.19:0/2901071122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581201f9ae0 con 0x5581201fc480 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d259c640 1 --2- 172.21.15.19:0/2901071122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1490 0x558120208200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33d2d9d640 1 --2- 172.21.15.19:0/2901071122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201fc480 0x5581201fb9a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f33cc005fd0 tx=0x7f33cc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72bad7e2c6a5916 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33bbfff640 1 -- 172.21.15.19:0/2901071122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc004d20 con 0x5581201fc480 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33bbfff640 1 -- 172.21.15.19:0/2901071122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc024070 con 0x5581201fc480 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.170+0000 7f33bbfff640 1 -- 172.21.15.19:0/2901071122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc005360 con 0x5581201fc480 2024-04-23T18:35:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 -- 172.21.15.19:0/2901071122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201fc480 msgr2=0x5581201fb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 --2- 172.21.15.19:0/2901071122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201fc480 0x5581201fb9a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f33cc005fd0 tx=0x7f33cc014000 comp rx=0 tx=0).stop 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 -- 172.21.15.19:0/2901071122 shutdown_connections 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 --2- 172.21.15.19:0/2901071122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201fc480 0x5581201fb9a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 --2- 172.21.15.19:0/2901071122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201f1db0 0x558120204af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 --2- 172.21.15.19:0/2901071122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1490 0x558120208200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 -- 172.21.15.19:0/2901071122 >> 172.21.15.19:0/2901071122 conn(0x5581201ed7b0 msgr2=0x5581201f43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 -- 172.21.15.19:0/2901071122 shutdown_connections 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 -- 172.21.15.19:0/2901071122 wait complete. 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 -- start start 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 Processor -- start 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1db0 0x5581201fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201fc480 0x5581201fd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41af050 con 0x5581201f1490 2024-04-23T18:35:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41ae980 con 0x5581201f1db0 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41abbe0 con 0x5581201fc480 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.171+0000 7f33db97d740 10 client.0 _ll_register_callbacks cb 0x7f33c40098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37708/0 (socket says 172.21.15.19:37708) 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d259c640 1 -- 172.21.15.19:0/726506409 learned_addr learned my addr 172.21.15.19:0/726506409 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1db0 0x5581201fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d2d9d640 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201fc480 0x5581201fd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d259c640 1 -- 172.21.15.19:0/726506409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201fc480 msgr2=0x5581201fd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d259c640 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201fc480 0x5581201fd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d259c640 1 -- 172.21.15.19:0/726506409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1db0 msgr2=0x5581201fcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d259c640 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1db0 0x5581201fcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d259c640 1 -- 172.21.15.19:0/726506409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581201fde10 con 0x5581201f1490 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33d259c640 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fca50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f33c400e700 tx=0x7f33c40092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8526962b3f9eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33b9ffb640 1 -- 172.21.15.19:0/726506409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4038070 con 0x5581201f1490 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33b9ffb640 1 -- 172.21.15.19:0/726506409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f33c401baf0 con 0x5581201f1490 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33db97d740 1 -- 172.21.15.19:0/726506409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33cc024070 con 0x5581201f1490 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33b9ffb640 1 -- 172.21.15.19:0/726506409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f33c4041050 con 0x5581201f1490 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33db97d740 10 client.16881 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.172+0000 7f33db97d740 1 -- 172.21.15.19:0/726506409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558120277300 con 0x5581201f1490 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33b9ffb640 1 -- 172.21.15.19:0/726506409 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc024070 con 0x5581201f1490 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33b9ffb640 1 -- 172.21.15.19:0/726506409 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f33c403b020 con 0x5581201f1490 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33db97d740 10 client.16881 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33db97d740 10 client.16881 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33db97d740 20 client.16881 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33db97d740 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558120277a70 0x558120279e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33db97d740 4 client.16881 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cbddd4d7", "release": true, "await": true}] 2024-04-23T18:35:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33db97d740 1 -- 172.21.15.19:0/726506409 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cbddd4d7", "release": true, "await": true}) -- 0x5581201640f0 con 0x558120277a70 2024-04-23T18:35:46.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558120277a70 0x558120279e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33d1d9b640 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558120277a70 0x558120279e90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5fbab9263cba1b server_cookie=7f95ed8719e4cac9 in_seq=0 out_seq=0 2024-04-23T18:35:46.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.173+0000 7f33b9ffb640 10 client.16881 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33b9ffb640 1 -- 172.21.15.19:0/726506409 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5581201640f0 con 0x558120277a70 2024-04-23T18:35:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33b9ffb640 10 client.16881 handle_command_reply: tid=0 2024-04-23T18:35:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 client.16881 shutdown 2024-04-23T18:35:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 -- 172.21.15.19:0/726506409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 msgr2=0x5581201fca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fca50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f33c400e700 tx=0x7f33c40092d0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 -- 172.21.15.19:0/726506409 shutdown_connections 2024-04-23T18:35:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558120277a70 0x558120279e90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581201f1db0 0x5581201fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581201fc480 0x5581201fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33b9ffb640 0 client.16881 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 --2- 172.21.15.19:0/726506409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581201f1490 0x5581201fca50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 -- 172.21.15.19:0/726506409 >> 172.21.15.19:0/726506409 conn(0x5581201ed7b0 msgr2=0x5581201ec140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 -- 172.21.15.19:0/726506409 shutdown_connections 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 1 -- 172.21.15.19:0/726506409 wait complete. 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.177+0000 7f33db97d740 20 client.16881 trim_cache size 0 max 16384 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:35:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 40, 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "cbddd4d7": { 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 40, 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 78.8, 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:35:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:35:46.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:35:46.175 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:35:46.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:46.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:46.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:46.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:46.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:46.175 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:35:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 -- 172.21.15.19:0/89852165 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f33bc0a3d20 msgr2=0x7f33bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f33bc0a3d20 0x7f33bc0a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0092f0 tx=0x7f33cc0096d0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 -- 172.21.15.19:0/89852165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4106160 msgr2=0x7f33d4198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4106160 0x7f33d4198c60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f33c8032fd0 tx=0x7f33c8004270 comp rx=0 tx=0).stop 2024-04-23T18:35:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 -- 172.21.15.19:0/89852165 shutdown_connections 2024-04-23T18:35:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f33bc0a3d20 0x7f33bc0a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33d419cfe0 0x7f33d419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4106160 0x7f33d4198c60 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 --2- 172.21.15.19:0/89852165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d4101090 0x7f33d4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 -- 172.21.15.19:0/89852165 >> 172.21.15.19:0/89852165 conn(0x7f33d4077f50 msgr2=0x7f33d40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 -- 172.21.15.19:0/89852165 shutdown_connections 2024-04-23T18:35:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.178+0000 7f33d8c6b640 1 -- 172.21.15.19:0/89852165 wait complete. 2024-04-23T18:35:46.183 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cbddd4d7', seconds elapsed: 19.9 2024-04-23T18:35:46.184 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.8 2024-04-23T18:35:46.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:35:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 -- 172.21.15.19:0/2871399806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f578010c520 msgr2=0x7f578010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 --2- 172.21.15.19:0/2871399806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f578010c520 0x7f578010e9c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5778009ca0 tx=0x7f577802a9e0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 -- 172.21.15.19:0/2871399806 shutdown_connections 2024-04-23T18:35:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 --2- 172.21.15.19:0/2871399806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5780103b00 0x7f578010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 --2- 172.21.15.19:0/2871399806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f578010c520 0x7f578010e9c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 --2- 172.21.15.19:0/2871399806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57801031e0 0x7f57801035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 -- 172.21.15.19:0/2871399806 >> 172.21.15.19:0/2871399806 conn(0x7f5780077f50 msgr2=0x7f5780078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 -- 172.21.15.19:0/2871399806 shutdown_connections 2024-04-23T18:35:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 -- 172.21.15.19:0/2871399806 wait complete. 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 Processor -- start 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.294+0000 7f57887a1640 1 -- start start 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f57887a1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57801031e0 0x7f5780198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f57887a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5780103b00 0x7f5780198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f57887a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f578010c520 0x7f578019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f57887a1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5780112190 con 0x7f578010c520 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f57887a1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5780112010 con 0x7f5780103b00 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f57887a1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5780112310 con 0x7f57801031e0 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5786d3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f578010c520 0x7f578019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5786d3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f578010c520 0x7f578019cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37734/0 (socket says 172.21.15.19:37734) 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5786d3e640 1 -- 172.21.15.19:0/2736213387 learned_addr learned my addr 172.21.15.19:0/2736213387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5785d3c640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5780103b00 0x7f5780198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f578653d640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57801031e0 0x7f5780198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5786d3e640 1 -- 172.21.15.19:0/2736213387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57801031e0 msgr2=0x7f5780198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5786d3e640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57801031e0 0x7f5780198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5786d3e640 1 -- 172.21.15.19:0/2736213387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5780103b00 msgr2=0x7f5780198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5786d3e640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5780103b00 0x7f5780198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5786d3e640 1 -- 172.21.15.19:0/2736213387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578019d6d0 con 0x7f578010c520 2024-04-23T18:35:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5785d3c640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5780103b00 0x7f5780198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f5786d3e640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f578010c520 0x7f578019cfa0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5778006010 tx=0x7f577802fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f57777fe640 1 -- 172.21.15.19:0/2736213387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778033040 con 0x7f578010c520 2024-04-23T18:35:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f57887a1640 1 -- 172.21.15.19:0/2736213387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578019d960 con 0x7f578010c520 2024-04-23T18:35:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.295+0000 7f57777fe640 1 -- 172.21.15.19:0/2736213387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577802e7a0 con 0x7f578010c520 2024-04-23T18:35:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.296+0000 7f57887a1640 1 -- 172.21.15.19:0/2736213387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57801a5250 con 0x7f578010c520 2024-04-23T18:35:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.296+0000 7f57777fe640 1 -- 172.21.15.19:0/2736213387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778043750 con 0x7f578010c520 2024-04-23T18:35:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.297+0000 7f57887a1640 1 -- 172.21.15.19:0/2736213387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5780068d70 con 0x7f578010c520 2024-04-23T18:35:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.298+0000 7f57777fe640 1 -- 172.21.15.19:0/2736213387 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5778036070 con 0x7f578010c520 2024-04-23T18:35:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.298+0000 7f57777fe640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f576c0a3d20 0x7f576c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.298+0000 7f578653d640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f576c0a3d20 0x7f576c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.298+0000 7f57777fe640 1 -- 172.21.15.19:0/2736213387 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f57780f1640 con 0x7f578010c520 2024-04-23T18:35:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.299+0000 7f578653d640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f576c0a3d20 0x7f576c0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f577c0058c0 tx=0x7f577c005c60 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.303+0000 7f57777fe640 1 -- 172.21.15.19:0/2736213387 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f577803e070 con 0x7f578010c520 2024-04-23T18:35:46.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:45 smithi044 ceph-mon[32458]: pgmap v297: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-04-23T18:35:46.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:45 smithi155 ceph-mon[33468]: pgmap v297: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-04-23T18:35:46.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.528+0000 7f57887a1640 1 -- 172.21.15.19:0/2736213387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f578002d5c0 con 0x7f578010c520 2024-04-23T18:35:46.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.529+0000 7f57777fe640 1 -- 172.21.15.19:0/2736213387 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f57780098b0 con 0x7f578010c520 2024-04-23T18:35:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:35:46.521 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:35:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:35:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.530+0000 7f57887a1640 1 -- 172.21.15.19:0/2736213387 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f576c0a3d20 msgr2=0x7f576c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.530+0000 7f57887a1640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f576c0a3d20 0x7f576c0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f577c0058c0 tx=0x7f577c005c60 comp rx=0 tx=0).stop 2024-04-23T18:35:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.530+0000 7f57887a1640 1 -- 172.21.15.19:0/2736213387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f578010c520 msgr2=0x7f578019cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.530+0000 7f57887a1640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f578010c520 0x7f578019cfa0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5778006010 tx=0x7f577802fc00 comp rx=0 tx=0).stop 2024-04-23T18:35:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.531+0000 7f57887a1640 1 -- 172.21.15.19:0/2736213387 shutdown_connections 2024-04-23T18:35:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.531+0000 7f57887a1640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f576c0a3d20 0x7f576c0a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.531+0000 7f57887a1640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5780103b00 0x7f5780198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.531+0000 7f57887a1640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f578010c520 0x7f578019cfa0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.531+0000 7f57887a1640 1 --2- 172.21.15.19:0/2736213387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57801031e0 0x7f5780198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.531+0000 7f57887a1640 1 -- 172.21.15.19:0/2736213387 >> 172.21.15.19:0/2736213387 conn(0x7f5780077f50 msgr2=0x7f57800fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.531+0000 7f57887a1640 1 -- 172.21.15.19:0/2736213387 shutdown_connections 2024-04-23T18:35:46.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.531+0000 7f57887a1640 1 -- 172.21.15.19:0/2736213387 wait complete. 2024-04-23T18:35:46.533 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 78.8 --await-for 42.0' 2024-04-23T18:35:46.533 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 78.8 --await-for 42.0 2024-04-23T18:35:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.653+0000 7fd09f1ad640 1 -- 172.21.15.19:0/3759260569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0981016d0 msgr2=0x7fd09810e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.653+0000 7fd09f1ad640 1 --2- 172.21.15.19:0/3759260569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0981016d0 0x7fd09810e420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd088009900 tx=0x7fd08802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.653+0000 7fd09f1ad640 1 -- 172.21.15.19:0/3759260569 shutdown_connections 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.653+0000 7fd09f1ad640 1 --2- 172.21.15.19:0/3759260569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09810e960 0x7fd098110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.653+0000 7fd09f1ad640 1 --2- 172.21.15.19:0/3759260569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0981016d0 0x7fd09810e420 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.653+0000 7fd09f1ad640 1 --2- 172.21.15.19:0/3759260569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098100db0 0x7fd098101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.653+0000 7fd09f1ad640 1 -- 172.21.15.19:0/3759260569 >> 172.21.15.19:0/3759260569 conn(0x7fd098077fb0 msgr2=0x7fd0980783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.653+0000 7fd09f1ad640 1 -- 172.21.15.19:0/3759260569 shutdown_connections 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.653+0000 7fd09f1ad640 1 -- 172.21.15.19:0/3759260569 wait complete. 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09f1ad640 1 Processor -- start 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09f1ad640 1 -- start start 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09f1ad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098100db0 0x7fd0981a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09f1ad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0981016d0 0x7fd0981a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09f1ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd09810e960 0x7fd09819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09f1ad640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd098114410 con 0x7fd09810e960 2024-04-23T18:35:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09f1ad640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd098114290 con 0x7fd098100db0 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09f1ad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd098114590 con 0x7fd0981016d0 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09d74a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd09810e960 0x7fd09819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09d74a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd09810e960 0x7fd09819a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37750/0 (socket says 172.21.15.19:37750) 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09d74a640 1 -- 172.21.15.19:0/2453865996 learned_addr learned my addr 172.21.15.19:0/2453865996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd08ffff640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0981016d0 0x7fd0981a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.654+0000 7fd09cf49640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098100db0 0x7fd0981a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd09d74a640 1 -- 172.21.15.19:0/2453865996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0981016d0 msgr2=0x7fd0981a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0981016d0 0x7fd0981a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd09d74a640 1 -- 172.21.15.19:0/2453865996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098100db0 msgr2=0x7fd0981a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098100db0 0x7fd0981a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd09d74a640 1 -- 172.21.15.19:0/2453865996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09819aff0 con 0x7fd09810e960 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd09cf49640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098100db0 0x7fd0981a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd09810e960 0x7fd09819a870 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd09000d9d0 tx=0x7fd09000de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd08dffb640 1 -- 172.21.15.19:0/2453865996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090017030 con 0x7fd09810e960 2024-04-23T18:35:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd09f1ad640 1 -- 172.21.15.19:0/2453865996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09819b2e0 con 0x7fd09810e960 2024-04-23T18:35:46.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd08dffb640 1 -- 172.21.15.19:0/2453865996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd090003e20 con 0x7fd09810e960 2024-04-23T18:35:46.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd08dffb640 1 -- 172.21.15.19:0/2453865996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0900047d0 con 0x7fd09810e960 2024-04-23T18:35:46.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.655+0000 7fd09f1ad640 1 -- 172.21.15.19:0/2453865996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0981a7580 con 0x7fd09810e960 2024-04-23T18:35:46.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.657+0000 7fd08dffb640 1 -- 172.21.15.19:0/2453865996 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd09001a070 con 0x7fd09810e960 2024-04-23T18:35:46.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd0a1fa0740 1 Processor -- start 2024-04-23T18:35:46.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd0a1fa0740 1 -- start start 2024-04-23T18:35:46.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd08dffb640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0800a3d20 0x7fd0800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc570a0 0x55bcadc57480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc579c0 0x55bcadc61b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc62090 0x55bcadc64480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bcadbbc370 con 0x55bcadc579c0 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bcadb592e0 con 0x55bcadc570a0 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bcadbb8330 con 0x55bcadc62090 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd08ffff640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0800a3d20 0x7fd0800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd08dffb640 1 -- 172.21.15.19:0/2453865996 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fd0900d04f0 con 0x7fd09810e960 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc579c0 0x55bcadc61b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd09d74a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc62090 0x55bcadc64480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc579c0 0x55bcadc61b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37764/0 (socket says 172.21.15.19:37764) 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd08ffff640 1 -- 172.21.15.19:0/3831257788 learned_addr learned my addr 172.21.15.19:0/3831257788 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.658+0000 7fd09cf49640 1 --2- 172.21.15.19:0/3831257788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc570a0 0x55bcadc57480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.659+0000 7fd09d74a640 1 -- 172.21.15.19:0/3831257788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc570a0 msgr2=0x55bcadc57480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.659+0000 7fd09d74a640 1 --2- 172.21.15.19:0/3831257788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc570a0 0x55bcadc57480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.659+0000 7fd09d74a640 1 -- 172.21.15.19:0/3831257788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc579c0 msgr2=0x55bcadc61b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.659+0000 7fd09d74a640 1 --2- 172.21.15.19:0/3831257788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc579c0 0x55bcadc61b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.659+0000 7fd09d74a640 1 -- 172.21.15.19:0/3831257788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcad99ad80 con 0x55bcadc62090 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.659+0000 7fd08ffff640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0800a3d20 0x7fd0800a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd08802a930 tx=0x7fd088032ee0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.659+0000 7fd09d74a640 1 --2- 172.21.15.19:0/3831257788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc62090 0x55bcadc64480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd0900cdbe0 tx=0x7fd09009c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dffb57e4dca50d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.659+0000 7fd07effd640 1 -- 172.21.15.19:0/3831257788 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090067da0 con 0x55bcadc62090 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.659+0000 7fd07effd640 1 -- 172.21.15.19:0/3831257788 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd09009c430 con 0x55bcadc62090 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.659+0000 7fd07effd640 1 -- 172.21.15.19:0/3831257788 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09000bb70 con 0x55bcadc62090 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/3831257788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc62090 msgr2=0x55bcadc64480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/3831257788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc62090 0x55bcadc64480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd0900cdbe0 tx=0x7fd09009c330 comp rx=0 tx=0).stop 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/3831257788 shutdown_connections 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/3831257788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc62090 0x55bcadc64480 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/3831257788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc579c0 0x55bcadc61b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/3831257788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc570a0 0x55bcadc57480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/3831257788 >> 172.21.15.19:0/3831257788 conn(0x55bcadc533c0 msgr2=0x55bcadc537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/3831257788 shutdown_connections 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/3831257788 wait complete. 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 -- start start 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.660+0000 7fd0a1fa0740 1 Processor -- start 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc570a0 0x55bcadc5efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 0x55bcadc5f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bcadc64e80 con 0x55bcadc62090 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bcadb56fb0 con 0x55bcadc579c0 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bcadc65000 con 0x55bcadc570a0 2024-04-23T18:35:46.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd0a1fa0740 10 client.0 _ll_register_callbacks cb 0x55bcadb569a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd09d74a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd09d74a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37776/0 (socket says 172.21.15.19:37776) 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd09d74a640 1 -- 172.21.15.19:0/2725431052 learned_addr learned my addr 172.21.15.19:0/2725431052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd09cf49640 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc570a0 0x55bcadc5efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd08ffff640 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 0x55bcadc5f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd09d74a640 1 -- 172.21.15.19:0/2725431052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc570a0 msgr2=0x55bcadc5efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc570a0 0x55bcadc5efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd09d74a640 1 -- 172.21.15.19:0/2725431052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 msgr2=0x55bcadc5f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 0x55bcadc5f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd09d74a640 1 -- 172.21.15.19:0/2725431052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcad9fb510 con 0x55bcadc62090 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd08ffff640 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 0x55bcadc5f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5fa40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd09000b870 tx=0x7fd090067310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc0eca4d2df440e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd077fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd077fff640 1 -- 172.21.15.19:0/2725431052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090098aa0 con 0x55bcadc62090 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.661+0000 7fd077fff640 1 -- 172.21.15.19:0/2725431052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fd0900cde60 con 0x55bcadc62090 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2725431052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09009c430 con 0x55bcadc62090 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd077fff640 1 -- 172.21.15.19:0/2725431052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd090098c40 con 0x55bcadc62090 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd0a1fa0740 10 client.16899 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2725431052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bcad88e670 con 0x55bcadc62090 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd077fff640 1 -- 172.21.15.19:0/2725431052 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd09009c430 con 0x55bcadc62090 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd077fff640 1 -- 172.21.15.19:0/2725431052 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd090024060 con 0x55bcadc62090 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd0a1fa0740 10 client.16899 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd0a1fa0740 10 client.16899 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd0a1fa0740 20 client.16899 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bcadc72240 0x55bcadc74630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd0a1fa0740 4 client.16899 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2725431052 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcadbc9d90 con 0x55bcadc72240 2024-04-23T18:35:46.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.662+0000 7fd09cf49640 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bcadc72240 0x55bcadc74630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.663+0000 7fd077fff640 10 client.16899 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:46.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.663+0000 7fd09cf49640 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bcadc72240 0x55bcadc74630 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d54b83254bf75d6 server_cookie=4fbc98c30eb149c1 in_seq=0 out_seq=0 2024-04-23T18:35:46.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd077fff640 1 -- 172.21.15.19:0/2725431052 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcadbc9d90 con 0x55bcadc72240 2024-04-23T18:35:46.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd077fff640 10 client.16899 handle_command_reply: tid=0 2024-04-23T18:35:46.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 client.16899 shutdown 2024-04-23T18:35:46.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2725431052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 msgr2=0x55bcadc5fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5fa40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd09000b870 tx=0x7fd090067310 comp rx=0 tx=0).stop 2024-04-23T18:35:46.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2725431052 shutdown_connections 2024-04-23T18:35:46.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bcadc72240 0x55bcadc74630 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 0x55bcadc5f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5fa40 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd077fff640 0 client.16899 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:46.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2725431052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc570a0 0x55bcadc5efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2725431052 >> 172.21.15.19:0/2725431052 conn(0x55bcadc533c0 msgr2=0x55bcadc52600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2725431052 shutdown_connections 2024-04-23T18:35:46.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2725431052 wait complete. 2024-04-23T18:35:46.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.664+0000 7fd0a1fa0740 20 client.16899 trim_cache size 0 max 16384 2024-04-23T18:35:46.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.681+0000 7fd0a1fa0740 1 Processor -- start 2024-04-23T18:35:46.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.681+0000 7fd0a1fa0740 1 -- start start 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc570a0 0x55bcadc5f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc579c0 0x55bcadc58f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981afe30 con 0x55bcadc62090 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981b1970 con 0x55bcadc570a0 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981b0500 con 0x55bcadc579c0 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd09d74a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd09d74a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37784/0 (socket says 172.21.15.19:37784) 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd09d74a640 1 -- 172.21.15.19:0/2409403175 learned_addr learned my addr 172.21.15.19:0/2409403175 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd08ffff640 1 --2- 172.21.15.19:0/2409403175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc579c0 0x55bcadc58f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd09cf49640 1 --2- 172.21.15.19:0/2409403175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc570a0 0x55bcadc5f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd09d74a640 1 -- 172.21.15.19:0/2409403175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc579c0 msgr2=0x55bcadc58f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2409403175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc579c0 0x55bcadc58f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd09d74a640 1 -- 172.21.15.19:0/2409403175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc570a0 msgr2=0x55bcadc5f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2409403175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc570a0 0x55bcadc5f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd09d74a640 1 -- 172.21.15.19:0/2409403175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcadc5dfb0 con 0x55bcadc62090 2024-04-23T18:35:46.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2409403175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5c280 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd090090290 tx=0x7fd09009c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4be07105c8ef764 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd07e7fc640 1 -- 172.21.15.19:0/2409403175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09009bd90 con 0x55bcadc62090 2024-04-23T18:35:46.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd07e7fc640 1 -- 172.21.15.19:0/2409403175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0900b5070 con 0x55bcadc62090 2024-04-23T18:35:46.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.682+0000 7fd07e7fc640 1 -- 172.21.15.19:0/2409403175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0900cde60 con 0x55bcadc62090 2024-04-23T18:35:46.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2409403175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 msgr2=0x55bcadc5c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2409403175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5c280 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd090090290 tx=0x7fd09009c330 comp rx=0 tx=0).stop 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2409403175 shutdown_connections 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2409403175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadc5c280 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2409403175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc579c0 0x55bcadc58f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2409403175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc570a0 0x55bcadc5f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2409403175 >> 172.21.15.19:0/2409403175 conn(0x55bcadc533c0 msgr2=0x55bcadc58830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2409403175 shutdown_connections 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2409403175 wait complete. 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 -- start start 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.683+0000 7fd0a1fa0740 1 Processor -- start 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc570a0 0x55bcadc758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 0x55bcadc75e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd0a1fa0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadcd13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981b12a0 con 0x55bcadc62090 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981b0bd0 con 0x55bcadc579c0 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd0a1fa0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981ade30 con 0x55bcadc570a0 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd0a1fa0740 10 client.0 _ll_register_callbacks cb 0x7fd0900092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd09d74a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadcd13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd09d74a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadcd13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37788/0 (socket says 172.21.15.19:37788) 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd09d74a640 1 -- 172.21.15.19:0/2210162107 learned_addr learned my addr 172.21.15.19:0/2210162107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd09cf49640 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc570a0 0x55bcadc758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd08ffff640 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 0x55bcadc75e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd09d74a640 1 -- 172.21.15.19:0/2210162107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc570a0 msgr2=0x55bcadc758d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc570a0 0x55bcadc758d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd09d74a640 1 -- 172.21.15.19:0/2210162107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 msgr2=0x55bcadc75e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 0x55bcadc75e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd09d74a640 1 -- 172.21.15.19:0/2210162107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcadcd19e0 con 0x55bcadc62090 2024-04-23T18:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd09d74a640 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadcd13f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd09009c130 tx=0x7fd0900980b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d14a938ba91d76e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.684+0000 7fd077fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd077fff640 1 -- 172.21.15.19:0/2210162107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09009bd90 con 0x55bcadc62090 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd077fff640 1 -- 172.21.15.19:0/2210162107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fd0900b5db0 con 0x55bcadc62090 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2210162107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0900b5070 con 0x55bcadc62090 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd077fff640 1 -- 172.21.15.19:0/2210162107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd0900a07e0 con 0x55bcadc62090 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd0a1fa0740 10 client.16905 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2210162107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bcadcd19e0 con 0x55bcadc62090 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd077fff640 1 -- 172.21.15.19:0/2210162107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0900b5070 con 0x55bcadc62090 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd077fff640 1 -- 172.21.15.19:0/2210162107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd090098180 con 0x55bcadc62090 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd0a1fa0740 10 client.16905 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd0a1fa0740 10 client.16905 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd0a1fa0740 20 client.16905 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bcadcd9100 0x55bcadcdb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd0a1fa0740 4 client.16905 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.8, "await_for": 42.0}] 2024-04-23T18:35:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2210162107 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.8, "await_for": 42.0}) -- 0x55bcadbc3c00 con 0x55bcadcd9100 2024-04-23T18:35:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.685+0000 7fd09cf49640 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bcadcd9100 0x55bcadcdb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.686+0000 7fd077fff640 10 client.16905 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.686+0000 7fd09cf49640 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bcadcd9100 0x55bcadcdb4f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202ece3b4110725d server_cookie=9e94b8db71983028 in_seq=0 out_seq=0 2024-04-23T18:35:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd077fff640 1 -- 172.21.15.19:0/2210162107 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55bcadbc3c00 con 0x55bcadcd9100 2024-04-23T18:35:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd077fff640 10 client.16905 handle_command_reply: tid=0 2024-04-23T18:35:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 client.16905 shutdown 2024-04-23T18:35:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2210162107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 msgr2=0x55bcadcd13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadcd13f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd09009c130 tx=0x7fd0900980b0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2210162107 shutdown_connections 2024-04-23T18:35:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bcadcd9100 0x55bcadcdb4f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcadc579c0 0x55bcadc75e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcadc62090 0x55bcadcd13f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd077fff640 0 client.16905 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 --2- 172.21.15.19:0/2210162107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bcadc570a0 0x55bcadc758d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2210162107 >> 172.21.15.19:0/2210162107 conn(0x55bcadc533c0 msgr2=0x55bcadc59080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2210162107 shutdown_connections 2024-04-23T18:35:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 1 -- 172.21.15.19:0/2210162107 wait complete. 2024-04-23T18:35:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.701+0000 7fd0a1fa0740 20 client.16905 trim_cache size 0 max 16384 2024-04-23T18:35:46.694 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 42, 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "3678ca7a": { 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 42, 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 78.8, 2024-04-23T18:35:46.695 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:35:46.696 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:35:46.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 -- 172.21.15.19:0/2453865996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0800a3d20 msgr2=0x7fd0800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0800a3d20 0x7fd0800a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd08802a930 tx=0x7fd088032ee0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 -- 172.21.15.19:0/2453865996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd09810e960 msgr2=0x7fd09819a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd09810e960 0x7fd09819a870 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd09000d9d0 tx=0x7fd09000de90 comp rx=0 tx=0).stop 2024-04-23T18:35:46.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 -- 172.21.15.19:0/2453865996 shutdown_connections 2024-04-23T18:35:46.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0800a3d20 0x7fd0800a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd09810e960 0x7fd09819a870 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0981016d0 0x7fd0981a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 --2- 172.21.15.19:0/2453865996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098100db0 0x7fd0981a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 -- 172.21.15.19:0/2453865996 >> 172.21.15.19:0/2453865996 conn(0x7fd098077fb0 msgr2=0x7fd09810ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 -- 172.21.15.19:0/2453865996 shutdown_connections 2024-04-23T18:35:46.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:35:46.702+0000 7fd09f1ad640 1 -- 172.21.15.19:0/2453865996 wait complete. 2024-04-23T18:35:46.704 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3678ca7a', quiesce took 0.5 seconds. Will release after: 18.8 2024-04-23T18:35:47.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:46 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1445937703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:47.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:46 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2736213387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:47.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:46 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1445937703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:47.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:46 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2736213387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:47.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:46 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1445937703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:47.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:46 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2736213387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:48.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:47 smithi019 ceph-mon[29215]: pgmap v298: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:35:48.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:47 smithi044 ceph-mon[32458]: pgmap v298: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:35:48.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:47 smithi155 ceph-mon[33468]: pgmap v298: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.363+0000 7fd9c8708640 1 -- 172.21.15.19:0/2662501449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c0101a90 msgr2=0x7fd9c0105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.363+0000 7fd9c8708640 1 --2- 172.21.15.19:0/2662501449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c0101a90 0x7fd9c0105ad0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009900 tx=0x7fd9b402a410 comp rx=0 tx=0).stop 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.363+0000 7fd9c8708640 1 -- 172.21.15.19:0/2662501449 shutdown_connections 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.363+0000 7fd9c8708640 1 --2- 172.21.15.19:0/2662501449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c01061b0 0x7fd9c010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.363+0000 7fd9c8708640 1 --2- 172.21.15.19:0/2662501449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c0101a90 0x7fd9c0105ad0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.363+0000 7fd9c8708640 1 --2- 172.21.15.19:0/2662501449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9c01010e0 0x7fd9c01014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.363+0000 7fd9c8708640 1 -- 172.21.15.19:0/2662501449 >> 172.21.15.19:0/2662501449 conn(0x7fd9c00771d0 msgr2=0x7fd9c00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.364+0000 7fd9c8708640 1 -- 172.21.15.19:0/2662501449 shutdown_connections 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.364+0000 7fd9c8708640 1 -- 172.21.15.19:0/2662501449 wait complete. 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.364+0000 7fd9c8708640 1 Processor -- start 2024-04-23T18:35:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.364+0000 7fd9c8708640 1 -- start start 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.364+0000 7fd9c8708640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9c01010e0 0x7fd9c0102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c8708640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c01061b0 0x7fd9c01030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c8708640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01035f0 0x7fd9c01a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c8708640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c01110f0 con 0x7fd9c01035f0 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c8708640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0110f70 con 0x7fd9c01061b0 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c8708640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0111270 con 0x7fd9c01010e0 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c6ca5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01035f0 0x7fd9c01a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c6ca5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01035f0 0x7fd9c01a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37798/0 (socket says 172.21.15.19:37798) 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c6ca5640 1 -- 172.21.15.19:0/1466207129 learned_addr learned my addr 172.21.15.19:0/1466207129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c01061b0 0x7fd9c01030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c64a4640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9c01010e0 0x7fd9c0102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c6ca5640 1 -- 172.21.15.19:0/1466207129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9c01010e0 msgr2=0x7fd9c0102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c6ca5640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9c01010e0 0x7fd9c0102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c6ca5640 1 -- 172.21.15.19:0/1466207129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c01061b0 msgr2=0x7fd9c01030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c6ca5640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c01061b0 0x7fd9c01030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c6ca5640 1 -- 172.21.15.19:0/1466207129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c01a6f30 con 0x7fd9c01035f0 2024-04-23T18:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c6ca5640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01035f0 0x7fd9c01a6830 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800d750 tx=0x7fd9b800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9b37fe640 1 -- 172.21.15.19:0/1466207129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8017070 con 0x7fd9c01035f0 2024-04-23T18:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9b37fe640 1 -- 172.21.15.19:0/1466207129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9b80054d0 con 0x7fd9c01035f0 2024-04-23T18:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9b37fe640 1 -- 172.21.15.19:0/1466207129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8004c60 con 0x7fd9c01035f0 2024-04-23T18:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c8708640 1 -- 172.21.15.19:0/1466207129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c01a7220 con 0x7fd9c01035f0 2024-04-23T18:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.365+0000 7fd9c8708640 1 -- 172.21.15.19:0/1466207129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c01a7740 con 0x7fd9c01035f0 2024-04-23T18:35:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.368+0000 7fd9b37fe640 1 -- 172.21.15.19:0/1466207129 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9b8009bd0 con 0x7fd9c01035f0 2024-04-23T18:35:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.368+0000 7fd9cb429740 1 Processor -- start 2024-04-23T18:35:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.368+0000 7fd9cb429740 1 -- start start 2024-04-23T18:35:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab5ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab68bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9cb429740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eadaac0970 con 0x55eadab5b620 2024-04-23T18:35:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9cb429740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55eadaa5d4a0 con 0x55eadab66800 2024-04-23T18:35:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9cb429740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eadaabc930 con 0x55eadab5bf40 2024-04-23T18:35:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c5ca3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab5ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c5ca3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab5ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37800/0 (socket says 172.21.15.19:37800) 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3055450092 learned_addr learned my addr 172.21.15.19:0/3055450092 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9b37fe640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9ac0a3e70 0x7fd9ac0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c64a4640 1 --2- 172.21.15.19:0/3055450092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/3055450092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab68bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c6ca5640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9ac0a3e70 0x7fd9ac0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3055450092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 msgr2=0x55eadab662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/3055450092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9b37fe640 1 -- 172.21.15.19:0/1466207129 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fd9b80d0190 con 0x7fd9c01035f0 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3055450092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 msgr2=0x55eadab68bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/3055450092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab68bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3055450092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eada7d3210 con 0x55eadab5b620 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.369+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/3055450092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab5ba00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x55eadab58cc0 tx=0x7fd9b4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e619ab652623f85b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9b0ff9640 1 -- 172.21.15.19:0/3055450092 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4035070 con 0x55eadab5b620 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9c6ca5640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9ac0a3e70 0x7fd9ac0a6330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8098530 tx=0x7fd9b8066c20 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9b0ff9640 1 -- 172.21.15.19:0/3055450092 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9b4004020 con 0x55eadab5b620 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9b0ff9640 1 -- 172.21.15.19:0/3055450092 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4005320 con 0x55eadab5b620 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9cb429740 1 -- 172.21.15.19:0/3055450092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 msgr2=0x55eadab5ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3055450092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab5ba00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x55eadab58cc0 tx=0x7fd9b4005b50 comp rx=0 tx=0).stop 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9cb429740 1 -- 172.21.15.19:0/3055450092 shutdown_connections 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3055450092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab68bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3055450092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3055450092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab5ba00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.370+0000 7fd9cb429740 1 -- 172.21.15.19:0/3055450092 >> 172.21.15.19:0/3055450092 conn(0x55eadab57b30 msgr2=0x55eadab57f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 1 -- 172.21.15.19:0/3055450092 shutdown_connections 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 1 -- 172.21.15.19:0/3055450092 wait complete. 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 1 -- start start 2024-04-23T18:35:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 1 Processor -- start 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5b620 0x55eadab661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5bf40 0x55eadab634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab63ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eadab69370 con 0x55eadab5bf40 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55eadaa5b0b0 con 0x55eadab66800 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eadab694f0 con 0x55eadab5b620 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9c64a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5bf40 0x55eadab634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9cb429740 10 client.0 _ll_register_callbacks cb 0x55eadaa48d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9c64a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5bf40 0x55eadab634c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37802/0 (socket says 172.21.15.19:37802) 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.371+0000 7fd9c64a4640 1 -- 172.21.15.19:0/272341216 learned_addr learned my addr 172.21.15.19:0/272341216 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5b620 0x55eadab661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab63ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9c64a4640 1 -- 172.21.15.19:0/272341216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5b620 msgr2=0x55eadab661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9c64a4640 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5b620 0x55eadab661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9c64a4640 1 -- 172.21.15.19:0/272341216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 msgr2=0x55eadab63ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9c64a4640 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab63ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9c64a4640 1 -- 172.21.15.19:0/272341216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eada89f620 con 0x55eadab5bf40 2024-04-23T18:35:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab63ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9c64a4640 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5bf40 0x55eadab634c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009fc0 tx=0x7fd9bc0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484a3e94637d652d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/272341216 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc004b20 con 0x55eadab5bf40 2024-04-23T18:35:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/272341216 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fd9bc004cc0 con 0x55eadab5bf40 2024-04-23T18:35:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9cb429740 1 -- 172.21.15.19:0/272341216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9b4004020 con 0x55eadab5bf40 2024-04-23T18:35:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/272341216 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd9bc0118d0 con 0x55eadab5bf40 2024-04-23T18:35:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9cb429740 10 client.16917 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.372+0000 7fd9cb429740 1 -- 172.21.15.19:0/272341216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eadab753b0 con 0x55eadab5bf40 2024-04-23T18:35:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.373+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/272341216 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9b4004020 con 0x55eadab5bf40 2024-04-23T18:35:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.373+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/272341216 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd9bc01bc20 con 0x55eadab5bf40 2024-04-23T18:35:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.373+0000 7fd9cb429740 10 client.16917 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.373+0000 7fd9cb429740 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.373+0000 7fd9cb429740 20 client.16917 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.373+0000 7fd9cb429740 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eadab768c0 0x55eadab78cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.373+0000 7fd9cb429740 4 client.16917 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.373+0000 7fd9cb429740 1 -- 172.21.15.19:0/272341216 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eadaace390 con 0x55eadab768c0 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.373+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eadab768c0 0x55eadab78cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.374+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eadab768c0 0x55eadab78cb0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31eccb421a763d96 server_cookie=1782a27324e469a8 in_seq=0 out_seq=0 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.374+0000 7fd9a9ffb640 10 client.16917 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.374+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/272341216 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eadaace390 con 0x55eadab768c0 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.374+0000 7fd9a9ffb640 10 client.16917 handle_command_reply: tid=0 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.374+0000 7fd9cb429740 1 client.16917 shutdown 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.374+0000 7fd9cb429740 1 -- 172.21.15.19:0/272341216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5bf40 msgr2=0x55eadab634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.374+0000 7fd9cb429740 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5bf40 0x55eadab634c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009fc0 tx=0x7fd9bc0044d0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.375+0000 7fd9cb429740 1 -- 172.21.15.19:0/272341216 shutdown_connections 2024-04-23T18:35:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.375+0000 7fd9cb429740 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eadab768c0 0x55eadab78cb0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.375+0000 7fd9cb429740 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab63ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.375+0000 7fd9cb429740 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5bf40 0x55eadab634c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.375+0000 7fd9a9ffb640 0 client.16917 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.375+0000 7fd9cb429740 1 --2- 172.21.15.19:0/272341216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5b620 0x55eadab661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.375+0000 7fd9cb429740 1 -- 172.21.15.19:0/272341216 >> 172.21.15.19:0/272341216 conn(0x55eadab57b30 msgr2=0x55eadab5d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.375+0000 7fd9cb429740 1 -- 172.21.15.19:0/272341216 shutdown_connections 2024-04-23T18:35:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.375+0000 7fd9cb429740 1 -- 172.21.15.19:0/272341216 wait complete. 2024-04-23T18:35:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.375+0000 7fd9cb429740 20 client.16917 trim_cache size 0 max 16384 2024-04-23T18:35:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9cb429740 1 Processor -- start 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9cb429740 1 -- start start 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab5b620 0x55eadab72230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab65d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab66800 0x55eadab7e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9cb429740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c01afbf0 con 0x55eadab66800 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9cb429740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c01b1730 con 0x55eadab5b620 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9cb429740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c01b02c0 con 0x55eadab5bf40 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9c5ca3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab66800 0x55eadab7e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9c5ca3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab66800 0x55eadab7e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37818/0 (socket says 172.21.15.19:37818) 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3236390620 learned_addr learned my addr 172.21.15.19:0/3236390620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3236390620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 msgr2=0x55eadab65d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/3236390620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab65d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3236390620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab5b620 msgr2=0x55eadab72230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/3236390620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab5b620 0x55eadab72230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.390+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3236390620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eadab5bd70 con 0x55eadab66800 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/3236390620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab66800 0x55eadab7e580 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009900 tx=0x7fd9b4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8965336e80c9124 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9abfff640 1 -- 172.21.15.19:0/3236390620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4035040 con 0x55eadab66800 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9abfff640 1 -- 172.21.15.19:0/3236390620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9b4031d60 con 0x55eadab66800 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9abfff640 1 -- 172.21.15.19:0/3236390620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40055d0 con 0x55eadab66800 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9cb429740 1 -- 172.21.15.19:0/3236390620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab66800 msgr2=0x55eadab7e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3236390620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab66800 0x55eadab7e580 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009900 tx=0x7fd9b4005850 comp rx=0 tx=0).stop 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9cb429740 1 -- 172.21.15.19:0/3236390620 shutdown_connections 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3236390620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab66800 0x55eadab7e580 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3236390620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab65d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3236390620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab5b620 0x55eadab72230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9cb429740 1 -- 172.21.15.19:0/3236390620 >> 172.21.15.19:0/3236390620 conn(0x55eadab57b30 msgr2=0x55eadab5eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9cb429740 1 -- 172.21.15.19:0/3236390620 shutdown_connections 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.391+0000 7fd9cb429740 1 -- 172.21.15.19:0/3236390620 wait complete. 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9cb429740 1 -- start start 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9cb429740 1 Processor -- start 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab6a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9cb429740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab6a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9cb429740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c01b1060 con 0x55eadab5b620 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9cb429740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c01b0990 con 0x55eadab66800 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9cb429740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c01adbf0 con 0x55eadab5bf40 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9cb429740 10 client.0 _ll_register_callbacks cb 0x7fd9bc003e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9c5ca3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9c5ca3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab6ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37820/0 (socket says 172.21.15.19:37820) 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3951377460 learned_addr learned my addr 172.21.15.19:0/3951377460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:35:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.392+0000 7fd9c64a4640 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab6a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3951377460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 msgr2=0x55eadab6a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab6a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3951377460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 msgr2=0x55eadab6a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab6a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9c5ca3640 1 -- 172.21.15.19:0/3951377460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eadabdbdf0 con 0x55eadab5b620 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9c64a4640 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab6a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9c5ca3640 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab6ecc0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4031600 tx=0x7fd9b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a5bef5c6a38c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/3951377460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4035070 con 0x55eadab5b620 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/3951377460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fd9b4005270 con 0x55eadab5b620 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9cb429740 1 -- 172.21.15.19:0/3951377460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9b4031d60 con 0x55eadab5b620 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/3951377460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd9b4005470 con 0x55eadab5b620 2024-04-23T18:35:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9cb429740 10 client.16923 fetch_fsmap learned FSMap version 27 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9cb429740 1 -- 172.21.15.19:0/3951377460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eadabdcfc0 con 0x55eadab5b620 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/3951377460 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9b4031d60 con 0x55eadab5b620 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/3951377460 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd9b403d4e0 con 0x55eadab5b620 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9cb429740 10 client.16923 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9cb429740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9cb429740 20 client.16923 populate_metadata read hostname 'smithi019' 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eadabdd240 0x55eadabdf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9cb429740 4 client.16923 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.393+0000 7fd9cb429740 1 -- 172.21.15.19:0/3951377460 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eadaac8200 con 0x55eadabdd240 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.394+0000 7fd9c64a4640 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eadabdd240 0x55eadabdf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.394+0000 7fd9c64a4640 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eadabdd240 0x55eadabdf630 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d311642ee9714bd server_cookie=25915aadbd6ba87c in_seq=0 out_seq=0 2024-04-23T18:35:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.394+0000 7fd9a9ffb640 10 client.16923 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.408+0000 7fd9a9ffb640 1 -- 172.21.15.19:0/3951377460 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eadaac8200 con 0x55eadabdd240 2024-04-23T18:35:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.408+0000 7fd9a9ffb640 10 client.16923 handle_command_reply: tid=0 2024-04-23T18:35:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 client.16923 shutdown 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 -- 172.21.15.19:0/3951377460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 msgr2=0x55eadab6ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab6ecc0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4031600 tx=0x7fd9b4000bc0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 -- 172.21.15.19:0/3951377460 shutdown_connections 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eadabdd240 0x55eadabdf630 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eadab66800 0x55eadab6a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eadab5bf40 0x55eadab6a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9a9ffb640 0 client.16923 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 --2- 172.21.15.19:0/3951377460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eadab5b620 0x55eadab6ecc0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 -- 172.21.15.19:0/3951377460 >> 172.21.15.19:0/3951377460 conn(0x55eadab57b30 msgr2=0x55eadab71220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 -- 172.21.15.19:0/3951377460 shutdown_connections 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 1 -- 172.21.15.19:0/3951377460 wait complete. 2024-04-23T18:35:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9cb429740 20 client.16923 trim_cache size 0 max 16384 2024-04-23T18:35:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:35:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:35:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:35:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9c8708640 1 -- 172.21.15.19:0/1466207129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9ac0a3e70 msgr2=0x7fd9ac0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9c8708640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9ac0a3e70 0x7fd9ac0a6330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8098530 tx=0x7fd9b8066c20 comp rx=0 tx=0).stop 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9c8708640 1 -- 172.21.15.19:0/1466207129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01035f0 msgr2=0x7fd9c01a6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.409+0000 7fd9c8708640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01035f0 0x7fd9c01a6830 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800d750 tx=0x7fd9b800dc10 comp rx=0 tx=0).stop 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.410+0000 7fd9c8708640 1 -- 172.21.15.19:0/1466207129 shutdown_connections 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.410+0000 7fd9c8708640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd9ac0a3e70 0x7fd9ac0a6330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.410+0000 7fd9c8708640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c01061b0 0x7fd9c01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.410+0000 7fd9c8708640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c01035f0 0x7fd9c01a6830 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.410+0000 7fd9c8708640 1 --2- 172.21.15.19:0/1466207129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9c01010e0 0x7fd9c0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.410+0000 7fd9c8708640 1 -- 172.21.15.19:0/1466207129 >> 172.21.15.19:0/1466207129 conn(0x7fd9c00771d0 msgr2=0x7fd9c006aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.410+0000 7fd9c8708640 1 -- 172.21.15.19:0/1466207129 shutdown_connections 2024-04-23T18:35:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:35:49.410+0000 7fd9c8708640 1 -- 172.21.15.19:0/1466207129 wait complete. 2024-04-23T18:35:50.124 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:49 smithi155 ceph-mon[33468]: pgmap v299: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:35:50.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:49 smithi019 ceph-mon[29215]: pgmap v299: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:35:50.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:49 smithi044 ceph-mon[32458]: pgmap v299: 97 pgs: 97 active+clean; 4.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:35:52.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:51 smithi019 ceph-mon[29215]: pgmap v300: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:35:52.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:51 smithi044 ceph-mon[32458]: pgmap v300: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:35:52.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:51 smithi155 ceph-mon[33468]: pgmap v300: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:35:54.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:53 smithi019 ceph-mon[29215]: pgmap v301: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-23T18:35:54.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:53 smithi044 ceph-mon[32458]: pgmap v301: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-23T18:35:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:53 smithi155 ceph-mon[33468]: pgmap v301: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-23T18:35:56.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:55 smithi019 ceph-mon[29215]: pgmap v302: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:35:56.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:55 smithi044 ceph-mon[32458]: pgmap v302: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:35:56.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:55 smithi155 ceph-mon[33468]: pgmap v302: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:35:58.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:57 smithi019 ceph-mon[29215]: pgmap v303: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T18:35:58.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:57 smithi044 ceph-mon[32458]: pgmap v303: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T18:35:58.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:57 smithi155 ceph-mon[33468]: pgmap v303: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T18:36:00.114 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:35:59 smithi044 ceph-mon[32458]: pgmap v304: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T18:36:00.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:35:59 smithi019 ceph-mon[29215]: pgmap v304: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T18:36:00.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:35:59 smithi155 ceph-mon[33468]: pgmap v304: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T18:36:02.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:01 smithi019 ceph-mon[29215]: pgmap v305: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:36:02.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:01 smithi044 ceph-mon[32458]: pgmap v305: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:36:02.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:01 smithi155 ceph-mon[33468]: pgmap v305: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:36:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 -- 172.21.15.19:0/3931646074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff51c101a90 msgr2=0x7ff51c105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 --2- 172.21.15.19:0/3931646074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff51c101a90 0x7ff51c105ad0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff508009900 tx=0x7ff50802a410 comp rx=0 tx=0).stop 2024-04-23T18:36:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 -- 172.21.15.19:0/3931646074 shutdown_connections 2024-04-23T18:36:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 --2- 172.21.15.19:0/3931646074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff51c1061b0 0x7ff51c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 --2- 172.21.15.19:0/3931646074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff51c101a90 0x7ff51c105ad0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 --2- 172.21.15.19:0/3931646074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff51c1010e0 0x7ff51c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 -- 172.21.15.19:0/3931646074 >> 172.21.15.19:0/3931646074 conn(0x7ff51c0771d0 msgr2=0x7ff51c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 -- 172.21.15.19:0/3931646074 shutdown_connections 2024-04-23T18:36:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 -- 172.21.15.19:0/3931646074 wait complete. 2024-04-23T18:36:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 Processor -- start 2024-04-23T18:36:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 -- start start 2024-04-23T18:36:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff51c1010e0 0x7ff51c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.509+0000 7ff5231ad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff51c1061b0 0x7ff51c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff5231ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff51c19cf00 0x7ff51c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff5231ad640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1110f0 con 0x7ff51c19cf00 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff5231ad640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c110f70 con 0x7ff51c1010e0 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff5231ad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c111270 con 0x7ff51c1061b0 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff52174a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff51c19cf00 0x7ff51c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff52174a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff51c19cf00 0x7ff51c19d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35104/0 (socket says 172.21.15.19:35104) 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff52174a640 1 -- 172.21.15.19:0/932745754 learned_addr learned my addr 172.21.15.19:0/932745754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff520f49640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff51c1010e0 0x7ff51c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff51bfff640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff51c1061b0 0x7ff51c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff52174a640 1 -- 172.21.15.19:0/932745754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff51c1061b0 msgr2=0x7ff51c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff52174a640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff51c1061b0 0x7ff51c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff52174a640 1 -- 172.21.15.19:0/932745754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff51c1010e0 msgr2=0x7ff51c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff52174a640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff51c1010e0 0x7ff51c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff52174a640 1 -- 172.21.15.19:0/932745754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c19d9f0 con 0x7ff51c19cf00 2024-04-23T18:36:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff52174a640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff51c19cf00 0x7ff51c19d3b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff51400eb10 tx=0x7ff51400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff519ffb640 1 -- 172.21.15.19:0/932745754 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514004b20 con 0x7ff51c19cf00 2024-04-23T18:36:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff519ffb640 1 -- 172.21.15.19:0/932745754 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff514004cc0 con 0x7ff51c19cf00 2024-04-23T18:36:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff5231ad640 1 -- 172.21.15.19:0/932745754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c19dce0 con 0x7ff51c19cf00 2024-04-23T18:36:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.510+0000 7ff519ffb640 1 -- 172.21.15.19:0/932745754 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514010670 con 0x7ff51c19cf00 2024-04-23T18:36:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.511+0000 7ff5231ad640 1 -- 172.21.15.19:0/932745754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c1a54f0 con 0x7ff51c19cf00 2024-04-23T18:36:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.513+0000 7ff519ffb640 1 -- 172.21.15.19:0/932745754 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff514005020 con 0x7ff51c19cf00 2024-04-23T18:36:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff525f9f740 1 Processor -- start 2024-04-23T18:36:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff525f9f740 1 -- start start 2024-04-23T18:36:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff519ffb640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff50c0a3e40 0x7ff50c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7a620 0x564480e7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7af40 0x564480e852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e87bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff525f9f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564480ddf970 con 0x564480e85800 2024-04-23T18:36:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff525f9f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564480d7c4a0 con 0x564480e7af40 2024-04-23T18:36:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff525f9f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564480ddb930 con 0x564480e7a620 2024-04-23T18:36:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff51bfff640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff50c0a3e40 0x7ff50c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff519ffb640 1 -- 172.21.15.19:0/932745754 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7ff5140d0d00 con 0x7ff51c19cf00 2024-04-23T18:36:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff52174a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e87bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff52174a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e87bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35106/0 (socket says 172.21.15.19:35106) 2024-04-23T18:36:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff52174a640 1 -- 172.21.15.19:0/2213278550 learned_addr learned my addr 172.21.15.19:0/2213278550 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff520f49640 1 --2- 172.21.15.19:0/2213278550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7a620 0x564480e7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff51bfff640 1 --2- 172.21.15.19:0/2213278550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7af40 0x564480e852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff52174a640 1 -- 172.21.15.19:0/2213278550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7a620 msgr2=0x564480e7aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff52174a640 1 --2- 172.21.15.19:0/2213278550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7a620 0x564480e7aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff52174a640 1 -- 172.21.15.19:0/2213278550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7af40 msgr2=0x564480e852c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff52174a640 1 --2- 172.21.15.19:0/2213278550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7af40 0x564480e852c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.514+0000 7ff52174a640 1 -- 172.21.15.19:0/2213278550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564480af2210 con 0x564480e85800 2024-04-23T18:36:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.515+0000 7ff51bfff640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff50c0a3e40 0x7ff50c0a6300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff508000c00 tx=0x7ff508032ce0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.515+0000 7ff52174a640 1 --2- 172.21.15.19:0/2213278550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e87bf0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff5140a53c0 tx=0x7ff51400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3203acda56adc6b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.515+0000 7ff506ffd640 1 -- 172.21.15.19:0/2213278550 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51409cc80 con 0x564480e85800 2024-04-23T18:36:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.515+0000 7ff506ffd640 1 -- 172.21.15.19:0/2213278550 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5140a1030 con 0x564480e85800 2024-04-23T18:36:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.515+0000 7ff506ffd640 1 -- 172.21.15.19:0/2213278550 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5140ce6b0 con 0x564480e85800 2024-04-23T18:36:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 -- 172.21.15.19:0/2213278550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 msgr2=0x564480e87bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 --2- 172.21.15.19:0/2213278550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e87bf0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff5140a53c0 tx=0x7ff51400cee0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 -- 172.21.15.19:0/2213278550 shutdown_connections 2024-04-23T18:36:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 --2- 172.21.15.19:0/2213278550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7af40 0x564480e852c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 --2- 172.21.15.19:0/2213278550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e87bf0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 --2- 172.21.15.19:0/2213278550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7a620 0x564480e7aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 -- 172.21.15.19:0/2213278550 >> 172.21.15.19:0/2213278550 conn(0x564480e76b30 msgr2=0x564480e76f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 -- 172.21.15.19:0/2213278550 shutdown_connections 2024-04-23T18:36:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 -- 172.21.15.19:0/2213278550 wait complete. 2024-04-23T18:36:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 -- start start 2024-04-23T18:36:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.516+0000 7ff525f9f740 1 Processor -- start 2024-04-23T18:36:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e7a620 0x564480e85110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7af40 0x564480e82400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e85800 0x564480e82940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff525f9f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564480e88370 con 0x564480e7a620 2024-04-23T18:36:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff525f9f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564480d7a0b0 con 0x564480e7af40 2024-04-23T18:36:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff525f9f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564480e884f0 con 0x564480e85800 2024-04-23T18:36:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff525f9f740 10 client.0 _ll_register_callbacks cb 0x564480d67d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff520f49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e7a620 0x564480e85110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff520f49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e7a620 0x564480e85110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35118/0 (socket says 172.21.15.19:35118) 2024-04-23T18:36:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff520f49640 1 -- 172.21.15.19:0/1818214348 learned_addr learned my addr 172.21.15.19:0/1818214348 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff51bfff640 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7af40 0x564480e82400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff52174a640 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e85800 0x564480e82940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff520f49640 1 -- 172.21.15.19:0/1818214348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e85800 msgr2=0x564480e82940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff520f49640 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e85800 0x564480e82940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff520f49640 1 -- 172.21.15.19:0/1818214348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7af40 msgr2=0x564480e82400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff520f49640 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7af40 0x564480e82400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff520f49640 1 -- 172.21.15.19:0/1818214348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564480bbe620 con 0x564480e7a620 2024-04-23T18:36:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff520f49640 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e7a620 0x564480e85110 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff510012680 tx=0x7ff510012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bfa26eb7862950 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff4fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:36:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1818214348 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51000f920 con 0x564480e7a620 2024-04-23T18:36:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1818214348 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7ff51000fac0 con 0x564480e7a620 2024-04-23T18:36:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.517+0000 7ff525f9f740 1 -- 172.21.15.19:0/1818214348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5140a1030 con 0x564480e7a620 2024-04-23T18:36:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1818214348 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff51000fcc0 con 0x564480e7a620 2024-04-23T18:36:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff525f9f740 10 client.16935 fetch_fsmap learned FSMap version 27 2024-04-23T18:36:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff525f9f740 1 -- 172.21.15.19:0/1818214348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564480e942b0 con 0x564480e7a620 2024-04-23T18:36:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1818214348 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5140a1030 con 0x564480e7a620 2024-04-23T18:36:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1818214348 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff51001c420 con 0x564480e7a620 2024-04-23T18:36:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff525f9f740 10 client.16935 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:36:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff525f9f740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff525f9f740 20 client.16935 populate_metadata read hostname 'smithi019' 2024-04-23T18:36:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564480e95770 0x564480e97ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff525f9f740 4 client.16935 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.518+0000 7ff525f9f740 1 -- 172.21.15.19:0/1818214348 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564480ded390 con 0x564480e95770 2024-04-23T18:36:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.519+0000 7ff51bfff640 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564480e95770 0x564480e97ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.519+0000 7ff51bfff640 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564480e95770 0x564480e97ba0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9815e2ca29639595 server_cookie=20438877625afbca in_seq=0 out_seq=0 2024-04-23T18:36:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.519+0000 7ff4fbfff640 10 client.16935 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1818214348 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564480ded390 con 0x564480e95770 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff4fbfff640 10 client.16935 handle_command_reply: tid=0 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 client.16935 shutdown 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 -- 172.21.15.19:0/1818214348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e7a620 msgr2=0x564480e85110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e7a620 0x564480e85110 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff510012680 tx=0x7ff510012b40 comp rx=0 tx=0).stop 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 -- 172.21.15.19:0/1818214348 shutdown_connections 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564480e95770 0x564480e97ba0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7af40 0x564480e82400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e85800 0x564480e82940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1818214348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e7a620 0x564480e85110 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff4fbfff640 0 client.16935 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 -- 172.21.15.19:0/1818214348 >> 172.21.15.19:0/1818214348 conn(0x564480e76b30 msgr2=0x564480e75cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 -- 172.21.15.19:0/1818214348 shutdown_connections 2024-04-23T18:36:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 1 -- 172.21.15.19:0/1818214348 wait complete. 2024-04-23T18:36:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.520+0000 7ff525f9f740 20 client.16935 trim_cache size 0 max 16384 2024-04-23T18:36:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.535+0000 7ff525f9f740 1 Processor -- start 2024-04-23T18:36:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.535+0000 7ff525f9f740 1 -- start start 2024-04-23T18:36:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 0x564480e91230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7af40 0x564480e84c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e9d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff525f9f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1adba0 con 0x564480e85800 2024-04-23T18:36:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff525f9f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1af6e0 con 0x564480e7a620 2024-04-23T18:36:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff525f9f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1ae270 con 0x564480e7af40 2024-04-23T18:36:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff52174a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e9d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff52174a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e9d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35120/0 (socket says 172.21.15.19:35120) 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff52174a640 1 -- 172.21.15.19:0/3712916829 learned_addr learned my addr 172.21.15.19:0/3712916829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff520f49640 1 --2- 172.21.15.19:0/3712916829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 0x564480e91230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff51bfff640 1 --2- 172.21.15.19:0/3712916829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7af40 0x564480e84c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff52174a640 1 -- 172.21.15.19:0/3712916829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7af40 msgr2=0x564480e84c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff52174a640 1 --2- 172.21.15.19:0/3712916829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7af40 0x564480e84c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff52174a640 1 -- 172.21.15.19:0/3712916829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 msgr2=0x564480e91230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff52174a640 1 --2- 172.21.15.19:0/3712916829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 0x564480e91230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff52174a640 1 -- 172.21.15.19:0/3712916829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564480e82ca0 con 0x564480e85800 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff52174a640 1 --2- 172.21.15.19:0/3712916829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e9d4b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff51409cc10 tx=0x7ff514009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a930c5425d0cc776 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff5067fc640 1 -- 172.21.15.19:0/3712916829 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514055030 con 0x564480e85800 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff5067fc640 1 -- 172.21.15.19:0/3712916829 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff514057050 con 0x564480e85800 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.536+0000 7ff5067fc640 1 -- 172.21.15.19:0/3712916829 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514099450 con 0x564480e85800 2024-04-23T18:36:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 -- 172.21.15.19:0/3712916829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 msgr2=0x564480e9d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 --2- 172.21.15.19:0/3712916829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e9d4b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff51409cc10 tx=0x7ff514009200 comp rx=0 tx=0).stop 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 -- 172.21.15.19:0/3712916829 shutdown_connections 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 --2- 172.21.15.19:0/3712916829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e9d4b0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 --2- 172.21.15.19:0/3712916829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7af40 0x564480e84c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 --2- 172.21.15.19:0/3712916829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 0x564480e91230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 -- 172.21.15.19:0/3712916829 >> 172.21.15.19:0/3712916829 conn(0x564480e76b30 msgr2=0x564480e75cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 -- 172.21.15.19:0/3712916829 shutdown_connections 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 -- 172.21.15.19:0/3712916829 wait complete. 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 -- start start 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.537+0000 7ff525f9f740 1 Processor -- start 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 0x564480e8dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7af40 0x564480e7eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff525f9f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e7f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff525f9f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1af010 con 0x564480e85800 2024-04-23T18:36:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff525f9f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1ae940 con 0x564480e7a620 2024-04-23T18:36:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff525f9f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1abba0 con 0x564480e7af40 2024-04-23T18:36:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff525f9f740 10 client.0 _ll_register_callbacks cb 0x7ff510009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff52174a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e7f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff52174a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e7f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35122/0 (socket says 172.21.15.19:35122) 2024-04-23T18:36:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff52174a640 1 -- 172.21.15.19:0/1654036876 learned_addr learned my addr 172.21.15.19:0/1654036876 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff520f49640 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 0x564480e8dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff52174a640 1 -- 172.21.15.19:0/1654036876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7af40 msgr2=0x564480e7eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff51bfff640 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7af40 0x564480e7eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff52174a640 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7af40 0x564480e7eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff52174a640 1 -- 172.21.15.19:0/1654036876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 msgr2=0x564480e8dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff52174a640 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 0x564480e8dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff52174a640 1 -- 172.21.15.19:0/1654036876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564480e7fc80 con 0x564480e85800 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.538+0000 7ff520f49640 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 0x564480e8dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff52174a640 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e7f450 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff5140049d0 tx=0x7ff514000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3f397202adab4f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff4fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1654036876 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514055070 con 0x564480e85800 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1654036876 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7ff51409cee0 con 0x564480e85800 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff525f9f740 1 -- 172.21.15.19:0/1654036876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff514057050 con 0x564480e85800 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1654036876 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5140ce8c0 con 0x564480e85800 2024-04-23T18:36:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff525f9f740 10 client.16941 fetch_fsmap learned FSMap version 27 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff525f9f740 1 -- 172.21.15.19:0/1654036876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564480efbdd0 con 0x564480e85800 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1654036876 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff514057050 con 0x564480e85800 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1654036876 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff5140ceca0 con 0x564480e85800 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff525f9f740 10 client.16941 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff525f9f740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff525f9f740 20 client.16941 populate_metadata read hostname 'smithi019' 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564480efc000 0x564480efe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff525f9f740 4 client.16941 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.539+0000 7ff525f9f740 1 -- 172.21.15.19:0/1654036876 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564480de7200 con 0x564480efc000 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.540+0000 7ff520f49640 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564480efc000 0x564480efe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.540+0000 7ff520f49640 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564480efc000 0x564480efe3f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1bbfa46f16b1ef7 server_cookie=1f18402fe5db7f6c in_seq=0 out_seq=0 2024-04-23T18:36:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.540+0000 7ff4fbfff640 10 client.16941 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff4fbfff640 1 -- 172.21.15.19:0/1654036876 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564480de7200 con 0x564480efc000 2024-04-23T18:36:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff4fbfff640 10 client.16941 handle_command_reply: tid=0 2024-04-23T18:36:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff525f9f740 1 client.16941 shutdown 2024-04-23T18:36:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff525f9f740 1 -- 172.21.15.19:0/1654036876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 msgr2=0x564480e7f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e7f450 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff5140049d0 tx=0x7ff514000f40 comp rx=0 tx=0).stop 2024-04-23T18:36:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff525f9f740 1 -- 172.21.15.19:0/1654036876 shutdown_connections 2024-04-23T18:36:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564480efc000 0x564480efe3f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564480e85800 0x564480e7f450 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564480e7af40 0x564480e7eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff4fbfff640 0 client.16941 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff525f9f740 1 --2- 172.21.15.19:0/1654036876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564480e7a620 0x564480e8dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.552+0000 7ff525f9f740 1 -- 172.21.15.19:0/1654036876 >> 172.21.15.19:0/1654036876 conn(0x564480e76b30 msgr2=0x564480e81c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff525f9f740 1 -- 172.21.15.19:0/1654036876 shutdown_connections 2024-04-23T18:36:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff525f9f740 1 -- 172.21.15.19:0/1654036876 wait complete. 2024-04-23T18:36:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff525f9f740 20 client.16941 trim_cache size 0 max 16384 2024-04-23T18:36:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:36:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:36:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:36:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 -- 172.21.15.19:0/932745754 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff50c0a3e40 msgr2=0x7ff50c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff50c0a3e40 0x7ff50c0a6300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff508000c00 tx=0x7ff508032ce0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 -- 172.21.15.19:0/932745754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff51c19cf00 msgr2=0x7ff51c19d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff51c19cf00 0x7ff51c19d3b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff51400eb10 tx=0x7ff51400c2d0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 -- 172.21.15.19:0/932745754 shutdown_connections 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff50c0a3e40 0x7ff50c0a6300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff51c19cf00 0x7ff51c19d3b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff51c1061b0 0x7ff51c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 --2- 172.21.15.19:0/932745754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff51c1010e0 0x7ff51c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 -- 172.21.15.19:0/932745754 >> 172.21.15.19:0/932745754 conn(0x7ff51c0771d0 msgr2=0x7ff51c06a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.553+0000 7ff5231ad640 1 -- 172.21.15.19:0/932745754 shutdown_connections 2024-04-23T18:36:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:02.554+0000 7ff5231ad640 1 -- 172.21.15.19:0/932745754 wait complete. 2024-04-23T18:36:04.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:03 smithi019 ceph-mon[29215]: pgmap v306: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-04-23T18:36:04.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:03 smithi044 ceph-mon[32458]: pgmap v306: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-04-23T18:36:04.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:03 smithi155 ceph-mon[33468]: pgmap v306: 97 pgs: 97 active+clean; 5.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-04-23T18:36:05.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:36:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.629+0000 7f1b4f76f640 1 -- 172.21.15.19:0/1970418056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b48102bf0 msgr2=0x7f1b48106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.629+0000 7f1b4f76f640 1 --2- 172.21.15.19:0/1970418056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b48102bf0 0x7f1b48106ba0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009900 tx=0x7f1b3802a410 comp rx=0 tx=0).stop 2024-04-23T18:36:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 -- 172.21.15.19:0/1970418056 shutdown_connections 2024-04-23T18:36:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 --2- 172.21.15.19:0/1970418056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b48107280 0x7f1b4810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 --2- 172.21.15.19:0/1970418056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b48102bf0 0x7f1b48106ba0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 --2- 172.21.15.19:0/1970418056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b48102240 0x7f1b48102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 -- 172.21.15.19:0/1970418056 >> 172.21.15.19:0/1970418056 conn(0x7f1b480790e0 msgr2=0x7f1b480794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 -- 172.21.15.19:0/1970418056 shutdown_connections 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 -- 172.21.15.19:0/1970418056 wait complete. 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 Processor -- start 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 -- start start 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b48102240 0x7f1b48198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b48107280 0x7f1b48198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.630+0000 7f1b4f76f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4819d010 0x7f1b4819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4f76f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48112210 con 0x7f1b48102240 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4f76f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48112090 con 0x7f1b48107280 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4f76f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48112390 con 0x7f1b4819d010 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4d50b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b48102240 0x7f1b48198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4d50b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b48102240 0x7f1b48198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35140/0 (socket says 172.21.15.19:35140) 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4d50b640 1 -- 172.21.15.19:0/3712379026 learned_addr learned my addr 172.21.15.19:0/3712379026 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4cd0a640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b48107280 0x7f1b48198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4dd0c640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4819d010 0x7f1b4819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4d50b640 1 -- 172.21.15.19:0/3712379026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4819d010 msgr2=0x7f1b4819d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4d50b640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4819d010 0x7f1b4819d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4d50b640 1 -- 172.21.15.19:0/3712379026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b48107280 msgr2=0x7f1b48198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4d50b640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b48107280 0x7f1b48198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4d50b640 1 -- 172.21.15.19:0/3712379026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4819db00 con 0x7f1b48102240 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4cd0a640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b48107280 0x7f1b48198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4d50b640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b48102240 0x7f1b48198750 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400d950 tx=0x7f1b4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b3e7fc640 1 -- 172.21.15.19:0/3712379026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44017070 con 0x7f1b48102240 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4f76f640 1 -- 172.21.15.19:0/3712379026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4819ddf0 con 0x7f1b48102240 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b3e7fc640 1 -- 172.21.15.19:0/3712379026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b44009c30 con 0x7f1b48102240 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b3e7fc640 1 -- 172.21.15.19:0/3712379026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400f620 con 0x7f1b48102240 2024-04-23T18:36:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.631+0000 7f1b4f76f640 1 -- 172.21.15.19:0/3712379026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b481a5600 con 0x7f1b48102240 2024-04-23T18:36:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.632+0000 7f1b4f76f640 1 -- 172.21.15.19:0/3712379026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b480691c0 con 0x7f1b48102240 2024-04-23T18:36:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.634+0000 7f1b3e7fc640 1 -- 172.21.15.19:0/3712379026 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1b44010460 con 0x7f1b48102240 2024-04-23T18:36:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.634+0000 7f1b3e7fc640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1b340a3d20 0x7f1b340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.634+0000 7f1b3e7fc640 1 -- 172.21.15.19:0/3712379026 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f1b440d04d0 con 0x7f1b48102240 2024-04-23T18:36:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.634+0000 7f1b4cd0a640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1b340a3d20 0x7f1b340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.635+0000 7f1b4cd0a640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1b340a3d20 0x7f1b340a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b3802aa00 tx=0x7f1b38000bc0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.638+0000 7f1b3e7fc640 1 -- 172.21.15.19:0/3712379026 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b4409cb70 con 0x7f1b48102240 2024-04-23T18:36:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.863+0000 7f1b4f76f640 1 -- 172.21.15.19:0/3712379026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b48199aa0 con 0x7f1b48102240 2024-04-23T18:36:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.863+0000 7f1b3e7fc640 1 -- 172.21.15.19:0/3712379026 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f1b44009de0 con 0x7f1b48102240 2024-04-23T18:36:05.856 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:36:05.856 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:36:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:36:05.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.865+0000 7f1b4f76f640 1 -- 172.21.15.19:0/3712379026 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1b340a3d20 msgr2=0x7f1b340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.865+0000 7f1b4f76f640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1b340a3d20 0x7f1b340a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b3802aa00 tx=0x7f1b38000bc0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.865+0000 7f1b4f76f640 1 -- 172.21.15.19:0/3712379026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b48102240 msgr2=0x7f1b48198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.865+0000 7f1b4f76f640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b48102240 0x7f1b48198750 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400d950 tx=0x7f1b4400de10 comp rx=0 tx=0).stop 2024-04-23T18:36:05.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.865+0000 7f1b4f76f640 1 -- 172.21.15.19:0/3712379026 shutdown_connections 2024-04-23T18:36:05.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.865+0000 7f1b4f76f640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1b340a3d20 0x7f1b340a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.865+0000 7f1b4f76f640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b48107280 0x7f1b48198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.865+0000 7f1b4f76f640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4819d010 0x7f1b4819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.865+0000 7f1b4f76f640 1 --2- 172.21.15.19:0/3712379026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b48102240 0x7f1b48198750 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.865+0000 7f1b4f76f640 1 -- 172.21.15.19:0/3712379026 >> 172.21.15.19:0/3712379026 conn(0x7f1b480790e0 msgr2=0x7f1b48100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:05.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.866+0000 7f1b4f76f640 1 -- 172.21.15.19:0/3712379026 shutdown_connections 2024-04-23T18:36:05.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.866+0000 7f1b4f76f640 1 -- 172.21.15.19:0/3712379026 wait complete. 2024-04-23T18:36:05.870 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id 3678ca7a --release --await' 2024-04-23T18:36:05.870 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id 3678ca7a --release --await 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.990+0000 7fad6a2fa640 1 -- 172.21.15.19:0/543426457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad6406b1f0 msgr2=0x7fad64106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.990+0000 7fad6a2fa640 1 --2- 172.21.15.19:0/543426457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad6406b1f0 0x7fad64106b90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fad54009900 tx=0x7fad5402a410 comp rx=0 tx=0).stop 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.990+0000 7fad6a2fa640 1 -- 172.21.15.19:0/543426457 shutdown_connections 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.990+0000 7fad6a2fa640 1 --2- 172.21.15.19:0/543426457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad64107270 0x7fad6410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.990+0000 7fad6a2fa640 1 --2- 172.21.15.19:0/543426457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad6406b1f0 0x7fad64106b90 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.990+0000 7fad6a2fa640 1 --2- 172.21.15.19:0/543426457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad6406a840 0x7fad6406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.990+0000 7fad6a2fa640 1 -- 172.21.15.19:0/543426457 >> 172.21.15.19:0/543426457 conn(0x7fad64077eb0 msgr2=0x7fad640782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.990+0000 7fad6a2fa640 1 -- 172.21.15.19:0/543426457 shutdown_connections 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.990+0000 7fad6a2fa640 1 -- 172.21.15.19:0/543426457 wait complete. 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.991+0000 7fad6a2fa640 1 Processor -- start 2024-04-23T18:36:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.991+0000 7fad6a2fa640 1 -- start start 2024-04-23T18:36:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.991+0000 7fad6a2fa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad6406a840 0x7fad64198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.991+0000 7fad6a2fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad64107270 0x7fad64198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.991+0000 7fad6a2fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6419d010 0x7fad6419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.991+0000 7fad6a2fa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad641121b0 con 0x7fad64107270 2024-04-23T18:36:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.991+0000 7fad6a2fa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad64112030 con 0x7fad6419d010 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.991+0000 7fad6a2fa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fad64112330 con 0x7fad6406a840 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad64107270 0x7fad64198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad64107270 0x7fad64198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35158/0 (socket says 172.21.15.19:35158) 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad637fe640 1 -- 172.21.15.19:0/1186089289 learned_addr learned my addr 172.21.15.19:0/1186089289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad63fff640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad6406a840 0x7fad64198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad68897640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6419d010 0x7fad6419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad637fe640 1 -- 172.21.15.19:0/1186089289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad6406a840 msgr2=0x7fad64198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad637fe640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad6406a840 0x7fad64198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad637fe640 1 -- 172.21.15.19:0/1186089289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6419d010 msgr2=0x7fad6419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad637fe640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6419d010 0x7fad6419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad637fe640 1 -- 172.21.15.19:0/1186089289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6419db00 con 0x7fad64107270 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad637fe640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad64107270 0x7fad64198c90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fad54006010 tx=0x7fad54004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad617fa640 1 -- 172.21.15.19:0/1186089289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5403f030 con 0x7fad64107270 2024-04-23T18:36:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad617fa640 1 -- 172.21.15.19:0/1186089289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad54003aa0 con 0x7fad64107270 2024-04-23T18:36:05.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad6a2fa640 1 -- 172.21.15.19:0/1186089289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6419dd90 con 0x7fad64107270 2024-04-23T18:36:05.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad617fa640 1 -- 172.21.15.19:0/1186089289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54035070 con 0x7fad64107270 2024-04-23T18:36:05.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.992+0000 7fad6a2fa640 1 -- 172.21.15.19:0/1186089289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad641a5550 con 0x7fad64107270 2024-04-23T18:36:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad6d014740 1 Processor -- start 2024-04-23T18:36:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad617fa640 1 -- 172.21.15.19:0/1186089289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad54042030 con 0x7fad64107270 2024-04-23T18:36:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad6d014740 1 -- start start 2024-04-23T18:36:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad6d014740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c611870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad6d014740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611db0 0x56008c61bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad6d014740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c61c480 0x56008c61e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad6d014740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56008c5766d0 con 0x56008c611490 2024-04-23T18:36:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad6d014740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56008c5131a0 con 0x56008c611db0 2024-04-23T18:36:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad6d014740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56008c572690 con 0x56008c61c480 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad68897640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c611870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad68897640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c611870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35170/0 (socket says 172.21.15.19:35170) 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad68897640 1 -- 172.21.15.19:0/1012134891 learned_addr learned my addr 172.21.15.19:0/1012134891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad63fff640 1 --2- 172.21.15.19:0/1012134891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611db0 0x56008c61bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad637fe640 1 --2- 172.21.15.19:0/1012134891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c61c480 0x56008c61e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad68897640 1 -- 172.21.15.19:0/1012134891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c61c480 msgr2=0x56008c61e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad617fa640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fad4c0a3d20 0x7fad4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad68897640 1 --2- 172.21.15.19:0/1012134891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c61c480 0x56008c61e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad68897640 1 -- 172.21.15.19:0/1012134891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611db0 msgr2=0x56008c61bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad68897640 1 --2- 172.21.15.19:0/1012134891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611db0 0x56008c61bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.995+0000 7fad68897640 1 -- 172.21.15.19:0/1012134891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56008c51d5b0 con 0x56008c611490 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.996+0000 7fad617fa640 1 -- 172.21.15.19:0/1186089289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8557+0+0 (secure 0 0 0) 0x7fad540f0bc0 con 0x7fad64107270 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.996+0000 7fad68897640 1 --2- 172.21.15.19:0/1012134891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c611870 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009f90 tx=0x7fad5c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df598de4e382a8c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.996+0000 7fad68897640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fad4c0a3d20 0x7fad4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.996+0000 7fad4a7fc640 1 -- 172.21.15.19:0/1012134891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c01f040 con 0x56008c611490 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.996+0000 7fad4a7fc640 1 -- 172.21.15.19:0/1012134891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c01a4b0 con 0x56008c611490 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.996+0000 7fad4a7fc640 1 -- 172.21.15.19:0/1012134891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0196f0 con 0x56008c611490 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.996+0000 7fad6d014740 1 -- 172.21.15.19:0/1012134891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 msgr2=0x56008c611870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.996+0000 7fad68897640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fad4c0a3d20 0x7fad4c0a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad5c011010 tx=0x7fad5c019e10 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.996+0000 7fad6d014740 1 --2- 172.21.15.19:0/1012134891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c611870 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009f90 tx=0x7fad5c010860 comp rx=0 tx=0).stop 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 -- 172.21.15.19:0/1012134891 shutdown_connections 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 --2- 172.21.15.19:0/1012134891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611db0 0x56008c61bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 --2- 172.21.15.19:0/1012134891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c61c480 0x56008c61e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 --2- 172.21.15.19:0/1012134891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c611870 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 -- 172.21.15.19:0/1012134891 >> 172.21.15.19:0/1012134891 conn(0x56008c60d7b0 msgr2=0x56008c60db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 -- 172.21.15.19:0/1012134891 shutdown_connections 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 -- 172.21.15.19:0/1012134891 wait complete. 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 -- start start 2024-04-23T18:36:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 Processor -- start 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611490 0x56008c61bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611db0 0x56008c619190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c6197a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56008c61f310 con 0x56008c61c480 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56008c510b20 con 0x56008c611db0 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56008c61f490 con 0x56008c611490 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad6d014740 10 client.0 _ll_register_callbacks cb 0x56008c510530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.997+0000 7fad68897640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c6197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad68897640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c6197a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35182/0 (socket says 172.21.15.19:35182) 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad68897640 1 -- 172.21.15.19:0/1424392652 learned_addr learned my addr 172.21.15.19:0/1424392652 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad63fff640 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611490 0x56008c61bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad637fe640 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611db0 0x56008c619190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad68897640 1 -- 172.21.15.19:0/1424392652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611490 msgr2=0x56008c61bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad68897640 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611490 0x56008c61bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad68897640 1 -- 172.21.15.19:0/1424392652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611db0 msgr2=0x56008c619190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad68897640 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611db0 0x56008c619190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad68897640 1 -- 172.21.15.19:0/1424392652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008c288920 con 0x56008c61c480 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad68897640 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c6197a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fad5c005b00 tx=0x7fad5c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c9db268181cb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad437fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad437fe640 1 -- 172.21.15.19:0/1424392652 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c036060 con 0x56008c61c480 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad437fe640 1 -- 172.21.15.19:0/1424392652 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8557+0+0 (secure 0 0 0) 0x7fad5c011ac0 con 0x56008c61c480 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad6d014740 1 -- 172.21.15.19:0/1424392652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad5c01a4b0 con 0x56008c61c480 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad437fe640 1 -- 172.21.15.19:0/1424392652 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad5c01f040 con 0x56008c61c480 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad6d014740 10 client.16959 fetch_fsmap learned FSMap version 27 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.998+0000 7fad6d014740 1 -- 172.21.15.19:0/1424392652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56008c354d30 con 0x56008c61c480 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad437fe640 1 -- 172.21.15.19:0/1424392652 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c01a4b0 con 0x56008c61c480 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad437fe640 1 -- 172.21.15.19:0/1424392652 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fad5c0188f0 con 0x56008c61c480 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad6d014740 10 client.16959 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:36:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad6d014740 10 client.16959 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad6d014740 20 client.16959 populate_metadata read hostname 'smithi019' 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad6d014740 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56008c62c650 0x56008c62ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad6d014740 4 client.16959 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad6d014740 1 -- 172.21.15.19:0/1424392652 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56008c534500 con 0x56008c62c650 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad63fff640 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56008c62c650 0x56008c62ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad63fff640 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56008c62c650 0x56008c62ea40 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58cad9b3780b2a10 server_cookie=dc0cba109ee7ee1f in_seq=0 out_seq=0 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:05.999+0000 7fad437fe640 10 client.16959 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad437fe640 1 -- 172.21.15.19:0/1424392652 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56008c534500 con 0x56008c62c650 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad437fe640 10 client.16959 handle_command_reply: tid=0 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 client.16959 shutdown 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 -- 172.21.15.19:0/1424392652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 msgr2=0x56008c6197a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c6197a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fad5c005b00 tx=0x7fad5c023000 comp rx=0 tx=0).stop 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 -- 172.21.15.19:0/1424392652 shutdown_connections 2024-04-23T18:36:05.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56008c62c650 0x56008c62ea40 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611db0 0x56008c619190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c6197a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad437fe640 0 client.16959 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:05.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 --2- 172.21.15.19:0/1424392652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611490 0x56008c61bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 -- 172.21.15.19:0/1424392652 >> 172.21.15.19:0/1424392652 conn(0x56008c60d7b0 msgr2=0x56008c60c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:05.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 -- 172.21.15.19:0/1424392652 shutdown_connections 2024-04-23T18:36:05.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.000+0000 7fad6d014740 1 -- 172.21.15.19:0/1424392652 wait complete. 2024-04-23T18:36:05.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.001+0000 7fad6d014740 20 client.16959 trim_cache size 0 max 16384 2024-04-23T18:36:06.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad6d014740 1 Processor -- start 2024-04-23T18:36:06.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad6d014740 1 -- start start 2024-04-23T18:36:06.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad6d014740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611490 0x56008c628200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad6d014740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611db0 0x56008c613bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad6d014740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c61b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad6d014740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad641adbe0 con 0x56008c61c480 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad6d014740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad641af720 con 0x56008c611490 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad6d014740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fad641ae2b0 con 0x56008c611db0 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad68897640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c61b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad68897640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c61b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35198/0 (socket says 172.21.15.19:35198) 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad68897640 1 -- 172.21.15.19:0/4227689286 learned_addr learned my addr 172.21.15.19:0/4227689286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad68897640 1 -- 172.21.15.19:0/4227689286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611db0 msgr2=0x56008c613bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad68897640 1 --2- 172.21.15.19:0/4227689286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611db0 0x56008c613bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad68897640 1 -- 172.21.15.19:0/4227689286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611490 msgr2=0x56008c628200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad68897640 1 --2- 172.21.15.19:0/4227689286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611490 0x56008c628200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad68897640 1 -- 172.21.15.19:0/4227689286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56008c61ebe0 con 0x56008c61c480 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.018+0000 7fad68897640 1 --2- 172.21.15.19:0/4227689286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c61b9d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009a80 tx=0x7fad5c018f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e7363bb998c8b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad49ffb640 1 -- 172.21.15.19:0/4227689286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c036070 con 0x56008c61c480 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad49ffb640 1 -- 172.21.15.19:0/4227689286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c02d040 con 0x56008c61c480 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad49ffb640 1 -- 172.21.15.19:0/4227689286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c011d20 con 0x56008c61c480 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad6d014740 1 -- 172.21.15.19:0/4227689286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 msgr2=0x56008c61b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad6d014740 1 --2- 172.21.15.19:0/4227689286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c61b9d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009a80 tx=0x7fad5c018f30 comp rx=0 tx=0).stop 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad6d014740 1 -- 172.21.15.19:0/4227689286 shutdown_connections 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad6d014740 1 --2- 172.21.15.19:0/4227689286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c61c480 0x56008c61b9d0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad6d014740 1 --2- 172.21.15.19:0/4227689286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611db0 0x56008c613bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad6d014740 1 --2- 172.21.15.19:0/4227689286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c611490 0x56008c628200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad6d014740 1 -- 172.21.15.19:0/4227689286 >> 172.21.15.19:0/4227689286 conn(0x56008c60d7b0 msgr2=0x56008c6134b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad6d014740 1 -- 172.21.15.19:0/4227689286 shutdown_connections 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad6d014740 1 -- 172.21.15.19:0/4227689286 wait complete. 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.019+0000 7fad6d014740 1 -- start start 2024-04-23T18:36:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad6d014740 1 Processor -- start 2024-04-23T18:36:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad6d014740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c616e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad6d014740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611db0 0x56008c624710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad6d014740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c61c480 0x56008c61d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad6d014740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad641af050 con 0x56008c611490 2024-04-23T18:36:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad6d014740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad641ae980 con 0x56008c61c480 2024-04-23T18:36:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad6d014740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fad641abbe0 con 0x56008c611db0 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad6d014740 10 client.0 _ll_register_callbacks cb 0x7fad5c005a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad63fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c616e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad63fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c616e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35200/0 (socket says 172.21.15.19:35200) 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad63fff640 1 -- 172.21.15.19:0/32148663 learned_addr learned my addr 172.21.15.19:0/32148663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad68897640 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c61c480 0x56008c61d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad63fff640 1 -- 172.21.15.19:0/32148663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611db0 msgr2=0x56008c624710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad63fff640 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611db0 0x56008c624710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad63fff640 1 -- 172.21.15.19:0/32148663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c61c480 msgr2=0x56008c61d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad63fff640 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c61c480 0x56008c61d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.020+0000 7fad63fff640 1 -- 172.21.15.19:0/32148663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008c61dac0 con 0x56008c611490 2024-04-23T18:36:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad63fff640 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c616e70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fad58004700 tx=0x7fad580108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e390c5ab5d035ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad437fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad437fe640 1 -- 172.21.15.19:0/32148663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad580035f0 con 0x56008c611490 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad437fe640 1 -- 172.21.15.19:0/32148663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8557+0+0 (secure 0 0 0) 0x7fad58003790 con 0x56008c611490 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad6d014740 1 -- 172.21.15.19:0/32148663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad5c02d040 con 0x56008c611490 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad437fe640 1 -- 172.21.15.19:0/32148663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad5802d050 con 0x56008c611490 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad6d014740 10 client.16965 fetch_fsmap learned FSMap version 27 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad6d014740 1 -- 172.21.15.19:0/32148663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56008c6931f0 con 0x56008c611490 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad437fe640 1 -- 172.21.15.19:0/32148663 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c02d040 con 0x56008c611490 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad437fe640 1 -- 172.21.15.19:0/32148663 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fad5801f5d0 con 0x56008c611490 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad6d014740 10 client.16965 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad6d014740 10 client.16965 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad6d014740 20 client.16965 populate_metadata read hostname 'smithi019' 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad6d014740 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56008c6938f0 0x56008c695d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad6d014740 4 client.16965 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3678ca7a", "release": true, "await": true}] 2024-04-23T18:36:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.021+0000 7fad6d014740 1 -- 172.21.15.19:0/32148663 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3678ca7a", "release": true, "await": true}) -- 0x56008c5840f0 con 0x56008c6938f0 2024-04-23T18:36:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.022+0000 7fad637fe640 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56008c6938f0 0x56008c695d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.022+0000 7fad437fe640 10 client.16965 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.022+0000 7fad637fe640 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56008c6938f0 0x56008c695d10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3df0927a02ba834 server_cookie=7fcfd38c4b55d776 in_seq=0 out_seq=0 2024-04-23T18:36:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad437fe640 1 -- 172.21.15.19:0/32148663 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56008c5840f0 con 0x56008c6938f0 2024-04-23T18:36:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad437fe640 10 client.16965 handle_command_reply: tid=0 2024-04-23T18:36:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 client.16965 shutdown 2024-04-23T18:36:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 -- 172.21.15.19:0/32148663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 msgr2=0x56008c616e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c616e70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fad58004700 tx=0x7fad580108c0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 -- 172.21.15.19:0/32148663 shutdown_connections 2024-04-23T18:36:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56008c6938f0 0x56008c695d10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56008c61c480 0x56008c61d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56008c611db0 0x56008c624710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad437fe640 0 client.16965 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 --2- 172.21.15.19:0/32148663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008c611490 0x56008c616e70 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 -- 172.21.15.19:0/32148663 >> 172.21.15.19:0/32148663 conn(0x56008c60d7b0 msgr2=0x56008c6271f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 -- 172.21.15.19:0/32148663 shutdown_connections 2024-04-23T18:36:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 1 -- 172.21.15.19:0/32148663 wait complete. 2024-04-23T18:36:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.026+0000 7fad6d014740 20 client.16965 trim_cache size 0 max 16384 2024-04-23T18:36:06.021 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:36:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-04-23T18:36:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:36:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 44, 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "3678ca7a": { 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 44, 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 78.8, 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:36:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-04-23T18:36:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:36:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:36:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:36:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:36:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:36:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:36:06.023 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 -- 172.21.15.19:0/1186089289 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fad4c0a3d20 msgr2=0x7fad4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fad4c0a3d20 0x7fad4c0a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad5c011010 tx=0x7fad5c019e10 comp rx=0 tx=0).stop 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 -- 172.21.15.19:0/1186089289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad64107270 msgr2=0x7fad64198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad64107270 0x7fad64198c90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fad54006010 tx=0x7fad54004060 comp rx=0 tx=0).stop 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 -- 172.21.15.19:0/1186089289 shutdown_connections 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fad4c0a3d20 0x7fad4c0a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6419d010 0x7fad6419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad64107270 0x7fad64198c90 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 --2- 172.21.15.19:0/1186089289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad6406a840 0x7fad64198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 -- 172.21.15.19:0/1186089289 >> 172.21.15.19:0/1186089289 conn(0x7fad64077eb0 msgr2=0x7fad640ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 -- 172.21.15.19:0/1186089289 shutdown_connections 2024-04-23T18:36:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:36:06.027+0000 7fad6a2fa640 1 -- 172.21.15.19:0/1186089289 wait complete. 2024-04-23T18:36:06.029 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3678ca7a', seconds elapsed: 19.8 2024-04-23T18:36:06.258 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:05 smithi019 ceph-mon[29215]: pgmap v307: 97 pgs: 97 active+clean; 5.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 56 KiB/s wr, 34 op/s 2024-04-23T18:36:06.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:05 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3712379026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:06.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:05 smithi044 ceph-mon[32458]: pgmap v307: 97 pgs: 97 active+clean; 5.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 56 KiB/s wr, 34 op/s 2024-04-23T18:36:06.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:05 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3712379026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:06.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:05 smithi155 ceph-mon[33468]: pgmap v307: 97 pgs: 97 active+clean; 5.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 56 KiB/s wr, 34 op/s 2024-04-23T18:36:06.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:05 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3712379026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:07.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:06 smithi019 ceph-mon[29215]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:36:07.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:06 smithi044 ceph-mon[32458]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:36:07.372 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:06 smithi155 ceph-mon[33468]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:36:08.258 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:07 smithi019 ceph-mon[29215]: pgmap v309: 97 pgs: 97 active+clean; 5.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T18:36:08.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:07 smithi044 ceph-mon[32458]: pgmap v309: 97 pgs: 97 active+clean; 5.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T18:36:08.364 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:07 smithi155 ceph-mon[33468]: pgmap v309: 97 pgs: 97 active+clean; 5.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T18:36:10.259 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:09 smithi019 ceph-mon[29215]: pgmap v310: 97 pgs: 97 active+clean; 5.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T18:36:10.481 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:09 smithi155 ceph-mon[33468]: pgmap v310: 97 pgs: 97 active+clean; 5.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T18:36:10.530 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:09 smithi044 ceph-mon[32458]: pgmap v310: 97 pgs: 97 active+clean; 5.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T18:36:12.258 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:11 smithi019 ceph-mon[29215]: pgmap v311: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-04-23T18:36:12.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:11 smithi044 ceph-mon[32458]: pgmap v311: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-04-23T18:36:12.364 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:11 smithi155 ceph-mon[33468]: pgmap v311: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-04-23T18:36:14.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:13 smithi044 ceph-mon[32458]: pgmap v312: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-04-23T18:36:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:13 smithi044 ceph-mon[32458]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:36:14.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:13 smithi155 ceph-mon[33468]: pgmap v312: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-04-23T18:36:14.206 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:13 smithi155 ceph-mon[33468]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:36:14.258 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:13 smithi019 ceph-mon[29215]: pgmap v312: 97 pgs: 97 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-04-23T18:36:14.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:13 smithi019 ceph-mon[29215]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:36:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.653+0000 7f4da98e1640 1 -- 172.21.15.19:0/2437596653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da410e910 msgr2=0x7f4da4110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.653+0000 7f4da98e1640 1 --2- 172.21.15.19:0/2437596653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da410e910 0x7f4da4110d00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c009ca0 tx=0x7f4d9c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:36:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.653+0000 7f4da98e1640 1 -- 172.21.15.19:0/2437596653 shutdown_connections 2024-04-23T18:36:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.653+0000 7f4da98e1640 1 --2- 172.21.15.19:0/2437596653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da410e910 0x7f4da4110d00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.653+0000 7f4da98e1640 1 --2- 172.21.15.19:0/2437596653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da4101680 0x7f4da410e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.653+0000 7f4da98e1640 1 --2- 172.21.15.19:0/2437596653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da4100d60 0x7f4da4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.653+0000 7f4da98e1640 1 -- 172.21.15.19:0/2437596653 >> 172.21.15.19:0/2437596653 conn(0x7f4da4077fd0 msgr2=0x7f4da40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.653+0000 7f4da98e1640 1 -- 172.21.15.19:0/2437596653 shutdown_connections 2024-04-23T18:36:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.653+0000 7f4da98e1640 1 -- 172.21.15.19:0/2437596653 wait complete. 2024-04-23T18:36:16.208 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:15 smithi155 ceph-mon[33468]: pgmap v314: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-23T18:36:16.321 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:15 smithi044 ceph-mon[32458]: pgmap v314: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-23T18:36:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.653+0000 7f4da98e1640 1 Processor -- start 2024-04-23T18:36:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da98e1640 1 -- start start 2024-04-23T18:36:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da98e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da4100d60 0x7f4da41a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da98e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da4101680 0x7f4da41a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da98e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da410e910 0x7f4da419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da98e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4da41143b0 con 0x7f4da4101680 2024-04-23T18:36:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da98e1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4da4114230 con 0x7f4da410e910 2024-04-23T18:36:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da98e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4da4114530 con 0x7f4da4100d60 2024-04-23T18:36:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da4101680 0x7f4da41a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da4101680 0x7f4da41a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52472/0 (socket says 172.21.15.19:52472) 2024-04-23T18:36:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da27fc640 1 -- 172.21.15.19:0/1615877321 learned_addr learned my addr 172.21.15.19:0/1615877321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da2ffd640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da4100d60 0x7f4da41a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da37fe640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da410e910 0x7f4da419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da27fc640 1 -- 172.21.15.19:0/1615877321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da4100d60 msgr2=0x7f4da41a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da27fc640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da4100d60 0x7f4da41a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da27fc640 1 -- 172.21.15.19:0/1615877321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da410e910 msgr2=0x7f4da419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da27fc640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da410e910 0x7f4da419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da27fc640 1 -- 172.21.15.19:0/1615877321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da419b030 con 0x7f4da4101680 2024-04-23T18:36:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.654+0000 7f4da37fe640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da410e910 0x7f4da419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.655+0000 7f4da27fc640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da4101680 0x7f4da41a0bf0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800d950 tx=0x7f4d9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.655+0000 7f4da88df640 1 -- 172.21.15.19:0/1615877321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98017070 con 0x7f4da4101680 2024-04-23T18:36:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.655+0000 7f4da98e1640 1 -- 172.21.15.19:0/1615877321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da419b320 con 0x7f4da4101680 2024-04-23T18:36:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.655+0000 7f4da88df640 1 -- 172.21.15.19:0/1615877321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d98009c30 con 0x7f4da4101680 2024-04-23T18:36:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.655+0000 7f4da88df640 1 -- 172.21.15.19:0/1615877321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800f620 con 0x7f4da4101680 2024-04-23T18:36:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.655+0000 7f4da98e1640 1 -- 172.21.15.19:0/1615877321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da41a7630 con 0x7f4da4101680 2024-04-23T18:36:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.657+0000 7f4da88df640 1 -- 172.21.15.19:0/1615877321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4d98010460 con 0x7f4da4101680 2024-04-23T18:36:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4dac5f4740 1 Processor -- start 2024-04-23T18:36:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4dac5f4740 1 -- start start 2024-04-23T18:36:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efad4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efadf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b0efa39970 con 0x55b0efadf800 2024-04-23T18:36:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef9d64a0 con 0x55b0efad4f40 2024-04-23T18:36:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b0efa35930 con 0x55b0efad4620 2024-04-23T18:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4da88df640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4d8c0a3dc0 0x7f4d8c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4da27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4da37fe640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4d8c0a3dc0 0x7f4d8c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4da27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52488/0 (socket says 172.21.15.19:52488) 2024-04-23T18:36:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4da27fc640 1 -- 172.21.15.19:0/3347593694 learned_addr learned my addr 172.21.15.19:0/3347593694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4da37fe640 1 --2- 172.21.15.19:0/3347593694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efad4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4da2ffd640 1 --2- 172.21.15.19:0/3347593694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efadf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.658+0000 7f4da88df640 1 -- 172.21.15.19:0/1615877321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f4d980d0460 con 0x7f4da4101680 2024-04-23T18:36:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.659+0000 7f4da27fc640 1 -- 172.21.15.19:0/3347593694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 msgr2=0x55b0efad4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.659+0000 7f4da27fc640 1 --2- 172.21.15.19:0/3347593694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efad4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.659+0000 7f4da27fc640 1 -- 172.21.15.19:0/3347593694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 msgr2=0x55b0efadf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.659+0000 7f4da27fc640 1 --2- 172.21.15.19:0/3347593694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efadf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.659+0000 7f4da27fc640 1 -- 172.21.15.19:0/3347593694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ef74c210 con 0x55b0efadf800 2024-04-23T18:36:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.659+0000 7f4da37fe640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4d8c0a3dc0 0x7f4d8c0a6280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c02f4c0 tx=0x7f4d9c02f450 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.659+0000 7f4da27fc640 1 --2- 172.21.15.19:0/3347593694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae1bf0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4d9806eec0 tx=0x7f4d9806eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec5624d065d8b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.659+0000 7f4d89ffb640 1 -- 172.21.15.19:0/3347593694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98020070 con 0x55b0efadf800 2024-04-23T18:36:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.659+0000 7f4d89ffb640 1 -- 172.21.15.19:0/3347593694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d9809cb00 con 0x55b0efadf800 2024-04-23T18:36:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.659+0000 7f4d89ffb640 1 -- 172.21.15.19:0/3347593694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98098a70 con 0x55b0efadf800 2024-04-23T18:36:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3347593694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 msgr2=0x55b0efae1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3347593694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae1bf0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4d9806eec0 tx=0x7f4d9806eef0 comp rx=0 tx=0).stop 2024-04-23T18:36:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3347593694 shutdown_connections 2024-04-23T18:36:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3347593694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efadf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3347593694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae1bf0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3347593694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efad4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3347593694 >> 172.21.15.19:0/3347593694 conn(0x55b0efad0b30 msgr2=0x55b0efad0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3347593694 shutdown_connections 2024-04-23T18:36:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3347593694 wait complete. 2024-04-23T18:36:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 -- start start 2024-04-23T18:36:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.660+0000 7f4dac5f4740 1 Processor -- start 2024-04-23T18:36:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4620 0x55b0efadf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4f40 0x55b0efadc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efadca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b0efae2430 con 0x55b0efadf800 2024-04-23T18:36:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef9d40b0 con 0x55b0efad4620 2024-04-23T18:36:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b0efae25b0 con 0x55b0efad4f40 2024-04-23T18:36:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4dac5f4740 10 client.0 _ll_register_callbacks cb 0x55b0ef9c1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efadca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efadca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52492/0 (socket says 172.21.15.19:52492) 2024-04-23T18:36:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da37fe640 1 -- 172.21.15.19:0/3673292653 learned_addr learned my addr 172.21.15.19:0/3673292653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da2ffd640 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4620 0x55b0efadf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da27fc640 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4f40 0x55b0efadc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da37fe640 1 -- 172.21.15.19:0/3673292653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4f40 msgr2=0x55b0efadc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da37fe640 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4f40 0x55b0efadc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da37fe640 1 -- 172.21.15.19:0/3673292653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4620 msgr2=0x55b0efadf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da37fe640 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4620 0x55b0efadf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da37fe640 1 -- 172.21.15.19:0/3673292653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ef818620 con 0x55b0efadf800 2024-04-23T18:36:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4da37fe640 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efadca30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c031df0 tx=0x7f4d9c047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8742476a03b638ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.661+0000 7f4d82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:36:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3673292653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c003b80 con 0x55b0efadf800 2024-04-23T18:36:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3673292653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d9809cb00 con 0x55b0efadf800 2024-04-23T18:36:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3673292653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f4d9c003d20 con 0x55b0efadf800 2024-04-23T18:36:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3673292653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d9c004040 con 0x55b0efadf800 2024-04-23T18:36:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4dac5f4740 10 client.16977 fetch_fsmap learned FSMap version 27 2024-04-23T18:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3673292653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0efaee3e0 con 0x55b0efadf800 2024-04-23T18:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3673292653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d9809cb00 con 0x55b0efadf800 2024-04-23T18:36:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3673292653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4d9c02f930 con 0x55b0efadf800 2024-04-23T18:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4dac5f4740 10 client.16977 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:36:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4dac5f4740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:20.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4dac5f4740 20 client.16977 populate_metadata read hostname 'smithi019' 2024-04-23T18:36:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b0efaef890 0x55b0efaf1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4dac5f4740 4 client.16977 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.662+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3673292653 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0efa47390 con 0x55b0efaef890 2024-04-23T18:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.663+0000 7f4da2ffd640 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b0efaef890 0x55b0efaf1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.663+0000 7f4da2ffd640 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b0efaef890 0x55b0efaf1cc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9df4043b86ecfa1 server_cookie=351aae9a01100049 in_seq=0 out_seq=0 2024-04-23T18:36:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.663+0000 7f4d82ffd640 10 client.16977 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3673292653 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0efa47390 con 0x55b0efaef890 2024-04-23T18:36:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4d82ffd640 10 client.16977 handle_command_reply: tid=0 2024-04-23T18:36:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 client.16977 shutdown 2024-04-23T18:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3673292653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 msgr2=0x55b0efadca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efadca30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c031df0 tx=0x7f4d9c047000 comp rx=0 tx=0).stop 2024-04-23T18:36:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3673292653 shutdown_connections 2024-04-23T18:36:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b0efaef890 0x55b0efaf1cc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efadca30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4f40 0x55b0efadc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3673292653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4620 0x55b0efadf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4d82ffd640 0 client.16977 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3673292653 >> 172.21.15.19:0/3673292653 conn(0x55b0efad0b30 msgr2=0x55b0efad68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3673292653 shutdown_connections 2024-04-23T18:36:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3673292653 wait complete. 2024-04-23T18:36:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.664+0000 7f4dac5f4740 20 client.16977 trim_cache size 0 max 16384 2024-04-23T18:36:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4dac5f4740 1 Processor -- start 2024-04-23T18:36:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4dac5f4740 1 -- start start 2024-04-23T18:36:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efaeb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efacfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efaf75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4da41afda0 con 0x55b0efadf800 2024-04-23T18:36:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4da41b18e0 con 0x55b0efad4f40 2024-04-23T18:36:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4da41b0470 con 0x55b0efad4620 2024-04-23T18:36:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4da37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efaf75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4da37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efaf75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52500/0 (socket says 172.21.15.19:52500) 2024-04-23T18:36:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4da37fe640 1 -- 172.21.15.19:0/3262993012 learned_addr learned my addr 172.21.15.19:0/3262993012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.679+0000 7f4da2ffd640 1 --2- 172.21.15.19:0/3262993012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efaeb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4da27fc640 1 --2- 172.21.15.19:0/3262993012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efacfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4da37fe640 1 -- 172.21.15.19:0/3262993012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 msgr2=0x55b0efaeb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4da37fe640 1 --2- 172.21.15.19:0/3262993012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efaeb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4da37fe640 1 -- 172.21.15.19:0/3262993012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 msgr2=0x55b0efacfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4da37fe640 1 --2- 172.21.15.19:0/3262993012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efacfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4da37fe640 1 -- 172.21.15.19:0/3262993012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0efaf2340 con 0x55b0efadf800 2024-04-23T18:36:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4da27fc640 1 --2- 172.21.15.19:0/3262993012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efacfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4da37fe640 1 --2- 172.21.15.19:0/3262993012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efaf75d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c005e30 tx=0x7f4d9c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920bc7413e607ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4d897fa640 1 -- 172.21.15.19:0/3262993012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c061030 con 0x55b0efadf800 2024-04-23T18:36:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4d897fa640 1 -- 172.21.15.19:0/3262993012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d9c036070 con 0x55b0efadf800 2024-04-23T18:36:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4d897fa640 1 -- 172.21.15.19:0/3262993012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c02f9d0 con 0x55b0efadf800 2024-04-23T18:36:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3262993012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 msgr2=0x55b0efaf75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.680+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3262993012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efaf75d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c005e30 tx=0x7f4d9c0093e0 comp rx=0 tx=0).stop 2024-04-23T18:36:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3262993012 shutdown_connections 2024-04-23T18:36:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3262993012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efacfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3262993012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efaf75d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3262993012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efaeb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3262993012 >> 172.21.15.19:0/3262993012 conn(0x55b0efad0b30 msgr2=0x55b0efacf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3262993012 shutdown_connections 2024-04-23T18:36:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3262993012 wait complete. 2024-04-23T18:36:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 -- start start 2024-04-23T18:36:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 Processor -- start 2024-04-23T18:36:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efae7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efae4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4da41b1210 con 0x55b0efadf800 2024-04-23T18:36:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4da41b0b40 con 0x55b0efad4f40 2024-04-23T18:36:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4da41adda0 con 0x55b0efad4620 2024-04-23T18:36:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.681+0000 7f4dac5f4740 10 client.0 _ll_register_callbacks cb 0x7f4d9c02af60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52502/0 (socket says 172.21.15.19:52502) 2024-04-23T18:36:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da37fe640 1 -- 172.21.15.19:0/3904816954 learned_addr learned my addr 172.21.15.19:0/3904816954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da2ffd640 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efae7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da27fc640 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efae4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da37fe640 1 -- 172.21.15.19:0/3904816954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 msgr2=0x55b0efae7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da37fe640 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efae7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da37fe640 1 -- 172.21.15.19:0/3904816954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 msgr2=0x55b0efae4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da37fe640 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efae4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da37fe640 1 -- 172.21.15.19:0/3904816954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0efae5c70 con 0x55b0efadf800 2024-04-23T18:36:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da2ffd640 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efae7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4da37fe640 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae5460 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c02fe70 tx=0x7f4d9c02fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87cd5a7356d4359 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4d82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:36:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3904816954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c0041e0 con 0x55b0efadf800 2024-04-23T18:36:23.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3904816954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f4d9c004380 con 0x55b0efadf800 2024-04-23T18:36:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3904816954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d9c036070 con 0x55b0efadf800 2024-04-23T18:36:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3904816954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d9c004580 con 0x55b0efadf800 2024-04-23T18:36:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4dac5f4740 10 client.16983 fetch_fsmap learned FSMap version 27 2024-04-23T18:36:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3904816954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0efb55da0 con 0x55b0efadf800 2024-04-23T18:36:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.682+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3904816954 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d9c036070 con 0x55b0efadf800 2024-04-23T18:36:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.683+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3904816954 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4d9c030c30 con 0x55b0efadf800 2024-04-23T18:36:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.683+0000 7f4dac5f4740 10 client.16983 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:36:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.683+0000 7f4dac5f4740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.683+0000 7f4dac5f4740 20 client.16983 populate_metadata read hostname 'smithi019' 2024-04-23T18:36:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.683+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b0efb56400 0x55b0efb58820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.683+0000 7f4dac5f4740 4 client.16983 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:36:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.683+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3904816954 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b0efa41200 con 0x55b0efb56400 2024-04-23T18:36:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.683+0000 7f4da2ffd640 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b0efb56400 0x55b0efb58820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.683+0000 7f4da2ffd640 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b0efb56400 0x55b0efb58820 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663d9d7915662118 server_cookie=be5344cba0491763 in_seq=0 out_seq=0 2024-04-23T18:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.683+0000 7f4d82ffd640 10 client.16983 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4d82ffd640 1 -- 172.21.15.19:0/3904816954 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b0efa41200 con 0x55b0efb56400 2024-04-23T18:36:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4d82ffd640 10 client.16983 handle_command_reply: tid=0 2024-04-23T18:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4dac5f4740 1 client.16983 shutdown 2024-04-23T18:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3904816954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 msgr2=0x55b0efae5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae5460 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c02fe70 tx=0x7f4d9c02fa00 comp rx=0 tx=0).stop 2024-04-23T18:36:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3904816954 shutdown_connections 2024-04-23T18:36:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b0efb56400 0x55b0efb58820 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0efad4f40 0x55b0efae4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0efadf800 0x55b0efae5460 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4d82ffd640 0 client.16983 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4dac5f4740 1 --2- 172.21.15.19:0/3904816954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0efad4620 0x55b0efae7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.695+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3904816954 >> 172.21.15.19:0/3904816954 conn(0x55b0efad0b30 msgr2=0x55b0efacf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3904816954 shutdown_connections 2024-04-23T18:36:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4dac5f4740 1 -- 172.21.15.19:0/3904816954 wait complete. 2024-04-23T18:36:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4dac5f4740 20 client.16983 trim_cache size 0 max 16384 2024-04-23T18:36:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:36:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:36:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:36:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:36:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 -- 172.21.15.19:0/1615877321 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4d8c0a3dc0 msgr2=0x7f4d8c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4d8c0a3dc0 0x7f4d8c0a6280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c02f4c0 tx=0x7f4d9c02f450 comp rx=0 tx=0).stop 2024-04-23T18:36:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 -- 172.21.15.19:0/1615877321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da4101680 msgr2=0x7f4da41a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da4101680 0x7f4da41a0bf0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800d950 tx=0x7f4d9800de10 comp rx=0 tx=0).stop 2024-04-23T18:36:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 -- 172.21.15.19:0/1615877321 shutdown_connections 2024-04-23T18:36:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4d8c0a3dc0 0x7f4d8c0a6280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da410e910 0x7f4da419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da4101680 0x7f4da41a0bf0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 --2- 172.21.15.19:0/1615877321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da4100d60 0x7f4da41a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 -- 172.21.15.19:0/1615877321 >> 172.21.15.19:0/1615877321 conn(0x7f4da4077fd0 msgr2=0x7f4da410fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 -- 172.21.15.19:0/1615877321 shutdown_connections 2024-04-23T18:36:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:15.696+0000 7f4da98e1640 1 -- 172.21.15.19:0/1615877321 wait complete. 2024-04-23T18:36:26.482 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:17 smithi155 ceph-mon[33468]: pgmap v315: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-23T18:36:26.515 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:19 smithi155 ceph-mon[33468]: pgmap v316: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-23T18:36:26.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:21 smithi155 ceph-mon[33468]: pgmap v317: 97 pgs: 97 active+clean; 1.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-23T18:36:26.617 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:21 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:26.648 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: pgmap v318: 97 pgs: 97 active+clean; 1.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-23T18:36:26.726 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:26.765 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:26.798 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:26.890 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:26.940 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:26.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:27.023 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.056 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.090 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.171 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:23 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.207 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-23T18:36:27.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:36:27.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.332 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.449 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.482 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.515 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.549 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.599 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:27.655 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-23T18:36:27.674 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-04-23T18:36:27.777 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:36:27.799 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:36:27.882 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:27.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:27.932 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:27.957 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:25 smithi155 ceph-mon[33468]: pgmap v319: 97 pgs: 97 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-23T18:36:27.983 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:17 smithi044 ceph-mon[32458]: pgmap v315: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-23T18:36:28.024 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:19 smithi044 ceph-mon[32458]: pgmap v316: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-23T18:36:28.073 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:21 smithi044 ceph-mon[32458]: pgmap v317: 97 pgs: 97 active+clean; 1.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-23T18:36:28.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:21 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:28.174 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: pgmap v318: 97 pgs: 97 active+clean; 1.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-23T18:36:28.207 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:28.241 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:28.274 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:28.393 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:28.433 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:28.466 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:28.499 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:28.533 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:28.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:28.599 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:23 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:28.658 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-23T18:36:28.691 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:36:28.724 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:28.764 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:28.808 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:28.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:28.866 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:28.900 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:28.933 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:29.041 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:29.128 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-23T18:36:29.167 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-04-23T18:36:29.200 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:36:29.266 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:36:29.300 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:29.333 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:29.367 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:29.400 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:25 smithi044 ceph-mon[32458]: pgmap v319: 97 pgs: 97 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-23T18:36:29.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:15 smithi019 ceph-mon[29215]: pgmap v314: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-23T18:36:29.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:17 smithi019 ceph-mon[29215]: pgmap v315: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-23T18:36:29.517 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:19 smithi019 ceph-mon[29215]: pgmap v316: 97 pgs: 97 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-23T18:36:29.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:21 smithi019 ceph-mon[29215]: pgmap v317: 97 pgs: 97 active+clean; 1.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-23T18:36:29.584 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:21 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:29.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: pgmap v318: 97 pgs: 97 active+clean; 1.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-23T18:36:29.725 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:29.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:29.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:29.825 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:29.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:29.892 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:29.925 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:29.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:30.026 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:23 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:30.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-23T18:36:30.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:36:30.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:30.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:30.209 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:30.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:30.334 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:30.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:30.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:30.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:30.593 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-23T18:36:30.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-04-23T18:36:30.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:36:30.726 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:36:30.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:30.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:30.893 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:36:30.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:25 smithi019 ceph-mon[29215]: pgmap v319: 97 pgs: 97 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-23T18:36:30.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:27 smithi019 ceph-mon[29215]: pgmap v320: 97 pgs: 97 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:36:31.010 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:27 smithi155 ceph-mon[33468]: pgmap v320: 97 pgs: 97 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:36:31.043 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:27 smithi044 ceph-mon[32458]: pgmap v320: 97 pgs: 97 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:36:31.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:29 smithi155 ceph-mon[33468]: pgmap v321: 97 pgs: 97 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:36:31.110 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:29 smithi044 ceph-mon[32458]: pgmap v321: 97 pgs: 97 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:36:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.797+0000 7f8d1e010640 1 -- 172.21.15.19:0/679345481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d18103200 msgr2=0x7f8d181035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.797+0000 7f8d1e010640 1 --2- 172.21.15.19:0/679345481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d18103200 0x7f8d181035e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009ca0 tx=0x7f8d0802a950 comp rx=0 tx=0).stop 2024-04-23T18:36:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.797+0000 7f8d1e010640 1 -- 172.21.15.19:0/679345481 shutdown_connections 2024-04-23T18:36:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.797+0000 7f8d1e010640 1 --2- 172.21.15.19:0/679345481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d1810c5d0 0x7f8d1810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.797+0000 7f8d1e010640 1 --2- 172.21.15.19:0/679345481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d18103b20 0x7f8d1810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.797+0000 7f8d1e010640 1 --2- 172.21.15.19:0/679345481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d18103200 0x7f8d181035e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.797+0000 7f8d1e010640 1 -- 172.21.15.19:0/679345481 >> 172.21.15.19:0/679345481 conn(0x7f8d180771d0 msgr2=0x7f8d180775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.797+0000 7f8d1e010640 1 -- 172.21.15.19:0/679345481 shutdown_connections 2024-04-23T18:36:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.797+0000 7f8d1e010640 1 -- 172.21.15.19:0/679345481 wait complete. 2024-04-23T18:36:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d1e010640 1 Processor -- start 2024-04-23T18:36:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d1e010640 1 -- start start 2024-04-23T18:36:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d1e010640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d18103200 0x7f8d18198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d1e010640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d18103b20 0x7f8d18198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d1e010640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d1810c5d0 0x7f8d1819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d1e010640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18112270 con 0x7f8d18103b20 2024-04-23T18:36:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d1e010640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181120f0 con 0x7f8d18103200 2024-04-23T18:36:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d1e010640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181123f0 con 0x7f8d1810c5d0 2024-04-23T18:36:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d18103b20 0x7f8d18198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d18103b20 0x7f8d18198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58184/0 (socket says 172.21.15.19:58184) 2024-04-23T18:36:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d16ffd640 1 -- 172.21.15.19:0/2785585879 learned_addr learned my addr 172.21.15.19:0/2785585879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d17fff640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d1810c5d0 0x7f8d1819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.798+0000 7f8d177fe640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d18103200 0x7f8d18198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d16ffd640 1 -- 172.21.15.19:0/2785585879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d1810c5d0 msgr2=0x7f8d1819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d1810c5d0 0x7f8d1819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d16ffd640 1 -- 172.21.15.19:0/2785585879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d18103200 msgr2=0x7f8d18198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d18103200 0x7f8d18198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d16ffd640 1 -- 172.21.15.19:0/2785585879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1819d660 con 0x7f8d18103b20 2024-04-23T18:36:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d18103b20 0x7f8d18198bb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00c970 tx=0x7f8d0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d14ff9640 1 -- 172.21.15.19:0/2785585879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c007bb0 con 0x7f8d18103b20 2024-04-23T18:36:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d1e010640 1 -- 172.21.15.19:0/2785585879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1819d950 con 0x7f8d18103b20 2024-04-23T18:36:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d14ff9640 1 -- 172.21.15.19:0/2785585879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0c007d50 con 0x7f8d18103b20 2024-04-23T18:36:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d14ff9640 1 -- 172.21.15.19:0/2785585879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c00f450 con 0x7f8d18103b20 2024-04-23T18:36:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.799+0000 7f8d1e010640 1 -- 172.21.15.19:0/2785585879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d181a51e0 con 0x7f8d18103b20 2024-04-23T18:36:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.801+0000 7f8d14ff9640 1 -- 172.21.15.19:0/2785585879 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8d0c004f20 con 0x7f8d18103b20 2024-04-23T18:36:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d20dff740 1 Processor -- start 2024-04-23T18:36:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d20dff740 1 -- start start 2024-04-23T18:36:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890b620 0x564ff890ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890bf40 0x564ff89162c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff8918bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d20dff740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ff8870970 con 0x564ff8916800 2024-04-23T18:36:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d20dff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564ff880d4a0 con 0x564ff890b620 2024-04-23T18:36:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d14ff9640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8d000a3dc0 0x7f8d000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d20dff740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564ff886c930 con 0x564ff890bf40 2024-04-23T18:36:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d17fff640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8d000a3dc0 0x7f8d000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff8918bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890bf40 0x564ff89162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff8918bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58188/0 (socket says 172.21.15.19:58188) 2024-04-23T18:36:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d16ffd640 1 -- 172.21.15.19:0/1927352361 learned_addr learned my addr 172.21.15.19:0/1927352361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.802+0000 7f8d17fff640 1 --2- 172.21.15.19:0/1927352361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890b620 0x564ff890ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8d14ff9640 1 -- 172.21.15.19:0/2785585879 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f8d0c0d04b0 con 0x7f8d18103b20 2024-04-23T18:36:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8d16ffd640 1 -- 172.21.15.19:0/1927352361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890bf40 msgr2=0x564ff89162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/1927352361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890bf40 0x564ff89162c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8d16ffd640 1 -- 172.21.15.19:0/1927352361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890b620 msgr2=0x564ff890ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/1927352361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890b620 0x564ff890ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8d16ffd640 1 -- 172.21.15.19:0/1927352361 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ff8583210 con 0x564ff8916800 2024-04-23T18:36:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8d17fff640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8d000a3dc0 0x7f8d000a6280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005dd0 tx=0x7f8d1000a9e0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/1927352361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff8918bf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c09c200 tx=0x7f8d0c06bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7009920a488151b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8cfdffb640 1 -- 172.21.15.19:0/1927352361 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c01d040 con 0x564ff8916800 2024-04-23T18:36:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8cfdffb640 1 -- 172.21.15.19:0/1927352361 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0c06be50 con 0x564ff8916800 2024-04-23T18:36:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.803+0000 7f8cfdffb640 1 -- 172.21.15.19:0/1927352361 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0cde80 con 0x564ff8916800 2024-04-23T18:36:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.804+0000 7f8d20dff740 1 -- 172.21.15.19:0/1927352361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 msgr2=0x564ff8918bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.804+0000 7f8d20dff740 1 --2- 172.21.15.19:0/1927352361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff8918bf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c09c200 tx=0x7f8d0c06bde0 comp rx=0 tx=0).stop 2024-04-23T18:36:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.804+0000 7f8d20dff740 1 -- 172.21.15.19:0/1927352361 shutdown_connections 2024-04-23T18:36:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.804+0000 7f8d20dff740 1 --2- 172.21.15.19:0/1927352361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff8918bf0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.804+0000 7f8d20dff740 1 --2- 172.21.15.19:0/1927352361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890bf40 0x564ff89162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.804+0000 7f8d20dff740 1 --2- 172.21.15.19:0/1927352361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890b620 0x564ff890ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.804+0000 7f8d20dff740 1 -- 172.21.15.19:0/1927352361 >> 172.21.15.19:0/1927352361 conn(0x564ff8907b30 msgr2=0x564ff8907f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.804+0000 7f8d20dff740 1 -- 172.21.15.19:0/1927352361 shutdown_connections 2024-04-23T18:36:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.804+0000 7f8d20dff740 1 -- 172.21.15.19:0/1927352361 wait complete. 2024-04-23T18:36:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.804+0000 7f8d20dff740 1 -- start start 2024-04-23T18:36:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d20dff740 1 Processor -- start 2024-04-23T18:36:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff8916150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8913440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff8916800 0x564ff89139a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d20dff740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ff8919430 con 0x564ff890bf40 2024-04-23T18:36:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d20dff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564ff880b0b0 con 0x564ff8916800 2024-04-23T18:36:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d20dff740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564ff89195b0 con 0x564ff890b620 2024-04-23T18:36:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d20dff740 10 client.0 _ll_register_callbacks cb 0x564ff87f8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8913440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8913440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58200/0 (socket says 172.21.15.19:58200) 2024-04-23T18:36:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d16ffd640 1 -- 172.21.15.19:0/2540754617 learned_addr learned my addr 172.21.15.19:0/2540754617 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d177fe640 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff8916150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d17fff640 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff8916800 0x564ff89139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d16ffd640 1 -- 172.21.15.19:0/2540754617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 msgr2=0x564ff8916150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff8916150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d16ffd640 1 -- 172.21.15.19:0/2540754617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff8916800 msgr2=0x564ff89139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff8916800 0x564ff89139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.805+0000 7f8d16ffd640 1 -- 172.21.15.19:0/2540754617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ff864f620 con 0x564ff890bf40 2024-04-23T18:36:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8913440 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c098960 tx=0x7f8d0c09aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaf44080e7c7246 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8cf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:36:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/2540754617 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c01d070 con 0x564ff890bf40 2024-04-23T18:36:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/2540754617 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f8d0c005210 con 0x564ff890bf40 2024-04-23T18:36:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8d20dff740 1 -- 172.21.15.19:0/2540754617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d0c06be50 con 0x564ff890bf40 2024-04-23T18:36:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/2540754617 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d0c005410 con 0x564ff890bf40 2024-04-23T18:36:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8d20dff740 10 client.16995 fetch_fsmap learned FSMap version 27 2024-04-23T18:36:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8d20dff740 1 -- 172.21.15.19:0/2540754617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ff8925360 con 0x564ff890bf40 2024-04-23T18:36:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/2540754617 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0c06be50 con 0x564ff890bf40 2024-04-23T18:36:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/2540754617 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d0c0a4c30 con 0x564ff890bf40 2024-04-23T18:36:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8d20dff740 10 client.16995 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:36:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8d20dff740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8d20dff740 20 client.16995 populate_metadata read hostname 'smithi019' 2024-04-23T18:36:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8d20dff740 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ff89266b0 0x564ff8928aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8d20dff740 4 client.16995 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.806+0000 7f8d20dff740 1 -- 172.21.15.19:0/2540754617 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ff887e390 con 0x564ff89266b0 2024-04-23T18:36:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.807+0000 7f8d177fe640 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ff89266b0 0x564ff8928aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.807+0000 7f8d177fe640 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ff89266b0 0x564ff8928aa0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76be393d2a48c653 server_cookie=c750a3f5e1396266 in_seq=0 out_seq=0 2024-04-23T18:36:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.807+0000 7f8cf6ffd640 10 client.16995 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/2540754617 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ff887e390 con 0x564ff89266b0 2024-04-23T18:36:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8cf6ffd640 10 client.16995 handle_command_reply: tid=0 2024-04-23T18:36:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 client.16995 shutdown 2024-04-23T18:36:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 -- 172.21.15.19:0/2540754617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 msgr2=0x564ff8913440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8913440 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c098960 tx=0x7f8d0c09aa10 comp rx=0 tx=0).stop 2024-04-23T18:36:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 -- 172.21.15.19:0/2540754617 shutdown_connections 2024-04-23T18:36:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ff89266b0 0x564ff8928aa0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff8916800 0x564ff89139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8913440 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8cf6ffd640 0 client.16995 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 --2- 172.21.15.19:0/2540754617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff8916150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 -- 172.21.15.19:0/2540754617 >> 172.21.15.19:0/2540754617 conn(0x564ff8907b30 msgr2=0x564ff890d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 -- 172.21.15.19:0/2540754617 shutdown_connections 2024-04-23T18:36:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 1 -- 172.21.15.19:0/2540754617 wait complete. 2024-04-23T18:36:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.808+0000 7f8d20dff740 20 client.16995 trim_cache size 0 max 16384 2024-04-23T18:36:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.823+0000 7f8d20dff740 1 Processor -- start 2024-04-23T18:36:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.823+0000 7f8d20dff740 1 -- start start 2024-04-23T18:36:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff892bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890bf40 0x564ff892c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:36.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff89106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d20dff740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181adb20 con 0x564ff8916800 2024-04-23T18:36:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d20dff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181af660 con 0x564ff890bf40 2024-04-23T18:36:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d20dff740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181ae1f0 con 0x564ff890b620 2024-04-23T18:36:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff89106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff89106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58216/0 (socket says 172.21.15.19:58216) 2024-04-23T18:36:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d17fff640 1 -- 172.21.15.19:0/3045837782 learned_addr learned my addr 172.21.15.19:0/3045837782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890bf40 0x564ff892c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d177fe640 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff892bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d17fff640 1 -- 172.21.15.19:0/3045837782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 msgr2=0x564ff892bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d17fff640 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff892bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d17fff640 1 -- 172.21.15.19:0/3045837782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890bf40 msgr2=0x564ff892c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d17fff640 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890bf40 0x564ff892c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d17fff640 1 -- 172.21.15.19:0/3045837782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ff89032e0 con 0x564ff8916800 2024-04-23T18:36:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d177fe640 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff892bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890bf40 0x564ff892c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8d17fff640 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff89106a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000a6a0 tx=0x7f8d10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997c639184eaa58a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8cfd7fa640 1 -- 172.21.15.19:0/3045837782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000fc60 con 0x564ff8916800 2024-04-23T18:36:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8cfd7fa640 1 -- 172.21.15.19:0/3045837782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d1000fe00 con 0x564ff8916800 2024-04-23T18:36:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.824+0000 7f8cfd7fa640 1 -- 172.21.15.19:0/3045837782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10004d90 con 0x564ff8916800 2024-04-23T18:36:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.825+0000 7f8d20dff740 1 -- 172.21.15.19:0/3045837782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 msgr2=0x564ff89106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.825+0000 7f8d20dff740 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff89106a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000a6a0 tx=0x7f8d10004730 comp rx=0 tx=0).stop 2024-04-23T18:36:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.825+0000 7f8d20dff740 1 -- 172.21.15.19:0/3045837782 shutdown_connections 2024-04-23T18:36:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.825+0000 7f8d20dff740 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff890bf40 0x564ff892c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.825+0000 7f8d20dff740 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff8916800 0x564ff89106a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.825+0000 7f8d20dff740 1 --2- 172.21.15.19:0/3045837782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff892bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.825+0000 7f8d20dff740 1 -- 172.21.15.19:0/3045837782 >> 172.21.15.19:0/3045837782 conn(0x564ff8907b30 msgr2=0x564ff8906cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.825+0000 7f8d20dff740 1 -- 172.21.15.19:0/3045837782 shutdown_connections 2024-04-23T18:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.825+0000 7f8d20dff740 1 -- 172.21.15.19:0/3045837782 wait complete. 2024-04-23T18:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.825+0000 7f8d20dff740 1 -- start start 2024-04-23T18:36:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d20dff740 1 Processor -- start 2024-04-23T18:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff8989590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8989ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d20dff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff8916800 0x564ff892b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d20dff740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181aef90 con 0x564ff890bf40 2024-04-23T18:36:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d20dff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181ae8c0 con 0x564ff8916800 2024-04-23T18:36:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d20dff740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181abb20 con 0x564ff890b620 2024-04-23T18:36:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d20dff740 10 client.0 _ll_register_callbacks cb 0x7f8d0c007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8989ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8989ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58232/0 (socket says 172.21.15.19:58232) 2024-04-23T18:36:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d16ffd640 1 -- 172.21.15.19:0/3290439354 learned_addr learned my addr 172.21.15.19:0/3290439354 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d17fff640 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff8916800 0x564ff892b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d177fe640 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff8989590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d16ffd640 1 -- 172.21.15.19:0/3290439354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 msgr2=0x564ff8989590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff8989590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d16ffd640 1 -- 172.21.15.19:0/3290439354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff8916800 msgr2=0x564ff892b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff8916800 0x564ff892b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.826+0000 7f8d16ffd640 1 -- 172.21.15.19:0/3290439354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ff8913d10 con 0x564ff890bf40 2024-04-23T18:36:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8d16ffd640 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8989ad0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0a4b90 tx=0x7f8d0c098120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e637ae6f0d7bba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8cf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:36:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/3290439354 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c01d070 con 0x564ff890bf40 2024-04-23T18:36:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/3290439354 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f8d0c0a4c30 con 0x564ff890bf40 2024-04-23T18:36:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8d20dff740 1 -- 172.21.15.19:0/3290439354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d1000fe00 con 0x564ff890bf40 2024-04-23T18:36:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/3290439354 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d0c0a50c0 con 0x564ff890bf40 2024-04-23T18:36:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8d20dff740 10 client.17001 fetch_fsmap learned FSMap version 27 2024-04-23T18:36:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8d20dff740 1 -- 172.21.15.19:0/3290439354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ff8913d10 con 0x564ff890bf40 2024-04-23T18:36:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/3290439354 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d1000fe00 con 0x564ff890bf40 2024-04-23T18:36:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/3290439354 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d0c09b3c0 con 0x564ff890bf40 2024-04-23T18:36:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8d20dff740 10 client.17001 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:36:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8d20dff740 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8d20dff740 20 client.17001 populate_metadata read hostname 'smithi019' 2024-04-23T18:36:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8d20dff740 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ff8990e80 0x564ff8993270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8d20dff740 4 client.17001 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:36:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.827+0000 7f8d20dff740 1 -- 172.21.15.19:0/3290439354 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ff8878200 con 0x564ff8990e80 2024-04-23T18:36:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.828+0000 7f8d177fe640 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ff8990e80 0x564ff8993270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.828+0000 7f8cf6ffd640 10 client.17001 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.828+0000 7f8d177fe640 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ff8990e80 0x564ff8993270 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2e967bbc7abb78 server_cookie=334e186fcb15184d in_seq=0 out_seq=0 2024-04-23T18:36:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8cf6ffd640 1 -- 172.21.15.19:0/3290439354 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ff8878200 con 0x564ff8990e80 2024-04-23T18:36:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8cf6ffd640 10 client.17001 handle_command_reply: tid=0 2024-04-23T18:36:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 client.17001 shutdown 2024-04-23T18:36:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 -- 172.21.15.19:0/3290439354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 msgr2=0x564ff8989ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8989ad0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0a4b90 tx=0x7f8d0c098120 comp rx=0 tx=0).stop 2024-04-23T18:36:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 -- 172.21.15.19:0/3290439354 shutdown_connections 2024-04-23T18:36:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564ff8990e80 0x564ff8993270 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ff8916800 0x564ff892b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8cf6ffd640 0 client.17001 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:36:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ff890bf40 0x564ff8989ad0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 --2- 172.21.15.19:0/3290439354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ff890b620 0x564ff8989590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 -- 172.21.15.19:0/3290439354 >> 172.21.15.19:0/3290439354 conn(0x564ff8907b30 msgr2=0x564ff8906cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 -- 172.21.15.19:0/3290439354 shutdown_connections 2024-04-23T18:36:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 1 -- 172.21.15.19:0/3290439354 wait complete. 2024-04-23T18:36:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.842+0000 7f8d20dff740 20 client.17001 trim_cache size 0 max 16384 2024-04-23T18:36:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:36:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:36:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:36:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:36:40.267 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:31 smithi155 ceph-mon[33468]: pgmap v322: 97 pgs: 97 active+clean; 2.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-04-23T18:36:40.300 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:33 smithi155 ceph-mon[33468]: pgmap v323: 97 pgs: 97 active+clean; 2.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:36:40.333 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:35 smithi155 ceph-mon[33468]: pgmap v324: 97 pgs: 97 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-23T18:36:40.413 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:37 smithi155 ceph-mon[33468]: pgmap v325: 97 pgs: 97 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:36:40.444 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:31 smithi044 ceph-mon[32458]: pgmap v322: 97 pgs: 97 active+clean; 2.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-04-23T18:36:40.475 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:33 smithi044 ceph-mon[32458]: pgmap v323: 97 pgs: 97 active+clean; 2.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:36:40.508 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:35 smithi044 ceph-mon[32458]: pgmap v324: 97 pgs: 97 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-23T18:36:40.542 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:37 smithi044 ceph-mon[32458]: pgmap v325: 97 pgs: 97 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:36:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 -- 172.21.15.19:0/2785585879 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8d000a3dc0 msgr2=0x7f8d000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8d000a3dc0 0x7f8d000a6280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005dd0 tx=0x7f8d1000a9e0 comp rx=0 tx=0).stop 2024-04-23T18:36:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 -- 172.21.15.19:0/2785585879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d18103b20 msgr2=0x7f8d18198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d18103b20 0x7f8d18198bb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00c970 tx=0x7f8d0c00ce30 comp rx=0 tx=0).stop 2024-04-23T18:36:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 -- 172.21.15.19:0/2785585879 shutdown_connections 2024-04-23T18:36:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8d000a3dc0 0x7f8d000a6280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d1810c5d0 0x7f8d1819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d18103b20 0x7f8d18198bb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 --2- 172.21.15.19:0/2785585879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d18103200 0x7f8d18198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 -- 172.21.15.19:0/2785585879 >> 172.21.15.19:0/2785585879 conn(0x7f8d180771d0 msgr2=0x7f8d180fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 -- 172.21.15.19:0/2785585879 shutdown_connections 2024-04-23T18:36:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:28.843+0000 7f8d1e010640 1 -- 172.21.15.19:0/2785585879 wait complete. 2024-04-23T18:36:41.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:29 smithi019 ceph-mon[29215]: pgmap v321: 97 pgs: 97 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T18:36:41.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:31 smithi019 ceph-mon[29215]: pgmap v322: 97 pgs: 97 active+clean; 2.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-04-23T18:36:41.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:33 smithi019 ceph-mon[29215]: pgmap v323: 97 pgs: 97 active+clean; 2.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:36:41.576 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:41.609 INFO:tasks.workunit.client.0.smithi019.stdout:313/600: open fd = 3 2024-04-23T18:36:41.659 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:41.688 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:41.779 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:41.801 INFO:tasks.workunit.client.0.smithi019.stdout:314/600: open fd = 3 2024-04-23T18:36:41.829 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:41.929 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:41.962 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:41.995 INFO:tasks.workunit.client.0.smithi019.stdout:315/600: open fd = 3 2024-04-23T18:36:42.029 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:42.062 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:42.095 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:42.129 INFO:tasks.workunit.client.0.smithi019.stdout:316/600: open fd = 3 2024-04-23T18:36:42.179 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:42.241 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:42.279 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:42.329 INFO:tasks.workunit.client.0.smithi019.stdout:317/600: open fd = 3 2024-04-23T18:36:42.446 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:42.496 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:42.546 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:42.596 INFO:tasks.workunit.client.0.smithi019.stdout:318/600: open fd = 3 2024-04-23T18:36:42.646 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:42.696 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:42.746 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:42.800 INFO:tasks.workunit.client.0.smithi019.stdout:319/600: open fd = 3 2024-04-23T18:36:42.862 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:42.896 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:42.946 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:43.080 INFO:tasks.workunit.client.0.smithi019.stdout:320/600: open fd = 3 2024-04-23T18:36:43.130 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:43.180 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:43.230 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:43.280 INFO:tasks.workunit.client.0.smithi019.stdout:321/600: open fd = 3 2024-04-23T18:36:43.330 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:43.380 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:43.455 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:43.505 INFO:tasks.workunit.client.0.smithi019.stdout:322/600: open fd = 3 2024-04-23T18:36:43.555 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:43.605 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:43.722 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:43.772 INFO:tasks.workunit.client.0.smithi019.stdout:323/600: open fd = 3 2024-04-23T18:36:43.822 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:43.872 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:43.930 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:43.980 INFO:tasks.workunit.client.0.smithi019.stdout:324/600: open fd = 3 2024-04-23T18:36:44.030 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:44.080 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:44.144 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:44.180 INFO:tasks.workunit.client.0.smithi019.stdout:325/600: open fd = 3 2024-04-23T18:36:44.230 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:44.364 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:44.414 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:44.463 INFO:tasks.workunit.client.0.smithi019.stdout:326/600: open fd = 3 2024-04-23T18:36:44.514 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:44.564 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:44.614 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:44.664 INFO:tasks.workunit.client.0.smithi019.stdout:327/600: open fd = 3 2024-04-23T18:36:44.714 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:44.785 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:44.839 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:44.889 INFO:tasks.workunit.client.0.smithi019.stdout:328/600: open fd = 3 2024-04-23T18:36:44.997 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:45.047 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:45.097 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:45.147 INFO:tasks.workunit.client.0.smithi019.stdout:329/600: open fd = 3 2024-04-23T18:36:45.197 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:45.247 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:45.298 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:45.347 INFO:tasks.workunit.client.0.smithi019.stdout:330/600: open fd = 3 2024-04-23T18:36:45.427 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:45.473 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:45.523 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:45.631 INFO:tasks.workunit.client.0.smithi019.stdout:331/600: open fd = 3 2024-04-23T18:36:45.681 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:45.723 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:45.764 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:45.798 INFO:tasks.workunit.client.0.smithi019.stdout:332/600: open fd = 3 2024-04-23T18:36:45.831 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:45.873 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:45.945 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:46.026 INFO:tasks.workunit.client.0.smithi019.stdout:333/600: open fd = 3 2024-04-23T18:36:46.065 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:46.098 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:46.190 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:46.223 INFO:tasks.workunit.client.0.smithi019.stdout:334/600: open fd = 3 2024-04-23T18:36:46.257 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:46.290 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:46.323 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:46.357 INFO:tasks.workunit.client.0.smithi019.stdout:335/600: open fd = 3 2024-04-23T18:36:46.398 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:46.446 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:46.536 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:46.565 INFO:tasks.workunit.client.0.smithi019.stdout:336/600: open fd = 3 2024-04-23T18:36:46.599 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:46.715 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:46.765 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:46.799 INFO:tasks.workunit.client.0.smithi019.stdout:337/600: open fd = 3 2024-04-23T18:36:46.832 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:46.865 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:46.899 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:46.932 INFO:tasks.workunit.client.0.smithi019.stdout:338/600: open fd = 3 2024-04-23T18:36:46.974 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.001 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.053 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:47.082 INFO:tasks.workunit.client.0.smithi019.stdout:339/600: open fd = 3 2024-04-23T18:36:47.191 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.224 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.257 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:47.291 INFO:tasks.workunit.client.0.smithi019.stdout:340/600: open fd = 3 2024-04-23T18:36:47.324 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.366 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.399 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:47.441 INFO:tasks.workunit.client.0.smithi019.stdout:341/600: open fd = 3 2024-04-23T18:36:47.466 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.543 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.574 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:47.683 INFO:tasks.workunit.client.0.smithi019.stdout:342/600: open fd = 3 2024-04-23T18:36:47.716 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.749 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.799 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:47.850 INFO:tasks.workunit.client.0.smithi019.stdout:343/600: open fd = 3 2024-04-23T18:36:47.900 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:47.950 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:48.020 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:48.140 INFO:tasks.workunit.client.0.smithi019.stdout:344/600: open fd = 3 2024-04-23T18:36:48.175 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:48.225 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:48.358 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:48.408 INFO:tasks.workunit.client.0.smithi019.stdout:345/600: open fd = 3 2024-04-23T18:36:48.458 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:48.508 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:48.558 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:48.608 INFO:tasks.workunit.client.0.smithi019.stdout:346/600: open fd = 3 2024-04-23T18:36:48.658 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:48.708 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:48.770 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:48.843 INFO:tasks.workunit.client.0.smithi019.stdout:347/600: open fd = 3 2024-04-23T18:36:48.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:48.992 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:49.042 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:49.092 INFO:tasks.workunit.client.0.smithi019.stdout:348/600: open fd = 3 2024-04-23T18:36:49.142 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:49.192 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:49.242 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:49.302 INFO:tasks.workunit.client.0.smithi019.stdout:349/600: open fd = 3 2024-04-23T18:36:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:49.400 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:49.450 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:49.518 INFO:tasks.workunit.client.0.smithi019.stdout:350/600: open fd = 3 2024-04-23T18:36:49.567 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:49.618 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:49.668 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:49.718 INFO:tasks.workunit.client.0.smithi019.stdout:351/600: open fd = 3 2024-04-23T18:36:49.768 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:49.818 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:49.975 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:50.026 INFO:tasks.workunit.client.0.smithi019.stdout:352/600: open fd = 3 2024-04-23T18:36:50.076 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:50.126 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:50.176 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:50.226 INFO:tasks.workunit.client.0.smithi019.stdout:353/600: open fd = 3 2024-04-23T18:36:50.276 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:50.326 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:50.376 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:50.426 INFO:tasks.workunit.client.0.smithi019.stdout:354/600: open fd = 3 2024-04-23T18:36:50.476 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:50.527 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:50.579 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:50.627 INFO:tasks.workunit.client.0.smithi019.stdout:355/600: open fd = 3 2024-04-23T18:36:50.676 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:50.727 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:50.860 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:50.902 INFO:tasks.workunit.client.0.smithi019.stdout:356/600: open fd = 3 2024-04-23T18:36:50.943 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:50.977 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:51.010 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:51.043 INFO:tasks.workunit.client.0.smithi019.stdout:357/600: open fd = 3 2024-04-23T18:36:51.077 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:51.110 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:51.519 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:51.665 INFO:tasks.workunit.client.0.smithi019.stdout:358/600: open fd = 3 2024-04-23T18:36:51.702 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:51.780 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:51.811 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:359/600: open fd = 3 2024-04-23T18:36:51.927 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:51.961 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:51.994 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:52.027 INFO:tasks.workunit.client.0.smithi019.stdout:360/600: open fd = 3 2024-04-23T18:36:52.061 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.094 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.152 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:52.178 INFO:tasks.workunit.client.0.smithi019.stdout:361/600: open fd = 3 2024-04-23T18:36:52.228 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.261 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.353 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:52.386 INFO:tasks.workunit.client.0.smithi019.stdout:362/600: open fd = 3 2024-04-23T18:36:52.419 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.453 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.486 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:52.520 INFO:tasks.workunit.client.0.smithi019.stdout:363/600: open fd = 3 2024-04-23T18:36:52.570 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.630 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.670 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:52.728 INFO:tasks.workunit.client.0.smithi019.stdout:364/600: open fd = 3 2024-04-23T18:36:52.762 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.845 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.878 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:52.911 INFO:tasks.workunit.client.0.smithi019.stdout:365/600: open fd = 3 2024-04-23T18:36:52.945 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:52.978 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:53.012 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:53.045 INFO:tasks.workunit.client.0.smithi019.stdout:366/600: open fd = 3 2024-04-23T18:36:53.121 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:53.162 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:53.221 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:53.262 INFO:tasks.workunit.client.0.smithi019.stdout:367/600: open fd = 3 2024-04-23T18:36:53.362 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:53.412 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:53.462 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:53.512 INFO:tasks.workunit.client.0.smithi019.stdout:368/600: open fd = 3 2024-04-23T18:36:53.562 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:53.612 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:53.662 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:53.755 INFO:tasks.workunit.client.0.smithi019.stdout:369/600: open fd = 3 2024-04-23T18:36:53.795 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:53.845 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:53.896 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:54.012 INFO:tasks.workunit.client.0.smithi019.stdout:370/600: open fd = 3 2024-04-23T18:36:54.062 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:54.112 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:54.162 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:54.212 INFO:tasks.workunit.client.0.smithi019.stdout:371/600: open fd = 3 2024-04-23T18:36:54.262 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:54.312 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:54.418 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:372/600: open fd = 3 2024-04-23T18:36:54.605 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:54.638 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:54.788 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:54.838 INFO:tasks.workunit.client.0.smithi019.stdout:373/600: open fd = 3 2024-04-23T18:36:54.888 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:54.938 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:54.988 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:55.038 INFO:tasks.workunit.client.0.smithi019.stdout:374/600: open fd = 3 2024-04-23T18:36:55.088 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:55.166 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:36:55.213 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:36:55.272 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:35 smithi019 ceph-mon[29215]: pgmap v324: 97 pgs: 97 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-23T18:36:55.313 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:37 smithi019 ceph-mon[29215]: pgmap v325: 97 pgs: 97 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:36:55.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:39 smithi019 ceph-mon[29215]: pgmap v326: 97 pgs: 97 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:36:55.458 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:39 smithi155 ceph-mon[33468]: pgmap v326: 97 pgs: 97 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:36:55.505 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:41 smithi155 ceph-mon[33468]: pgmap v327: 97 pgs: 97 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:36:55.630 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:43 smithi155 ceph-mon[33468]: pgmap v328: 97 pgs: 97 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-04-23T18:36:55.688 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:45 smithi155 ceph-mon[33468]: pgmap v329: 97 pgs: 97 active+clean; 2.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:36:55.738 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:47 smithi155 ceph-mon[33468]: pgmap v330: 97 pgs: 97 active+clean; 2.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T18:36:55.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:49 smithi155 ceph-mon[33468]: pgmap v331: 97 pgs: 97 active+clean; 2.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T18:36:55.838 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:51 smithi155 ceph-mon[33468]: pgmap v332: 97 pgs: 97 active+clean; 2.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:36:55.889 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:53 smithi155 ceph-mon[33468]: pgmap v333: 97 pgs: 97 active+clean; 2.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:36:55.938 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:39 smithi044 ceph-mon[32458]: pgmap v326: 97 pgs: 97 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:36:55.989 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:41 smithi044 ceph-mon[32458]: pgmap v327: 97 pgs: 97 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:36:56.155 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:43 smithi044 ceph-mon[32458]: pgmap v328: 97 pgs: 97 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-04-23T18:36:56.205 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:45 smithi044 ceph-mon[32458]: pgmap v329: 97 pgs: 97 active+clean; 2.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:36:56.247 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:47 smithi044 ceph-mon[32458]: pgmap v330: 97 pgs: 97 active+clean; 2.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T18:36:56.280 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:49 smithi044 ceph-mon[32458]: pgmap v331: 97 pgs: 97 active+clean; 2.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T18:36:56.322 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:51 smithi044 ceph-mon[32458]: pgmap v332: 97 pgs: 97 active+clean; 2.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:36:56.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:53 smithi044 ceph-mon[32458]: pgmap v333: 97 pgs: 97 active+clean; 2.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:36:56.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:55 smithi155 ceph-mon[33468]: pgmap v334: 97 pgs: 97 active+clean; 3.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:36:56.507 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:55 smithi044 ceph-mon[32458]: pgmap v334: 97 pgs: 97 active+clean; 3.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:36:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.945+0000 7efcbf7a2640 1 -- 172.21.15.19:0/3201466739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb8102b80 msgr2=0x7efcb8106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.945+0000 7efcbf7a2640 1 --2- 172.21.15.19:0/3201466739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb8102b80 0x7efcb8106bc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7efca8009900 tx=0x7efca802a410 comp rx=0 tx=0).stop 2024-04-23T18:36:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.946+0000 7efcbf7a2640 1 -- 172.21.15.19:0/3201466739 shutdown_connections 2024-04-23T18:36:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.946+0000 7efcbf7a2640 1 --2- 172.21.15.19:0/3201466739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb81072a0 0x7efcb810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.946+0000 7efcbf7a2640 1 --2- 172.21.15.19:0/3201466739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb8102b80 0x7efcb8106bc0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.946+0000 7efcbf7a2640 1 --2- 172.21.15.19:0/3201466739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcb81021d0 0x7efcb81025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.946+0000 7efcbf7a2640 1 -- 172.21.15.19:0/3201466739 >> 172.21.15.19:0/3201466739 conn(0x7efcb80793e0 msgr2=0x7efcb80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.946+0000 7efcbf7a2640 1 -- 172.21.15.19:0/3201466739 shutdown_connections 2024-04-23T18:36:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.946+0000 7efcbf7a2640 1 -- 172.21.15.19:0/3201466739 wait complete. 2024-04-23T18:36:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.946+0000 7efcbf7a2640 1 Processor -- start 2024-04-23T18:36:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.946+0000 7efcbf7a2640 1 -- start start 2024-04-23T18:36:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbf7a2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb81021d0 0x7efcb81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbf7a2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcb81072a0 0x7efcb8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbf7a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb819cfa0 0x7efcb819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbf7a2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81121e0 con 0x7efcb819cfa0 2024-04-23T18:36:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbf7a2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8112060 con 0x7efcb81021d0 2024-04-23T18:36:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbf7a2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8112360 con 0x7efcb81072a0 2024-04-23T18:36:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbdd3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb819cfa0 0x7efcb819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbdd3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb819cfa0 0x7efcb819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58336/0 (socket says 172.21.15.19:58336) 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbdd3f640 1 -- 172.21.15.19:0/1040879990 learned_addr learned my addr 172.21.15.19:0/1040879990 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:36:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbd53e640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb81021d0 0x7efcb81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcb81072a0 0x7efcb8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbdd3f640 1 -- 172.21.15.19:0/1040879990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcb81072a0 msgr2=0x7efcb8198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbdd3f640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcb81072a0 0x7efcb8198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbdd3f640 1 -- 172.21.15.19:0/1040879990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb81021d0 msgr2=0x7efcb81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbdd3f640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb81021d0 0x7efcb81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbdd3f640 1 -- 172.21.15.19:0/1040879990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb819da90 con 0x7efcb819cfa0 2024-04-23T18:36:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbdd3f640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb819cfa0 0x7efcb819d450 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7efcb000cab0 tx=0x7efcb000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcae7fc640 1 -- 172.21.15.19:0/1040879990 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb000adc0 con 0x7efcb819cfa0 2024-04-23T18:36:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.947+0000 7efcbf7a2640 1 -- 172.21.15.19:0/1040879990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb819dd80 con 0x7efcb819cfa0 2024-04-23T18:36:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.948+0000 7efcae7fc640 1 -- 172.21.15.19:0/1040879990 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb000f900 con 0x7efcb819cfa0 2024-04-23T18:36:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.948+0000 7efcae7fc640 1 -- 172.21.15.19:0/1040879990 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb00189a0 con 0x7efcb819cfa0 2024-04-23T18:36:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.948+0000 7efcbf7a2640 1 -- 172.21.15.19:0/1040879990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb81a5540 con 0x7efcb819cfa0 2024-04-23T18:36:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.950+0000 7efcae7fc640 1 -- 172.21.15.19:0/1040879990 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efcb000f450 con 0x7efcb819cfa0 2024-04-23T18:36:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.950+0000 7efcc2596740 1 Processor -- start 2024-04-23T18:36:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.950+0000 7efcc2596740 1 -- start start 2024-04-23T18:36:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcc2596740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c9823a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcc2596740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c9823f40 0x55d2c982e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcc2596740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c982e800 0x55d2c9830bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcc2596740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9788970 con 0x55d2c9823620 2024-04-23T18:36:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcc2596740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d2c97254a0 con 0x55d2c982e800 2024-04-23T18:36:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcc2596740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9784930 con 0x55d2c9823f40 2024-04-23T18:36:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcae7fc640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efca40a3dc0 0x7efca40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbcd3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c9823a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbcd3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c9823a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58344/0 (socket says 172.21.15.19:58344) 2024-04-23T18:37:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbcd3d640 1 -- 172.21.15.19:0/477933383 learned_addr learned my addr 172.21.15.19:0/477933383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbdd3f640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efca40a3dc0 0x7efca40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbd53e640 1 --2- 172.21.15.19:0/477933383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c9823f40 0x55d2c982e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcae7fc640 1 -- 172.21.15.19:0/1040879990 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7efcb00d0b30 con 0x7efcb819cfa0 2024-04-23T18:37:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/477933383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c982e800 0x55d2c9830bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbcd3d640 1 -- 172.21.15.19:0/477933383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c9823f40 msgr2=0x55d2c982e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/477933383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c9823f40 0x55d2c982e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbcd3d640 1 -- 172.21.15.19:0/477933383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c982e800 msgr2=0x55d2c9830bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/477933383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c982e800 0x55d2c9830bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbcd3d640 1 -- 172.21.15.19:0/477933383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c949b210 con 0x55d2c9823620 2024-04-23T18:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.951+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/477933383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c9823a00 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efca8005ab0 tx=0x7efca80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8766004055d2b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efcbdd3f640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efca40a3dc0 0x7efca40a6280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efcb00ce880 tx=0x7efcb006e510 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efca37fe640 1 -- 172.21.15.19:0/477933383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8035050 con 0x55d2c9823620 2024-04-23T18:37:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efca37fe640 1 -- 172.21.15.19:0/477933383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca8031870 con 0x55d2c9823620 2024-04-23T18:37:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efca37fe640 1 -- 172.21.15.19:0/477933383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8004de0 con 0x55d2c9823620 2024-04-23T18:37:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efcc2596740 1 -- 172.21.15.19:0/477933383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 msgr2=0x55d2c9823a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efcc2596740 1 --2- 172.21.15.19:0/477933383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c9823a00 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efca8005ab0 tx=0x7efca80058b0 comp rx=0 tx=0).stop 2024-04-23T18:37:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efcc2596740 1 -- 172.21.15.19:0/477933383 shutdown_connections 2024-04-23T18:37:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efcc2596740 1 --2- 172.21.15.19:0/477933383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c982e800 0x55d2c9830bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efcc2596740 1 --2- 172.21.15.19:0/477933383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c9823f40 0x55d2c982e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efcc2596740 1 --2- 172.21.15.19:0/477933383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c9823a00 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efcc2596740 1 -- 172.21.15.19:0/477933383 >> 172.21.15.19:0/477933383 conn(0x55d2c981fb30 msgr2=0x55d2c981ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efcc2596740 1 -- 172.21.15.19:0/477933383 shutdown_connections 2024-04-23T18:37:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.952+0000 7efcc2596740 1 -- 172.21.15.19:0/477933383 wait complete. 2024-04-23T18:37:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.953+0000 7efcc2596740 1 -- start start 2024-04-23T18:37:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.953+0000 7efcc2596740 1 Processor -- start 2024-04-23T18:37:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.953+0000 7efcc2596740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823620 0x55d2c982e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.953+0000 7efcc2596740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823f40 0x55d2c982b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.953+0000 7efcc2596740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c982e800 0x55d2c982b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.953+0000 7efcc2596740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d2c98313d0 con 0x55d2c9823f40 2024-04-23T18:37:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.953+0000 7efcc2596740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d2c97230b0 con 0x55d2c9823620 2024-04-23T18:37:03.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.953+0000 7efcc2596740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9831550 con 0x55d2c982e800 2024-04-23T18:37:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.953+0000 7efcc2596740 10 client.0 _ll_register_callbacks cb 0x55d2c9710d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbd53e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823f40 0x55d2c982b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbd53e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823f40 0x55d2c982b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58360/0 (socket says 172.21.15.19:58360) 2024-04-23T18:37:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbd53e640 1 -- 172.21.15.19:0/726421936 learned_addr learned my addr 172.21.15.19:0/726421936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823620 0x55d2c982e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c982e800 0x55d2c982b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbd53e640 1 -- 172.21.15.19:0/726421936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c982e800 msgr2=0x55d2c982b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbd53e640 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c982e800 0x55d2c982b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbd53e640 1 -- 172.21.15.19:0/726421936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823620 msgr2=0x55d2c982e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:04.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbd53e640 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823620 0x55d2c982e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbd53e640 1 -- 172.21.15.19:0/726421936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c9567620 con 0x55d2c9823f40 2024-04-23T18:37:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823620 0x55d2c982e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:37:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcbd53e640 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823f40 0x55d2c982b450 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efcb4011800 tx=0x7efcb4011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbcd2abb8655710 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efca0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efca0ff9640 1 -- 172.21.15.19:0/726421936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb401e060 con 0x55d2c9823f40 2024-04-23T18:37:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efca0ff9640 1 -- 172.21.15.19:0/726421936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7efcb400fdb0 con 0x55d2c9823f40 2024-04-23T18:37:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.954+0000 7efcc2596740 1 -- 172.21.15.19:0/726421936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca8031870 con 0x55d2c9823f40 2024-04-23T18:37:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efca0ff9640 1 -- 172.21.15.19:0/726421936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efcb4022410 con 0x55d2c9823f40 2024-04-23T18:37:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efcc2596740 10 client.17013 fetch_fsmap learned FSMap version 27 2024-04-23T18:37:05.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efcc2596740 1 -- 172.21.15.19:0/726421936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2c983d360 con 0x55d2c9823f40 2024-04-23T18:37:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efca0ff9640 1 -- 172.21.15.19:0/726421936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca8031870 con 0x55d2c9823f40 2024-04-23T18:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efca0ff9640 1 -- 172.21.15.19:0/726421936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efcb4018030 con 0x55d2c9823f40 2024-04-23T18:37:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efcc2596740 10 client.17013 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:37:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efcc2596740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efcc2596740 20 client.17013 populate_metadata read hostname 'smithi019' 2024-04-23T18:37:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efcc2596740 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d2c983e870 0x55d2c9840c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efcc2596740 4 client.17013 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:37:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efcc2596740 1 -- 172.21.15.19:0/726421936 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2c9796390 con 0x55d2c983e870 2024-04-23T18:37:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.955+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d2c983e870 0x55d2c9840c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.956+0000 7efca0ff9640 10 client.17013 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:37:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.956+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d2c983e870 0x55d2c9840c60 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3755f180f96d8e59 server_cookie=3529d50b461d31a8 in_seq=0 out_seq=0 2024-04-23T18:37:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.956+0000 7efca0ff9640 1 -- 172.21.15.19:0/726421936 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2c9796390 con 0x55d2c983e870 2024-04-23T18:37:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.956+0000 7efca0ff9640 10 client.17013 handle_command_reply: tid=0 2024-04-23T18:37:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.956+0000 7efcc2596740 1 client.17013 shutdown 2024-04-23T18:37:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 1 -- 172.21.15.19:0/726421936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823f40 msgr2=0x55d2c982b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823f40 0x55d2c982b450 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efcb4011800 tx=0x7efcb4011cc0 comp rx=0 tx=0).stop 2024-04-23T18:37:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 1 -- 172.21.15.19:0/726421936 shutdown_connections 2024-04-23T18:37:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d2c983e870 0x55d2c9840c60 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c982e800 0x55d2c982b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efca0ff9640 0 client.17013 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:37:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823f40 0x55d2c982b450 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 1 --2- 172.21.15.19:0/726421936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823620 0x55d2c982e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 1 -- 172.21.15.19:0/726421936 >> 172.21.15.19:0/726421936 conn(0x55d2c981fb30 msgr2=0x55d2c98258b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 1 -- 172.21.15.19:0/726421936 shutdown_connections 2024-04-23T18:37:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 1 -- 172.21.15.19:0/726421936 wait complete. 2024-04-23T18:37:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.957+0000 7efcc2596740 20 client.17013 trim_cache size 0 max 16384 2024-04-23T18:37:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcc2596740 1 Processor -- start 2024-04-23T18:37:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcc2596740 1 -- start start 2024-04-23T18:37:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcc2596740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c983a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcc2596740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 0x55d2c9846550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcc2596740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c982e800 0x55d2c9846a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcc2596740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81ada10 con 0x55d2c9823620 2024-04-23T18:37:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcc2596740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81af550 con 0x55d2c9823f40 2024-04-23T18:37:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcc2596740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81ae0e0 con 0x55d2c982e800 2024-04-23T18:37:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcbcd3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c983a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcbcd3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c983a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58364/0 (socket says 172.21.15.19:58364) 2024-04-23T18:37:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcbcd3d640 1 -- 172.21.15.19:0/2048014198 learned_addr learned my addr 172.21.15.19:0/2048014198 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcbd53e640 1 --2- 172.21.15.19:0/2048014198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 0x55d2c9846550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/2048014198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c982e800 0x55d2c9846a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcbcd3d640 1 -- 172.21.15.19:0/2048014198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c982e800 msgr2=0x55d2c9846a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/2048014198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c982e800 0x55d2c9846a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcbcd3d640 1 -- 172.21.15.19:0/2048014198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 msgr2=0x55d2c9846550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/2048014198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 0x55d2c9846550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.973+0000 7efcbcd3d640 1 -- 172.21.15.19:0/2048014198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c982a400 con 0x55d2c9823620 2024-04-23T18:37:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/2048014198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c983a290 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efca8005d90 tx=0x7efca8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f724f9e5600c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efca2ffd640 1 -- 172.21.15.19:0/2048014198 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8035040 con 0x55d2c9823620 2024-04-23T18:37:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efca2ffd640 1 -- 172.21.15.19:0/2048014198 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca8052030 con 0x55d2c9823620 2024-04-23T18:37:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efca2ffd640 1 -- 172.21.15.19:0/2048014198 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca803a540 con 0x55d2c9823620 2024-04-23T18:37:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efcc2596740 1 -- 172.21.15.19:0/2048014198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 msgr2=0x55d2c983a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efcc2596740 1 --2- 172.21.15.19:0/2048014198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c983a290 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efca8005d90 tx=0x7efca8005e80 comp rx=0 tx=0).stop 2024-04-23T18:37:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efcc2596740 1 -- 172.21.15.19:0/2048014198 shutdown_connections 2024-04-23T18:37:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efcc2596740 1 --2- 172.21.15.19:0/2048014198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 0x55d2c9846550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efcc2596740 1 --2- 172.21.15.19:0/2048014198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c982e800 0x55d2c9846a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efcc2596740 1 --2- 172.21.15.19:0/2048014198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c9823620 0x55d2c983a290 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efcc2596740 1 -- 172.21.15.19:0/2048014198 >> 172.21.15.19:0/2048014198 conn(0x55d2c981fb30 msgr2=0x55d2c981ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efcc2596740 1 -- 172.21.15.19:0/2048014198 shutdown_connections 2024-04-23T18:37:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.974+0000 7efcc2596740 1 -- 172.21.15.19:0/2048014198 wait complete. 2024-04-23T18:37:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.975+0000 7efcc2596740 1 -- start start 2024-04-23T18:37:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.975+0000 7efcc2596740 1 Processor -- start 2024-04-23T18:37:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.975+0000 7efcc2596740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c9823620 0x55d2c9836a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.975+0000 7efcc2596740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 0x55d2c9833df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.975+0000 7efcc2596740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c982e800 0x55d2c9834330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.975+0000 7efcc2596740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81aee80 con 0x55d2c982e800 2024-04-23T18:37:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.975+0000 7efcc2596740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81ae7b0 con 0x55d2c9823f40 2024-04-23T18:37:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.975+0000 7efcc2596740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81aba10 con 0x55d2c9823620 2024-04-23T18:37:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.975+0000 7efcc2596740 10 client.0 _ll_register_callbacks cb 0x7efcb4011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.975+0000 7efcbcd3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c982e800 0x55d2c9834330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbcd3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c982e800 0x55d2c9834330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58368/0 (socket says 172.21.15.19:58368) 2024-04-23T18:37:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbd53e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 0x55d2c9833df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbcd3d640 1 -- 172.21.15.19:0/2923408292 learned_addr learned my addr 172.21.15.19:0/2923408292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c9823620 0x55d2c9836a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbd53e640 1 -- 172.21.15.19:0/2923408292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c9823620 msgr2=0x55d2c9836a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbd53e640 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c9823620 0x55d2c9836a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbd53e640 1 -- 172.21.15.19:0/2923408292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c982e800 msgr2=0x55d2c9834330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbd53e640 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c982e800 0x55d2c9834330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbd53e640 1 -- 172.21.15.19:0/2923408292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c982bd20 con 0x55d2c9823f40 2024-04-23T18:37:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c982e800 0x55d2c9834330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:37:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcbd53e640 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 0x55d2c9833df0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e6c0 tx=0x7efcb40052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34486d5a2ce562f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efca0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-23T18:37:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efca0ff9640 1 -- 172.21.15.19:0/2923408292 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4005c00 con 0x55d2c9823f40 2024-04-23T18:37:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efcc2596740 1 -- 172.21.15.19:0/2923408292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca8052030 con 0x55d2c9823f40 2024-04-23T18:37:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.976+0000 7efca0ff9640 1 -- 172.21.15.19:0/2923408292 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7efcb400e940 con 0x55d2c9823f40 2024-04-23T18:37:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efca0ff9640 1 -- 172.21.15.19:0/2923408292 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efcb4039050 con 0x55d2c9823f40 2024-04-23T18:37:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efcc2596740 10 client.26288 fetch_fsmap learned FSMap version 27 2024-04-23T18:37:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efcc2596740 1 -- 172.21.15.19:0/2923408292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2c9840fd0 con 0x55d2c9823f40 2024-04-23T18:37:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efca0ff9640 1 -- 172.21.15.19:0/2923408292 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca8052030 con 0x55d2c9823f40 2024-04-23T18:37:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efca0ff9640 1 -- 172.21.15.19:0/2923408292 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efcb4018050 con 0x55d2c9823f40 2024-04-23T18:37:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efcc2596740 10 client.26288 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:37:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efcc2596740 10 client.26288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efcc2596740 20 client.26288 populate_metadata read hostname 'smithi019' 2024-04-23T18:37:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efcc2596740 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d2c983e870 0x55d2c98a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efcc2596740 4 client.26288 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:37:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efcc2596740 1 -- 172.21.15.19:0/2923408292 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2c9790200 con 0x55d2c983e870 2024-04-23T18:37:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.977+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d2c983e870 0x55d2c98a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.978+0000 7efcbcd3d640 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d2c983e870 0x55d2c98a7090 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3adb872f59cbe9 server_cookie=cfb526f4bd4fc5b8 in_seq=0 out_seq=0 2024-04-23T18:37:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.978+0000 7efca0ff9640 10 client.26288 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:37:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efca0ff9640 1 -- 172.21.15.19:0/2923408292 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2c9790200 con 0x55d2c983e870 2024-04-23T18:37:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efca0ff9640 10 client.26288 handle_command_reply: tid=0 2024-04-23T18:37:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 client.26288 shutdown 2024-04-23T18:37:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 -- 172.21.15.19:0/2923408292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 msgr2=0x55d2c9833df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 0x55d2c9833df0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e6c0 tx=0x7efcb40052f0 comp rx=0 tx=0).stop 2024-04-23T18:37:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 -- 172.21.15.19:0/2923408292 shutdown_connections 2024-04-23T18:37:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d2c983e870 0x55d2c98a7090 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2c9823f40 0x55d2c9833df0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efca0ff9640 0 client.26288 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:37:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2c982e800 0x55d2c9834330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 --2- 172.21.15.19:0/2923408292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2c9823620 0x55d2c9836a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 -- 172.21.15.19:0/2923408292 >> 172.21.15.19:0/2923408292 conn(0x55d2c981fb30 msgr2=0x55d2c981ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 -- 172.21.15.19:0/2923408292 shutdown_connections 2024-04-23T18:37:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 1 -- 172.21.15.19:0/2923408292 wait complete. 2024-04-23T18:37:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.992+0000 7efcc2596740 20 client.26288 trim_cache size 0 max 16384 2024-04-23T18:37:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:37:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:37:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:37:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:37:15.887 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 31.5 2024-04-23T18:37:16.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:37:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 -- 172.21.15.19:0/1040879990 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efca40a3dc0 msgr2=0x7efca40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efca40a3dc0 0x7efca40a6280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efcb00ce880 tx=0x7efcb006e510 comp rx=0 tx=0).stop 2024-04-23T18:37:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 -- 172.21.15.19:0/1040879990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb819cfa0 msgr2=0x7efcb819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb819cfa0 0x7efcb819d450 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7efcb000cab0 tx=0x7efcb000cf70 comp rx=0 tx=0).stop 2024-04-23T18:37:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 -- 172.21.15.19:0/1040879990 shutdown_connections 2024-04-23T18:37:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efca40a3dc0 0x7efca40a6280 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb819cfa0 0x7efcb819d450 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcb81072a0 0x7efcb8198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 --2- 172.21.15.19:0/1040879990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb81021d0 0x7efcb81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 -- 172.21.15.19:0/1040879990 >> 172.21.15.19:0/1040879990 conn(0x7efcb80793e0 msgr2=0x7efcb80ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 -- 172.21.15.19:0/1040879990 shutdown_connections 2024-04-23T18:37:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:41.993+0000 7efcbf7a2640 1 -- 172.21.15.19:0/1040879990 wait complete. 2024-04-23T18:37:17.267 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:41 smithi019 ceph-mon[29215]: pgmap v327: 97 pgs: 97 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:37:17.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:43 smithi019 ceph-mon[29215]: pgmap v328: 97 pgs: 97 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-04-23T18:37:17.521 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:45 smithi019 ceph-mon[29215]: pgmap v329: 97 pgs: 97 active+clean; 2.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:37:17.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:47 smithi019 ceph-mon[29215]: pgmap v330: 97 pgs: 97 active+clean; 2.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T18:37:17.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:49 smithi019 ceph-mon[29215]: pgmap v331: 97 pgs: 97 active+clean; 2.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T18:37:17.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:51 smithi019 ceph-mon[29215]: pgmap v332: 97 pgs: 97 active+clean; 2.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:37:17.935 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:53 smithi019 ceph-mon[29215]: pgmap v333: 97 pgs: 97 active+clean; 2.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:37:18.021 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:57 smithi155 ceph-mon[33468]: pgmap v335: 97 pgs: 97 active+clean; 3.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:37:18.104 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:36:58 smithi155 ceph-mon[33468]: pgmap v336: 97 pgs: 97 active+clean; 3.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:37:18.253 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:01 smithi155 ceph-mon[33468]: pgmap v337: 97 pgs: 97 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:37:18.337 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:03 smithi155 ceph-mon[33468]: pgmap v338: 97 pgs: 97 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:37:18.420 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:05 smithi155 ceph-mon[33468]: pgmap v339: 97 pgs: 97 active+clean; 3.5 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-23T18:37:18.503 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:06 smithi155 ceph-mon[33468]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:37:18.587 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:07 smithi155 ceph-mon[33468]: pgmap v341: 97 pgs: 97 active+clean; 3.5 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-23T18:37:18.670 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:09 smithi155 ceph-mon[33468]: pgmap v342: 97 pgs: 97 active+clean; 3.5 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-23T18:37:18.753 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:11 smithi155 ceph-mon[33468]: pgmap v343: 97 pgs: 97 active+clean; 3.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-23T18:37:18.837 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:13 smithi155 ceph-mon[33468]: pgmap v344: 97 pgs: 97 active+clean; 3.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-23T18:37:18.920 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:13 smithi155 ceph-mon[33468]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:37:19.003 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:15 smithi155 ceph-mon[33468]: pgmap v346: 97 pgs: 97 active+clean; 3.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 20 op/s 2024-04-23T18:37:19.087 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:57 smithi044 ceph-mon[32458]: pgmap v335: 97 pgs: 97 active+clean; 3.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:37:19.212 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:36:58 smithi044 ceph-mon[32458]: pgmap v336: 97 pgs: 97 active+clean; 3.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:37:19.295 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:01 smithi044 ceph-mon[32458]: pgmap v337: 97 pgs: 97 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:37:19.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:03 smithi044 ceph-mon[32458]: pgmap v338: 97 pgs: 97 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:37:19.462 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:05 smithi044 ceph-mon[32458]: pgmap v339: 97 pgs: 97 active+clean; 3.5 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-23T18:37:19.554 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:06 smithi044 ceph-mon[32458]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:37:19.637 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:07 smithi044 ceph-mon[32458]: pgmap v341: 97 pgs: 97 active+clean; 3.5 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-23T18:37:19.729 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:09 smithi044 ceph-mon[32458]: pgmap v342: 97 pgs: 97 active+clean; 3.5 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-23T18:37:19.787 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:11 smithi044 ceph-mon[32458]: pgmap v343: 97 pgs: 97 active+clean; 3.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-23T18:37:19.887 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:13 smithi044 ceph-mon[32458]: pgmap v344: 97 pgs: 97 active+clean; 3.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-23T18:37:19.979 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:13 smithi044 ceph-mon[32458]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:37:20.037 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:15 smithi044 ceph-mon[32458]: pgmap v346: 97 pgs: 97 active+clean; 3.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 20 op/s 2024-04-23T18:37:20.182 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:17 smithi044 ceph-mon[32458]: pgmap v347: 97 pgs: 97 active+clean; 3.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:37:20.268 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:19 smithi044 ceph-mon[32458]: pgmap v348: 97 pgs: 97 active+clean; 3.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:37:20.329 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:17 smithi155 ceph-mon[33468]: pgmap v347: 97 pgs: 97 active+clean; 3.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:37:20.412 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:19 smithi155 ceph-mon[33468]: pgmap v348: 97 pgs: 97 active+clean; 3.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:37:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.092+0000 7f6151072640 1 -- 172.21.15.19:0/3657061750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f614c102b80 msgr2=0x7f614c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.092+0000 7f6151072640 1 --2- 172.21.15.19:0/3657061750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f614c102b80 0x7f614c106bc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6140009900 tx=0x7f614002a410 comp rx=0 tx=0).stop 2024-04-23T18:37:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.092+0000 7f6151072640 1 -- 172.21.15.19:0/3657061750 shutdown_connections 2024-04-23T18:37:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.092+0000 7f6151072640 1 --2- 172.21.15.19:0/3657061750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c1072a0 0x7f614c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.092+0000 7f6151072640 1 --2- 172.21.15.19:0/3657061750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f614c102b80 0x7f614c106bc0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.092+0000 7f6151072640 1 --2- 172.21.15.19:0/3657061750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f614c1021d0 0x7f614c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.092+0000 7f6151072640 1 -- 172.21.15.19:0/3657061750 >> 172.21.15.19:0/3657061750 conn(0x7f614c0fda10 msgr2=0x7f614c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.092+0000 7f6151072640 1 -- 172.21.15.19:0/3657061750 shutdown_connections 2024-04-23T18:37:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.092+0000 7f6151072640 1 -- 172.21.15.19:0/3657061750 wait complete. 2024-04-23T18:37:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.093+0000 7f6151072640 1 Processor -- start 2024-04-23T18:37:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.093+0000 7f6151072640 1 -- start start 2024-04-23T18:37:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.093+0000 7f6151072640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c1021d0 0x7f614c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.093+0000 7f6151072640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f614c1072a0 0x7f614c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.093+0000 7f6151072640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f614c19cf50 0x7f614c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.093+0000 7f6151072640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f614c1121c0 con 0x7f614c1072a0 2024-04-23T18:37:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.093+0000 7f6151072640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f614c112040 con 0x7f614c1021d0 2024-04-23T18:37:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.093+0000 7f6151072640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f614c112340 con 0x7f614c19cf50 2024-04-23T18:37:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f614c1072a0 0x7f614c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c1021d0 0x7f614c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f614c1072a0 0x7f614c198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41364/0 (socket says 172.21.15.19:41364) 2024-04-23T18:37:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614a59c640 1 -- 172.21.15.19:0/2038362497 learned_addr learned my addr 172.21.15.19:0/2038362497 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614b59e640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f614c19cf50 0x7f614c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614a59c640 1 -- 172.21.15.19:0/2038362497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f614c19cf50 msgr2=0x7f614c19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614a59c640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f614c19cf50 0x7f614c19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614a59c640 1 -- 172.21.15.19:0/2038362497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c1021d0 msgr2=0x7f614c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614a59c640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c1021d0 0x7f614c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614a59c640 1 -- 172.21.15.19:0/2038362497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c19da40 con 0x7f614c1072a0 2024-04-23T18:37:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f614a59c640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f614c1072a0 0x7f614c198c20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6140032fd0 tx=0x7f6140004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f6133fff640 1 -- 172.21.15.19:0/2038362497 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614003f030 con 0x7f614c1072a0 2024-04-23T18:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f6151072640 1 -- 172.21.15.19:0/2038362497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c19dcd0 con 0x7f614c1072a0 2024-04-23T18:37:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f6133fff640 1 -- 172.21.15.19:0/2038362497 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6140004490 con 0x7f614c1072a0 2024-04-23T18:37:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.094+0000 7f6133fff640 1 -- 172.21.15.19:0/2038362497 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140035070 con 0x7f614c1072a0 2024-04-23T18:37:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.095+0000 7f6151072640 1 -- 172.21.15.19:0/2038362497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c1a54f0 con 0x7f614c1072a0 2024-04-23T18:37:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.097+0000 7f6133fff640 1 -- 172.21.15.19:0/2038362497 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6140042030 con 0x7f614c1072a0 2024-04-23T18:37:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f6153d85740 1 Processor -- start 2024-04-23T18:37:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f6153d85740 1 -- start start 2024-04-23T18:37:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f6153d85740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599a620 0x55564599aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f6153d85740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f6153d85740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f6153d85740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5556458ff970 con 0x55564599af40 2024-04-23T18:37:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f6153d85740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55564589c4a0 con 0x55564599a620 2024-04-23T18:37:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f6153d85740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5556458fb930 con 0x5556459a5800 2024-04-23T18:37:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f614ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f614ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41372/0 (socket says 172.21.15.19:41372) 2024-04-23T18:37:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f614ad9d640 1 -- 172.21.15.19:0/2058427001 learned_addr learned my addr 172.21.15.19:0/2058427001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f6133fff640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f61340a3dc0 0x7f61340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f614b59e640 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599a620 0x55564599aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f614a59c640 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f614b59e640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f61340a3dc0 0x7f61340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.098+0000 7f614ad9d640 1 -- 172.21.15.19:0/2058427001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 msgr2=0x5556459a7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f6133fff640 1 -- 172.21.15.19:0/2038362497 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f61400f0c60 con 0x7f614c1072a0 2024-04-23T18:37:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f614ad9d640 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459a7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f614ad9d640 1 -- 172.21.15.19:0/2058427001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599a620 msgr2=0x55564599aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f614ad9d640 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599a620 0x55564599aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f614ad9d640 1 -- 172.21.15.19:0/2058427001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555645612210 con 0x55564599af40 2024-04-23T18:37:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f614b59e640 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599a620 0x55564599aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f614a59c640 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:37:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f614ad9d640 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459a52c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f613c00bcd0 tx=0x7f613c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf847b2e27573895 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f614b59e640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f61340a3dc0 0x7f61340a6280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6144005fd0 tx=0x7f6144009600 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f61317fa640 1 -- 172.21.15.19:0/2058427001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c005220 con 0x55564599af40 2024-04-23T18:37:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f61317fa640 1 -- 172.21.15.19:0/2058427001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f613c004c50 con 0x55564599af40 2024-04-23T18:37:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.099+0000 7f61317fa640 1 -- 172.21.15.19:0/2058427001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c011660 con 0x55564599af40 2024-04-23T18:37:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 -- 172.21.15.19:0/2058427001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 msgr2=0x5556459a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459a52c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f613c00bcd0 tx=0x7f613c0107a0 comp rx=0 tx=0).stop 2024-04-23T18:37:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 -- 172.21.15.19:0/2058427001 shutdown_connections 2024-04-23T18:37:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459a52c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 --2- 172.21.15.19:0/2058427001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599a620 0x55564599aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 -- 172.21.15.19:0/2058427001 >> 172.21.15.19:0/2058427001 conn(0x555645996b30 msgr2=0x555645996f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 -- 172.21.15.19:0/2058427001 shutdown_connections 2024-04-23T18:37:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 -- 172.21.15.19:0/2058427001 wait complete. 2024-04-23T18:37:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 -- start start 2024-04-23T18:37:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 Processor -- start 2024-04-23T18:37:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.100+0000 7f6153d85740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f6153d85740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599af40 0x5556459a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f6153d85740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f6153d85740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5556459a8430 con 0x55564599a620 2024-04-23T18:37:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f6153d85740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55564589a0b0 con 0x55564599af40 2024-04-23T18:37:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f6153d85740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5556459a85b0 con 0x5556459a5800 2024-04-23T18:37:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f6153d85740 10 client.0 _ll_register_callbacks cb 0x555645887d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41380/0 (socket says 172.21.15.19:41380) 2024-04-23T18:37:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614ad9d640 1 -- 172.21.15.19:0/2182538587 learned_addr learned my addr 172.21.15.19:0/2182538587 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614a59c640 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599af40 0x5556459a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614b59e640 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614ad9d640 1 -- 172.21.15.19:0/2182538587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 msgr2=0x5556459a29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614ad9d640 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459a29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614ad9d640 1 -- 172.21.15.19:0/2182538587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599af40 msgr2=0x5556459a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614ad9d640 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599af40 0x5556459a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614ad9d640 1 -- 172.21.15.19:0/2182538587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556456de620 con 0x55564599a620 2024-04-23T18:37:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614b59e640 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:37:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f614ad9d640 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a5160 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f613c005690 tx=0x7f613c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83877cf1da1c04c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f612eff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:37:30.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f612eff5640 1 -- 172.21.15.19:0/2182538587 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c005010 con 0x55564599a620 2024-04-23T18:37:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f612eff5640 1 -- 172.21.15.19:0/2182538587 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f613c0051b0 con 0x55564599a620 2024-04-23T18:37:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.101+0000 7f6153d85740 1 -- 172.21.15.19:0/2182538587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f613c004c50 con 0x55564599a620 2024-04-23T18:37:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f612eff5640 1 -- 172.21.15.19:0/2182538587 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f613c0053b0 con 0x55564599a620 2024-04-23T18:37:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f6153d85740 10 client.17031 fetch_fsmap learned FSMap version 27 2024-04-23T18:37:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f6153d85740 1 -- 172.21.15.19:0/2182538587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5556459b42d0 con 0x55564599a620 2024-04-23T18:37:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f612eff5640 1 -- 172.21.15.19:0/2182538587 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f613c004c50 con 0x55564599a620 2024-04-23T18:37:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f612eff5640 1 -- 172.21.15.19:0/2182538587 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f613c003ca0 con 0x55564599a620 2024-04-23T18:37:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f6153d85740 10 client.17031 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:37:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f6153d85740 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f6153d85740 20 client.17031 populate_metadata read hostname 'smithi019' 2024-04-23T18:37:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f6153d85740 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5556459b5680 0x5556459b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f6153d85740 4 client.17031 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:37:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.102+0000 7f6153d85740 1 -- 172.21.15.19:0/2182538587 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55564590d390 con 0x5556459b5680 2024-04-23T18:37:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.103+0000 7f614a59c640 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5556459b5680 0x5556459b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.103+0000 7f614a59c640 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5556459b5680 0x5556459b7a70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dafc5479ec2e4a61 server_cookie=900a162e4c46943c in_seq=0 out_seq=0 2024-04-23T18:37:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.103+0000 7f612eff5640 10 client.17031 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:37:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f612eff5640 1 -- 172.21.15.19:0/2182538587 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55564590d390 con 0x5556459b5680 2024-04-23T18:37:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f612eff5640 10 client.17031 handle_command_reply: tid=0 2024-04-23T18:37:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 client.17031 shutdown 2024-04-23T18:37:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 -- 172.21.15.19:0/2182538587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 msgr2=0x5556459a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a5160 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f613c005690 tx=0x7f613c0048f0 comp rx=0 tx=0).stop 2024-04-23T18:37:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 -- 172.21.15.19:0/2182538587 shutdown_connections 2024-04-23T18:37:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5556459b5680 0x5556459b7a70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599af40 0x5556459a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 --2- 172.21.15.19:0/2182538587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a5160 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f612eff5640 0 client.17031 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:37:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 -- 172.21.15.19:0/2182538587 >> 172.21.15.19:0/2182538587 conn(0x555645996b30 msgr2=0x55564599c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 -- 172.21.15.19:0/2182538587 shutdown_connections 2024-04-23T18:37:32.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 1 -- 172.21.15.19:0/2182538587 wait complete. 2024-04-23T18:37:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.104+0000 7f6153d85740 20 client.17031 trim_cache size 0 max 16384 2024-04-23T18:37:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.119+0000 7f6153d85740 1 Processor -- start 2024-04-23T18:37:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.119+0000 7f6153d85740 1 -- start start 2024-04-23T18:37:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f6153d85740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f6153d85740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599af40 0x5556459bd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f6153d85740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459bd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f6153d85740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f614c1adb30 con 0x55564599a620 2024-04-23T18:37:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f6153d85740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f614c1af670 con 0x55564599af40 2024-04-23T18:37:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f6153d85740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f614c1ae200 con 0x5556459a5800 2024-04-23T18:37:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f614ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f614ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41394/0 (socket says 172.21.15.19:41394) 2024-04-23T18:37:33.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f614ad9d640 1 -- 172.21.15.19:0/3698003165 learned_addr learned my addr 172.21.15.19:0/3698003165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f614ad9d640 1 -- 172.21.15.19:0/3698003165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 msgr2=0x5556459bd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f614ad9d640 1 --2- 172.21.15.19:0/3698003165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f614ad9d640 1 -- 172.21.15.19:0/3698003165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599af40 msgr2=0x5556459bd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f614ad9d640 1 --2- 172.21.15.19:0/3698003165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599af40 0x5556459bd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f614ad9d640 1 -- 172.21.15.19:0/3698003165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556459a2d20 con 0x55564599a620 2024-04-23T18:37:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f614ad9d640 1 --2- 172.21.15.19:0/3698003165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a4c90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f613c00b470 tx=0x7f613c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ccefe04a0109bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f6130ff9640 1 -- 172.21.15.19:0/3698003165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c004960 con 0x55564599a620 2024-04-23T18:37:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f6130ff9640 1 -- 172.21.15.19:0/3698003165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f613c004b00 con 0x55564599a620 2024-04-23T18:37:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.120+0000 7f6130ff9640 1 -- 172.21.15.19:0/3698003165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c0266d0 con 0x55564599a620 2024-04-23T18:37:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 -- 172.21.15.19:0/3698003165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 msgr2=0x5556459a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 --2- 172.21.15.19:0/3698003165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a4c90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f613c00b470 tx=0x7f613c0107a0 comp rx=0 tx=0).stop 2024-04-23T18:37:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 -- 172.21.15.19:0/3698003165 shutdown_connections 2024-04-23T18:37:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 --2- 172.21.15.19:0/3698003165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55564599af40 0x5556459bd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 --2- 172.21.15.19:0/3698003165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556459a5800 0x5556459bd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 --2- 172.21.15.19:0/3698003165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599a620 0x5556459a4c90 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 -- 172.21.15.19:0/3698003165 >> 172.21.15.19:0/3698003165 conn(0x555645996b30 msgr2=0x5556459a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 -- 172.21.15.19:0/3698003165 shutdown_connections 2024-04-23T18:37:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 -- 172.21.15.19:0/3698003165 wait complete. 2024-04-23T18:37:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 -- start start 2024-04-23T18:37:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.121+0000 7f6153d85740 1 Processor -- start 2024-04-23T18:37:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f6153d85740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55564599a620 0x5556459ad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f6153d85740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f6153d85740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556459a5800 0x5556459ab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f6153d85740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f614c1aefa0 con 0x55564599af40 2024-04-23T18:37:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f6153d85740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f614c1ae8d0 con 0x5556459a5800 2024-04-23T18:37:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f6153d85740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f614c1abb30 con 0x55564599a620 2024-04-23T18:37:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f6153d85740 10 client.0 _ll_register_callbacks cb 0x7f613c00b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459aadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41400/0 (socket says 172.21.15.19:41400) 2024-04-23T18:37:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614a59c640 1 -- 172.21.15.19:0/3350517893 learned_addr learned my addr 172.21.15.19:0/3350517893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614b59e640 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556459a5800 0x5556459ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614a59c640 1 -- 172.21.15.19:0/3350517893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55564599a620 msgr2=0x5556459ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614ad9d640 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55564599a620 0x5556459ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614a59c640 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55564599a620 0x5556459ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614a59c640 1 -- 172.21.15.19:0/3350517893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556459a5800 msgr2=0x5556459ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614a59c640 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556459a5800 0x5556459ab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614a59c640 1 -- 172.21.15.19:0/3350517893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556459ab850 con 0x55564599af40 2024-04-23T18:37:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614ad9d640 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55564599a620 0x5556459ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:37:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f614a59c640 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459aadd0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f61400b9620 tx=0x7f6140047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40315cdbd0fcf73 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f612eff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:37:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f612eff5640 1 -- 172.21.15.19:0/3350517893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140049040 con 0x55564599af40 2024-04-23T18:37:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f612eff5640 1 -- 172.21.15.19:0/3350517893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f61400dd0d0 con 0x55564599af40 2024-04-23T18:37:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.122+0000 7f6153d85740 1 -- 172.21.15.19:0/3350517893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f613c004b00 con 0x55564599af40 2024-04-23T18:37:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f612eff5640 1 -- 172.21.15.19:0/3350517893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61400c5810 con 0x55564599af40 2024-04-23T18:37:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f6153d85740 10 client.17037 fetch_fsmap learned FSMap version 27 2024-04-23T18:37:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f6153d85740 1 -- 172.21.15.19:0/3350517893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555645a1b620 con 0x55564599af40 2024-04-23T18:37:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f612eff5640 1 -- 172.21.15.19:0/3350517893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f613c004b00 con 0x55564599af40 2024-04-23T18:37:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f612eff5640 1 -- 172.21.15.19:0/3350517893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f614003bc20 con 0x55564599af40 2024-04-23T18:37:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f6153d85740 10 client.17037 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:37:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f6153d85740 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f6153d85740 20 client.17037 populate_metadata read hostname 'smithi019' 2024-04-23T18:37:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f6153d85740 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555645a1bd50 0x555645a1e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f6153d85740 4 client.17037 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:37:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f6153d85740 1 -- 172.21.15.19:0/3350517893 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555645907200 con 0x555645a1bd50 2024-04-23T18:37:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.123+0000 7f614ad9d640 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555645a1bd50 0x555645a1e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.124+0000 7f612eff5640 10 client.17037 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:37:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.124+0000 7f614ad9d640 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555645a1bd50 0x555645a1e170 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d04e9ad56ff76860 server_cookie=aa0aa9f09c0deca1 in_seq=0 out_seq=0 2024-04-23T18:37:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f612eff5640 1 -- 172.21.15.19:0/3350517893 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555645907200 con 0x555645a1bd50 2024-04-23T18:37:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f612eff5640 10 client.17037 handle_command_reply: tid=0 2024-04-23T18:37:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 client.17037 shutdown 2024-04-23T18:37:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 -- 172.21.15.19:0/3350517893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 msgr2=0x5556459aadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459aadd0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f61400b9620 tx=0x7f6140047000 comp rx=0 tx=0).stop 2024-04-23T18:37:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 -- 172.21.15.19:0/3350517893 shutdown_connections 2024-04-23T18:37:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x555645a1bd50 0x555645a1e170 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556459a5800 0x5556459ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55564599af40 0x5556459aadd0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f612eff5640 0 client.17037 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:37:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 --2- 172.21.15.19:0/3350517893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55564599a620 0x5556459ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 -- 172.21.15.19:0/3350517893 >> 172.21.15.19:0/3350517893 conn(0x555645996b30 msgr2=0x555645995420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 -- 172.21.15.19:0/3350517893 shutdown_connections 2024-04-23T18:37:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 1 -- 172.21.15.19:0/3350517893 wait complete. 2024-04-23T18:37:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.138+0000 7f6153d85740 20 client.17037 trim_cache size 0 max 16384 2024-04-23T18:37:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:37:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:37:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:37:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 -- 172.21.15.19:0/2038362497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f61340a3dc0 msgr2=0x7f61340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f61340a3dc0 0x7f61340a6280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6144005fd0 tx=0x7f6144009600 comp rx=0 tx=0).stop 2024-04-23T18:37:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 -- 172.21.15.19:0/2038362497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f614c1072a0 msgr2=0x7f614c198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f614c1072a0 0x7f614c198c20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6140032fd0 tx=0x7f6140004270 comp rx=0 tx=0).stop 2024-04-23T18:37:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 -- 172.21.15.19:0/2038362497 shutdown_connections 2024-04-23T18:37:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f61340a3dc0 0x7f61340a6280 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f614c19cf50 0x7f614c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f614c1072a0 0x7f614c198c20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 --2- 172.21.15.19:0/2038362497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c1021d0 0x7f614c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 -- 172.21.15.19:0/2038362497 >> 172.21.15.19:0/2038362497 conn(0x7f614c0fda10 msgr2=0x7f614c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 -- 172.21.15.19:0/2038362497 shutdown_connections 2024-04-23T18:37:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:36:55.139+0000 7f6151072640 1 -- 172.21.15.19:0/2038362497 wait complete. 2024-04-23T18:37:39.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:55 smithi019 ceph-mon[29215]: pgmap v334: 97 pgs: 97 active+clean; 3.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:37:39.211 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:57 smithi019 ceph-mon[29215]: pgmap v335: 97 pgs: 97 active+clean; 3.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:37:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:36:58 smithi019 ceph-mon[29215]: pgmap v336: 97 pgs: 97 active+clean; 3.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:37:39.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:01 smithi019 ceph-mon[29215]: pgmap v337: 97 pgs: 97 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:37:39.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:03 smithi019 ceph-mon[29215]: pgmap v338: 97 pgs: 97 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:37:39.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:05 smithi019 ceph-mon[29215]: pgmap v339: 97 pgs: 97 active+clean; 3.5 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-23T18:37:39.486 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:06 smithi019 ceph-mon[29215]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:37:39.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:07 smithi019 ceph-mon[29215]: pgmap v341: 97 pgs: 97 active+clean; 3.5 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-23T18:37:39.696 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:21 smithi155 ceph-mon[33468]: pgmap v349: 97 pgs: 97 active+clean; 4.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:37:39.770 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:23 smithi155 ceph-mon[33468]: pgmap v350: 97 pgs: 97 active+clean; 4.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:37:39.861 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:37:39.928 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:39.995 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:40.063 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:40.128 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:40.245 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:40.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:24 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:40.461 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:25 smithi155 ceph-mon[33468]: pgmap v351: 97 pgs: 97 active+clean; 4.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-04-23T18:37:40.512 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:25 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:37:40.578 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:25 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:37:40.645 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:25 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:40.703 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:27 smithi155 ceph-mon[33468]: pgmap v352: 97 pgs: 97 active+clean; 4.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:37:40.778 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:29 smithi155 ceph-mon[33468]: pgmap v353: 97 pgs: 97 active+clean; 4.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:37:40.862 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:31 smithi155 ceph-mon[33468]: pgmap v354: 97 pgs: 97 active+clean; 4.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:37:40.937 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:33 smithi155 ceph-mon[33468]: pgmap v355: 97 pgs: 97 active+clean; 4.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:37:41.004 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:35 smithi155 ceph-mon[33468]: pgmap v356: 97 pgs: 97 active+clean; 4.7 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:37:41.054 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:37 smithi155 ceph-mon[33468]: pgmap v357: 97 pgs: 97 active+clean; 4.7 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:37:41.120 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:21 smithi044 ceph-mon[32458]: pgmap v349: 97 pgs: 97 active+clean; 4.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:37:41.187 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:23 smithi044 ceph-mon[32458]: pgmap v350: 97 pgs: 97 active+clean; 4.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:37:41.320 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:37:41.395 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:41.454 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:41.537 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:41.600 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:41.662 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:41.737 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:24 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:41.804 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:25 smithi044 ceph-mon[32458]: pgmap v351: 97 pgs: 97 active+clean; 4.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-04-23T18:37:41.854 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:25 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:37:41.954 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:25 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:37:42.038 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:25 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:37:42.179 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:27 smithi044 ceph-mon[32458]: pgmap v352: 97 pgs: 97 active+clean; 4.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:37:42.254 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:29 smithi044 ceph-mon[32458]: pgmap v353: 97 pgs: 97 active+clean; 4.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:37:42.329 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:31 smithi044 ceph-mon[32458]: pgmap v354: 97 pgs: 97 active+clean; 4.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:37:42.396 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:33 smithi044 ceph-mon[32458]: pgmap v355: 97 pgs: 97 active+clean; 4.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:37:42.485 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:35 smithi044 ceph-mon[32458]: pgmap v356: 97 pgs: 97 active+clean; 4.7 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:37:42.521 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:37 smithi044 ceph-mon[32458]: pgmap v357: 97 pgs: 97 active+clean; 4.7 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:37:42.573 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:39 smithi044 ceph-mon[32458]: pgmap v358: 97 pgs: 97 active+clean; 4.7 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:37:42.674 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:41 smithi044 ceph-mon[32458]: pgmap v359: 97 pgs: 97 active+clean; 5.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T18:37:42.796 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:39 smithi155 ceph-mon[33468]: pgmap v358: 97 pgs: 97 active+clean; 4.7 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:37:42.879 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:41 smithi155 ceph-mon[33468]: pgmap v359: 97 pgs: 97 active+clean; 5.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T18:37:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3386288578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f34107280 msgr2=0x7f5f3410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 --2- 172.21.15.19:0/3386288578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f34107280 0x7f5f3410ee50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009ca0 tx=0x7f5f2c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:37:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3386288578 shutdown_connections 2024-04-23T18:37:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 --2- 172.21.15.19:0/3386288578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f34101070 0x7f5f34106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 --2- 172.21.15.19:0/3386288578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f34107280 0x7f5f3410ee50 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 --2- 172.21.15.19:0/3386288578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f341006c0 0x7f5f34100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3386288578 >> 172.21.15.19:0/3386288578 conn(0x7f5f34077390 msgr2=0x7f5f340fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3386288578 shutdown_connections 2024-04-23T18:37:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3386288578 wait complete. 2024-04-23T18:37:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 Processor -- start 2024-04-23T18:37:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.240+0000 7f5f3b3f9640 1 -- start start 2024-04-23T18:37:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f3b3f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f341006c0 0x7f5f34101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f3b3f9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f34107280 0x7f5f34101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f3b3f9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341020b0 0x7f5f341a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f3b3f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341121f0 con 0x7f5f341006c0 2024-04-23T18:37:44.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f3b3f9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34112070 con 0x7f5f341020b0 2024-04-23T18:37:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f3b3f9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34112370 con 0x7f5f34107280 2024-04-23T18:37:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f39195640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f341006c0 0x7f5f34101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f39195640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f341006c0 0x7f5f34101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42566/0 (socket says 172.21.15.19:42566) 2024-04-23T18:37:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f39195640 1 -- 172.21.15.19:0/3631332730 learned_addr learned my addr 172.21.15.19:0/3631332730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f39996640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341020b0 0x7f5f341a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f38994640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f34107280 0x7f5f34101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f39195640 1 -- 172.21.15.19:0/3631332730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f34107280 msgr2=0x7f5f34101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f39195640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f34107280 0x7f5f34101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f39195640 1 -- 172.21.15.19:0/3631332730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341020b0 msgr2=0x7f5f341a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f39195640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341020b0 0x7f5f341a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f39195640 1 -- 172.21.15.19:0/3631332730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341a6f80 con 0x7f5f341006c0 2024-04-23T18:37:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.241+0000 7f5f39195640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f341006c0 0x7f5f34101630 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000c970 tx=0x7f5f3000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.242+0000 7f5f2a7fc640 1 -- 172.21.15.19:0/3631332730 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30007bb0 con 0x7f5f341006c0 2024-04-23T18:37:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.242+0000 7f5f2a7fc640 1 -- 172.21.15.19:0/3631332730 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f30007d50 con 0x7f5f341006c0 2024-04-23T18:37:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.242+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3631332730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f341a7270 con 0x7f5f341006c0 2024-04-23T18:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.242+0000 7f5f2a7fc640 1 -- 172.21.15.19:0/3631332730 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30004de0 con 0x7f5f341006c0 2024-04-23T18:37:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.242+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3631332730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f341a77c0 con 0x7f5f341006c0 2024-04-23T18:37:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.244+0000 7f5f2a7fc640 1 -- 172.21.15.19:0/3631332730 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f30004fe0 con 0x7f5f341006c0 2024-04-23T18:37:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f3e1e0740 1 Processor -- start 2024-04-23T18:37:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f3e1e0740 1 -- start start 2024-04-23T18:37:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 0x55d5fc9adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f2a7fc640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5f200a3e70 0x7f5f200a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d5fc905970 con 0x55d5fc9a0620 2024-04-23T18:37:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d5fc8a24a0 con 0x55d5fc9ab800 2024-04-23T18:37:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5fc901930 con 0x55d5fc9a0f40 2024-04-23T18:37:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f39996640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f38994640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5f200a3e70 0x7f5f200a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f39996640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42570/0 (socket says 172.21.15.19:42570) 2024-04-23T18:37:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f39996640 1 -- 172.21.15.19:0/1508199884 learned_addr learned my addr 172.21.15.19:0/1508199884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f2a7fc640 1 -- 172.21.15.19:0/3631332730 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8557+0+0 (secure 0 0 0) 0x7f5f300d03c0 con 0x7f5f341006c0 2024-04-23T18:37:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f39996640 1 --2- 172.21.15.19:0/1508199884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 0x55d5fc9adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f39195640 1 --2- 172.21.15.19:0/1508199884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f39996640 1 -- 172.21.15.19:0/1508199884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0f40 msgr2=0x55d5fc9ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f39996640 1 --2- 172.21.15.19:0/1508199884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f39996640 1 -- 172.21.15.19:0/1508199884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 msgr2=0x55d5fc9adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f39996640 1 --2- 172.21.15.19:0/1508199884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 0x55d5fc9adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.245+0000 7f5f39996640 1 -- 172.21.15.19:0/1508199884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5fc618210 con 0x55d5fc9a0620 2024-04-23T18:37:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.246+0000 7f5f39996640 1 --2- 172.21.15.19:0/1508199884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9a0a00 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c005fe0 tx=0x7f5f2c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30d87896e47f339 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.246+0000 7f5f38994640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5f200a3e70 0x7f5f200a6330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f341030c0 tx=0x7f5f2400a2d0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.246+0000 7f5f1f7fe640 1 -- 172.21.15.19:0/1508199884 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c049070 con 0x55d5fc9a0620 2024-04-23T18:37:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.246+0000 7f5f1f7fe640 1 -- 172.21.15.19:0/1508199884 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f2c036040 con 0x55d5fc9a0620 2024-04-23T18:37:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.246+0000 7f5f1f7fe640 1 -- 172.21.15.19:0/1508199884 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c004250 con 0x55d5fc9a0620 2024-04-23T18:37:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.246+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1508199884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 msgr2=0x55d5fc9a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.246+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/1508199884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9a0a00 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c005fe0 tx=0x7f5f2c005ef0 comp rx=0 tx=0).stop 2024-04-23T18:37:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1508199884 shutdown_connections 2024-04-23T18:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/1508199884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 0x55d5fc9adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/1508199884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/1508199884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9a0a00 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1508199884 >> 172.21.15.19:0/1508199884 conn(0x55d5fc99cb30 msgr2=0x55d5fc99cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1508199884 shutdown_connections 2024-04-23T18:37:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1508199884 wait complete. 2024-04-23T18:37:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 -- start start 2024-04-23T18:37:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 Processor -- start 2024-04-23T18:37:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9a8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.247+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9ab800 0x55d5fc9a8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d5fc9ae3d0 con 0x55d5fc9a0620 2024-04-23T18:37:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d5fc8a00b0 con 0x55d5fc9a0f40 2024-04-23T18:37:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5fc9ae550 con 0x55d5fc9ab800 2024-04-23T18:37:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f3e1e0740 10 client.0 _ll_register_callbacks cb 0x55d5fc88dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39195640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39195640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9ab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42582/0 (socket says 172.21.15.19:42582) 2024-04-23T18:37:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39195640 1 -- 172.21.15.19:0/1087281996 learned_addr learned my addr 172.21.15.19:0/1087281996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f38994640 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9a8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39996640 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9ab800 0x55d5fc9a8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39195640 1 -- 172.21.15.19:0/1087281996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9ab800 msgr2=0x55d5fc9a8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39195640 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9ab800 0x55d5fc9a8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39195640 1 -- 172.21.15.19:0/1087281996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9a0f40 msgr2=0x55d5fc9a8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39195640 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9a8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39195640 1 -- 172.21.15.19:0/1087281996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5fc6e4620 con 0x55d5fc9a0620 2024-04-23T18:37:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39996640 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9ab800 0x55d5fc9a8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f39195640 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9ab130 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5f300987a0 tx=0x7f5f3009bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14b50a82a0dfb33 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:37:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f1cff9640 1 -- 172.21.15.19:0/1087281996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300bb060 con 0x55d5fc9a0620 2024-04-23T18:37:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f1cff9640 1 -- 172.21.15.19:0/1087281996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8557+0+0 (secure 0 0 0) 0x7f5f300b65d0 con 0x55d5fc9a0620 2024-04-23T18:37:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1087281996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2c036040 con 0x55d5fc9a0620 2024-04-23T18:37:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.248+0000 7f5f1cff9640 1 -- 172.21.15.19:0/1087281996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f3001d040 con 0x55d5fc9a0620 2024-04-23T18:37:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f3e1e0740 10 client.17049 fetch_fsmap learned FSMap version 27 2024-04-23T18:37:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1087281996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5fc9ba250 con 0x55d5fc9a0620 2024-04-23T18:37:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f1cff9640 1 -- 172.21.15.19:0/1087281996 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f2c036040 con 0x55d5fc9a0620 2024-04-23T18:37:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f1cff9640 1 -- 172.21.15.19:0/1087281996 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5f300a4b20 con 0x55d5fc9a0620 2024-04-23T18:37:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f3e1e0740 10 client.17049 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:37:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f3e1e0740 10 client.17049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f3e1e0740 20 client.17049 populate_metadata read hostname 'smithi019' 2024-04-23T18:37:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d5fc9bb710 0x55d5fc9bdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f3e1e0740 4 client.17049 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:37:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1087281996 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5fc913390 con 0x55d5fc9bb710 2024-04-23T18:37:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.249+0000 7f5f38994640 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d5fc9bb710 0x55d5fc9bdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.250+0000 7f5f1cff9640 10 client.17049 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:37:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.250+0000 7f5f38994640 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d5fc9bb710 0x55d5fc9bdb60 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa6bd0cdef06818c server_cookie=fb7dd68e972f0671 in_seq=0 out_seq=0 2024-04-23T18:37:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.250+0000 7f5f1cff9640 1 -- 172.21.15.19:0/1087281996 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5fc913390 con 0x55d5fc9bb710 2024-04-23T18:37:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.250+0000 7f5f1cff9640 10 client.17049 handle_command_reply: tid=0 2024-04-23T18:37:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.250+0000 7f5f3e1e0740 1 client.17049 shutdown 2024-04-23T18:37:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1087281996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 msgr2=0x55d5fc9ab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9ab130 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5f300987a0 tx=0x7f5f3009bd60 comp rx=0 tx=0).stop 2024-04-23T18:37:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1087281996 shutdown_connections 2024-04-23T18:37:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d5fc9bb710 0x55d5fc9bdb60 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9a8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f1cff9640 0 client.17049 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:37:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9ab800 0x55d5fc9a8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/1087281996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0620 0x55d5fc9ab130 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1087281996 >> 172.21.15.19:0/1087281996 conn(0x55d5fc99cb30 msgr2=0x55d5fc9a28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1087281996 shutdown_connections 2024-04-23T18:37:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/1087281996 wait complete. 2024-04-23T18:37:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.251+0000 7f5f3e1e0740 20 client.17049 trim_cache size 0 max 16384 2024-04-23T18:37:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f3e1e0740 1 Processor -- start 2024-04-23T18:37:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f3e1e0740 1 -- start start 2024-04-23T18:37:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 0x55d5fc9b7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9ab800 0x55d5fc9c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341afc40 con 0x55d5fc9ab800 2024-04-23T18:37:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b1780 con 0x55d5fc9a0f40 2024-04-23T18:37:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b0310 con 0x55d5fc9a0620 2024-04-23T18:37:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f39996640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9ab800 0x55d5fc9c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f39996640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9ab800 0x55d5fc9c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42598/0 (socket says 172.21.15.19:42598) 2024-04-23T18:37:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f39996640 1 -- 172.21.15.19:0/2847486696 learned_addr learned my addr 172.21.15.19:0/2847486696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f39195640 1 --2- 172.21.15.19:0/2847486696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 0x55d5fc9b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f39996640 1 -- 172.21.15.19:0/2847486696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 msgr2=0x55d5fc9b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f39996640 1 --2- 172.21.15.19:0/2847486696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 0x55d5fc9b7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f39996640 1 -- 172.21.15.19:0/2847486696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9a0f40 msgr2=0x55d5fc9aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f39996640 1 --2- 172.21.15.19:0/2847486696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f39996640 1 -- 172.21.15.19:0/2847486696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5fc9a18b0 con 0x55d5fc9ab800 2024-04-23T18:37:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.267+0000 7f5f39195640 1 --2- 172.21.15.19:0/2847486696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 0x55d5fc9b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.268+0000 7f5f39996640 1 --2- 172.21.15.19:0/2847486696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9ab800 0x55d5fc9c3410 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c02ade0 tx=0x7f5f2c009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598e8357d06961e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.268+0000 7f5f1effd640 1 -- 172.21.15.19:0/2847486696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c040050 con 0x55d5fc9ab800 2024-04-23T18:37:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.268+0000 7f5f1effd640 1 -- 172.21.15.19:0/2847486696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f2c036040 con 0x55d5fc9ab800 2024-04-23T18:37:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.268+0000 7f5f1effd640 1 -- 172.21.15.19:0/2847486696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c049040 con 0x55d5fc9ab800 2024-04-23T18:37:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.268+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/2847486696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9ab800 msgr2=0x55d5fc9c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.268+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/2847486696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9ab800 0x55d5fc9c3410 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c02ade0 tx=0x7f5f2c009890 comp rx=0 tx=0).stop 2024-04-23T18:37:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/2847486696 shutdown_connections 2024-04-23T18:37:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/2847486696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/2847486696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9ab800 0x55d5fc9c3410 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/2847486696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 0x55d5fc9b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/2847486696 >> 172.21.15.19:0/2847486696 conn(0x55d5fc99cb30 msgr2=0x55d5fc99bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/2847486696 shutdown_connections 2024-04-23T18:37:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/2847486696 wait complete. 2024-04-23T18:37:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 -- start start 2024-04-23T18:37:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 Processor -- start 2024-04-23T18:37:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 0x55d5fc9b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 0x55d5fc9af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b10b0 con 0x55d5fc9a0f40 2024-04-23T18:37:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b09e0 con 0x55d5fc9ab800 2024-04-23T18:37:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341adc40 con 0x55d5fc9a0620 2024-04-23T18:37:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.269+0000 7f5f3e1e0740 10 client.0 _ll_register_callbacks cb 0x7f5f3009c110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f38994640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f38994640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42614/0 (socket says 172.21.15.19:42614) 2024-04-23T18:37:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f38994640 1 -- 172.21.15.19:0/188792551 learned_addr learned my addr 172.21.15.19:0/188792551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:37:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f39195640 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 0x55d5fc9b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f39996640 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 0x55d5fc9af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f38994640 1 -- 172.21.15.19:0/188792551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 msgr2=0x55d5fc9b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f38994640 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 0x55d5fc9b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f38994640 1 -- 172.21.15.19:0/188792551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 msgr2=0x55d5fc9af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f38994640 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 0x55d5fc9af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f38994640 1 -- 172.21.15.19:0/188792551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5fc9afba0 con 0x55d5fc9a0f40 2024-04-23T18:38:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f39996640 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 0x55d5fc9af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:38:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f38994640 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9aee60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x55d5fc9a7c80 tx=0x7f5f240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdbf4d605e1065c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:38:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f1cff9640 1 -- 172.21.15.19:0/188792551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24016070 con 0x55d5fc9a0f40 2024-04-23T18:38:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f1cff9640 1 -- 172.21.15.19:0/188792551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8557+0+0 (secure 0 0 0) 0x7f5f24028250 con 0x55d5fc9a0f40 2024-04-23T18:38:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/188792551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2c036040 con 0x55d5fc9a0f40 2024-04-23T18:38:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f1cff9640 1 -- 172.21.15.19:0/188792551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f24028450 con 0x55d5fc9a0f40 2024-04-23T18:38:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.270+0000 7f5f3e1e0740 10 client.17055 fetch_fsmap learned FSMap version 27 2024-04-23T18:38:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.271+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/188792551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5fca21cb0 con 0x55d5fc9a0f40 2024-04-23T18:38:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.271+0000 7f5f1cff9640 1 -- 172.21.15.19:0/188792551 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f2c036040 con 0x55d5fc9a0f40 2024-04-23T18:38:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.271+0000 7f5f1cff9640 1 -- 172.21.15.19:0/188792551 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5f24031c10 con 0x55d5fc9a0f40 2024-04-23T18:38:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.271+0000 7f5f3e1e0740 10 client.17055 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:38:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.271+0000 7f5f3e1e0740 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.271+0000 7f5f3e1e0740 20 client.17055 populate_metadata read hostname 'smithi019' 2024-04-23T18:38:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.271+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d5fca22460 0x55d5fca24880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.271+0000 7f5f3e1e0740 4 client.17055 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:38:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.271+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/188792551 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5fc90d200 con 0x55d5fca22460 2024-04-23T18:38:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.271+0000 7f5f39195640 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d5fca22460 0x55d5fca24880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.272+0000 7f5f39195640 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d5fca22460 0x55d5fca24880 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb4e316f083ebd server_cookie=4c056c5340011d91 in_seq=0 out_seq=0 2024-04-23T18:38:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.272+0000 7f5f1cff9640 10 client.17055 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f1cff9640 1 -- 172.21.15.19:0/188792551 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5fc90d200 con 0x55d5fca22460 2024-04-23T18:38:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f1cff9640 10 client.17055 handle_command_reply: tid=0 2024-04-23T18:38:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f3e1e0740 1 client.17055 shutdown 2024-04-23T18:38:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/188792551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0f40 msgr2=0x55d5fc9aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9aee60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x55d5fc9a7c80 tx=0x7f5f240046c0 comp rx=0 tx=0).stop 2024-04-23T18:38:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/188792551 shutdown_connections 2024-04-23T18:38:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55d5fca22460 0x55d5fca24880 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5fc9ab800 0x55d5fc9af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5fc9a0f40 0x55d5fc9aee60 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f1cff9640 0 client.17055 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f3e1e0740 1 --2- 172.21.15.19:0/188792551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5fc9a0620 0x55d5fc9b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.286+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/188792551 >> 172.21.15.19:0/188792551 conn(0x55d5fc99cb30 msgr2=0x55d5fc99b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/188792551 shutdown_connections 2024-04-23T18:38:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3e1e0740 1 -- 172.21.15.19:0/188792551 wait complete. 2024-04-23T18:38:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3e1e0740 20 client.17055 trim_cache size 0 max 16384 2024-04-23T18:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:38:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:38:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:38:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:38:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3631332730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5f200a3e70 msgr2=0x7f5f200a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5f200a3e70 0x7f5f200a6330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f341030c0 tx=0x7f5f2400a2d0 comp rx=0 tx=0).stop 2024-04-23T18:38:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3631332730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f341006c0 msgr2=0x7f5f34101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f341006c0 0x7f5f34101630 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000c970 tx=0x7f5f3000cd70 comp rx=0 tx=0).stop 2024-04-23T18:38:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3631332730 shutdown_connections 2024-04-23T18:38:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f5f200a3e70 0x7f5f200a6330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341020b0 0x7f5f341a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f34107280 0x7f5f34101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 --2- 172.21.15.19:0/3631332730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f341006c0 0x7f5f34101630 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3631332730 >> 172.21.15.19:0/3631332730 conn(0x7f5f34077390 msgr2=0x7f5f34105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3631332730 shutdown_connections 2024-04-23T18:38:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:08.287+0000 7f5f3b3f9640 1 -- 172.21.15.19:0/3631332730 wait complete. 2024-04-23T18:38:05.268 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:09 smithi019 ceph-mon[29215]: pgmap v342: 97 pgs: 97 active+clean; 3.5 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-23T18:38:05.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:11 smithi019 ceph-mon[29215]: pgmap v343: 97 pgs: 97 active+clean; 3.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-23T18:38:05.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:13 smithi019 ceph-mon[29215]: pgmap v344: 97 pgs: 97 active+clean; 3.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-23T18:38:05.458 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:13 smithi019 ceph-mon[29215]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:38:05.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:15 smithi019 ceph-mon[29215]: pgmap v346: 97 pgs: 97 active+clean; 3.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 20 op/s 2024-04-23T18:38:05.600 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:17 smithi019 ceph-mon[29215]: pgmap v347: 97 pgs: 97 active+clean; 3.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:38:05.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:19 smithi019 ceph-mon[29215]: pgmap v348: 97 pgs: 97 active+clean; 3.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:38:05.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:43 smithi155 ceph-mon[33468]: pgmap v360: 97 pgs: 97 active+clean; 5.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:38:05.874 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:45 smithi155 ceph-mon[33468]: pgmap v361: 97 pgs: 97 active+clean; 5.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:05.962 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:47 smithi155 ceph-mon[33468]: pgmap v362: 97 pgs: 97 active+clean; 5.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:38:06.024 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:49 smithi155 ceph-mon[33468]: pgmap v363: 97 pgs: 97 active+clean; 5.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:38:06.124 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:51 smithi155 ceph-mon[33468]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-23T18:38:06.241 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:53 smithi155 ceph-mon[33468]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-23T18:38:06.308 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:55 smithi155 ceph-mon[33468]: pgmap v366: 97 pgs: 97 active+clean; 1.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:06.375 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:57 smithi155 ceph-mon[33468]: pgmap v367: 97 pgs: 97 active+clean; 1.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:38:06.433 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:37:59 smithi155 ceph-mon[33468]: pgmap v368: 97 pgs: 97 active+clean; 1.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:38:06.483 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:01 smithi155 ceph-mon[33468]: pgmap v369: 97 pgs: 97 active+clean; 1.8 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:06.600 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:03 smithi155 ceph-mon[33468]: pgmap v370: 97 pgs: 97 active+clean; 1.8 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:38:06.650 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:43 smithi044 ceph-mon[32458]: pgmap v360: 97 pgs: 97 active+clean; 5.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:38:06.700 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:45 smithi044 ceph-mon[32458]: pgmap v361: 97 pgs: 97 active+clean; 5.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:06.776 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:47 smithi044 ceph-mon[32458]: pgmap v362: 97 pgs: 97 active+clean; 5.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:38:06.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:49 smithi044 ceph-mon[32458]: pgmap v363: 97 pgs: 97 active+clean; 5.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:38:06.883 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:51 smithi044 ceph-mon[32458]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-23T18:38:06.948 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:53 smithi044 ceph-mon[32458]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-23T18:38:06.983 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:55 smithi044 ceph-mon[32458]: pgmap v366: 97 pgs: 97 active+clean; 1.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:07.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:57 smithi044 ceph-mon[32458]: pgmap v367: 97 pgs: 97 active+clean; 1.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:38:07.088 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:37:59 smithi044 ceph-mon[32458]: pgmap v368: 97 pgs: 97 active+clean; 1.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:38:07.134 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:01 smithi044 ceph-mon[32458]: pgmap v369: 97 pgs: 97 active+clean; 1.8 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:07.292 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:03 smithi044 ceph-mon[32458]: pgmap v370: 97 pgs: 97 active+clean; 1.8 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:38:07.345 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:05 smithi044 ceph-mon[32458]: pgmap v371: 97 pgs: 97 active+clean; 2.1 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 106 KiB/s wr, 39 op/s 2024-04-23T18:38:07.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:05 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2796105966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:38:07.459 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:06 smithi044 ceph-mon[32458]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:38:07.501 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:05 smithi155 ceph-mon[33468]: pgmap v371: 97 pgs: 97 active+clean; 2.1 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 106 KiB/s wr, 39 op/s 2024-04-23T18:38:07.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:05 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2796105966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:38:07.606 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:06 smithi155 ceph-mon[33468]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:38:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.386+0000 7fd116744640 1 -- 172.21.15.19:0/3667375246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1101021a0 msgr2=0x7fd110102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.386+0000 7fd116744640 1 --2- 172.21.15.19:0/3667375246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1101021a0 0x7fd110102580 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd100009900 tx=0x7fd10002a410 comp rx=0 tx=0).stop 2024-04-23T18:38:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.386+0000 7fd116744640 1 -- 172.21.15.19:0/3667375246 shutdown_connections 2024-04-23T18:38:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.386+0000 7fd116744640 1 --2- 172.21.15.19:0/3667375246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd110107270 0x7fd11010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.386+0000 7fd116744640 1 --2- 172.21.15.19:0/3667375246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd110102b50 0x7fd110106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.386+0000 7fd116744640 1 --2- 172.21.15.19:0/3667375246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1101021a0 0x7fd110102580 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.386+0000 7fd116744640 1 -- 172.21.15.19:0/3667375246 >> 172.21.15.19:0/3667375246 conn(0x7fd1100793e0 msgr2=0x7fd1100797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.386+0000 7fd116744640 1 -- 172.21.15.19:0/3667375246 shutdown_connections 2024-04-23T18:38:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.386+0000 7fd116744640 1 -- 172.21.15.19:0/3667375246 wait complete. 2024-04-23T18:38:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.387+0000 7fd116744640 1 Processor -- start 2024-04-23T18:38:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.387+0000 7fd116744640 1 -- start start 2024-04-23T18:38:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd116744640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1101021a0 0x7fd110074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd116744640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd110107270 0x7fd110074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd116744640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd110074cd0 0x7fd1101a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd116744640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101121b0 con 0x7fd1101021a0 2024-04-23T18:38:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd116744640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd110112030 con 0x7fd110074cd0 2024-04-23T18:38:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd116744640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd110112330 con 0x7fd110107270 2024-04-23T18:38:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1101021a0 0x7fd110074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1101021a0 0x7fd110074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39860/0 (socket says 172.21.15.19:39860) 2024-04-23T18:38:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10ffff640 1 -- 172.21.15.19:0/3978661430 learned_addr learned my addr 172.21.15.19:0/3978661430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd114ce1640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd110074cd0 0x7fd1101a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd110107270 0x7fd110074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10ffff640 1 -- 172.21.15.19:0/3978661430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd110107270 msgr2=0x7fd110074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10ffff640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd110107270 0x7fd110074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10ffff640 1 -- 172.21.15.19:0/3978661430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd110074cd0 msgr2=0x7fd1101a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10ffff640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd110074cd0 0x7fd1101a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10ffff640 1 -- 172.21.15.19:0/3978661430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1101a6f60 con 0x7fd1101021a0 2024-04-23T18:38:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd110107270 0x7fd110074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:38:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10ffff640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1101021a0 0x7fd110074250 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd100005940 tx=0x7fd1000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10d7fa640 1 -- 172.21.15.19:0/3978661430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10003f030 con 0x7fd1101021a0 2024-04-23T18:38:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.388+0000 7fd10d7fa640 1 -- 172.21.15.19:0/3978661430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd100003cb0 con 0x7fd1101021a0 2024-04-23T18:38:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.389+0000 7fd116744640 1 -- 172.21.15.19:0/3978661430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1101a71f0 con 0x7fd1101021a0 2024-04-23T18:38:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.389+0000 7fd10d7fa640 1 -- 172.21.15.19:0/3978661430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100035070 con 0x7fd1101021a0 2024-04-23T18:38:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.389+0000 7fd116744640 1 -- 172.21.15.19:0/3978661430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1101a76e0 con 0x7fd1101021a0 2024-04-23T18:38:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.391+0000 7fd10d7fa640 1 -- 172.21.15.19:0/3978661430 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd100042030 con 0x7fd1101021a0 2024-04-23T18:38:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.391+0000 7fd11945d740 1 Processor -- start 2024-04-23T18:38:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.391+0000 7fd11945d740 1 -- start start 2024-04-23T18:38:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.391+0000 7fd11945d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe1b620 0x55e74fe1ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.391+0000 7fd11945d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe262c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.391+0000 7fd11945d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 0x55e74fe28bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.391+0000 7fd11945d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e74fd80970 con 0x55e74fe1bf40 2024-04-23T18:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.391+0000 7fd11945d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e74fd1d4a0 con 0x55e74fe1b620 2024-04-23T18:38:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.391+0000 7fd11945d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e74fd7c930 con 0x55e74fe26800 2024-04-23T18:38:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10d7fa640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0f80a3dc0 0x7fd0f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe262c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe262c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39872/0 (socket says 172.21.15.19:39872) 2024-04-23T18:38:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10f7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 0x55e74fe28bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10ffff640 1 -- 172.21.15.19:0/2142953280 learned_addr learned my addr 172.21.15.19:0/2142953280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd114ce1640 1 --2- 172.21.15.19:0/2142953280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe1b620 0x55e74fe1ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd114ce1640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0f80a3dc0 0x7fd0f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:10.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10d7fa640 1 -- 172.21.15.19:0/3978661430 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fd1000f1190 con 0x7fd1101021a0 2024-04-23T18:38:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10ffff640 1 -- 172.21.15.19:0/2142953280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 msgr2=0x55e74fe28bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10ffff640 1 --2- 172.21.15.19:0/2142953280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 0x55e74fe28bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10ffff640 1 -- 172.21.15.19:0/2142953280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe1b620 msgr2=0x55e74fe1ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10ffff640 1 --2- 172.21.15.19:0/2142953280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe1b620 0x55e74fe1ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10ffff640 1 -- 172.21.15.19:0/2142953280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74fa93210 con 0x55e74fe1bf40 2024-04-23T18:38:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd114ce1640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0f80a3dc0 0x7fd0f80a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd108005fd0 tx=0x7fd108009a10 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd10ffff640 1 --2- 172.21.15.19:0/2142953280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe262c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd1000c5810 tx=0x7fd100032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66a74be1485b0d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd0f67fc640 1 -- 172.21.15.19:0/2142953280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100049070 con 0x55e74fe1bf40 2024-04-23T18:38:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd0f67fc640 1 -- 172.21.15.19:0/2142953280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1000bd7d0 con 0x55e74fe1bf40 2024-04-23T18:38:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.392+0000 7fd0f67fc640 1 -- 172.21.15.19:0/2142953280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1000eeb00 con 0x55e74fe1bf40 2024-04-23T18:38:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.393+0000 7fd11945d740 1 -- 172.21.15.19:0/2142953280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 msgr2=0x55e74fe262c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.393+0000 7fd11945d740 1 --2- 172.21.15.19:0/2142953280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe262c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd1000c5810 tx=0x7fd100032f60 comp rx=0 tx=0).stop 2024-04-23T18:38:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.393+0000 7fd11945d740 1 -- 172.21.15.19:0/2142953280 shutdown_connections 2024-04-23T18:38:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.393+0000 7fd11945d740 1 --2- 172.21.15.19:0/2142953280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 0x55e74fe28bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.393+0000 7fd11945d740 1 --2- 172.21.15.19:0/2142953280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe262c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.393+0000 7fd11945d740 1 --2- 172.21.15.19:0/2142953280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe1b620 0x55e74fe1ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.393+0000 7fd11945d740 1 -- 172.21.15.19:0/2142953280 >> 172.21.15.19:0/2142953280 conn(0x55e74fe17b30 msgr2=0x55e74fe17f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.393+0000 7fd11945d740 1 -- 172.21.15.19:0/2142953280 shutdown_connections 2024-04-23T18:38:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.393+0000 7fd11945d740 1 -- 172.21.15.19:0/2142953280 wait complete. 2024-04-23T18:38:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd11945d740 1 -- start start 2024-04-23T18:38:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd11945d740 1 Processor -- start 2024-04-23T18:38:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd11945d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1b620 0x55e74fe261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd11945d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe1bf40 0x55e74fe234a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd11945d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 0x55e74fe23a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd11945d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e74fd80970 con 0x55e74fe1b620 2024-04-23T18:38:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd11945d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e74fd1b0b0 con 0x55e74fe1bf40 2024-04-23T18:38:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd11945d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e74fe291f0 con 0x55e74fe26800 2024-04-23T18:38:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd11945d740 10 client.0 _ll_register_callbacks cb 0x55e74fd08d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd10ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1b620 0x55e74fe261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd10ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1b620 0x55e74fe261b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39888/0 (socket says 172.21.15.19:39888) 2024-04-23T18:38:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd10ffff640 1 -- 172.21.15.19:0/1768264387 learned_addr learned my addr 172.21.15.19:0/1768264387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.394+0000 7fd114ce1640 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 0x55e74fe23a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd10ffff640 1 -- 172.21.15.19:0/1768264387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 msgr2=0x55e74fe23a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe1bf40 0x55e74fe234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd10ffff640 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 0x55e74fe23a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd10ffff640 1 -- 172.21.15.19:0/1768264387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe1bf40 msgr2=0x55e74fe234a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd10ffff640 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe1bf40 0x55e74fe234a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd10ffff640 1 -- 172.21.15.19:0/1768264387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74fb5f620 con 0x55e74fe1b620 2024-04-23T18:38:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd114ce1640 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 0x55e74fe23a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:38:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd10ffff640 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1b620 0x55e74fe261b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fd1000bd060 tx=0x7fd1000b9970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43949e87161633f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd0ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:38:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/1768264387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100049040 con 0x55e74fe1b620 2024-04-23T18:38:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/1768264387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fd1000c1120 con 0x55e74fe1b620 2024-04-23T18:38:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd11945d740 1 -- 172.21.15.19:0/1768264387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1000bd7d0 con 0x55e74fe1b620 2024-04-23T18:38:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/1768264387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1000eed10 con 0x55e74fe1b620 2024-04-23T18:38:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd11945d740 10 client.17067 fetch_fsmap learned FSMap version 27 2024-04-23T18:38:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd11945d740 1 -- 172.21.15.19:0/1768264387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e74fe34fe0 con 0x55e74fe1b620 2024-04-23T18:38:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.395+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/1768264387 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1000bd7d0 con 0x55e74fe1b620 2024-04-23T18:38:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.396+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/1768264387 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd10003bbf0 con 0x55e74fe1b620 2024-04-23T18:38:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.396+0000 7fd11945d740 10 client.17067 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:38:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.396+0000 7fd11945d740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.396+0000 7fd11945d740 20 client.17067 populate_metadata read hostname 'smithi019' 2024-04-23T18:38:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.396+0000 7fd11945d740 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e74fe36380 0x55e74fe38770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.396+0000 7fd11945d740 4 client.17067 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:38:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.396+0000 7fd11945d740 1 -- 172.21.15.19:0/1768264387 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e74fd8e390 con 0x55e74fe36380 2024-04-23T18:38:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.396+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e74fe36380 0x55e74fe38770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.396+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e74fe36380 0x55e74fe38770 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=675fc32f62916c83 server_cookie=c5e71bd1f80ddc31 in_seq=0 out_seq=0 2024-04-23T18:38:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.396+0000 7fd0ef7fe640 10 client.17067 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/1768264387 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e74fd8e390 con 0x55e74fe36380 2024-04-23T18:38:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd0ef7fe640 10 client.17067 handle_command_reply: tid=0 2024-04-23T18:38:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd11945d740 1 client.17067 shutdown 2024-04-23T18:38:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd11945d740 1 -- 172.21.15.19:0/1768264387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1b620 msgr2=0x55e74fe261b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd11945d740 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1b620 0x55e74fe261b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fd1000bd060 tx=0x7fd1000b9970 comp rx=0 tx=0).stop 2024-04-23T18:38:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd11945d740 1 -- 172.21.15.19:0/1768264387 shutdown_connections 2024-04-23T18:38:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd11945d740 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e74fe36380 0x55e74fe38770 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd11945d740 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe1bf40 0x55e74fe234a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd11945d740 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe26800 0x55e74fe23a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd11945d740 1 --2- 172.21.15.19:0/1768264387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1b620 0x55e74fe261b0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:14.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd11945d740 1 -- 172.21.15.19:0/1768264387 >> 172.21.15.19:0/1768264387 conn(0x55e74fe17b30 msgr2=0x55e74fe1d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd0ef7fe640 0 client.17067 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.397+0000 7fd11945d740 1 -- 172.21.15.19:0/1768264387 shutdown_connections 2024-04-23T18:38:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.398+0000 7fd11945d740 1 -- 172.21.15.19:0/1768264387 wait complete. 2024-04-23T18:38:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.398+0000 7fd11945d740 20 client.17067 trim_cache size 0 max 16384 2024-04-23T18:38:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.412+0000 7fd11945d740 1 Processor -- start 2024-04-23T18:38:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.412+0000 7fd11945d740 1 -- start start 2024-04-23T18:38:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd11945d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe1b620 0x55e74fe25ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:14.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd11945d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe3dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd11945d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe26800 0x55e74fe3e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd11945d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101afd10 con 0x55e74fe1bf40 2024-04-23T18:38:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd11945d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b1850 con 0x55e74fe26800 2024-04-23T18:38:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd11945d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b03e0 con 0x55e74fe1b620 2024-04-23T18:38:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd10f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe3dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd10f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe3dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39890/0 (socket says 172.21.15.19:39890) 2024-04-23T18:38:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd10f7fe640 1 -- 172.21.15.19:0/2859387349 learned_addr learned my addr 172.21.15.19:0/2859387349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd114ce1640 1 --2- 172.21.15.19:0/2859387349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe26800 0x55e74fe3e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd10ffff640 1 --2- 172.21.15.19:0/2859387349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe1b620 0x55e74fe25ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd10f7fe640 1 -- 172.21.15.19:0/2859387349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe1b620 msgr2=0x55e74fe25ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/2859387349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe1b620 0x55e74fe25ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd10f7fe640 1 -- 172.21.15.19:0/2859387349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe26800 msgr2=0x55e74fe3e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/2859387349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe26800 0x55e74fe3e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd10f7fe640 1 -- 172.21.15.19:0/2859387349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74fe23db0 con 0x55e74fe1bf40 2024-04-23T18:38:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/2859387349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe3dfd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd104005fd0 tx=0x7fd1040094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d78eed6c749525 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd0f5ffb640 1 -- 172.21.15.19:0/2859387349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104004220 con 0x55e74fe1bf40 2024-04-23T18:38:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.413+0000 7fd0f5ffb640 1 -- 172.21.15.19:0/2859387349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1040043c0 con 0x55e74fe1bf40 2024-04-23T18:38:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd0f5ffb640 1 -- 172.21.15.19:0/2859387349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1040138b0 con 0x55e74fe1bf40 2024-04-23T18:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd11945d740 1 -- 172.21.15.19:0/2859387349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 msgr2=0x55e74fe3dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd11945d740 1 --2- 172.21.15.19:0/2859387349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe3dfd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd104005fd0 tx=0x7fd1040094a0 comp rx=0 tx=0).stop 2024-04-23T18:38:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd11945d740 1 -- 172.21.15.19:0/2859387349 shutdown_connections 2024-04-23T18:38:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd11945d740 1 --2- 172.21.15.19:0/2859387349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe26800 0x55e74fe3e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd11945d740 1 --2- 172.21.15.19:0/2859387349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe3dfd0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd11945d740 1 --2- 172.21.15.19:0/2859387349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe1b620 0x55e74fe25ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd11945d740 1 -- 172.21.15.19:0/2859387349 >> 172.21.15.19:0/2859387349 conn(0x55e74fe17b30 msgr2=0x55e74fe232b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd11945d740 1 -- 172.21.15.19:0/2859387349 shutdown_connections 2024-04-23T18:38:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd11945d740 1 -- 172.21.15.19:0/2859387349 wait complete. 2024-04-23T18:38:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.414+0000 7fd11945d740 1 -- start start 2024-04-23T18:38:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd11945d740 1 Processor -- start 2024-04-23T18:38:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd11945d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe1b620 0x55e74fe2e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd11945d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe2b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd11945d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe26800 0x55e74fe2be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd11945d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b1180 con 0x55e74fe1bf40 2024-04-23T18:38:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd11945d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b0ab0 con 0x55e74fe26800 2024-04-23T18:38:16.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd11945d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101add10 con 0x55e74fe1b620 2024-04-23T18:38:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd11945d740 10 client.0 _ll_register_callbacks cb 0x7fd1000bcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd10f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe2b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd10f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe2b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39906/0 (socket says 172.21.15.19:39906) 2024-04-23T18:38:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd10f7fe640 1 -- 172.21.15.19:0/2312204972 learned_addr learned my addr 172.21.15.19:0/2312204972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd114ce1640 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe26800 0x55e74fe2be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd10ffff640 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe1b620 0x55e74fe2e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd10f7fe640 1 -- 172.21.15.19:0/2312204972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe1b620 msgr2=0x55e74fe2e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe1b620 0x55e74fe2e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd10f7fe640 1 -- 172.21.15.19:0/2312204972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe26800 msgr2=0x55e74fe2be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe26800 0x55e74fe2be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.415+0000 7fd10f7fe640 1 -- 172.21.15.19:0/2312204972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74fe2c3d0 con 0x55e74fe1bf40 2024-04-23T18:38:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd10f7fe640 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe2b950 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd104009310 tx=0x7fd10401a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ef6161c209af9e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd0ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:38:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/2312204972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104004120 con 0x55e74fe1bf40 2024-04-23T18:38:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/2312204972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fd10401a6e0 con 0x55e74fe1bf40 2024-04-23T18:38:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd11945d740 1 -- 172.21.15.19:0/2312204972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1040043c0 con 0x55e74fe1bf40 2024-04-23T18:38:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/2312204972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd10401a8e0 con 0x55e74fe1bf40 2024-04-23T18:38:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd11945d740 10 client.17073 fetch_fsmap learned FSMap version 27 2024-04-23T18:38:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd11945d740 1 -- 172.21.15.19:0/2312204972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e74fe9c510 con 0x55e74fe1bf40 2024-04-23T18:38:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/2312204972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1040043c0 con 0x55e74fe1bf40 2024-04-23T18:38:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/2312204972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd10400f040 con 0x55e74fe1bf40 2024-04-23T18:38:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd11945d740 10 client.17073 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:38:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd11945d740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd11945d740 20 client.17073 populate_metadata read hostname 'smithi019' 2024-04-23T18:38:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd11945d740 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e74fe9cbb0 0x55e74fe9efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd11945d740 4 client.17073 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:38:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.416+0000 7fd11945d740 1 -- 172.21.15.19:0/2312204972 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e74fd88200 con 0x55e74fe9cbb0 2024-04-23T18:38:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.417+0000 7fd10ffff640 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e74fe9cbb0 0x55e74fe9efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.417+0000 7fd0ef7fe640 10 client.17073 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.417+0000 7fd10ffff640 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e74fe9cbb0 0x55e74fe9efd0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd1ea62919e6b7cd server_cookie=4dd9792849de15a9 in_seq=0 out_seq=0 2024-04-23T18:38:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.432+0000 7fd0ef7fe640 1 -- 172.21.15.19:0/2312204972 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e74fd88200 con 0x55e74fe9cbb0 2024-04-23T18:38:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd0ef7fe640 10 client.17073 handle_command_reply: tid=0 2024-04-23T18:38:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 client.17073 shutdown 2024-04-23T18:38:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 -- 172.21.15.19:0/2312204972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 msgr2=0x55e74fe2b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe2b950 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd104009310 tx=0x7fd10401a4b0 comp rx=0 tx=0).stop 2024-04-23T18:38:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 -- 172.21.15.19:0/2312204972 shutdown_connections 2024-04-23T18:38:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e74fe9cbb0 0x55e74fe9efd0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e74fe26800 0x55e74fe2be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e74fe1bf40 0x55e74fe2b950 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 --2- 172.21.15.19:0/2312204972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e74fe1b620 0x55e74fe2e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd0ef7fe640 0 client.17073 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 -- 172.21.15.19:0/2312204972 >> 172.21.15.19:0/2312204972 conn(0x55e74fe17b30 msgr2=0x55e74fe16420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 -- 172.21.15.19:0/2312204972 shutdown_connections 2024-04-23T18:38:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 1 -- 172.21.15.19:0/2312204972 wait complete. 2024-04-23T18:38:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.433+0000 7fd11945d740 20 client.17073 trim_cache size 0 max 16384 2024-04-23T18:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:38:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:38:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:38:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:38:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 -- 172.21.15.19:0/3978661430 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0f80a3dc0 msgr2=0x7fd0f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0f80a3dc0 0x7fd0f80a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd108005fd0 tx=0x7fd108009a10 comp rx=0 tx=0).stop 2024-04-23T18:38:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 -- 172.21.15.19:0/3978661430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1101021a0 msgr2=0x7fd110074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1101021a0 0x7fd110074250 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd100005940 tx=0x7fd1000042f0 comp rx=0 tx=0).stop 2024-04-23T18:38:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 -- 172.21.15.19:0/3978661430 shutdown_connections 2024-04-23T18:38:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0f80a3dc0 0x7fd0f80a6280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd110074cd0 0x7fd1101a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd110107270 0x7fd110074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 --2- 172.21.15.19:0/3978661430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1101021a0 0x7fd110074250 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 -- 172.21.15.19:0/3978661430 >> 172.21.15.19:0/3978661430 conn(0x7fd1100793e0 msgr2=0x7fd1100ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 -- 172.21.15.19:0/3978661430 shutdown_connections 2024-04-23T18:38:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:21.434+0000 7fd116744640 1 -- 172.21.15.19:0/3978661430 wait complete. 2024-04-23T18:38:20.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:21 smithi019 ceph-mon[29215]: pgmap v349: 97 pgs: 97 active+clean; 4.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:38:20.837 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:23 smithi019 ceph-mon[29215]: pgmap v350: 97 pgs: 97 active+clean; 4.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:38:20.939 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:38:21.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:38:21.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:38:21.101 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:38:21.151 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:38:21.201 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:38:21.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:24 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:38:21.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:25 smithi019 ceph-mon[29215]: pgmap v351: 97 pgs: 97 active+clean; 4.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-04-23T18:38:21.351 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:25 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:38:21.402 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:25 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:38:21.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:25 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:38:21.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:27 smithi019 ceph-mon[29215]: pgmap v352: 97 pgs: 97 active+clean; 4.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:38:21.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:29 smithi019 ceph-mon[29215]: pgmap v353: 97 pgs: 97 active+clean; 4.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:38:21.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:31 smithi019 ceph-mon[29215]: pgmap v354: 97 pgs: 97 active+clean; 4.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:21.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:33 smithi019 ceph-mon[29215]: pgmap v355: 97 pgs: 97 active+clean; 4.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:38:21.805 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:07 smithi155 ceph-mon[33468]: pgmap v373: 97 pgs: 97 active+clean; 2.1 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T18:38:21.835 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:09 smithi155 ceph-mon[33468]: pgmap v374: 97 pgs: 97 active+clean; 2.1 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T18:38:21.871 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:11 smithi155 ceph-mon[33468]: pgmap v375: 97 pgs: 97 active+clean; 2.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 38 op/s 2024-04-23T18:38:21.902 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:13 smithi155 ceph-mon[33468]: pgmap v376: 97 pgs: 97 active+clean; 2.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 38 op/s 2024-04-23T18:38:21.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:13 smithi155 ceph-mon[33468]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:38:21.969 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:15 smithi155 ceph-mon[33468]: pgmap v378: 97 pgs: 97 active+clean; 2.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-04-23T18:38:22.002 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:17 smithi155 ceph-mon[33468]: pgmap v379: 97 pgs: 97 active+clean; 2.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:38:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:19 smithi155 ceph-mon[33468]: pgmap v380: 97 pgs: 97 active+clean; 2.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:38:22.085 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:07 smithi044 ceph-mon[32458]: pgmap v373: 97 pgs: 97 active+clean; 2.1 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T18:38:22.118 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:09 smithi044 ceph-mon[32458]: pgmap v374: 97 pgs: 97 active+clean; 2.1 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T18:38:22.173 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:11 smithi044 ceph-mon[32458]: pgmap v375: 97 pgs: 97 active+clean; 2.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 38 op/s 2024-04-23T18:38:22.240 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:13 smithi044 ceph-mon[32458]: pgmap v376: 97 pgs: 97 active+clean; 2.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 38 op/s 2024-04-23T18:38:22.327 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:13 smithi044 ceph-mon[32458]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:38:22.378 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:15 smithi044 ceph-mon[32458]: pgmap v378: 97 pgs: 97 active+clean; 2.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-04-23T18:38:22.427 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:17 smithi044 ceph-mon[32458]: pgmap v379: 97 pgs: 97 active+clean; 2.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:38:22.478 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:19 smithi044 ceph-mon[32458]: pgmap v380: 97 pgs: 97 active+clean; 2.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:38:22.528 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:21 smithi155 ceph-mon[33468]: pgmap v381: 97 pgs: 97 active+clean; 2.7 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:38:22.578 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:21 smithi044 ceph-mon[32458]: pgmap v381: 97 pgs: 97 active+clean; 2.7 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:38:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.534+0000 7f101a1fe640 1 -- 172.21.15.19:0/25310217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10141010f0 msgr2=0x7f10141014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.534+0000 7f101a1fe640 1 --2- 172.21.15.19:0/25310217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10141010f0 0x7f10141014d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1004009ca0 tx=0x7f100402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:38:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.534+0000 7f101a1fe640 1 -- 172.21.15.19:0/25310217 shutdown_connections 2024-04-23T18:38:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.534+0000 7f101a1fe640 1 --2- 172.21.15.19:0/25310217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1014106170 0x7f101410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.534+0000 7f101a1fe640 1 --2- 172.21.15.19:0/25310217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1014101aa0 0x7f1014105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.534+0000 7f101a1fe640 1 --2- 172.21.15.19:0/25310217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10141010f0 0x7f10141014d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.534+0000 7f101a1fe640 1 -- 172.21.15.19:0/25310217 >> 172.21.15.19:0/25310217 conn(0x7f1014077fd0 msgr2=0x7f10140783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.534+0000 7f101a1fe640 1 -- 172.21.15.19:0/25310217 shutdown_connections 2024-04-23T18:38:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.534+0000 7f101a1fe640 1 -- 172.21.15.19:0/25310217 wait complete. 2024-04-23T18:38:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f101a1fe640 1 Processor -- start 2024-04-23T18:38:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f101a1fe640 1 -- start start 2024-04-23T18:38:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f101a1fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10141010f0 0x7f1014198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f101a1fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1014106170 0x7f1014198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f101a1fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f101419cff0 0x7f101419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f101a1fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10141110d0 con 0x7f10141010f0 2024-04-23T18:38:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f101a1fe640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1014110f50 con 0x7f1014106170 2024-04-23T18:38:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f101a1fe640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1014111250 con 0x7f101419cff0 2024-04-23T18:38:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f10137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10141010f0 0x7f1014198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f10137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10141010f0 0x7f1014198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55968/0 (socket says 172.21.15.19:55968) 2024-04-23T18:38:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f10137fe640 1 -- 172.21.15.19:0/4146041890 learned_addr learned my addr 172.21.15.19:0/4146041890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f1013fff640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f101419cff0 0x7f101419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.535+0000 7f1012ffd640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1014106170 0x7f1014198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f10137fe640 1 -- 172.21.15.19:0/4146041890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f101419cff0 msgr2=0x7f101419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f10137fe640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f101419cff0 0x7f101419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f10137fe640 1 -- 172.21.15.19:0/4146041890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1014106170 msgr2=0x7f1014198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f10137fe640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1014106170 0x7f1014198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f10137fe640 1 -- 172.21.15.19:0/4146041890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101419dae0 con 0x7f10141010f0 2024-04-23T18:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f10137fe640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10141010f0 0x7f1014198780 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f100402ade0 tx=0x7f1004031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f1010ff9640 1 -- 172.21.15.19:0/4146041890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004003b70 con 0x7f10141010f0 2024-04-23T18:38:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f101a1fe640 1 -- 172.21.15.19:0/4146041890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101419dd70 con 0x7f10141010f0 2024-04-23T18:38:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f1010ff9640 1 -- 172.21.15.19:0/4146041890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1004003d10 con 0x7f10141010f0 2024-04-23T18:38:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f1010ff9640 1 -- 172.21.15.19:0/4146041890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100402fa50 con 0x7f10141010f0 2024-04-23T18:38:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.536+0000 7f101a1fe640 1 -- 172.21.15.19:0/4146041890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10141a55c0 con 0x7f10141010f0 2024-04-23T18:38:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.538+0000 7f1010ff9640 1 -- 172.21.15.19:0/4146041890 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1004004580 con 0x7f10141010f0 2024-04-23T18:38:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f101cfed740 1 Processor -- start 2024-04-23T18:38:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f101cfed740 1 -- start start 2024-04-23T18:38:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f101cfed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 0x560eca799900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f101cfed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799e40 0x560eca7a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f101cfed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7ab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f101cfed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560eca6fe970 con 0x560eca7a8c10 2024-04-23T18:38:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f101cfed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560eca69b4a0 con 0x560eca799520 2024-04-23T18:38:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f101cfed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560eca6fa930 con 0x560eca799e40 2024-04-23T18:38:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1012ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1010ff9640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ffc0a3dc0 0x7f0ffc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1012ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7ab000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55980/0 (socket says 172.21.15.19:55980) 2024-04-23T18:38:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1012ffd640 1 -- 172.21.15.19:0/2964867201 learned_addr learned my addr 172.21.15.19:0/2964867201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f10137fe640 1 --2- 172.21.15.19:0/2964867201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799e40 0x560eca7a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1013fff640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ffc0a3dc0 0x7f0ffc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1013fff640 1 --2- 172.21.15.19:0/2964867201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 0x560eca799900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1010ff9640 1 -- 172.21.15.19:0/4146041890 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f10040f1aa0 con 0x7f10141010f0 2024-04-23T18:38:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1012ffd640 1 -- 172.21.15.19:0/2964867201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799e40 msgr2=0x560eca7a86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1012ffd640 1 --2- 172.21.15.19:0/2964867201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799e40 0x560eca7a86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1012ffd640 1 -- 172.21.15.19:0/2964867201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 msgr2=0x560eca799900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1012ffd640 1 --2- 172.21.15.19:0/2964867201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 0x560eca799900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1012ffd640 1 -- 172.21.15.19:0/2964867201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eca411210 con 0x560eca7a8c10 2024-04-23T18:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.539+0000 7f1013fff640 1 --2- 172.21.15.19:0/2964867201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 0x560eca799900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:38:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.540+0000 7f1012ffd640 1 --2- 172.21.15.19:0/2964867201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7ab000 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f100800b6c0 tx=0x7f100800bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b3eab8211f60b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.540+0000 7f1013fff640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ffc0a3dc0 0x7f0ffc0a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f100c005f60 tx=0x7f100c012040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.540+0000 7f0ff9ffb640 1 -- 172.21.15.19:0/2964867201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008003bd0 con 0x560eca7a8c10 2024-04-23T18:38:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.540+0000 7f0ff9ffb640 1 -- 172.21.15.19:0/2964867201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1008015070 con 0x560eca7a8c10 2024-04-23T18:38:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.540+0000 7f0ff9ffb640 1 -- 172.21.15.19:0/2964867201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008004d10 con 0x560eca7a8c10 2024-04-23T18:38:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.540+0000 7f101cfed740 1 -- 172.21.15.19:0/2964867201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 msgr2=0x560eca7ab000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 --2- 172.21.15.19:0/2964867201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7ab000 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f100800b6c0 tx=0x7f100800bb80 comp rx=0 tx=0).stop 2024-04-23T18:38:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 -- 172.21.15.19:0/2964867201 shutdown_connections 2024-04-23T18:38:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 --2- 172.21.15.19:0/2964867201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7ab000 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 --2- 172.21.15.19:0/2964867201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799e40 0x560eca7a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 --2- 172.21.15.19:0/2964867201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 0x560eca799900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 -- 172.21.15.19:0/2964867201 >> 172.21.15.19:0/2964867201 conn(0x560eca795b30 msgr2=0x560eca795f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 -- 172.21.15.19:0/2964867201 shutdown_connections 2024-04-23T18:38:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 -- 172.21.15.19:0/2964867201 wait complete. 2024-04-23T18:38:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 -- start start 2024-04-23T18:38:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 Processor -- start 2024-04-23T18:38:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799520 0x560eca7a8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.541+0000 7f101cfed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799e40 0x560eca7a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f101cfed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f101cfed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560eca7ab840 con 0x560eca7a8c10 2024-04-23T18:38:26.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f101cfed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560eca6990b0 con 0x560eca799e40 2024-04-23T18:38:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f101cfed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560eca7ab9c0 con 0x560eca799520 2024-04-23T18:38:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f101cfed740 10 client.0 _ll_register_callbacks cb 0x560eca686d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1013fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1013fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55988/0 (socket says 172.21.15.19:55988) 2024-04-23T18:38:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1012ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799e40 0x560eca7a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1013fff640 1 -- 172.21.15.19:0/881059663 learned_addr learned my addr 172.21.15.19:0/881059663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f10137fe640 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799520 0x560eca7a8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1013fff640 1 -- 172.21.15.19:0/881059663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799520 msgr2=0x560eca7a8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1013fff640 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799520 0x560eca7a8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1013fff640 1 -- 172.21.15.19:0/881059663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799e40 msgr2=0x560eca7a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1013fff640 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799e40 0x560eca7a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1013fff640 1 -- 172.21.15.19:0/881059663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eca4dd620 con 0x560eca7a8c10 2024-04-23T18:38:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1012ffd640 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799e40 0x560eca7a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:38:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f1013fff640 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a5db0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f100c009250 tx=0x7f100c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb7419a098b8511 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f0ff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:38:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.542+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/881059663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c02c060 con 0x560eca7a8c10 2024-04-23T18:38:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/881059663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f100c01dc20 con 0x560eca7a8c10 2024-04-23T18:38:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f101cfed740 1 -- 172.21.15.19:0/881059663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1008015070 con 0x560eca7a8c10 2024-04-23T18:38:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/881059663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f100c01de20 con 0x560eca7a8c10 2024-04-23T18:38:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f101cfed740 10 client.17085 fetch_fsmap learned FSMap version 27 2024-04-23T18:38:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f101cfed740 1 -- 172.21.15.19:0/881059663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560eca7b7850 con 0x560eca7a8c10 2024-04-23T18:38:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/881059663 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1008015070 con 0x560eca7a8c10 2024-04-23T18:38:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/881059663 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f100c00e450 con 0x560eca7a8c10 2024-04-23T18:38:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f101cfed740 10 client.17085 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f101cfed740 10 client.17085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f101cfed740 20 client.17085 populate_metadata read hostname 'smithi019' 2024-04-23T18:38:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f101cfed740 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560eca7b8d10 0x560eca7bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f101cfed740 4 client.17085 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:38:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f101cfed740 1 -- 172.21.15.19:0/881059663 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eca70c390 con 0x560eca7b8d10 2024-04-23T18:38:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.543+0000 7f10137fe640 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560eca7b8d10 0x560eca7bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.544+0000 7f0ff2ffd640 10 client.17085 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.544+0000 7f10137fe640 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560eca7b8d10 0x560eca7bb120 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f1b4be0f5ba80c9 server_cookie=22f493f14267bdd1 in_seq=0 out_seq=0 2024-04-23T18:38:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/881059663 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560eca70c390 con 0x560eca7b8d10 2024-04-23T18:38:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f0ff2ffd640 10 client.17085 handle_command_reply: tid=0 2024-04-23T18:38:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 client.17085 shutdown 2024-04-23T18:38:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 -- 172.21.15.19:0/881059663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 msgr2=0x560eca7a5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a5db0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f100c009250 tx=0x7f100c0046c0 comp rx=0 tx=0).stop 2024-04-23T18:38:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 -- 172.21.15.19:0/881059663 shutdown_connections 2024-04-23T18:38:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560eca7b8d10 0x560eca7bb120 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799e40 0x560eca7a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a5db0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 --2- 172.21.15.19:0/881059663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799520 0x560eca7a8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f0ff2ffd640 0 client.17085 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 -- 172.21.15.19:0/881059663 >> 172.21.15.19:0/881059663 conn(0x560eca795b30 msgr2=0x560eca79b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 -- 172.21.15.19:0/881059663 shutdown_connections 2024-04-23T18:38:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 1 -- 172.21.15.19:0/881059663 wait complete. 2024-04-23T18:38:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.545+0000 7f101cfed740 20 client.17085 trim_cache size 0 max 16384 2024-04-23T18:38:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f101cfed740 1 Processor -- start 2024-04-23T18:38:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f101cfed740 1 -- start start 2024-04-23T18:38:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f101cfed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799520 0x560eca7b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f101cfed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca799e40 0x560eca795f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f101cfed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca7a8c10 0x560eca7a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f101cfed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10141ab820 con 0x560eca799e40 2024-04-23T18:38:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f101cfed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10141ad360 con 0x560eca7a8c10 2024-04-23T18:38:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f101cfed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f10141abef0 con 0x560eca799520 2024-04-23T18:38:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f1012ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca799e40 0x560eca795f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f1012ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca799e40 0x560eca795f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55992/0 (socket says 172.21.15.19:55992) 2024-04-23T18:38:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.560+0000 7f1012ffd640 1 -- 172.21.15.19:0/2673456391 learned_addr learned my addr 172.21.15.19:0/2673456391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f1013fff640 1 --2- 172.21.15.19:0/2673456391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca7a8c10 0x560eca7a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f10137fe640 1 --2- 172.21.15.19:0/2673456391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799520 0x560eca7b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f1012ffd640 1 -- 172.21.15.19:0/2673456391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799520 msgr2=0x560eca7b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f1012ffd640 1 --2- 172.21.15.19:0/2673456391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799520 0x560eca7b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f1012ffd640 1 -- 172.21.15.19:0/2673456391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca7a8c10 msgr2=0x560eca7a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f1012ffd640 1 --2- 172.21.15.19:0/2673456391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca7a8c10 0x560eca7a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f1012ffd640 1 -- 172.21.15.19:0/2673456391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eca7a4800 con 0x560eca799e40 2024-04-23T18:38:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f1012ffd640 1 --2- 172.21.15.19:0/2673456391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca799e40 0x560eca795f10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f10080039c0 tx=0x7f100800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea22ff4996cff5e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f0ff97fa640 1 -- 172.21.15.19:0/2673456391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008003dd0 con 0x560eca799e40 2024-04-23T18:38:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f0ff97fa640 1 -- 172.21.15.19:0/2673456391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1008029070 con 0x560eca799e40 2024-04-23T18:38:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.561+0000 7f0ff97fa640 1 -- 172.21.15.19:0/2673456391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10080116a0 con 0x560eca799e40 2024-04-23T18:38:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 -- 172.21.15.19:0/2673456391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca799e40 msgr2=0x560eca795f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 --2- 172.21.15.19:0/2673456391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca799e40 0x560eca795f10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f10080039c0 tx=0x7f100800be80 comp rx=0 tx=0).stop 2024-04-23T18:38:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 -- 172.21.15.19:0/2673456391 shutdown_connections 2024-04-23T18:38:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 --2- 172.21.15.19:0/2673456391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca7a8c10 0x560eca7a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 --2- 172.21.15.19:0/2673456391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca799e40 0x560eca795f10 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 --2- 172.21.15.19:0/2673456391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799520 0x560eca7b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 -- 172.21.15.19:0/2673456391 >> 172.21.15.19:0/2673456391 conn(0x560eca795b30 msgr2=0x560eca7a0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 -- 172.21.15.19:0/2673456391 shutdown_connections 2024-04-23T18:38:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 -- 172.21.15.19:0/2673456391 wait complete. 2024-04-23T18:38:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 -- start start 2024-04-23T18:38:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 Processor -- start 2024-04-23T18:38:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.562+0000 7f101cfed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 0x560eca7b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f101cfed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799e40 0x560eca7a2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f101cfed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f101cfed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10141acc90 con 0x560eca7a8c10 2024-04-23T18:38:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f101cfed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10141ac5c0 con 0x560eca799520 2024-04-23T18:38:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f101cfed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f10141a9820 con 0x560eca799e40 2024-04-23T18:38:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f101cfed740 10 client.0 _ll_register_callbacks cb 0x7f100c01f830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f1013fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f1013fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56000/0 (socket says 172.21.15.19:56000) 2024-04-23T18:38:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f1013fff640 1 -- 172.21.15.19:0/2972691151 learned_addr learned my addr 172.21.15.19:0/2972691151 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f10137fe640 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 0x560eca7b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f1012ffd640 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799e40 0x560eca7a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f1013fff640 1 -- 172.21.15.19:0/2972691151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799e40 msgr2=0x560eca7a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f1013fff640 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799e40 0x560eca7a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f1013fff640 1 -- 172.21.15.19:0/2972691151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 msgr2=0x560eca7b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f1013fff640 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 0x560eca7b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f1013fff640 1 -- 172.21.15.19:0/2972691151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eca7a2de0 con 0x560eca7a8c10 2024-04-23T18:38:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f1013fff640 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a26a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f100c012010 tx=0x7f100c00e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbe8812ecb8e6f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f0ff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:38:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/2972691151 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c01d7a0 con 0x560eca7a8c10 2024-04-23T18:38:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/2972691151 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f100c009ad0 con 0x560eca7a8c10 2024-04-23T18:38:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f101cfed740 1 -- 172.21.15.19:0/2972691151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1008029070 con 0x560eca7a8c10 2024-04-23T18:38:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/2972691151 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f100c02c030 con 0x560eca7a8c10 2024-04-23T18:38:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f101cfed740 10 client.17091 fetch_fsmap learned FSMap version 27 2024-04-23T18:38:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.563+0000 7f101cfed740 1 -- 172.21.15.19:0/2972691151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560eca81f1e0 con 0x560eca7a8c10 2024-04-23T18:38:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/2972691151 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1008029070 con 0x560eca7a8c10 2024-04-23T18:38:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/2972691151 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f100c003b30 con 0x560eca7a8c10 2024-04-23T18:38:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f101cfed740 10 client.17091 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:38:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f101cfed740 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f101cfed740 20 client.17091 populate_metadata read hostname 'smithi019' 2024-04-23T18:38:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f101cfed740 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560eca81f8f0 0x560eca821ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f101cfed740 4 client.17091 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:38:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f101cfed740 1 -- 172.21.15.19:0/2972691151 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560eca706200 con 0x560eca81f8f0 2024-04-23T18:38:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f10137fe640 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560eca81f8f0 0x560eca821ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f0ff2ffd640 10 client.17091 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.564+0000 7f10137fe640 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560eca81f8f0 0x560eca821ce0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4255d5331917a9d server_cookie=911f97f29de56400 in_seq=0 out_seq=0 2024-04-23T18:38:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f0ff2ffd640 1 -- 172.21.15.19:0/2972691151 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560eca706200 con 0x560eca81f8f0 2024-04-23T18:38:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f0ff2ffd640 10 client.17091 handle_command_reply: tid=0 2024-04-23T18:38:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 client.17091 shutdown 2024-04-23T18:38:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 -- 172.21.15.19:0/2972691151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 msgr2=0x560eca7a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a26a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f100c012010 tx=0x7f100c00e780 comp rx=0 tx=0).stop 2024-04-23T18:38:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 -- 172.21.15.19:0/2972691151 shutdown_connections 2024-04-23T18:38:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560eca81f8f0 0x560eca821ce0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560eca7a8c10 0x560eca7a26a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eca799e40 0x560eca7a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f0ff2ffd640 0 client.17091 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 --2- 172.21.15.19:0/2972691151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560eca799520 0x560eca7b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 -- 172.21.15.19:0/2972691151 >> 172.21.15.19:0/2972691151 conn(0x560eca795b30 msgr2=0x560eca7a0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 -- 172.21.15.19:0/2972691151 shutdown_connections 2024-04-23T18:38:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 1 -- 172.21.15.19:0/2972691151 wait complete. 2024-04-23T18:38:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.579+0000 7f101cfed740 20 client.17091 trim_cache size 0 max 16384 2024-04-23T18:38:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:38:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:38:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:38:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:38:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 -- 172.21.15.19:0/4146041890 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ffc0a3dc0 msgr2=0x7f0ffc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ffc0a3dc0 0x7f0ffc0a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f100c005f60 tx=0x7f100c012040 comp rx=0 tx=0).stop 2024-04-23T18:38:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 -- 172.21.15.19:0/4146041890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10141010f0 msgr2=0x7f1014198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10141010f0 0x7f1014198780 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f100402ade0 tx=0x7f1004031770 comp rx=0 tx=0).stop 2024-04-23T18:38:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 -- 172.21.15.19:0/4146041890 shutdown_connections 2024-04-23T18:38:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0ffc0a3dc0 0x7f0ffc0a6280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1014106170 0x7f1014198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f101419cff0 0x7f101419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 --2- 172.21.15.19:0/4146041890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10141010f0 0x7f1014198780 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 -- 172.21.15.19:0/4146041890 >> 172.21.15.19:0/4146041890 conn(0x7f1014077fd0 msgr2=0x7f1014069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 -- 172.21.15.19:0/4146041890 shutdown_connections 2024-04-23T18:38:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:34.580+0000 7f101a1fe640 1 -- 172.21.15.19:0/4146041890 wait complete. 2024-04-23T18:38:33.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:35 smithi019 ceph-mon[29215]: pgmap v356: 97 pgs: 97 active+clean; 4.7 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:33.445 INFO:tasks.workunit.client.0.smithi019.stdout:375/600: open fd = 3 2024-04-23T18:38:33.465 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:33.481 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:33.576 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:33.598 INFO:tasks.workunit.client.0.smithi019.stdout:376/600: open fd = 3 2024-04-23T18:38:33.623 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:33.732 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:33.748 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:33.765 INFO:tasks.workunit.client.0.smithi019.stdout:377/600: open fd = 3 2024-04-23T18:38:33.782 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:33.823 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:33.857 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:33.890 INFO:tasks.workunit.client.0.smithi019.stdout:378/600: open fd = 3 2024-04-23T18:38:33.923 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:33.957 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.045 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:34.116 INFO:tasks.workunit.client.0.smithi019.stdout:379/600: open fd = 3 2024-04-23T18:38:34.141 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.174 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.207 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:34.241 INFO:tasks.workunit.client.0.smithi019.stdout:380/600: open fd = 3 2024-04-23T18:38:34.274 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.307 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.341 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:34.374 INFO:tasks.workunit.client.0.smithi019.stdout:381/600: open fd = 3 2024-04-23T18:38:34.407 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.449 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.474 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:34.508 INFO:tasks.workunit.client.0.smithi019.stdout:382/600: open fd = 3 2024-04-23T18:38:34.541 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.574 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.608 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:34.641 INFO:tasks.workunit.client.0.smithi019.stdout:383/600: open fd = 3 2024-04-23T18:38:34.675 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.708 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:34.849 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:35.149 INFO:tasks.workunit.client.0.smithi019.stdout:384/600: open fd = 3 2024-04-23T18:38:35.183 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:35.249 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:35.286 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:35.333 INFO:tasks.workunit.client.0.smithi019.stdout:385/600: open fd = 3 2024-04-23T18:38:35.366 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:35.399 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:35.466 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:35.533 INFO:tasks.workunit.client.0.smithi019.stdout:386/600: open fd = 3 2024-04-23T18:38:35.566 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:35.600 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:35.663 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:35.800 INFO:tasks.workunit.client.0.smithi019.stdout:387/600: open fd = 3 2024-04-23T18:38:35.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:35.992 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.025 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:36.058 INFO:tasks.workunit.client.0.smithi019.stdout:388/600: open fd = 3 2024-04-23T18:38:36.092 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.125 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.192 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:36.225 INFO:tasks.workunit.client.0.smithi019.stdout:389/600: open fd = 3 2024-04-23T18:38:36.258 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.292 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.325 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:36.450 INFO:tasks.workunit.client.0.smithi019.stdout:390/600: open fd = 3 2024-04-23T18:38:36.484 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.517 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.550 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:36.584 INFO:tasks.workunit.client.0.smithi019.stdout:391/600: open fd = 3 2024-04-23T18:38:36.617 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.651 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.684 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:36.717 INFO:tasks.workunit.client.0.smithi019.stdout:392/600: open fd = 3 2024-04-23T18:38:36.750 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.807 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:36.940 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:36.976 INFO:tasks.workunit.client.0.smithi019.stdout:393/600: open fd = 3 2024-04-23T18:38:37.009 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.042 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.092 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:37.126 INFO:tasks.workunit.client.0.smithi019.stdout:394/600: open fd = 3 2024-04-23T18:38:37.159 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.267 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.301 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:37.334 INFO:tasks.workunit.client.0.smithi019.stdout:395/600: open fd = 3 2024-04-23T18:38:37.368 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.451 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.484 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:37.518 INFO:tasks.workunit.client.0.smithi019.stdout:396/600: open fd = 3 2024-04-23T18:38:37.551 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.584 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.652 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:37.685 INFO:tasks.workunit.client.0.smithi019.stdout:397/600: open fd = 3 2024-04-23T18:38:37.718 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.751 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.784 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:37.818 INFO:tasks.workunit.client.0.smithi019.stdout:398/600: open fd = 3 2024-04-23T18:38:37.896 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.934 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:37.968 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:38.001 INFO:tasks.workunit.client.0.smithi019.stdout:399/600: open fd = 3 2024-04-23T18:38:38.035 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.051 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.076 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:38.101 INFO:tasks.workunit.client.0.smithi019.stdout:400/600: open fd = 3 2024-04-23T18:38:38.135 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.177 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.202 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:38.302 INFO:tasks.workunit.client.0.smithi019.stdout:401/600: open fd = 3 2024-04-23T18:38:38.318 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.368 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.402 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:38.418 INFO:tasks.workunit.client.0.smithi019.stdout:402/600: open fd = 3 2024-04-23T18:38:38.435 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.452 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.468 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:38.485 INFO:tasks.workunit.client.0.smithi019.stdout:403/600: open fd = 3 2024-04-23T18:38:38.502 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.519 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.617 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:38.652 INFO:tasks.workunit.client.0.smithi019.stdout:404/600: open fd = 3 2024-04-23T18:38:38.685 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.719 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.752 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:38.786 INFO:tasks.workunit.client.0.smithi019.stdout:405/600: open fd = 3 2024-04-23T18:38:38.819 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.852 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:38.885 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:38.919 INFO:tasks.workunit.client.0.smithi019.stdout:406/600: open fd = 3 2024-04-23T18:38:39.011 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.107 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.144 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:39.178 INFO:tasks.workunit.client.0.smithi019.stdout:407/600: open fd = 3 2024-04-23T18:38:39.211 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.244 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.277 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:39.311 INFO:tasks.workunit.client.0.smithi019.stdout:408/600: open fd = 3 2024-04-23T18:38:39.345 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.378 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.411 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:39.444 INFO:tasks.workunit.client.0.smithi019.stdout:409/600: open fd = 3 2024-04-23T18:38:39.520 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.553 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.586 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:39.619 INFO:tasks.workunit.client.0.smithi019.stdout:410/600: open fd = 3 2024-04-23T18:38:39.653 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.686 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.719 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:39.753 INFO:tasks.workunit.client.0.smithi019.stdout:411/600: open fd = 3 2024-04-23T18:38:39.786 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.820 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:39.853 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:39.966 INFO:tasks.workunit.client.0.smithi019.stdout:412/600: open fd = 3 2024-04-23T18:38:40.057 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:40.120 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:40.171 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:40.221 INFO:tasks.workunit.client.0.smithi019.stdout:413/600: open fd = 3 2024-04-23T18:38:40.273 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:40.389 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:40.583 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:40.653 INFO:tasks.workunit.client.0.smithi019.stdout:414/600: open fd = 3 2024-04-23T18:38:40.762 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:40.828 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:40.904 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:40.948 INFO:tasks.workunit.client.0.smithi019.stdout:415/600: open fd = 3 2024-04-23T18:38:40.979 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.012 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.045 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:41.079 INFO:tasks.workunit.client.0.smithi019.stdout:416/600: open fd = 3 2024-04-23T18:38:41.129 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.162 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.195 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:41.237 INFO:tasks.workunit.client.0.smithi019.stdout:417/600: open fd = 3 2024-04-23T18:38:41.312 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.399 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.429 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:41.462 INFO:tasks.workunit.client.0.smithi019.stdout:418/600: open fd = 3 2024-04-23T18:38:41.495 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.596 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.629 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:41.662 INFO:tasks.workunit.client.0.smithi019.stdout:419/600: open fd = 3 2024-04-23T18:38:41.696 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.729 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.787 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:41.846 INFO:tasks.workunit.client.0.smithi019.stdout:420/600: open fd = 3 2024-04-23T18:38:41.901 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.939 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:41.979 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:42.012 INFO:tasks.workunit.client.0.smithi019.stdout:421/600: open fd = 3 2024-04-23T18:38:42.046 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:42.079 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:42.113 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:42.146 INFO:tasks.workunit.client.0.smithi019.stdout:422/600: open fd = 3 2024-04-23T18:38:42.179 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:42.213 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:42.314 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:42.425 INFO:tasks.workunit.client.0.smithi019.stdout:423/600: open fd = 3 2024-04-23T18:38:42.463 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:42.496 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:42.530 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:42.563 INFO:tasks.workunit.client.0.smithi019.stdout:424/600: open fd = 3 2024-04-23T18:38:42.596 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:42.630 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:42.663 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:42.696 INFO:tasks.workunit.client.0.smithi019.stdout:425/600: open fd = 3 2024-04-23T18:38:42.780 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:42.813 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:42.978 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:43.005 INFO:tasks.workunit.client.0.smithi019.stdout:426/600: open fd = 3 2024-04-23T18:38:43.038 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.071 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.105 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:43.138 INFO:tasks.workunit.client.0.smithi019.stdout:427/600: open fd = 3 2024-04-23T18:38:43.172 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.228 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.343 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:43.380 INFO:tasks.workunit.client.0.smithi019.stdout:428/600: open fd = 3 2024-04-23T18:38:43.413 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.554 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.580 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:43.597 INFO:tasks.workunit.client.0.smithi019.stdout:429/600: open fd = 3 2024-04-23T18:38:43.614 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.630 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.655 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:43.672 INFO:tasks.workunit.client.0.smithi019.stdout:430/600: open fd = 3 2024-04-23T18:38:43.697 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.780 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.797 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:43.814 INFO:tasks.workunit.client.0.smithi019.stdout:431/600: open fd = 3 2024-04-23T18:38:43.917 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.940 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:43.956 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:43.973 INFO:tasks.workunit.client.0.smithi019.stdout:432/600: open fd = 3 2024-04-23T18:38:43.990 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:44.006 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:44.061 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:44.081 INFO:tasks.workunit.client.0.smithi019.stdout:433/600: open fd = 3 2024-04-23T18:38:44.186 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:44.206 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:44.223 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:44.358 INFO:tasks.workunit.client.0.smithi019.stdout:434/600: open fd = 3 2024-04-23T18:38:44.373 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:44.390 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:44.407 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:44.423 INFO:tasks.workunit.client.0.smithi019.stdout:435/600: open fd = 3 2024-04-23T18:38:44.440 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:44.484 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:44.535 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:44.585 INFO:tasks.workunit.client.0.smithi019.stdout:436/600: open fd = 3 2024-04-23T18:38:44.635 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:44.685 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:38:44.785 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:38:44.835 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:37 smithi019 ceph-mon[29215]: pgmap v357: 97 pgs: 97 active+clean; 4.7 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:38:44.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:39 smithi019 ceph-mon[29215]: pgmap v358: 97 pgs: 97 active+clean; 4.7 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:38:44.935 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:41 smithi019 ceph-mon[29215]: pgmap v359: 97 pgs: 97 active+clean; 5.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T18:38:45.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:43 smithi019 ceph-mon[29215]: pgmap v360: 97 pgs: 97 active+clean; 5.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:38:45.101 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:45 smithi019 ceph-mon[29215]: pgmap v361: 97 pgs: 97 active+clean; 5.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:45.156 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:23 smithi044 ceph-mon[32458]: pgmap v382: 97 pgs: 97 active+clean; 2.7 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:38:45.202 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:25 smithi044 ceph-mon[32458]: pgmap v383: 97 pgs: 97 active+clean; 2.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-23T18:38:45.252 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:25 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:38:45.310 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:26 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:38:45.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:26 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:38:45.485 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:26 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:38:45.577 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:27 smithi044 ceph-mon[32458]: pgmap v384: 97 pgs: 97 active+clean; 2.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-23T18:38:45.710 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:29 smithi044 ceph-mon[32458]: pgmap v385: 97 pgs: 97 active+clean; 2.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-23T18:38:45.777 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:31 smithi044 ceph-mon[32458]: pgmap v386: 97 pgs: 97 active+clean; 3.1 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T18:38:45.861 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:33 smithi044 ceph-mon[32458]: pgmap v387: 97 pgs: 97 active+clean; 3.1 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:38:45.910 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:35 smithi044 ceph-mon[32458]: pgmap v388: 97 pgs: 97 active+clean; 3.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T18:38:45.960 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:37 smithi044 ceph-mon[32458]: pgmap v389: 97 pgs: 97 active+clean; 3.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:38:46.010 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:39 smithi044 ceph-mon[32458]: pgmap v390: 97 pgs: 97 active+clean; 3.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:38:46.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:41 smithi044 ceph-mon[32458]: pgmap v391: 97 pgs: 97 active+clean; 3.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:46.110 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:43 smithi044 ceph-mon[32458]: pgmap v392: 97 pgs: 97 active+clean; 3.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-23T18:38:46.186 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:23 smithi155 ceph-mon[33468]: pgmap v382: 97 pgs: 97 active+clean; 2.7 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:38:46.258 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:25 smithi155 ceph-mon[33468]: pgmap v383: 97 pgs: 97 active+clean; 2.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-23T18:38:46.310 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:25 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:38:46.361 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:26 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:38:46.411 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:26 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:38:46.461 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:26 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:38:46.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:27 smithi155 ceph-mon[33468]: pgmap v384: 97 pgs: 97 active+clean; 2.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-23T18:38:46.561 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:29 smithi155 ceph-mon[33468]: pgmap v385: 97 pgs: 97 active+clean; 2.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-23T18:38:46.611 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:31 smithi155 ceph-mon[33468]: pgmap v386: 97 pgs: 97 active+clean; 3.1 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T18:38:46.661 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:33 smithi155 ceph-mon[33468]: pgmap v387: 97 pgs: 97 active+clean; 3.1 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:38:46.797 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:35 smithi155 ceph-mon[33468]: pgmap v388: 97 pgs: 97 active+clean; 3.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T18:38:46.844 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:37 smithi155 ceph-mon[33468]: pgmap v389: 97 pgs: 97 active+clean; 3.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:38:46.894 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:39 smithi155 ceph-mon[33468]: pgmap v390: 97 pgs: 97 active+clean; 3.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:38:46.944 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:41 smithi155 ceph-mon[33468]: pgmap v391: 97 pgs: 97 active+clean; 3.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:46.994 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:43 smithi155 ceph-mon[33468]: pgmap v392: 97 pgs: 97 active+clean; 3.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-23T18:38:47.046 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:45 smithi044 ceph-mon[32458]: pgmap v393: 97 pgs: 97 active+clean; 3.8 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:47.094 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:45 smithi155 ceph-mon[33468]: pgmap v393: 97 pgs: 97 active+clean; 3.8 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:38:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.679+0000 7f50a8f37640 1 -- 172.21.15.19:0/2633312231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50a41061b0 msgr2=0x7f50a410dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.679+0000 7f50a8f37640 1 --2- 172.21.15.19:0/2633312231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50a41061b0 0x7f50a410dd80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f509c009ca0 tx=0x7f509c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:38:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.679+0000 7f50a8f37640 1 -- 172.21.15.19:0/2633312231 shutdown_connections 2024-04-23T18:38:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.679+0000 7f50a8f37640 1 --2- 172.21.15.19:0/2633312231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50a41061b0 0x7f50a410dd80 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.679+0000 7f50a8f37640 1 --2- 172.21.15.19:0/2633312231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50a4101a90 0x7f50a4105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:47.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.679+0000 7f50a8f37640 1 --2- 172.21.15.19:0/2633312231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a41010e0 0x7f50a41014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.679+0000 7f50a8f37640 1 -- 172.21.15.19:0/2633312231 >> 172.21.15.19:0/2633312231 conn(0x7f50a40771d0 msgr2=0x7f50a40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.680+0000 7f50a8f37640 1 -- 172.21.15.19:0/2633312231 shutdown_connections 2024-04-23T18:38:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.680+0000 7f50a8f37640 1 -- 172.21.15.19:0/2633312231 wait complete. 2024-04-23T18:38:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.680+0000 7f50a8f37640 1 Processor -- start 2024-04-23T18:38:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.680+0000 7f50a8f37640 1 -- start start 2024-04-23T18:38:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a8f37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a41010e0 0x7f50a4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a8f37640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50a41061b0 0x7f50a4198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a8f37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50a419cf00 0x7f50a419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a8f37640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50a4111150 con 0x7f50a419cf00 2024-04-23T18:38:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a8f37640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50a4110fd0 con 0x7f50a41010e0 2024-04-23T18:38:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a8f37640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41112d0 con 0x7f50a41061b0 2024-04-23T18:38:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50a419cf00 0x7f50a419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50a419cf00 0x7f50a419d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36896/0 (socket says 172.21.15.19:36896) 2024-04-23T18:38:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a2d9d640 1 -- 172.21.15.19:0/3698944981 learned_addr learned my addr 172.21.15.19:0/3698944981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a259c640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a41010e0 0x7f50a4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50a41061b0 0x7f50a4198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:48.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a2d9d640 1 -- 172.21.15.19:0/3698944981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50a41061b0 msgr2=0x7f50a4198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a2d9d640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50a41061b0 0x7f50a4198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a2d9d640 1 -- 172.21.15.19:0/3698944981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a41010e0 msgr2=0x7f50a4198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a2d9d640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a41010e0 0x7f50a4198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a2d9d640 1 -- 172.21.15.19:0/3698944981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a419d9f0 con 0x7f50a419cf00 2024-04-23T18:38:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.681+0000 7f50a2d9d640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50a419cf00 0x7f50a419d3b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f509c009430 tx=0x7f509c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.682+0000 7f50937fe640 1 -- 172.21.15.19:0/3698944981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c003c30 con 0x7f50a419cf00 2024-04-23T18:38:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.682+0000 7f50937fe640 1 -- 172.21.15.19:0/3698944981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f509c003dd0 con 0x7f50a419cf00 2024-04-23T18:38:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.682+0000 7f50a8f37640 1 -- 172.21.15.19:0/3698944981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a419dc80 con 0x7f50a419cf00 2024-04-23T18:38:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.682+0000 7f50937fe640 1 -- 172.21.15.19:0/3698944981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c02faa0 con 0x7f50a419cf00 2024-04-23T18:38:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.682+0000 7f50a8f37640 1 -- 172.21.15.19:0/3698944981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a41a54a0 con 0x7f50a419cf00 2024-04-23T18:38:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.684+0000 7f50937fe640 1 -- 172.21.15.19:0/3698944981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f509c02fc40 con 0x7f50a419cf00 2024-04-23T18:38:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.684+0000 7f50abc4d740 1 Processor -- start 2024-04-23T18:38:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50abc4d740 1 -- start start 2024-04-23T18:38:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f78a30 0x55ae61f78e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f79350 0x55ae61f836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f83c10 0x55ae61f86000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50abc4d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ae61ed9970 con 0x55ae61f78a30 2024-04-23T18:38:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50abc4d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ae61e764a0 con 0x55ae61f79350 2024-04-23T18:38:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50abc4d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ae61ed5930 con 0x55ae61f83c10 2024-04-23T18:38:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50937fe640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f508c0a3dc0 0x7f508c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50a1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f78a30 0x55ae61f78e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50a1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f78a30 0x55ae61f78e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36904/0 (socket says 172.21.15.19:36904) 2024-04-23T18:38:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50a1d9b640 1 -- 172.21.15.19:0/547862617 learned_addr learned my addr 172.21.15.19:0/547862617 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50a2d9d640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f508c0a3dc0 0x7f508c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/547862617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f83c10 0x55ae61f86000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.685+0000 7f50937fe640 1 -- 172.21.15.19:0/3698944981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f509c0f1ce0 con 0x7f50a419cf00 2024-04-23T18:38:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f50a259c640 1 --2- 172.21.15.19:0/547862617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f79350 0x55ae61f836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f50a1d9b640 1 -- 172.21.15.19:0/547862617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f83c10 msgr2=0x55ae61f86000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/547862617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f83c10 0x55ae61f86000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f50a1d9b640 1 -- 172.21.15.19:0/547862617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f79350 msgr2=0x55ae61f836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/547862617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f79350 0x55ae61f836d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f50a1d9b640 1 -- 172.21.15.19:0/547862617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae61bec210 con 0x55ae61f78a30 2024-04-23T18:38:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/547862617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f78a30 0x55ae61f78e10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f509800dfd0 tx=0x7f509800d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbb76ba1bf4e242 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f50a2d9d640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f508c0a3dc0 0x7f508c0a6280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f509c0ba700 tx=0x7f509c0095c0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:50.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f5090ff9640 1 -- 172.21.15.19:0/547862617 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098004030 con 0x55ae61f78a30 2024-04-23T18:38:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f5090ff9640 1 -- 172.21.15.19:0/547862617 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50980041d0 con 0x55ae61f78a30 2024-04-23T18:38:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.686+0000 7f5090ff9640 1 -- 172.21.15.19:0/547862617 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50980106c0 con 0x55ae61f78a30 2024-04-23T18:38:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 -- 172.21.15.19:0/547862617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f78a30 msgr2=0x55ae61f78e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 --2- 172.21.15.19:0/547862617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f78a30 0x55ae61f78e10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f509800dfd0 tx=0x7f509800d5d0 comp rx=0 tx=0).stop 2024-04-23T18:38:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 -- 172.21.15.19:0/547862617 shutdown_connections 2024-04-23T18:38:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 --2- 172.21.15.19:0/547862617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f79350 0x55ae61f836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 --2- 172.21.15.19:0/547862617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f83c10 0x55ae61f86000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 --2- 172.21.15.19:0/547862617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f78a30 0x55ae61f78e10 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 -- 172.21.15.19:0/547862617 >> 172.21.15.19:0/547862617 conn(0x55ae61f74f40 msgr2=0x55ae61f75320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 -- 172.21.15.19:0/547862617 shutdown_connections 2024-04-23T18:38:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 -- 172.21.15.19:0/547862617 wait complete. 2024-04-23T18:38:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 -- start start 2024-04-23T18:38:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.687+0000 7f50abc4d740 1 Processor -- start 2024-04-23T18:38:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f78a30 0x55ae61f835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f83c10 0x55ae61f80e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50abc4d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ae61f86840 con 0x55ae61f79350 2024-04-23T18:38:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50abc4d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ae61e740b0 con 0x55ae61f83c10 2024-04-23T18:38:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50abc4d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ae61f869c0 con 0x55ae61f78a30 2024-04-23T18:38:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50abc4d740 10 client.0 _ll_register_callbacks cb 0x55ae61e61d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f808a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f808a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36918/0 (socket says 172.21.15.19:36918) 2024-04-23T18:38:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a259c640 1 -- 172.21.15.19:0/1462105375 learned_addr learned my addr 172.21.15.19:0/1462105375 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f83c10 0x55ae61f80e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f78a30 0x55ae61f835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a259c640 1 -- 172.21.15.19:0/1462105375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f78a30 msgr2=0x55ae61f835b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a259c640 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f78a30 0x55ae61f835b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a259c640 1 -- 172.21.15.19:0/1462105375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f83c10 msgr2=0x55ae61f80e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a259c640 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f83c10 0x55ae61f80e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a259c640 1 -- 172.21.15.19:0/1462105375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae61cb8620 con 0x55ae61f79350 2024-04-23T18:38:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f50a259c640 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f808a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f509400ba20 tx=0x7f509400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d40389fb1b1467 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.688+0000 7f5089ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:38:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f5089ffb640 1 -- 172.21.15.19:0/1462105375 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509401b640 con 0x55ae61f79350 2024-04-23T18:38:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f5089ffb640 1 -- 172.21.15.19:0/1462105375 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f509401b7e0 con 0x55ae61f79350 2024-04-23T18:38:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f50abc4d740 1 -- 172.21.15.19:0/1462105375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50980041d0 con 0x55ae61f79350 2024-04-23T18:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f5089ffb640 1 -- 172.21.15.19:0/1462105375 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f509401b9e0 con 0x55ae61f79350 2024-04-23T18:38:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f50abc4d740 10 client.17103 fetch_fsmap learned FSMap version 27 2024-04-23T18:38:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f50abc4d740 1 -- 172.21.15.19:0/1462105375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae61f928a0 con 0x55ae61f79350 2024-04-23T18:38:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f5089ffb640 1 -- 172.21.15.19:0/1462105375 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50980041d0 con 0x55ae61f79350 2024-04-23T18:38:53.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f5089ffb640 1 -- 172.21.15.19:0/1462105375 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5094019020 con 0x55ae61f79350 2024-04-23T18:38:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f50abc4d740 10 client.17103 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:38:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f50abc4d740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f50abc4d740 20 client.17103 populate_metadata read hostname 'smithi019' 2024-04-23T18:38:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ae61f93db0 0x55ae61f961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f50abc4d740 4 client.17103 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:38:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.689+0000 7f50abc4d740 1 -- 172.21.15.19:0/1462105375 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae61ee7390 con 0x55ae61f93db0 2024-04-23T18:38:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.690+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ae61f93db0 0x55ae61f961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.690+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ae61f93db0 0x55ae61f961a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee163e56348116a server_cookie=c6308a2c20bf4d4 in_seq=0 out_seq=0 2024-04-23T18:38:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.690+0000 7f5089ffb640 10 client.17103 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f5089ffb640 1 -- 172.21.15.19:0/1462105375 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae61ee7390 con 0x55ae61f93db0 2024-04-23T18:38:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f5089ffb640 10 client.17103 handle_command_reply: tid=0 2024-04-23T18:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 client.17103 shutdown 2024-04-23T18:38:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 -- 172.21.15.19:0/1462105375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 msgr2=0x55ae61f808a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f808a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f509400ba20 tx=0x7f509400bee0 comp rx=0 tx=0).stop 2024-04-23T18:38:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 -- 172.21.15.19:0/1462105375 shutdown_connections 2024-04-23T18:38:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ae61f93db0 0x55ae61f961a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f83c10 0x55ae61f80e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f5089ffb640 0 client.17103 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f808a0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1462105375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f78a30 0x55ae61f835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 -- 172.21.15.19:0/1462105375 >> 172.21.15.19:0/1462105375 conn(0x55ae61f74f40 msgr2=0x55ae61f7acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 -- 172.21.15.19:0/1462105375 shutdown_connections 2024-04-23T18:38:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 1 -- 172.21.15.19:0/1462105375 wait complete. 2024-04-23T18:38:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.691+0000 7f50abc4d740 20 client.17103 trim_cache size 0 max 16384 2024-04-23T18:38:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50abc4d740 1 Processor -- start 2024-04-23T18:38:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50abc4d740 1 -- start start 2024-04-23T18:38:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f78a30 0x55ae61f8f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f79350 0x55ae61f7bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f83c10 0x55ae61f830e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50abc4d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41ab860 con 0x55ae61f83c10 2024-04-23T18:38:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50abc4d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41ad3a0 con 0x55ae61f79350 2024-04-23T18:38:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50abc4d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41abf30 con 0x55ae61f78a30 2024-04-23T18:38:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50a1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f83c10 0x55ae61f830e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50a1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f83c10 0x55ae61f830e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36920/0 (socket says 172.21.15.19:36920) 2024-04-23T18:38:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.706+0000 7f50a1d9b640 1 -- 172.21.15.19:0/3539710904 learned_addr learned my addr 172.21.15.19:0/3539710904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f50a1d9b640 1 -- 172.21.15.19:0/3539710904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f78a30 msgr2=0x55ae61f8f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/3539710904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f78a30 0x55ae61f8f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f50a1d9b640 1 -- 172.21.15.19:0/3539710904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f79350 msgr2=0x55ae61f7bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/3539710904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f79350 0x55ae61f7bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f50a1d9b640 1 -- 172.21.15.19:0/3539710904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae61f96510 con 0x55ae61f83c10 2024-04-23T18:38:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/3539710904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f83c10 0x55ae61f830e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5098013420 tx=0x7f509800de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ee90ba3508b25e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f508bfff640 1 -- 172.21.15.19:0/3539710904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50980041d0 con 0x55ae61f83c10 2024-04-23T18:38:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f508bfff640 1 -- 172.21.15.19:0/3539710904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50980054e0 con 0x55ae61f83c10 2024-04-23T18:38:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f508bfff640 1 -- 172.21.15.19:0/3539710904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509802a7a0 con 0x55ae61f83c10 2024-04-23T18:38:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f50abc4d740 1 -- 172.21.15.19:0/3539710904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f83c10 msgr2=0x55ae61f830e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.707+0000 7f50abc4d740 1 --2- 172.21.15.19:0/3539710904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f83c10 0x55ae61f830e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5098013420 tx=0x7f509800de90 comp rx=0 tx=0).stop 2024-04-23T18:38:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 -- 172.21.15.19:0/3539710904 shutdown_connections 2024-04-23T18:38:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 --2- 172.21.15.19:0/3539710904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f79350 0x55ae61f7bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 --2- 172.21.15.19:0/3539710904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f83c10 0x55ae61f830e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 --2- 172.21.15.19:0/3539710904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f78a30 0x55ae61f8f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 -- 172.21.15.19:0/3539710904 >> 172.21.15.19:0/3539710904 conn(0x55ae61f74f40 msgr2=0x55ae61f7b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 -- 172.21.15.19:0/3539710904 shutdown_connections 2024-04-23T18:38:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 -- 172.21.15.19:0/3539710904 wait complete. 2024-04-23T18:38:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 -- start start 2024-04-23T18:38:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 Processor -- start 2024-04-23T18:38:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f78a30 0x55ae61f8c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f87410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f83c10 0x55ae61f87a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41accd0 con 0x55ae61f79350 2024-04-23T18:38:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41ac600 con 0x55ae61f78a30 2024-04-23T18:38:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41a9860 con 0x55ae61f83c10 2024-04-23T18:38:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50abc4d740 10 client.0 _ll_register_callbacks cb 0x7f50940044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f87410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f87410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36924/0 (socket says 172.21.15.19:36924) 2024-04-23T18:38:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50a259c640 1 -- 172.21.15.19:0/1776777211 learned_addr learned my addr 172.21.15.19:0/1776777211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:38:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.708+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f83c10 0x55ae61f87a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f78a30 0x55ae61f8c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50a259c640 1 -- 172.21.15.19:0/1776777211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f83c10 msgr2=0x55ae61f87a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50a259c640 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f83c10 0x55ae61f87a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50a259c640 1 -- 172.21.15.19:0/1776777211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f78a30 msgr2=0x55ae61f8c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50a259c640 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f78a30 0x55ae61f8c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50a259c640 1 -- 172.21.15.19:0/1776777211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae61f79180 con 0x55ae61f79350 2024-04-23T18:38:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f78a30 0x55ae61f8c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:38:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50a259c640 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f87410 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x55ae61f80100 tx=0x7f509400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63314324e19ac40a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f5089ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:38:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f5089ffb640 1 -- 172.21.15.19:0/1776777211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094028070 con 0x55ae61f79350 2024-04-23T18:38:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f5089ffb640 1 -- 172.21.15.19:0/1776777211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f509401b440 con 0x55ae61f79350 2024-04-23T18:38:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50abc4d740 1 -- 172.21.15.19:0/1776777211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50980054e0 con 0x55ae61f79350 2024-04-23T18:38:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f5089ffb640 1 -- 172.21.15.19:0/1776777211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f509401b660 con 0x55ae61f79350 2024-04-23T18:38:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50abc4d740 10 client.17109 fetch_fsmap learned FSMap version 27 2024-04-23T18:38:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50abc4d740 1 -- 172.21.15.19:0/1776777211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae61ffa1e0 con 0x55ae61f79350 2024-04-23T18:38:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f5089ffb640 1 -- 172.21.15.19:0/1776777211 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50980054e0 con 0x55ae61f79350 2024-04-23T18:38:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f5089ffb640 1 -- 172.21.15.19:0/1776777211 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5094019040 con 0x55ae61f79350 2024-04-23T18:38:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50abc4d740 10 client.17109 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:38:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50abc4d740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.709+0000 7f50abc4d740 20 client.17109 populate_metadata read hostname 'smithi019' 2024-04-23T18:38:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.710+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ae61ffa930 0x55ae61ffcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.710+0000 7f50abc4d740 4 client.17109 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:38:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.710+0000 7f50abc4d740 1 -- 172.21.15.19:0/1776777211 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae61ee1200 con 0x55ae61ffa930 2024-04-23T18:38:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.710+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ae61ffa930 0x55ae61ffcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.710+0000 7f5089ffb640 10 client.17109 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.710+0000 7f50a1d9b640 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ae61ffa930 0x55ae61ffcd50 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73c81e18df413d3c server_cookie=1fa0e2f4f8a55bea in_seq=0 out_seq=0 2024-04-23T18:38:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f5089ffb640 1 -- 172.21.15.19:0/1776777211 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae61ee1200 con 0x55ae61ffa930 2024-04-23T18:38:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f5089ffb640 10 client.17109 handle_command_reply: tid=0 2024-04-23T18:38:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 client.17109 shutdown 2024-04-23T18:38:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 -- 172.21.15.19:0/1776777211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 msgr2=0x55ae61f87410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f87410 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x55ae61f80100 tx=0x7f509400b5e0 comp rx=0 tx=0).stop 2024-04-23T18:38:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 -- 172.21.15.19:0/1776777211 shutdown_connections 2024-04-23T18:38:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ae61ffa930 0x55ae61ffcd50 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae61f83c10 0x55ae61f87a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae61f79350 0x55ae61f87410 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 --2- 172.21.15.19:0/1776777211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae61f78a30 0x55ae61f8c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f5089ffb640 0 client.17109 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:38:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 -- 172.21.15.19:0/1776777211 >> 172.21.15.19:0/1776777211 conn(0x55ae61f74f40 msgr2=0x55ae61f6fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:59.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 -- 172.21.15.19:0/1776777211 shutdown_connections 2024-04-23T18:38:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 1 -- 172.21.15.19:0/1776777211 wait complete. 2024-04-23T18:38:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.725+0000 7f50abc4d740 20 client.17109 trim_cache size 0 max 16384 2024-04-23T18:38:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:38:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:38:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:38:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:39:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 -- 172.21.15.19:0/3698944981 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f508c0a3dc0 msgr2=0x7f508c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f508c0a3dc0 0x7f508c0a6280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f509c0ba700 tx=0x7f509c0095c0 comp rx=0 tx=0).stop 2024-04-23T18:39:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 -- 172.21.15.19:0/3698944981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50a419cf00 msgr2=0x7f50a419d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50a419cf00 0x7f50a419d3b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f509c009430 tx=0x7f509c0319f0 comp rx=0 tx=0).stop 2024-04-23T18:39:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 -- 172.21.15.19:0/3698944981 shutdown_connections 2024-04-23T18:39:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f508c0a3dc0 0x7f508c0a6280 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50a419cf00 0x7f50a419d3b0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50a41061b0 0x7f50a4198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 --2- 172.21.15.19:0/3698944981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a41010e0 0x7f50a4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 -- 172.21.15.19:0/3698944981 >> 172.21.15.19:0/3698944981 conn(0x7f50a40771d0 msgr2=0x7f50a406a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 -- 172.21.15.19:0/3698944981 shutdown_connections 2024-04-23T18:39:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:37:47.726+0000 7f50a8f37640 1 -- 172.21.15.19:0/3698944981 wait complete. 2024-04-23T18:39:00.539 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:47 smithi019 ceph-mon[29215]: pgmap v362: 97 pgs: 97 active+clean; 5.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:00.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:49 smithi019 ceph-mon[29215]: pgmap v363: 97 pgs: 97 active+clean; 5.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:00.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:51 smithi019 ceph-mon[29215]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-23T18:39:00.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:53 smithi019 ceph-mon[29215]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-23T18:39:00.829 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:55 smithi019 ceph-mon[29215]: pgmap v366: 97 pgs: 97 active+clean; 1.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:39:00.870 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:57 smithi019 ceph-mon[29215]: pgmap v367: 97 pgs: 97 active+clean; 1.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:01.002 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:37:59 smithi019 ceph-mon[29215]: pgmap v368: 97 pgs: 97 active+clean; 1.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:01.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:47 smithi155 ceph-mon[33468]: pgmap v394: 97 pgs: 97 active+clean; 3.8 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:01.312 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:49 smithi155 ceph-mon[33468]: pgmap v395: 97 pgs: 97 active+clean; 3.8 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:01.370 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:51 smithi155 ceph-mon[33468]: pgmap v396: 97 pgs: 97 active+clean; 3.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:39:01.404 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:53 smithi155 ceph-mon[33468]: pgmap v397: 97 pgs: 97 active+clean; 3.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:39:01.437 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:55 smithi155 ceph-mon[33468]: pgmap v398: 97 pgs: 97 active+clean; 4.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-23T18:39:01.562 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:57 smithi155 ceph-mon[33468]: pgmap v399: 97 pgs: 97 active+clean; 4.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:01.596 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:38:59 smithi155 ceph-mon[33468]: pgmap v400: 97 pgs: 97 active+clean; 4.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:01.720 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:47 smithi044 ceph-mon[32458]: pgmap v394: 97 pgs: 97 active+clean; 3.8 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:01.746 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:49 smithi044 ceph-mon[32458]: pgmap v395: 97 pgs: 97 active+clean; 3.8 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:01.779 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:51 smithi044 ceph-mon[32458]: pgmap v396: 97 pgs: 97 active+clean; 3.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:39:01.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:53 smithi044 ceph-mon[32458]: pgmap v397: 97 pgs: 97 active+clean; 3.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:39:01.846 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:55 smithi044 ceph-mon[32458]: pgmap v398: 97 pgs: 97 active+clean; 4.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-23T18:39:01.879 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:57 smithi044 ceph-mon[32458]: pgmap v399: 97 pgs: 97 active+clean; 4.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:01.913 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:38:59 smithi044 ceph-mon[32458]: pgmap v400: 97 pgs: 97 active+clean; 4.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 -- 172.21.15.19:0/2527016986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ac1021d0 msgr2=0x7f52ac1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 --2- 172.21.15.19:0/2527016986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ac1021d0 0x7f52ac1025b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009900 tx=0x7f52a802a410 comp rx=0 tx=0).stop 2024-04-23T18:39:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 -- 172.21.15.19:0/2527016986 shutdown_connections 2024-04-23T18:39:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 --2- 172.21.15.19:0/2527016986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52ac102b80 0x7f52ac106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 --2- 172.21.15.19:0/2527016986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52ac1072a0 0x7f52ac10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 --2- 172.21.15.19:0/2527016986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ac1021d0 0x7f52ac1025b0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 -- 172.21.15.19:0/2527016986 >> 172.21.15.19:0/2527016986 conn(0x7f52ac0793e0 msgr2=0x7f52ac0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 -- 172.21.15.19:0/2527016986 shutdown_connections 2024-04-23T18:39:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 -- 172.21.15.19:0/2527016986 wait complete. 2024-04-23T18:39:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 Processor -- start 2024-04-23T18:39:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.827+0000 7f52b3c59640 1 -- start start 2024-04-23T18:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b3c59640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52ac1021d0 0x7f52ac1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b3c59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ac1072a0 0x7f52ac1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b3c59640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52ac19a6e0 0x7f52ac19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b3c59640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac112230 con 0x7f52ac1072a0 2024-04-23T18:39:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b3c59640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1120b0 con 0x7f52ac19a6e0 2024-04-23T18:39:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b3c59640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1123b0 con 0x7f52ac1021d0 2024-04-23T18:39:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b11f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ac1072a0 0x7f52ac1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b11f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ac1072a0 0x7f52ac1a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36486/0 (socket says 172.21.15.19:36486) 2024-04-23T18:39:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b11f4640 1 -- 172.21.15.19:0/2979936632 learned_addr learned my addr 172.21.15.19:0/2979936632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b21f6640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52ac19a6e0 0x7f52ac19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b19f5640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52ac1021d0 0x7f52ac1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b11f4640 1 -- 172.21.15.19:0/2979936632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52ac1021d0 msgr2=0x7f52ac1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b11f4640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52ac1021d0 0x7f52ac1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b11f4640 1 -- 172.21.15.19:0/2979936632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52ac19a6e0 msgr2=0x7f52ac19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b11f4640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52ac19a6e0 0x7f52ac19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b11f4640 1 -- 172.21.15.19:0/2979936632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac19b2e0 con 0x7f52ac1072a0 2024-04-23T18:39:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.828+0000 7f52b11f4640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ac1072a0 0x7f52ac1a0b60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f529c00e9c0 tx=0x7f529c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.829+0000 7f52a2ffd640 1 -- 172.21.15.19:0/2979936632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c00cd80 con 0x7f52ac1072a0 2024-04-23T18:39:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.829+0000 7f52a2ffd640 1 -- 172.21.15.19:0/2979936632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c0054d0 con 0x7f52ac1072a0 2024-04-23T18:39:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.829+0000 7f52b3c59640 1 -- 172.21.15.19:0/2979936632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac1a73c0 con 0x7f52ac1072a0 2024-04-23T18:39:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.829+0000 7f52a2ffd640 1 -- 172.21.15.19:0/2979936632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c010430 con 0x7f52ac1072a0 2024-04-23T18:39:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.829+0000 7f52b3c59640 1 -- 172.21.15.19:0/2979936632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac1a77c0 con 0x7f52ac1072a0 2024-04-23T18:39:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.831+0000 7f52a2ffd640 1 -- 172.21.15.19:0/2979936632 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f529c004fd0 con 0x7f52ac1072a0 2024-04-23T18:39:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.831+0000 7f52b696e740 1 Processor -- start 2024-04-23T18:39:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b696e740 1 -- start start 2024-04-23T18:39:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b696e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4525a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b696e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4525f40 0x562fa45302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b696e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4530800 0x562fa4532bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b696e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562fa448a970 con 0x562fa4525620 2024-04-23T18:39:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b696e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562fa44274a0 con 0x562fa4530800 2024-04-23T18:39:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b696e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562fa4486930 con 0x562fa4525f40 2024-04-23T18:39:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52a2ffd640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f52980a3dc0 0x7f52980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b21f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4525a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b21f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4525a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36492/0 (socket says 172.21.15.19:36492) 2024-04-23T18:39:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b21f6640 1 -- 172.21.15.19:0/2787981056 learned_addr learned my addr 172.21.15.19:0/2787981056 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b21f6640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f52980a3dc0 0x7f52980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b19f5640 1 --2- 172.21.15.19:0/2787981056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4525f40 0x562fa45302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52a2ffd640 1 -- 172.21.15.19:0/2979936632 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f529c0d0c00 con 0x7f52ac1072a0 2024-04-23T18:39:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.832+0000 7f52b11f4640 1 --2- 172.21.15.19:0/2787981056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4530800 0x562fa4532bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.833+0000 7f52b21f6640 1 -- 172.21.15.19:0/2787981056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4525f40 msgr2=0x562fa45302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.833+0000 7f52b21f6640 1 --2- 172.21.15.19:0/2787981056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4525f40 0x562fa45302c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:05.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.833+0000 7f52b21f6640 1 -- 172.21.15.19:0/2787981056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4530800 msgr2=0x562fa4532bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.833+0000 7f52b21f6640 1 --2- 172.21.15.19:0/2787981056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4530800 0x562fa4532bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.833+0000 7f52b21f6640 1 -- 172.21.15.19:0/2787981056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa419d210 con 0x562fa4525620 2024-04-23T18:39:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.833+0000 7f52b21f6640 1 --2- 172.21.15.19:0/2787981056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4525a00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f52a4014e70 tx=0x7f52a40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3500c82f38aae5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.833+0000 7f52b21f6640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f52980a3dc0 0x7f52980a6280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52a4011cb0 tx=0x7f52a4016000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.833+0000 7f5297fff640 1 -- 172.21.15.19:0/2787981056 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4014050 con 0x562fa4525620 2024-04-23T18:39:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.833+0000 7f5297fff640 1 -- 172.21.15.19:0/2787981056 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a4007c50 con 0x562fa4525620 2024-04-23T18:39:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.833+0000 7f5297fff640 1 -- 172.21.15.19:0/2787981056 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40055d0 con 0x562fa4525620 2024-04-23T18:39:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 -- 172.21.15.19:0/2787981056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 msgr2=0x562fa4525a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 --2- 172.21.15.19:0/2787981056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4525a00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f52a4014e70 tx=0x7f52a40117b0 comp rx=0 tx=0).stop 2024-04-23T18:39:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 -- 172.21.15.19:0/2787981056 shutdown_connections 2024-04-23T18:39:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 --2- 172.21.15.19:0/2787981056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4530800 0x562fa4532bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 --2- 172.21.15.19:0/2787981056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4525f40 0x562fa45302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 --2- 172.21.15.19:0/2787981056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4525a00 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 -- 172.21.15.19:0/2787981056 >> 172.21.15.19:0/2787981056 conn(0x562fa4521b30 msgr2=0x562fa4521f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 -- 172.21.15.19:0/2787981056 shutdown_connections 2024-04-23T18:39:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 -- 172.21.15.19:0/2787981056 wait complete. 2024-04-23T18:39:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 -- start start 2024-04-23T18:39:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.834+0000 7f52b696e740 1 Processor -- start 2024-04-23T18:39:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b696e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4530200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b696e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525f40 0x562fa452d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b696e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 0x562fa452db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b696e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562fa45333d0 con 0x562fa4525620 2024-04-23T18:39:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b696e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562fa44250b0 con 0x562fa4525f40 2024-04-23T18:39:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b696e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562fa4533550 con 0x562fa4530800 2024-04-23T18:39:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b696e740 10 client.0 _ll_register_callbacks cb 0x562fa4412d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b19f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4530200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b19f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4530200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36496/0 (socket says 172.21.15.19:36496) 2024-04-23T18:39:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b19f5640 1 -- 172.21.15.19:0/2796330026 learned_addr learned my addr 172.21.15.19:0/2796330026 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b11f4640 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525f40 0x562fa452d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b21f6640 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 0x562fa452db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b19f5640 1 -- 172.21.15.19:0/2796330026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 msgr2=0x562fa452db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b19f5640 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 0x562fa452db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b19f5640 1 -- 172.21.15.19:0/2796330026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525f40 msgr2=0x562fa452d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b19f5640 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525f40 0x562fa452d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b19f5640 1 -- 172.21.15.19:0/2796330026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa4269620 con 0x562fa4525620 2024-04-23T18:39:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.835+0000 7f52b21f6640 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 0x562fa452db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52b19f5640 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4530200 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009230 tx=0x7f52a8004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f774eed85e1b143 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52957fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:39:07.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52957fa640 1 -- 172.21.15.19:0/2796330026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8035040 con 0x562fa4525620 2024-04-23T18:39:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52957fa640 1 -- 172.21.15.19:0/2796330026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f52a8004770 con 0x562fa4525620 2024-04-23T18:39:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52b696e740 1 -- 172.21.15.19:0/2796330026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a4007c50 con 0x562fa4525620 2024-04-23T18:39:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52957fa640 1 -- 172.21.15.19:0/2796330026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f52a8005610 con 0x562fa4525620 2024-04-23T18:39:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52b696e740 10 client.17121 fetch_fsmap learned FSMap version 27 2024-04-23T18:39:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52b696e740 1 -- 172.21.15.19:0/2796330026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562fa453f380 con 0x562fa4525620 2024-04-23T18:39:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52957fa640 1 -- 172.21.15.19:0/2796330026 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a4007c50 con 0x562fa4525620 2024-04-23T18:39:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52957fa640 1 -- 172.21.15.19:0/2796330026 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f52a8003c60 con 0x562fa4525620 2024-04-23T18:39:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52b696e740 10 client.17121 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:39:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52b696e740 10 client.17121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.836+0000 7f52b696e740 20 client.17121 populate_metadata read hostname 'smithi019' 2024-04-23T18:39:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.837+0000 7f52b696e740 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562fa45407f0 0x562fa4542be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.837+0000 7f52b696e740 4 client.17121 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:39:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.837+0000 7f52b696e740 1 -- 172.21.15.19:0/2796330026 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fa4498390 con 0x562fa45407f0 2024-04-23T18:39:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.837+0000 7f52b11f4640 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562fa45407f0 0x562fa4542be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.837+0000 7f52957fa640 10 client.17121 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.837+0000 7f52b11f4640 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562fa45407f0 0x562fa4542be0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=190a06265e0bd5f7 server_cookie=4d41e1098b343f17 in_seq=0 out_seq=0 2024-04-23T18:39:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52957fa640 1 -- 172.21.15.19:0/2796330026 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562fa4498390 con 0x562fa45407f0 2024-04-23T18:39:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52957fa640 10 client.17121 handle_command_reply: tid=0 2024-04-23T18:39:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 client.17121 shutdown 2024-04-23T18:39:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 -- 172.21.15.19:0/2796330026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 msgr2=0x562fa4530200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4530200 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009230 tx=0x7f52a8004580 comp rx=0 tx=0).stop 2024-04-23T18:39:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 -- 172.21.15.19:0/2796330026 shutdown_connections 2024-04-23T18:39:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562fa45407f0 0x562fa4542be0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525f40 0x562fa452d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 0x562fa452db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 --2- 172.21.15.19:0/2796330026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525620 0x562fa4530200 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52957fa640 0 client.17121 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 -- 172.21.15.19:0/2796330026 >> 172.21.15.19:0/2796330026 conn(0x562fa4521b30 msgr2=0x562fa4527860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 -- 172.21.15.19:0/2796330026 shutdown_connections 2024-04-23T18:39:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 1 -- 172.21.15.19:0/2796330026 wait complete. 2024-04-23T18:39:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.838+0000 7f52b696e740 20 client.17121 trim_cache size 0 max 16384 2024-04-23T18:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.853+0000 7f52b696e740 1 Processor -- start 2024-04-23T18:39:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.853+0000 7f52b696e740 1 -- start start 2024-04-23T18:39:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b696e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4525620 0x562fa453c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b696e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525f40 0x562fa4520b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b696e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4530800 0x562fa452fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b696e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1afc60 con 0x562fa4530800 2024-04-23T18:39:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b696e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1b17a0 con 0x562fa4525f40 2024-04-23T18:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b696e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1b0330 con 0x562fa4525620 2024-04-23T18:39:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b21f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4530800 0x562fa452fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b21f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4530800 0x562fa452fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36498/0 (socket says 172.21.15.19:36498) 2024-04-23T18:39:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b21f6640 1 -- 172.21.15.19:0/1606672111 learned_addr learned my addr 172.21.15.19:0/1606672111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b11f4640 1 --2- 172.21.15.19:0/1606672111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525f40 0x562fa4520b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b19f5640 1 --2- 172.21.15.19:0/1606672111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4525620 0x562fa453c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b21f6640 1 -- 172.21.15.19:0/1606672111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4525620 msgr2=0x562fa453c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b21f6640 1 --2- 172.21.15.19:0/1606672111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4525620 0x562fa453c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b21f6640 1 -- 172.21.15.19:0/1606672111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525f40 msgr2=0x562fa4520b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b21f6640 1 --2- 172.21.15.19:0/1606672111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525f40 0x562fa4520b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b21f6640 1 -- 172.21.15.19:0/1606672111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa452ba60 con 0x562fa4530800 2024-04-23T18:39:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52b21f6640 1 --2- 172.21.15.19:0/1606672111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4530800 0x562fa452fd30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f52a4014e70 tx=0x7f52a4007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c33af1019866ada server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52977fe640 1 -- 172.21.15.19:0/1606672111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4014050 con 0x562fa4530800 2024-04-23T18:39:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52977fe640 1 -- 172.21.15.19:0/1606672111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a4003e20 con 0x562fa4530800 2024-04-23T18:39:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.854+0000 7f52977fe640 1 -- 172.21.15.19:0/1606672111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40054d0 con 0x562fa4530800 2024-04-23T18:39:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 -- 172.21.15.19:0/1606672111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4530800 msgr2=0x562fa452fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 --2- 172.21.15.19:0/1606672111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4530800 0x562fa452fd30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f52a4014e70 tx=0x7f52a4007990 comp rx=0 tx=0).stop 2024-04-23T18:39:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 -- 172.21.15.19:0/1606672111 shutdown_connections 2024-04-23T18:39:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 --2- 172.21.15.19:0/1606672111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525f40 0x562fa4520b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 --2- 172.21.15.19:0/1606672111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4530800 0x562fa452fd30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 --2- 172.21.15.19:0/1606672111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4525620 0x562fa453c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 -- 172.21.15.19:0/1606672111 >> 172.21.15.19:0/1606672111 conn(0x562fa4521b30 msgr2=0x562fa4520420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 -- 172.21.15.19:0/1606672111 shutdown_connections 2024-04-23T18:39:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 -- 172.21.15.19:0/1606672111 wait complete. 2024-04-23T18:39:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 -- start start 2024-04-23T18:39:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.855+0000 7f52b696e740 1 Processor -- start 2024-04-23T18:39:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b696e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525620 0x562fa452a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b696e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525f40 0x562fa452a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b696e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 0x562fa452ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b696e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1b10d0 con 0x562fa4525f40 2024-04-23T18:39:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b696e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1b0a00 con 0x562fa4525620 2024-04-23T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b696e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1adc60 con 0x562fa4530800 2024-04-23T18:39:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b696e740 10 client.0 _ll_register_callbacks cb 0x7f52a8004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b11f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525f40 0x562fa452a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b11f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525f40 0x562fa452a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36504/0 (socket says 172.21.15.19:36504) 2024-04-23T18:39:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b11f4640 1 -- 172.21.15.19:0/298165245 learned_addr learned my addr 172.21.15.19:0/298165245 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b19f5640 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525620 0x562fa452a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b11f4640 1 -- 172.21.15.19:0/298165245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 msgr2=0x562fa452ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b21f6640 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 0x562fa452ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b11f4640 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 0x562fa452ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b11f4640 1 -- 172.21.15.19:0/298165245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525620 msgr2=0x562fa452a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b11f4640 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525620 0x562fa452a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b11f4640 1 -- 172.21.15.19:0/298165245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa45a5e50 con 0x562fa4525f40 2024-04-23T18:39:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b19f5640 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525620 0x562fa452a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:39:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.856+0000 7f52b21f6640 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 0x562fa452ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:39:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52b11f4640 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525f40 0x562fa452a8e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f529c099570 tx=0x7f529c0bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53249fb7c67cbd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52957fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:39:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52957fa640 1 -- 172.21.15.19:0/298165245 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c09cba0 con 0x562fa4525f40 2024-04-23T18:39:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52957fa640 1 -- 172.21.15.19:0/298165245 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f529c0bd0b0 con 0x562fa4525f40 2024-04-23T18:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52b696e740 1 -- 172.21.15.19:0/298165245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a4003e20 con 0x562fa4525f40 2024-04-23T18:39:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52957fa640 1 -- 172.21.15.19:0/298165245 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f529c0a0f50 con 0x562fa4525f40 2024-04-23T18:39:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52b696e740 10 client.17127 fetch_fsmap learned FSMap version 27 2024-04-23T18:39:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52b696e740 1 -- 172.21.15.19:0/298165245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562fa45a71a0 con 0x562fa4525f40 2024-04-23T18:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52957fa640 1 -- 172.21.15.19:0/298165245 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a4003e20 con 0x562fa4525f40 2024-04-23T18:39:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52957fa640 1 -- 172.21.15.19:0/298165245 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f529c0b59d0 con 0x562fa4525f40 2024-04-23T18:39:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52b696e740 10 client.17127 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:39:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52b696e740 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52b696e740 20 client.17127 populate_metadata read hostname 'smithi019' 2024-04-23T18:39:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52b696e740 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562fa45a78d0 0x562fa45a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52b696e740 4 client.17127 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:39:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.857+0000 7f52b696e740 1 -- 172.21.15.19:0/298165245 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fa4492200 con 0x562fa45a78d0 2024-04-23T18:39:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.858+0000 7f52b19f5640 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562fa45a78d0 0x562fa45a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.858+0000 7f52b19f5640 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562fa45a78d0 0x562fa45a9cf0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e6ad615b670ee47 server_cookie=29a4420a9747248c in_seq=0 out_seq=0 2024-04-23T18:39:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.858+0000 7f52957fa640 10 client.17127 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.873+0000 7f52957fa640 1 -- 172.21.15.19:0/298165245 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562fa4492200 con 0x562fa45a78d0 2024-04-23T18:39:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.873+0000 7f52957fa640 10 client.17127 handle_command_reply: tid=0 2024-04-23T18:39:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.873+0000 7f52b696e740 1 client.17127 shutdown 2024-04-23T18:39:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.873+0000 7f52b696e740 1 -- 172.21.15.19:0/298165245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525f40 msgr2=0x562fa452a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.873+0000 7f52b696e740 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525f40 0x562fa452a8e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f529c099570 tx=0x7f529c0bd040 comp rx=0 tx=0).stop 2024-04-23T18:39:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b696e740 1 -- 172.21.15.19:0/298165245 shutdown_connections 2024-04-23T18:39:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b696e740 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562fa45a78d0 0x562fa45a9cf0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b696e740 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fa4530800 0x562fa452ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b696e740 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fa4525f40 0x562fa452a8e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b696e740 1 --2- 172.21.15.19:0/298165245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562fa4525620 0x562fa452a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52957fa640 0 client.17127 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b696e740 1 -- 172.21.15.19:0/298165245 >> 172.21.15.19:0/298165245 conn(0x562fa4521b30 msgr2=0x562fa4527860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:13.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b696e740 1 -- 172.21.15.19:0/298165245 shutdown_connections 2024-04-23T18:39:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b696e740 1 -- 172.21.15.19:0/298165245 wait complete. 2024-04-23T18:39:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b696e740 20 client.17127 trim_cache size 0 max 16384 2024-04-23T18:39:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:39:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:39:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:39:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:39:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b3c59640 1 -- 172.21.15.19:0/2979936632 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f52980a3dc0 msgr2=0x7f52980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b3c59640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f52980a3dc0 0x7f52980a6280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52a4011cb0 tx=0x7f52a4016000 comp rx=0 tx=0).stop 2024-04-23T18:39:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b3c59640 1 -- 172.21.15.19:0/2979936632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ac1072a0 msgr2=0x7f52ac1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b3c59640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ac1072a0 0x7f52ac1a0b60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f529c00e9c0 tx=0x7f529c00ee80 comp rx=0 tx=0).stop 2024-04-23T18:39:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b3c59640 1 -- 172.21.15.19:0/2979936632 shutdown_connections 2024-04-23T18:39:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b3c59640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f52980a3dc0 0x7f52980a6280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b3c59640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52ac19a6e0 0x7f52ac19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b3c59640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52ac1072a0 0x7f52ac1a0b60 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b3c59640 1 --2- 172.21.15.19:0/2979936632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52ac1021d0 0x7f52ac1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.874+0000 7f52b3c59640 1 -- 172.21.15.19:0/2979936632 >> 172.21.15.19:0/2979936632 conn(0x7f52ac0793e0 msgr2=0x7f52ac0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.875+0000 7f52b3c59640 1 -- 172.21.15.19:0/2979936632 shutdown_connections 2024-04-23T18:39:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:00.875+0000 7f52b3c59640 1 -- 172.21.15.19:0/2979936632 wait complete. 2024-04-23T18:39:14.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:01 smithi019 ceph-mon[29215]: pgmap v369: 97 pgs: 97 active+clean; 1.8 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:39:14.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:03 smithi019 ceph-mon[29215]: pgmap v370: 97 pgs: 97 active+clean; 1.8 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T18:39:14.471 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:01 smithi155 ceph-mon[33468]: pgmap v401: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:39:14.550 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:03 smithi155 ceph-mon[33468]: pgmap v402: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:14.578 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:05 smithi155 ceph-mon[33468]: pgmap v403: 97 pgs: 97 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T18:39:14.612 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:06 smithi155 ceph-mon[33468]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T18:39:14.645 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:07 smithi155 ceph-mon[33468]: pgmap v405: 97 pgs: 97 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-04-23T18:39:14.679 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:09 smithi155 ceph-mon[33468]: pgmap v406: 97 pgs: 97 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-04-23T18:39:14.770 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:11 smithi155 ceph-mon[33468]: pgmap v407: 97 pgs: 97 active+clean; 4.9 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:39:14.803 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:13 smithi155 ceph-mon[33468]: pgmap v408: 97 pgs: 97 active+clean; 4.9 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:39:14.837 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:13 smithi155 ceph-mon[33468]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T18:39:14.870 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:01 smithi044 ceph-mon[32458]: pgmap v401: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:39:14.904 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:03 smithi044 ceph-mon[32458]: pgmap v402: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:39:14.995 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:05 smithi044 ceph-mon[32458]: pgmap v403: 97 pgs: 97 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T18:39:15.067 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:06 smithi044 ceph-mon[32458]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T18:39:15.124 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:07 smithi044 ceph-mon[32458]: pgmap v405: 97 pgs: 97 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-04-23T18:39:15.162 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:09 smithi044 ceph-mon[32458]: pgmap v406: 97 pgs: 97 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-04-23T18:39:15.195 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:11 smithi044 ceph-mon[32458]: pgmap v407: 97 pgs: 97 active+clean; 4.9 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:39:15.271 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:13 smithi044 ceph-mon[32458]: pgmap v408: 97 pgs: 97 active+clean; 4.9 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:39:15.304 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:13 smithi044 ceph-mon[32458]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T18:39:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.393+0000 7fcc35472640 1 -- 172.21.15.19:0/529475700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc30102240 msgr2=0x7fcc30102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:15.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.393+0000 7fcc35472640 1 --2- 172.21.15.19:0/529475700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc30102240 0x7fcc30102620 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009900 tx=0x7fcc2002a410 comp rx=0 tx=0).stop 2024-04-23T18:39:15.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.393+0000 7fcc35472640 1 -- 172.21.15.19:0/529475700 shutdown_connections 2024-04-23T18:39:15.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.393+0000 7fcc35472640 1 --2- 172.21.15.19:0/529475700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30102bf0 0x7fcc30106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.393+0000 7fcc35472640 1 --2- 172.21.15.19:0/529475700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc30107280 0x7fcc3010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.393+0000 7fcc35472640 1 --2- 172.21.15.19:0/529475700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc30102240 0x7fcc30102620 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:15.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.393+0000 7fcc35472640 1 -- 172.21.15.19:0/529475700 >> 172.21.15.19:0/529475700 conn(0x7fcc300790e0 msgr2=0x7fcc300794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:15.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc35472640 1 -- 172.21.15.19:0/529475700 shutdown_connections 2024-04-23T18:39:15.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc35472640 1 -- 172.21.15.19:0/529475700 wait complete. 2024-04-23T18:39:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc35472640 1 Processor -- start 2024-04-23T18:39:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc35472640 1 -- start start 2024-04-23T18:39:15.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc35472640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30102240 0x7fcc30198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:15.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc35472640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc30107280 0x7fcc30198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc35472640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc3019d010 0x7fcc3019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:15.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc35472640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc30112210 con 0x7fcc30107280 2024-04-23T18:39:15.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc35472640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc30112090 con 0x7fcc30102240 2024-04-23T18:39:15.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc35472640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc30112390 con 0x7fcc3019d010 2024-04-23T18:39:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.394+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc30107280 0x7fcc30198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:16.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc30107280 0x7fcc30198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36514/0 (socket says 172.21.15.19:36514) 2024-04-23T18:39:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc27fff640 1 -- 172.21.15.19:0/2796105966 learned_addr learned my addr 172.21.15.19:0/2796105966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc2f7fe640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc3019d010 0x7fcc3019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:16.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc2effd640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30102240 0x7fcc30198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:16.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc27fff640 1 -- 172.21.15.19:0/2796105966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc3019d010 msgr2=0x7fcc3019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:16.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc27fff640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc3019d010 0x7fcc3019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:16.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc27fff640 1 -- 172.21.15.19:0/2796105966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30102240 msgr2=0x7fcc30198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc27fff640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30102240 0x7fcc30198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:16.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc27fff640 1 -- 172.21.15.19:0/2796105966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3019db00 con 0x7fcc30107280 2024-04-23T18:39:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc2f7fe640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc3019d010 0x7fcc3019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc2effd640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30102240 0x7fcc30198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:16.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc27fff640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc30107280 0x7fcc30198c90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800e9f0 tx=0x7fcc1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc2cff9640 1 -- 172.21.15.19:0/2796105966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1800ccc0 con 0x7fcc30107280 2024-04-23T18:39:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc35472640 1 -- 172.21.15.19:0/2796105966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3019ddf0 con 0x7fcc30107280 2024-04-23T18:39:16.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc2cff9640 1 -- 172.21.15.19:0/2796105966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc180108e0 con 0x7fcc30107280 2024-04-23T18:39:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc2cff9640 1 -- 172.21.15.19:0/2796105966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18018860 con 0x7fcc30107280 2024-04-23T18:39:16.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.395+0000 7fcc35472640 1 -- 172.21.15.19:0/2796105966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc301a5630 con 0x7fcc30107280 2024-04-23T18:39:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.396+0000 7fcc35472640 1 -- 172.21.15.19:0/2796105966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc300691c0 con 0x7fcc30107280 2024-04-23T18:39:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.398+0000 7fcc2cff9640 1 -- 172.21.15.19:0/2796105966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcc18010430 con 0x7fcc30107280 2024-04-23T18:39:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.398+0000 7fcc2cff9640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc1c0a3d20 0x7fcc1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.398+0000 7fcc2effd640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc1c0a3d20 0x7fcc1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:17.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.398+0000 7fcc2cff9640 1 -- 172.21.15.19:0/2796105966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fcc180d0bf0 con 0x7fcc30107280 2024-04-23T18:39:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.399+0000 7fcc2effd640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc1c0a3d20 0x7fcc1c0a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc20031440 tx=0x7fcc20005a20 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.403+0000 7fcc2cff9640 1 -- 172.21.15.19:0/2796105966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc1809d290 con 0x7fcc30107280 2024-04-23T18:39:17.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.628+0000 7fcc35472640 1 -- 172.21.15.19:0/2796105966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc30102bf0 con 0x7fcc30107280 2024-04-23T18:39:17.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.629+0000 7fcc2cff9640 1 -- 172.21.15.19:0/2796105966 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7fcc1809cb90 con 0x7fcc30107280 2024-04-23T18:39:17.372 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:39:17.422 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:26:06.874631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/659103876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":659103876},{"type":"v1","addr":"172.21.15.44:6835","nonce":659103876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:39:17.474 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-23T18:39:17.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 -- 172.21.15.19:0/2796105966 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc1c0a3d20 msgr2=0x7fcc1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:17.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc1c0a3d20 0x7fcc1c0a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc20031440 tx=0x7fcc20005a20 comp rx=0 tx=0).stop 2024-04-23T18:39:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 -- 172.21.15.19:0/2796105966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc30107280 msgr2=0x7fcc30198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc30107280 0x7fcc30198c90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800e9f0 tx=0x7fcc1800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:39:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 -- 172.21.15.19:0/2796105966 shutdown_connections 2024-04-23T18:39:17.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc1c0a3d20 0x7fcc1c0a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:17.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc3019d010 0x7fcc3019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:18.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc30107280 0x7fcc30198c90 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:18.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 --2- 172.21.15.19:0/2796105966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30102240 0x7fcc30198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 -- 172.21.15.19:0/2796105966 >> 172.21.15.19:0/2796105966 conn(0x7fcc300790e0 msgr2=0x7fcc301003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 -- 172.21.15.19:0/2796105966 shutdown_connections 2024-04-23T18:39:18.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:38:05.631+0000 7fcc35472640 1 -- 172.21.15.19:0/2796105966 wait complete. 2024-04-23T18:39:18.273 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db / --timeout 60 --expiration 91.5 --await-for 42.0' 2024-04-23T18:39:18.331 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db / --timeout 60 --expiration 91.5 --await-for 42.0 2024-04-23T18:39:18.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:05 smithi019 ceph-mon[29215]: pgmap v371: 97 pgs: 97 active+clean; 2.1 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 106 KiB/s wr, 39 op/s 2024-04-23T18:39:18.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:05 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2796105966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:39:18.487 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:06 smithi019 ceph-mon[29215]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:39:18.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:07 smithi019 ceph-mon[29215]: pgmap v373: 97 pgs: 97 active+clean; 2.1 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T18:39:18.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:09 smithi019 ceph-mon[29215]: pgmap v374: 97 pgs: 97 active+clean; 2.1 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T18:39:18.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:11 smithi019 ceph-mon[29215]: pgmap v375: 97 pgs: 97 active+clean; 2.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 38 op/s 2024-04-23T18:39:18.740 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:15 smithi155 ceph-mon[33468]: pgmap v410: 97 pgs: 97 active+clean; 5.1 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:39:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:17 smithi155 ceph-mon[33468]: pgmap v411: 97 pgs: 97 active+clean; 5.1 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-23T18:39:18.839 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:15 smithi044 ceph-mon[32458]: pgmap v410: 97 pgs: 97 active+clean; 5.1 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:39:18.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:17 smithi044 ceph-mon[32458]: pgmap v411: 97 pgs: 97 active+clean; 5.1 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-23T18:39:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.975+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2671161813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dcc1031e0 msgr2=0x7f9dcc1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.975+0000 7f9dd19fe640 1 --2- 172.21.15.19:0/2671161813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dcc1031e0 0x7f9dcc1035c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009900 tx=0x7f9dbc02a510 comp rx=0 tx=0).stop 2024-04-23T18:39:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.975+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2671161813 shutdown_connections 2024-04-23T18:39:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.975+0000 7f9dd19fe640 1 --2- 172.21.15.19:0/2671161813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc103b00 0x7f9dcc10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.975+0000 7f9dd19fe640 1 --2- 172.21.15.19:0/2671161813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dcc10c520 0x7f9dcc10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.975+0000 7f9dd19fe640 1 --2- 172.21.15.19:0/2671161813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dcc1031e0 0x7f9dcc1035c0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.975+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2671161813 >> 172.21.15.19:0/2671161813 conn(0x7f9dcc077fd0 msgr2=0x7f9dcc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.975+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2671161813 shutdown_connections 2024-04-23T18:39:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.975+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2671161813 wait complete. 2024-04-23T18:39:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.975+0000 7f9dd19fe640 1 Processor -- start 2024-04-23T18:39:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dd19fe640 1 -- start start 2024-04-23T18:39:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dd19fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dcc1031e0 0x7f9dcc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dd19fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc103b00 0x7f9dcc198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dd19fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dcc10c520 0x7f9dcc19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dd19fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc112200 con 0x7f9dcc1031e0 2024-04-23T18:39:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dd19fe640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc112080 con 0x7f9dcc103b00 2024-04-23T18:39:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dd19fe640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc112380 con 0x7f9dcc10c520 2024-04-23T18:39:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dcaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dcc1031e0 0x7f9dcc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dcaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dcc1031e0 0x7f9dcc1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48566/0 (socket says 172.21.15.19:48566) 2024-04-23T18:39:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dcaffd640 1 -- 172.21.15.19:0/2865065396 learned_addr learned my addr 172.21.15.19:0/2865065396 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc103b00 0x7f9dcc198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dcb7fe640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dcc10c520 0x7f9dcc19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dcaffd640 1 -- 172.21.15.19:0/2865065396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dcc10c520 msgr2=0x7f9dcc19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dcc10c520 0x7f9dcc19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dcaffd640 1 -- 172.21.15.19:0/2865065396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc103b00 msgr2=0x7f9dcc198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc103b00 0x7f9dcc198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dcaffd640 1 -- 172.21.15.19:0/2865065396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc19d6d0 con 0x7f9dcc1031e0 2024-04-23T18:39:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.976+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dcc1031e0 0x7f9dcc1986e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc005ab0 tx=0x7f9dbc003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.977+0000 7f9dd09fc640 1 -- 172.21.15.19:0/2865065396 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc03f070 con 0x7f9dcc1031e0 2024-04-23T18:39:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.977+0000 7f9dd09fc640 1 -- 172.21.15.19:0/2865065396 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dbc031d60 con 0x7f9dcc1031e0 2024-04-23T18:39:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.977+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2865065396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc19d960 con 0x7f9dcc1031e0 2024-04-23T18:39:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.977+0000 7f9dd09fc640 1 -- 172.21.15.19:0/2865065396 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc035030 con 0x7f9dcc1031e0 2024-04-23T18:39:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.977+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2865065396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc1a5250 con 0x7f9dcc1031e0 2024-04-23T18:39:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.979+0000 7f9dd09fc640 1 -- 172.21.15.19:0/2865065396 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9dbc042070 con 0x7f9dcc1031e0 2024-04-23T18:39:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.979+0000 7f9dd47ed740 1 Processor -- start 2024-04-23T18:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dd47ed740 1 -- start start 2024-04-23T18:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4a4a30 0x564bbb4a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4afc10 0x564bbb4b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564bbb405970 con 0x564bbb4a5350 2024-04-23T18:39:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564bbb3a24a0 con 0x564bbb4afc10 2024-04-23T18:39:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564bbb401930 con 0x564bbb4a4a30 2024-04-23T18:39:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dd09fc640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9db40a3dc0 0x7f9db40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48580/0 (socket says 172.21.15.19:48580) 2024-04-23T18:39:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcaffd640 1 -- 172.21.15.19:0/710760988 learned_addr learned my addr 172.21.15.19:0/710760988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/710760988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4afc10 0x564bbb4b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcb7fe640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9db40a3dc0 0x7f9db40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcb7fe640 1 --2- 172.21.15.19:0/710760988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4a4a30 0x564bbb4a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dd09fc640 1 -- 172.21.15.19:0/2865065396 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f9dbc0f1110 con 0x7f9dcc1031e0 2024-04-23T18:39:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcaffd640 1 -- 172.21.15.19:0/710760988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4a4a30 msgr2=0x564bbb4a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/710760988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4a4a30 0x564bbb4a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcaffd640 1 -- 172.21.15.19:0/710760988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4afc10 msgr2=0x564bbb4b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/710760988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4afc10 0x564bbb4b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcaffd640 1 -- 172.21.15.19:0/710760988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bbb118210 con 0x564bbb4a5350 2024-04-23T18:39:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.980+0000 7f9dcb7fe640 1 --2- 172.21.15.19:0/710760988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4a4a30 0x564bbb4a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.981+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/710760988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4af6d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0b94b0 tx=0x7f9dbc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171e37ab25393371 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.981+0000 7f9dcb7fe640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9db40a3dc0 0x7f9db40a6280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009fd0 tx=0x7f9dc4009500 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.981+0000 7f9db1ffb640 1 -- 172.21.15.19:0/710760988 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc049070 con 0x564bbb4a5350 2024-04-23T18:39:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.981+0000 7f9db1ffb640 1 -- 172.21.15.19:0/710760988 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dbc0bd750 con 0x564bbb4a5350 2024-04-23T18:39:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.981+0000 7f9db1ffb640 1 -- 172.21.15.19:0/710760988 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc0c5830 con 0x564bbb4a5350 2024-04-23T18:39:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.981+0000 7f9dd47ed740 1 -- 172.21.15.19:0/710760988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 msgr2=0x564bbb4af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.981+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/710760988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4af6d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0b94b0 tx=0x7f9dbc031400 comp rx=0 tx=0).stop 2024-04-23T18:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 -- 172.21.15.19:0/710760988 shutdown_connections 2024-04-23T18:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/710760988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4afc10 0x564bbb4b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/710760988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4af6d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/710760988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4a4a30 0x564bbb4a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 -- 172.21.15.19:0/710760988 >> 172.21.15.19:0/710760988 conn(0x564bbb4a0f40 msgr2=0x564bbb4a1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 -- 172.21.15.19:0/710760988 shutdown_connections 2024-04-23T18:39:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 -- 172.21.15.19:0/710760988 wait complete. 2024-04-23T18:39:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 -- start start 2024-04-23T18:39:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 Processor -- start 2024-04-23T18:39:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 0x564bbb4af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.982+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564bbb4b2600 con 0x564bbb4a5350 2024-04-23T18:39:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564bbb3a00b0 con 0x564bbb4a4a30 2024-04-23T18:39:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564bbb4b2780 con 0x564bbb4afc10 2024-04-23T18:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dd47ed740 10 client.0 _ll_register_callbacks cb 0x564bbb38dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dca7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dcaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 0x564bbb4af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dca7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48584/0 (socket says 172.21.15.19:48584) 2024-04-23T18:39:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dca7fc640 1 -- 172.21.15.19:0/63869615 learned_addr learned my addr 172.21.15.19:0/63869615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dcb7fe640 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dcaffd640 1 -- 172.21.15.19:0/63869615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 msgr2=0x564bbb4ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dcaffd640 1 -- 172.21.15.19:0/63869615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 msgr2=0x564bbb4ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dcaffd640 1 -- 172.21.15.19:0/63869615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bbb1e4620 con 0x564bbb4a4a30 2024-04-23T18:39:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:39:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 0x564bbb4af5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0b9a20 tx=0x7f9dbc031f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a47df3e8003aeefb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9daaffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-23T18:39:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9daaffd640 1 -- 172.21.15.19:0/63869615 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc049040 con 0x564bbb4a4a30 2024-04-23T18:39:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9daaffd640 1 -- 172.21.15.19:0/63869615 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f9dbc0c5830 con 0x564bbb4a4a30 2024-04-23T18:39:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.983+0000 7f9dd47ed740 1 -- 172.21.15.19:0/63869615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dbc0bd750 con 0x564bbb4a4a30 2024-04-23T18:39:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9daaffd640 1 -- 172.21.15.19:0/63869615 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9dbc0d3080 con 0x564bbb4a4a30 2024-04-23T18:39:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9dd47ed740 10 client.26390 fetch_fsmap learned FSMap version 27 2024-04-23T18:39:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9dd47ed740 1 -- 172.21.15.19:0/63869615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564bbb4b2600 con 0x564bbb4a4a30 2024-04-23T18:39:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9daaffd640 1 -- 172.21.15.19:0/63869615 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dbc0bd750 con 0x564bbb4a4a30 2024-04-23T18:39:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9daaffd640 1 -- 172.21.15.19:0/63869615 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9dbc0b7bc0 con 0x564bbb4a4a30 2024-04-23T18:39:23.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9dd47ed740 10 client.26390 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:39:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9dd47ed740 10 client.26390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9dd47ed740 20 client.26390 populate_metadata read hostname 'smithi019' 2024-04-23T18:39:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564bbb4bf710 0x564bbb4c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9dd47ed740 4 client.26390 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:39:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9dd47ed740 1 -- 172.21.15.19:0/63869615 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bbb413390 con 0x564bbb4bf710 2024-04-23T18:39:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.984+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564bbb4bf710 0x564bbb4c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.985+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564bbb4bf710 0x564bbb4c1b00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a97c663da72379 server_cookie=8e4b2855975fc6b7 in_seq=0 out_seq=0 2024-04-23T18:39:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.985+0000 7f9daaffd640 10 client.26390 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9daaffd640 1 -- 172.21.15.19:0/63869615 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564bbb413390 con 0x564bbb4bf710 2024-04-23T18:39:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9daaffd640 10 client.26390 handle_command_reply: tid=0 2024-04-23T18:39:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 client.26390 shutdown 2024-04-23T18:39:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 -- 172.21.15.19:0/63869615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 msgr2=0x564bbb4af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 0x564bbb4af5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0b9a20 tx=0x7f9dbc031f10 comp rx=0 tx=0).stop 2024-04-23T18:39:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 -- 172.21.15.19:0/63869615 shutdown_connections 2024-04-23T18:39:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564bbb4bf710 0x564bbb4c1b00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/63869615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 0x564bbb4af5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9daaffd640 0 client.26390 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 -- 172.21.15.19:0/63869615 >> 172.21.15.19:0/63869615 conn(0x564bbb4a0f40 msgr2=0x564bbb49bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 -- 172.21.15.19:0/63869615 shutdown_connections 2024-04-23T18:39:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 1 -- 172.21.15.19:0/63869615 wait complete. 2024-04-23T18:39:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:13.986+0000 7f9dd47ed740 20 client.26390 trim_cache size 0 max 16384 2024-04-23T18:39:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dd47ed740 1 Processor -- start 2024-04-23T18:39:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dd47ed740 1 -- start start 2024-04-23T18:39:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 0x564bbb4c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1ab910 con 0x564bbb4a5350 2024-04-23T18:39:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1ad450 con 0x564bbb4a4a30 2024-04-23T18:39:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1abfe0 con 0x564bbb4afc10 2024-04-23T18:39:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dca7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dca7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48596/0 (socket says 172.21.15.19:48596) 2024-04-23T18:39:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.001+0000 7f9dca7fc640 1 -- 172.21.15.19:0/424511435 learned_addr learned my addr 172.21.15.19:0/424511435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/424511435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 0x564bbb4c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9dcb7fe640 1 --2- 172.21.15.19:0/424511435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9dca7fc640 1 -- 172.21.15.19:0/424511435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 msgr2=0x564bbb4a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/424511435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9dca7fc640 1 -- 172.21.15.19:0/424511435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 msgr2=0x564bbb4c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/424511435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 0x564bbb4c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9dca7fc640 1 -- 172.21.15.19:0/424511435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bbb4a5180 con 0x564bbb4a5350 2024-04-23T18:39:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/424511435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4c78e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00105c0 tx=0x7f9dc000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488646baad033266 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9db17fa640 1 -- 172.21.15.19:0/424511435 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc000f950 con 0x564bbb4a5350 2024-04-23T18:39:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9db17fa640 1 -- 172.21.15.19:0/424511435 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc0024020 con 0x564bbb4a5350 2024-04-23T18:39:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.002+0000 7f9db17fa640 1 -- 172.21.15.19:0/424511435 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00208f0 con 0x564bbb4a5350 2024-04-23T18:39:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 -- 172.21.15.19:0/424511435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 msgr2=0x564bbb4c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/424511435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4c78e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00105c0 tx=0x7f9dc000a2d0 comp rx=0 tx=0).stop 2024-04-23T18:39:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 -- 172.21.15.19:0/424511435 shutdown_connections 2024-04-23T18:39:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/424511435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/424511435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a5350 0x564bbb4c78e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/424511435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a4a30 0x564bbb4c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 -- 172.21.15.19:0/424511435 >> 172.21.15.19:0/424511435 conn(0x564bbb4a0f40 msgr2=0x564bbb49bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 -- 172.21.15.19:0/424511435 shutdown_connections 2024-04-23T18:39:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 -- 172.21.15.19:0/424511435 wait complete. 2024-04-23T18:39:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 -- start start 2024-04-23T18:39:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.003+0000 7f9dd47ed740 1 Processor -- start 2024-04-23T18:39:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a4a30 0x564bbb4b5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a5350 0x564bbb4b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dd47ed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1acd80 con 0x564bbb4a4a30 2024-04-23T18:39:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1ac6b0 con 0x564bbb4a5350 2024-04-23T18:39:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dd47ed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1a9910 con 0x564bbb4afc10 2024-04-23T18:39:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dd47ed740 10 client.0 _ll_register_callbacks cb 0x7f9dbc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dcaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a4a30 0x564bbb4b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dcaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a4a30 0x564bbb4b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48612/0 (socket says 172.21.15.19:48612) 2024-04-23T18:39:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dcaffd640 1 -- 172.21.15.19:0/2330060515 learned_addr learned my addr 172.21.15.19:0/2330060515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dcb7fe640 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a5350 0x564bbb4b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dcaffd640 1 -- 172.21.15.19:0/2330060515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 msgr2=0x564bbb4b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dcaffd640 1 -- 172.21.15.19:0/2330060515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a5350 msgr2=0x564bbb4b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a5350 0x564bbb4b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dcaffd640 1 -- 172.21.15.19:0/2330060515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bbb524c20 con 0x564bbb4a4a30 2024-04-23T18:39:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dcaffd640 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a4a30 0x564bbb4b5380 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0bd540 tx=0x7f9dbc0ee8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d804e7f4f42a154 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9daaffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:39:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9daaffd640 1 -- 172.21.15.19:0/2330060515 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc049040 con 0x564bbb4a4a30 2024-04-23T18:39:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9daaffd640 1 -- 172.21.15.19:0/2330060515 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f9dbc0c5830 con 0x564bbb4a4a30 2024-04-23T18:39:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9dd47ed740 1 -- 172.21.15.19:0/2330060515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dc0024020 con 0x564bbb4a4a30 2024-04-23T18:39:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.004+0000 7f9daaffd640 1 -- 172.21.15.19:0/2330060515 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9dbc0bd750 con 0x564bbb4a4a30 2024-04-23T18:39:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9dd47ed740 10 client.17151 fetch_fsmap learned FSMap version 27 2024-04-23T18:39:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9dd47ed740 1 -- 172.21.15.19:0/2330060515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564bbb524c20 con 0x564bbb4a4a30 2024-04-23T18:39:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9daaffd640 1 -- 172.21.15.19:0/2330060515 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc0024020 con 0x564bbb4a4a30 2024-04-23T18:39:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9daaffd640 1 -- 172.21.15.19:0/2330060515 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9dbc0d1af0 con 0x564bbb4a4a30 2024-04-23T18:39:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9dd47ed740 10 client.17151 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:39:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9dd47ed740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9dd47ed740 20 client.17151 populate_metadata read hostname 'smithi019' 2024-04-23T18:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564bbb526270 0x564bbb528660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9dd47ed740 4 client.17151 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:39:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9dd47ed740 1 -- 172.21.15.19:0/2330060515 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bbb40d200 con 0x564bbb526270 2024-04-23T18:39:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.005+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564bbb526270 0x564bbb528660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.006+0000 7f9daaffd640 10 client.17151 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.006+0000 7f9dca7fc640 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564bbb526270 0x564bbb528660 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e4e2b97231308e server_cookie=7ff48d4aa100042d in_seq=0 out_seq=0 2024-04-23T18:39:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9daaffd640 1 -- 172.21.15.19:0/2330060515 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bbb40d200 con 0x564bbb526270 2024-04-23T18:39:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9daaffd640 10 client.17151 handle_command_reply: tid=0 2024-04-23T18:39:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 client.17151 shutdown 2024-04-23T18:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 -- 172.21.15.19:0/2330060515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a4a30 msgr2=0x564bbb4b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a4a30 0x564bbb4b5380 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0bd540 tx=0x7f9dbc0ee8e0 comp rx=0 tx=0).stop 2024-04-23T18:39:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 -- 172.21.15.19:0/2330060515 shutdown_connections 2024-04-23T18:39:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x564bbb526270 0x564bbb528660 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bbb4a5350 0x564bbb4b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbb4afc10 0x564bbb4b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 --2- 172.21.15.19:0/2330060515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564bbb4a4a30 0x564bbb4b5380 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9daaffd640 0 client.17151 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 -- 172.21.15.19:0/2330060515 >> 172.21.15.19:0/2330060515 conn(0x564bbb4a0f40 msgr2=0x564bbb4ba420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 -- 172.21.15.19:0/2330060515 shutdown_connections 2024-04-23T18:39:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 1 -- 172.21.15.19:0/2330060515 wait complete. 2024-04-23T18:39:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.021+0000 7f9dd47ed740 20 client.17151 trim_cache size 0 max 16384 2024-04-23T18:39:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:39:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:39:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:39:30.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:39:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2865065396 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9db40a3dc0 msgr2=0x7f9db40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9db40a3dc0 0x7f9db40a6280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009fd0 tx=0x7f9dc4009500 comp rx=0 tx=0).stop 2024-04-23T18:39:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2865065396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dcc1031e0 msgr2=0x7f9dcc1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dcc1031e0 0x7f9dcc1986e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc005ab0 tx=0x7f9dbc003cd0 comp rx=0 tx=0).stop 2024-04-23T18:39:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2865065396 shutdown_connections 2024-04-23T18:39:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9db40a3dc0 0x7f9db40a6280 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc103b00 0x7f9dcc198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dcc10c520 0x7f9dcc19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 --2- 172.21.15.19:0/2865065396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dcc1031e0 0x7f9dcc1986e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2865065396 >> 172.21.15.19:0/2865065396 conn(0x7f9dcc077fd0 msgr2=0x7f9dcc0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2865065396 shutdown_connections 2024-04-23T18:39:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:14.022+0000 7f9dd19fe640 1 -- 172.21.15.19:0/2865065396 wait complete. 2024-04-23T18:39:31.031 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:13 smithi019 ceph-mon[29215]: pgmap v376: 97 pgs: 97 active+clean; 2.3 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 38 op/s 2024-04-23T18:39:31.130 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:13 smithi019 ceph-mon[29215]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:39:31.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:15 smithi019 ceph-mon[29215]: pgmap v378: 97 pgs: 97 active+clean; 2.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-04-23T18:39:31.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:17 smithi019 ceph-mon[29215]: pgmap v379: 97 pgs: 97 active+clean; 2.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:39:31.272 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:19 smithi019 ceph-mon[29215]: pgmap v380: 97 pgs: 97 active+clean; 2.5 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:39:31.313 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:21 smithi019 ceph-mon[29215]: pgmap v381: 97 pgs: 97 active+clean; 2.7 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:39:31.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:23 smithi019 ceph-mon[29215]: pgmap v382: 97 pgs: 97 active+clean; 2.7 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-23T18:39:31.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:25 smithi019 ceph-mon[29215]: pgmap v383: 97 pgs: 97 active+clean; 2.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-23T18:39:31.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:25 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:39:31.592 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:19 smithi044 ceph-mon[32458]: pgmap v412: 97 pgs: 97 active+clean; 5.1 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-23T18:39:31.675 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:19 smithi044 ceph-mon[32458]: daemon mds.e restarted 2024-04-23T18:39:31.727 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:39:19 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: ignoring --setuser ceph since I am not root 2024-04-23T18:39:31.800 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:39:19 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: ignoring --setgroup ceph since I am not root 2024-04-23T18:39:31.831 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:39:19 smithi044 ceph-mds[62205]: -- 172.21.15.44:0/2425425654 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e5a937a40 con 0x556e5a90ac00 2024-04-23T18:39:31.888 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:39:19 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: starting mds.e at 2024-04-23T18:39:31.933 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:19 smithi155 ceph-mon[33468]: pgmap v412: 97 pgs: 97 active+clean; 5.1 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-23T18:39:31.964 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:19 smithi155 ceph-mon[33468]: daemon mds.e restarted 2024-04-23T18:39:32.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:20 smithi155 ceph-mon[33468]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T18:39:32.081 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:20 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6834/942982921,v1:172.21.15.44:6835/942982921] up:boot 2024-04-23T18:39:32.148 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:20 smithi155 ceph-mon[33468]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-23T18:39:32.173 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:20 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:39:32.206 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:20 smithi155 ceph-mon[33468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:39:32.281 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:21 smithi155 ceph-mon[33468]: pgmap v414: 97 pgs: 97 active+clean; 5.3 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T18:39:32.331 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:23 smithi155 ceph-mon[33468]: pgmap v415: 97 pgs: 97 active+clean; 5.3 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-04-23T18:39:32.458 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:25 smithi155 ceph-mon[33468]: pgmap v416: 97 pgs: 97 active+clean; 1.5 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-23T18:39:32.555 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:26 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:39:32.590 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:27 smithi155 ceph-mon[33468]: pgmap v417: 97 pgs: 97 active+clean; 1.5 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-23T18:39:32.623 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:27 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:39:32.665 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:27 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:39:32.698 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:27 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:39:32.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:29 smithi155 ceph-mon[33468]: pgmap v418: 97 pgs: 97 active+clean; 1.5 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-23T18:39:32.880 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:31 smithi155 ceph-mon[33468]: pgmap v419: 97 pgs: 97 active+clean; 1.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-04-23T18:39:33.015 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:20 smithi044 ceph-mon[32458]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T18:39:33.048 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:20 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6834/942982921,v1:172.21.15.44:6835/942982921] up:boot 2024-04-23T18:39:33.081 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:20 smithi044 ceph-mon[32458]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-23T18:39:33.115 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:20 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:39:33.227 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:20 smithi044 ceph-mon[32458]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:39:33.265 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:21 smithi044 ceph-mon[32458]: pgmap v414: 97 pgs: 97 active+clean; 5.3 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T18:39:33.298 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:23 smithi044 ceph-mon[32458]: pgmap v415: 97 pgs: 97 active+clean; 5.3 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-04-23T18:39:33.332 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:25 smithi044 ceph-mon[32458]: pgmap v416: 97 pgs: 97 active+clean; 1.5 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-23T18:39:33.408 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:26 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:39:33.491 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:27 smithi044 ceph-mon[32458]: pgmap v417: 97 pgs: 97 active+clean; 1.5 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-23T18:39:33.516 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:27 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:39:33.577 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:27 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:39:33.666 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:27 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:39:33.699 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:29 smithi044 ceph-mon[32458]: pgmap v418: 97 pgs: 97 active+clean; 1.5 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-23T18:39:33.733 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:31 smithi044 ceph-mon[32458]: pgmap v419: 97 pgs: 97 active+clean; 1.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-04-23T18:39:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.121+0000 7f6a2029a640 1 -- 172.21.15.19:0/2013604086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a18101a90 msgr2=0x7f6a18105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.121+0000 7f6a2029a640 1 --2- 172.21.15.19:0/2013604086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a18101a90 0x7f6a18105ad0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009900 tx=0x7f6a0802a3f0 comp rx=0 tx=0).stop 2024-04-23T18:39:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.121+0000 7f6a2029a640 1 -- 172.21.15.19:0/2013604086 shutdown_connections 2024-04-23T18:39:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.121+0000 7f6a2029a640 1 --2- 172.21.15.19:0/2013604086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a181061b0 0x7f6a1810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.121+0000 7f6a2029a640 1 --2- 172.21.15.19:0/2013604086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a18101a90 0x7f6a18105ad0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.121+0000 7f6a2029a640 1 --2- 172.21.15.19:0/2013604086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a181010e0 0x7f6a181014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.121+0000 7f6a2029a640 1 -- 172.21.15.19:0/2013604086 >> 172.21.15.19:0/2013604086 conn(0x7f6a180771d0 msgr2=0x7f6a180775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.121+0000 7f6a2029a640 1 -- 172.21.15.19:0/2013604086 shutdown_connections 2024-04-23T18:39:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.121+0000 7f6a2029a640 1 -- 172.21.15.19:0/2013604086 wait complete. 2024-04-23T18:39:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a2029a640 1 Processor -- start 2024-04-23T18:39:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a2029a640 1 -- start start 2024-04-23T18:39:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a2029a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a181010e0 0x7f6a181a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a2029a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a181061b0 0x7f6a181a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a2029a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1819a680 0x7f6a1819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a2029a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18112230 con 0x7f6a181061b0 2024-04-23T18:39:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a2029a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a181120b0 con 0x7f6a181010e0 2024-04-23T18:39:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a2029a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a181123b0 con 0x7f6a1819a680 2024-04-23T18:39:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a1d835640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a181061b0 0x7f6a181a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a1d835640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a181061b0 0x7f6a181a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58112/0 (socket says 172.21.15.19:58112) 2024-04-23T18:39:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.122+0000 7f6a1d835640 1 -- 172.21.15.19:0/1281598668 learned_addr learned my addr 172.21.15.19:0/1281598668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a1e837640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1819a680 0x7f6a1819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a1e036640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a181010e0 0x7f6a181a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a1d835640 1 -- 172.21.15.19:0/1281598668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1819a680 msgr2=0x7f6a1819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a1d835640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1819a680 0x7f6a1819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a1d835640 1 -- 172.21.15.19:0/1281598668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a181010e0 msgr2=0x7f6a181a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a1d835640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a181010e0 0x7f6a181a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a1d835640 1 -- 172.21.15.19:0/1281598668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1819b280 con 0x7f6a181061b0 2024-04-23T18:39:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a1e036640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a181010e0 0x7f6a181a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a1d835640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a181061b0 0x7f6a181a0b00 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6a08032fd0 tx=0x7f6a08004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a0f7fe640 1 -- 172.21.15.19:0/1281598668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0803f030 con 0x7f6a181061b0 2024-04-23T18:39:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a2029a640 1 -- 172.21.15.19:0/1281598668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1819b510 con 0x7f6a181061b0 2024-04-23T18:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a0f7fe640 1 -- 172.21.15.19:0/1281598668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a08004490 con 0x7f6a181061b0 2024-04-23T18:39:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a0f7fe640 1 -- 172.21.15.19:0/1281598668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08035070 con 0x7f6a181061b0 2024-04-23T18:39:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.123+0000 7f6a2029a640 1 -- 172.21.15.19:0/1281598668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a181a7650 con 0x7f6a181061b0 2024-04-23T18:39:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.126+0000 7f6a0f7fe640 1 -- 172.21.15.19:0/1281598668 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a08042030 con 0x7f6a181061b0 2024-04-23T18:39:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.126+0000 7f6a22fb1740 1 Processor -- start 2024-04-23T18:39:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a22fb1740 1 -- start start 2024-04-23T18:39:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd920620 0x5584cd920a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd92b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 0x5584cd92dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5584cd885970 con 0x5584cd920f40 2024-04-23T18:39:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5584cd8224a0 con 0x5584cd920620 2024-04-23T18:39:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5584cd881930 con 0x5584cd92b800 2024-04-23T18:39:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a0f7fe640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6a040a3dc0 0x7f6a040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a1e036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd92b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a1e036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd92b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58122/0 (socket says 172.21.15.19:58122) 2024-04-23T18:39:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a1e036640 1 -- 172.21.15.19:0/79994300 learned_addr learned my addr 172.21.15.19:0/79994300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a1e837640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6a040a3dc0 0x7f6a040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a0f7fe640 1 -- 172.21.15.19:0/1281598668 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f6a080f0cb0 con 0x7f6a181061b0 2024-04-23T18:39:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a1d835640 1 --2- 172.21.15.19:0/79994300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 0x5584cd92dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.127+0000 7f6a1e837640 1 --2- 172.21.15.19:0/79994300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd920620 0x5584cd920a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.128+0000 7f6a1e036640 1 -- 172.21.15.19:0/79994300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 msgr2=0x5584cd92dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.128+0000 7f6a1e036640 1 --2- 172.21.15.19:0/79994300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 0x5584cd92dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.128+0000 7f6a1e036640 1 -- 172.21.15.19:0/79994300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd920620 msgr2=0x5584cd920a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.128+0000 7f6a1e036640 1 --2- 172.21.15.19:0/79994300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd920620 0x5584cd920a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.128+0000 7f6a1e036640 1 -- 172.21.15.19:0/79994300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584cd598210 con 0x5584cd920f40 2024-04-23T18:39:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.128+0000 7f6a1e837640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6a040a3dc0 0x7f6a040a6280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a10007fd0 tx=0x7f6a100072a0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.128+0000 7f6a1e036640 1 --2- 172.21.15.19:0/79994300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd92b2c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400bcc0 tx=0x7f6a1400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f99e2dac192bae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.128+0000 7f6a0cbfe640 1 -- 172.21.15.19:0/79994300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14005020 con 0x5584cd920f40 2024-04-23T18:39:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.128+0000 7f6a0cbfe640 1 -- 172.21.15.19:0/79994300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a140047a0 con 0x5584cd920f40 2024-04-23T18:39:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.128+0000 7f6a0cbfe640 1 -- 172.21.15.19:0/79994300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14011680 con 0x5584cd920f40 2024-04-23T18:39:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 -- 172.21.15.19:0/79994300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 msgr2=0x5584cd92b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/79994300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd92b2c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400bcc0 tx=0x7f6a1400f730 comp rx=0 tx=0).stop 2024-04-23T18:39:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 -- 172.21.15.19:0/79994300 shutdown_connections 2024-04-23T18:39:37.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/79994300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 0x5584cd92dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/79994300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd92b2c0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/79994300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd920620 0x5584cd920a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 -- 172.21.15.19:0/79994300 >> 172.21.15.19:0/79994300 conn(0x5584cd91cb30 msgr2=0x5584cd91cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 -- 172.21.15.19:0/79994300 shutdown_connections 2024-04-23T18:39:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 -- 172.21.15.19:0/79994300 wait complete. 2024-04-23T18:39:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 -- start start 2024-04-23T18:39:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.129+0000 7f6a22fb1740 1 Processor -- start 2024-04-23T18:39:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd920620 0x5584cd92b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd9283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 0x5584cd9288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5584cd92e430 con 0x5584cd920f40 2024-04-23T18:39:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5584cd8200b0 con 0x5584cd920620 2024-04-23T18:39:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5584cd92e5b0 con 0x5584cd92b800 2024-04-23T18:39:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a22fb1740 10 client.0 _ll_register_callbacks cb 0x5584cd80dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1d835640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd9283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1d835640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd9283a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58128/0 (socket says 172.21.15.19:58128) 2024-04-23T18:39:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1d835640 1 -- 172.21.15.19:0/2152631090 learned_addr learned my addr 172.21.15.19:0/2152631090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1e036640 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd920620 0x5584cd92b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1e837640 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 0x5584cd9288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1d835640 1 -- 172.21.15.19:0/2152631090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 msgr2=0x5584cd9288e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1d835640 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 0x5584cd9288e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1d835640 1 -- 172.21.15.19:0/2152631090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd920620 msgr2=0x5584cd92b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1d835640 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd920620 0x5584cd92b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1d835640 1 -- 172.21.15.19:0/2152631090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584cd664620 con 0x5584cd920f40 2024-04-23T18:39:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1e837640 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 0x5584cd9288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a1d835640 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd9283a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6a080b90b0 tx=0x7f6a080bcae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaf062a78c36030 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.130+0000 7f6a01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:39:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a01ffb640 1 -- 172.21.15.19:0/2152631090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08049040 con 0x5584cd920f40 2024-04-23T18:39:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2152631090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a140047a0 con 0x5584cd920f40 2024-04-23T18:39:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a01ffb640 1 -- 172.21.15.19:0/2152631090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f6a080ee680 con 0x5584cd920f40 2024-04-23T18:39:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a01ffb640 1 -- 172.21.15.19:0/2152631090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a080ee880 con 0x5584cd920f40 2024-04-23T18:39:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a22fb1740 10 client.17163 fetch_fsmap learned FSMap version 27 2024-04-23T18:39:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2152631090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5584cd93a2a0 con 0x5584cd920f40 2024-04-23T18:39:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a01ffb640 1 -- 172.21.15.19:0/2152631090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a140047a0 con 0x5584cd920f40 2024-04-23T18:39:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a01ffb640 1 -- 172.21.15.19:0/2152631090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a080dd0b0 con 0x5584cd920f40 2024-04-23T18:39:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a22fb1740 10 client.17163 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:39:39.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a22fb1740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a22fb1740 20 client.17163 populate_metadata read hostname 'smithi019' 2024-04-23T18:39:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5584cd93b750 0x5584cd93db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a22fb1740 4 client.17163 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:39:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2152631090 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584cd893390 con 0x5584cd93b750 2024-04-23T18:39:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.131+0000 7f6a1e036640 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5584cd93b750 0x5584cd93db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.132+0000 7f6a1e036640 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5584cd93b750 0x5584cd93db80 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c01ee25d56e106e server_cookie=b8515f126b2ebd1a in_seq=0 out_seq=0 2024-04-23T18:39:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.132+0000 7f6a01ffb640 10 client.17163 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a01ffb640 1 -- 172.21.15.19:0/2152631090 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584cd893390 con 0x5584cd93b750 2024-04-23T18:39:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a01ffb640 10 client.17163 handle_command_reply: tid=0 2024-04-23T18:39:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 client.17163 shutdown 2024-04-23T18:39:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2152631090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 msgr2=0x5584cd9283a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd9283a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6a080b90b0 tx=0x7f6a080bcae0 comp rx=0 tx=0).stop 2024-04-23T18:39:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2152631090 shutdown_connections 2024-04-23T18:39:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5584cd93b750 0x5584cd93db80 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd92b800 0x5584cd9288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920f40 0x5584cd9283a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a01ffb640 0 client.17163 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/2152631090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd920620 0x5584cd92b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2152631090 >> 172.21.15.19:0/2152631090 conn(0x5584cd91cb30 msgr2=0x5584cd9228b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2152631090 shutdown_connections 2024-04-23T18:39:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2152631090 wait complete. 2024-04-23T18:39:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.133+0000 7f6a22fb1740 20 client.17163 trim_cache size 0 max 16384 2024-04-23T18:39:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.148+0000 7f6a22fb1740 1 Processor -- start 2024-04-23T18:39:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.148+0000 7f6a22fb1740 1 -- start start 2024-04-23T18:39:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.148+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd9372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.148+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 0x5584cd9238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.148+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 0x5584cd92abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.148+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a181afd40 con 0x5584cd920620 2024-04-23T18:39:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.148+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a181b1880 con 0x5584cd92b800 2024-04-23T18:39:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.148+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a181b0410 con 0x5584cd920f40 2024-04-23T18:39:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd9372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd9372f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58138/0 (socket says 172.21.15.19:58138) 2024-04-23T18:39:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e036640 1 -- 172.21.15.19:0/2131196573 learned_addr learned my addr 172.21.15.19:0/2131196573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e837640 1 --2- 172.21.15.19:0/2131196573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 0x5584cd92abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1d835640 1 --2- 172.21.15.19:0/2131196573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 0x5584cd9238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e036640 1 -- 172.21.15.19:0/2131196573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 msgr2=0x5584cd9238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e036640 1 --2- 172.21.15.19:0/2131196573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 0x5584cd9238e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e036640 1 -- 172.21.15.19:0/2131196573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 msgr2=0x5584cd92abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e036640 1 --2- 172.21.15.19:0/2131196573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 0x5584cd92abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e036640 1 -- 172.21.15.19:0/2131196573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584cd9268f0 con 0x5584cd920620 2024-04-23T18:39:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e837640 1 --2- 172.21.15.19:0/2131196573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 0x5584cd92abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.149+0000 7f6a1e036640 1 --2- 172.21.15.19:0/2131196573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd9372f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x5584cd921bc0 tx=0x7f6a1400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b369d24b04abd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.150+0000 7f6a03fff640 1 -- 172.21.15.19:0/2131196573 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14004040 con 0x5584cd920620 2024-04-23T18:39:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.150+0000 7f6a03fff640 1 -- 172.21.15.19:0/2131196573 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a140035f0 con 0x5584cd920620 2024-04-23T18:39:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.150+0000 7f6a03fff640 1 -- 172.21.15.19:0/2131196573 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a140292e0 con 0x5584cd920620 2024-04-23T18:39:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2131196573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 msgr2=0x5584cd9372f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/2131196573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd9372f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x5584cd921bc0 tx=0x7f6a1400b5e0 comp rx=0 tx=0).stop 2024-04-23T18:39:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2131196573 shutdown_connections 2024-04-23T18:39:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/2131196573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 0x5584cd92abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/2131196573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 0x5584cd9238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/2131196573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd9372f0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2131196573 >> 172.21.15.19:0/2131196573 conn(0x5584cd91cb30 msgr2=0x5584cd9231d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2131196573 shutdown_connections 2024-04-23T18:39:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 -- 172.21.15.19:0/2131196573 wait complete. 2024-04-23T18:39:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 -- start start 2024-04-23T18:39:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.151+0000 7f6a22fb1740 1 Processor -- start 2024-04-23T18:39:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd933c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 0x5584cd924e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a22fb1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 0x5584cd925360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a181b11b0 con 0x5584cd920620 2024-04-23T18:39:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a181b0ae0 con 0x5584cd92b800 2024-04-23T18:39:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a22fb1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a181add40 con 0x5584cd920f40 2024-04-23T18:39:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a22fb1740 10 client.0 _ll_register_callbacks cb 0x7f6a080b8ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1e036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd933c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1e036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd933c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58140/0 (socket says 172.21.15.19:58140) 2024-04-23T18:39:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1e036640 1 -- 172.21.15.19:0/1388482255 learned_addr learned my addr 172.21.15.19:0/1388482255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:39:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1e837640 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 0x5584cd925360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1d835640 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 0x5584cd924e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1e036640 1 -- 172.21.15.19:0/1388482255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 msgr2=0x5584cd924e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1e036640 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 0x5584cd924e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1e036640 1 -- 172.21.15.19:0/1388482255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 msgr2=0x5584cd925360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1e036640 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 0x5584cd925360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1e036640 1 -- 172.21.15.19:0/1388482255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584cd925b90 con 0x5584cd920620 2024-04-23T18:39:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1d835640 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 0x5584cd924e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a1e036640 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd933c50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400fc30 tx=0x7f6a140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd06b53992dd43a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:39:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a01ffb640 1 -- 172.21.15.19:0/1388482255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14036060 con 0x5584cd920620 2024-04-23T18:39:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.152+0000 7f6a01ffb640 1 -- 172.21.15.19:0/1388482255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f6a14004250 con 0x5584cd920620 2024-04-23T18:39:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a22fb1740 1 -- 172.21.15.19:0/1388482255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a140035f0 con 0x5584cd920620 2024-04-23T18:39:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a01ffb640 1 -- 172.21.15.19:0/1388482255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a14004450 con 0x5584cd920620 2024-04-23T18:39:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a22fb1740 10 client.17169 fetch_fsmap learned FSMap version 27 2024-04-23T18:39:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a22fb1740 1 -- 172.21.15.19:0/1388482255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5584cd9a1da0 con 0x5584cd920620 2024-04-23T18:39:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a01ffb640 1 -- 172.21.15.19:0/1388482255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a140035f0 con 0x5584cd920620 2024-04-23T18:39:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a01ffb640 1 -- 172.21.15.19:0/1388482255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a14029550 con 0x5584cd920620 2024-04-23T18:39:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a22fb1740 10 client.17169 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:39:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a22fb1740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a22fb1740 20 client.17169 populate_metadata read hostname 'smithi019' 2024-04-23T18:39:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5584cd9a22a0 0x5584cd9a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a22fb1740 4 client.17169 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:39:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a22fb1740 1 -- 172.21.15.19:0/1388482255 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584cd928c70 con 0x5584cd9a22a0 2024-04-23T18:39:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.153+0000 7f6a1d835640 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5584cd9a22a0 0x5584cd9a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.154+0000 7f6a1d835640 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5584cd9a22a0 0x5584cd9a46c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c106a8ceb451cdad server_cookie=dd37bc9977a28446 in_seq=0 out_seq=0 2024-04-23T18:39:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.154+0000 7f6a01ffb640 10 client.17169 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.169+0000 7f6a01ffb640 1 -- 172.21.15.19:0/1388482255 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584cd928c70 con 0x5584cd9a22a0 2024-04-23T18:39:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.169+0000 7f6a01ffb640 10 client.17169 handle_command_reply: tid=0 2024-04-23T18:39:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.169+0000 7f6a22fb1740 1 client.17169 shutdown 2024-04-23T18:39:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 1 -- 172.21.15.19:0/1388482255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 msgr2=0x5584cd933c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd933c50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400fc30 tx=0x7f6a140046c0 comp rx=0 tx=0).stop 2024-04-23T18:39:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 1 -- 172.21.15.19:0/1388482255 shutdown_connections 2024-04-23T18:39:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5584cd9a22a0 0x5584cd9a46c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584cd92b800 0x5584cd925360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a01ffb640 0 client.17169 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:39:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584cd920f40 0x5584cd924e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 1 --2- 172.21.15.19:0/1388482255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5584cd920620 0x5584cd933c50 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 1 -- 172.21.15.19:0/1388482255 >> 172.21.15.19:0/1388482255 conn(0x5584cd91cb30 msgr2=0x5584cd922830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 1 -- 172.21.15.19:0/1388482255 shutdown_connections 2024-04-23T18:39:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 1 -- 172.21.15.19:0/1388482255 wait complete. 2024-04-23T18:39:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a22fb1740 20 client.17169 trim_cache size 0 max 16384 2024-04-23T18:39:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:39:45.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:39:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:39:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:39:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a2029a640 1 -- 172.21.15.19:0/1281598668 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6a040a3dc0 msgr2=0x7f6a040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a2029a640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6a040a3dc0 0x7f6a040a6280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a10007fd0 tx=0x7f6a100072a0 comp rx=0 tx=0).stop 2024-04-23T18:39:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a2029a640 1 -- 172.21.15.19:0/1281598668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a181061b0 msgr2=0x7f6a181a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a2029a640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a181061b0 0x7f6a181a0b00 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6a08032fd0 tx=0x7f6a08004270 comp rx=0 tx=0).stop 2024-04-23T18:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a2029a640 1 -- 172.21.15.19:0/1281598668 shutdown_connections 2024-04-23T18:39:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.170+0000 7f6a2029a640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f6a040a3dc0 0x7f6a040a6280 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.171+0000 7f6a2029a640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1819a680 0x7f6a1819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.171+0000 7f6a2029a640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a181061b0 0x7f6a181a0b00 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.171+0000 7f6a2029a640 1 --2- 172.21.15.19:0/1281598668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a181010e0 0x7f6a181a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.171+0000 7f6a2029a640 1 -- 172.21.15.19:0/1281598668 >> 172.21.15.19:0/1281598668 conn(0x7f6a180771d0 msgr2=0x7f6a1810c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:46.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.171+0000 7f6a2029a640 1 -- 172.21.15.19:0/1281598668 shutdown_connections 2024-04-23T18:39:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:27.171+0000 7f6a2029a640 1 -- 172.21.15.19:0/1281598668 wait complete. 2024-04-23T18:39:46.786 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:26 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:39:46.826 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:26 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:39:46.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:26 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:39:46.893 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:27 smithi019 ceph-mon[29215]: pgmap v384: 97 pgs: 97 active+clean; 2.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-23T18:39:46.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:29 smithi019 ceph-mon[29215]: pgmap v385: 97 pgs: 97 active+clean; 2.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-23T18:39:47.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:31 smithi019 ceph-mon[29215]: pgmap v386: 97 pgs: 97 active+clean; 3.1 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T18:39:47.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:33 smithi019 ceph-mon[29215]: pgmap v387: 97 pgs: 97 active+clean; 3.1 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T18:39:47.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:35 smithi019 ceph-mon[29215]: pgmap v388: 97 pgs: 97 active+clean; 3.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T18:39:47.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:37 smithi019 ceph-mon[29215]: pgmap v389: 97 pgs: 97 active+clean; 3.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:39:47.210 INFO:tasks.workunit.client.0.smithi019.stdout:437/600: open fd = 3 2024-04-23T18:39:47.251 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:47.305 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:47.343 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:47.376 INFO:tasks.workunit.client.0.smithi019.stdout:438/600: open fd = 3 2024-04-23T18:39:47.410 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:47.443 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:47.518 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:47.551 INFO:tasks.workunit.client.0.smithi019.stdout:439/600: open fd = 3 2024-04-23T18:39:47.585 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:47.618 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:47.651 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:47.737 INFO:tasks.workunit.client.0.smithi019.stdout:440/600: open fd = 3 2024-04-23T18:39:47.784 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:47.823 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:47.868 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:47.902 INFO:tasks.workunit.client.0.smithi019.stdout:441/600: open fd = 3 2024-04-23T18:39:47.985 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.019 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.052 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:48.085 INFO:tasks.workunit.client.0.smithi019.stdout:442/600: open fd = 3 2024-04-23T18:39:48.119 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.152 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.188 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:48.219 INFO:tasks.workunit.client.0.smithi019.stdout:443/600: open fd = 3 2024-04-23T18:39:48.300 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.335 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.394 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:48.427 INFO:tasks.workunit.client.0.smithi019.stdout:444/600: open fd = 3 2024-04-23T18:39:48.527 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.560 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.594 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:48.627 INFO:tasks.workunit.client.0.smithi019.stdout:445/600: open fd = 3 2024-04-23T18:39:48.661 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.694 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.727 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:48.805 INFO:tasks.workunit.client.0.smithi019.stdout:446/600: open fd = 3 2024-04-23T18:39:48.844 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.878 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:48.919 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:49.019 INFO:tasks.workunit.client.0.smithi019.stdout:447/600: open fd = 3 2024-04-23T18:39:49.069 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:49.119 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:49.169 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:49.219 INFO:tasks.workunit.client.0.smithi019.stdout:448/600: open fd = 3 2024-04-23T18:39:49.269 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:49.319 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:49.375 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:49.432 INFO:tasks.workunit.client.0.smithi019.stdout:449/600: open fd = 3 2024-04-23T18:39:49.470 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:49.520 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:49.636 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:49.686 INFO:tasks.workunit.client.0.smithi019.stdout:450/600: open fd = 3 2024-04-23T18:39:49.736 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:49.786 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:49.836 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:49.886 INFO:tasks.workunit.client.0.smithi019.stdout:451/600: open fd = 3 2024-04-23T18:39:49.936 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:50.015 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:50.053 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:50.128 INFO:tasks.workunit.client.0.smithi019.stdout:452/600: open fd = 3 2024-04-23T18:39:50.179 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:50.278 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:50.328 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:50.378 INFO:tasks.workunit.client.0.smithi019.stdout:453/600: open fd = 3 2024-04-23T18:39:50.437 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:50.487 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:50.537 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:50.587 INFO:tasks.workunit.client.0.smithi019.stdout:454/600: open fd = 3 2024-04-23T18:39:50.672 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:50.712 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:50.762 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:50.812 INFO:tasks.workunit.client.0.smithi019.stdout:455/600: open fd = 3 2024-04-23T18:39:50.912 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:50.962 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:51.025 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:51.062 INFO:tasks.workunit.client.0.smithi019.stdout:456/600: open fd = 3 2024-04-23T18:39:51.112 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:51.162 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:51.212 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:51.282 INFO:tasks.workunit.client.0.smithi019.stdout:457/600: open fd = 3 2024-04-23T18:39:51.362 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:51.412 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:51.523 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:51.571 INFO:tasks.workunit.client.0.smithi019.stdout:458/600: open fd = 3 2024-04-23T18:39:51.620 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:51.671 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:51.721 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:51.771 INFO:tasks.workunit.client.0.smithi019.stdout:459/600: open fd = 3 2024-04-23T18:39:51.822 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:51.871 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:51.961 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:51.996 INFO:tasks.workunit.client.0.smithi019.stdout:460/600: open fd = 3 2024-04-23T18:39:52.046 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:52.113 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:52.379 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:52.463 INFO:tasks.workunit.client.0.smithi019.stdout:461/600: open fd = 3 2024-04-23T18:39:52.513 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:52.563 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:52.613 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:52.663 INFO:tasks.workunit.client.0.smithi019.stdout:462/600: open fd = 3 2024-04-23T18:39:52.713 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:52.795 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:52.841 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:52.888 INFO:tasks.workunit.client.0.smithi019.stdout:463/600: open fd = 3 2024-04-23T18:39:52.938 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:52.988 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.071 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:53.121 INFO:tasks.workunit.client.0.smithi019.stdout:464/600: open fd = 3 2024-04-23T18:39:53.171 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.221 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.280 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:53.330 INFO:tasks.workunit.client.0.smithi019.stdout:465/600: open fd = 3 2024-04-23T18:39:53.363 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.428 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.455 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:53.488 INFO:tasks.workunit.client.0.smithi019.stdout:466/600: open fd = 3 2024-04-23T18:39:53.522 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.605 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.638 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:53.672 INFO:tasks.workunit.client.0.smithi019.stdout:467/600: open fd = 3 2024-04-23T18:39:53.705 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.747 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.780 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:53.814 INFO:tasks.workunit.client.0.smithi019.stdout:468/600: open fd = 3 2024-04-23T18:39:53.898 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.947 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:53.980 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:54.014 INFO:tasks.workunit.client.0.smithi019.stdout:469/600: open fd = 3 2024-04-23T18:39:54.047 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.080 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.114 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:54.147 INFO:tasks.workunit.client.0.smithi019.stdout:470/600: open fd = 3 2024-04-23T18:39:54.180 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.300 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.339 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:54.385 INFO:tasks.workunit.client.0.smithi019.stdout:471/600: open fd = 3 2024-04-23T18:39:54.414 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.447 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.481 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:54.514 INFO:tasks.workunit.client.0.smithi019.stdout:472/600: open fd = 3 2024-04-23T18:39:54.548 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.581 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.623 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:54.647 INFO:tasks.workunit.client.0.smithi019.stdout:473/600: open fd = 3 2024-04-23T18:39:54.681 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.767 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.806 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:54.889 INFO:tasks.workunit.client.0.smithi019.stdout:474/600: open fd = 3 2024-04-23T18:39:54.923 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.956 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:54.990 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:55.023 INFO:tasks.workunit.client.0.smithi019.stdout:475/600: open fd = 3 2024-04-23T18:39:55.056 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.090 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.123 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:55.198 INFO:tasks.workunit.client.0.smithi019.stdout:476/600: open fd = 3 2024-04-23T18:39:55.264 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.290 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.323 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:55.390 INFO:tasks.workunit.client.0.smithi019.stdout:477/600: open fd = 3 2024-04-23T18:39:55.423 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.457 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.490 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:55.523 INFO:tasks.workunit.client.0.smithi019.stdout:478/600: open fd = 3 2024-04-23T18:39:55.556 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.591 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.637 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:55.689 INFO:tasks.workunit.client.0.smithi019.stdout:479/600: open fd = 3 2024-04-23T18:39:55.731 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.765 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.849 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:55.899 INFO:tasks.workunit.client.0.smithi019.stdout:480/600: open fd = 3 2024-04-23T18:39:55.932 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.966 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:55.999 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:56.033 INFO:tasks.workunit.client.0.smithi019.stdout:481/600: open fd = 3 2024-04-23T18:39:56.066 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:56.134 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:56.229 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:56.274 INFO:tasks.workunit.client.0.smithi019.stdout:482/600: open fd = 3 2024-04-23T18:39:56.358 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:56.408 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:56.458 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:56.508 INFO:tasks.workunit.client.0.smithi019.stdout:483/600: open fd = 3 2024-04-23T18:39:56.566 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:56.616 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:56.666 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:56.716 INFO:tasks.workunit.client.0.smithi019.stdout:484/600: open fd = 3 2024-04-23T18:39:56.766 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:56.816 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:56.866 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:56.916 INFO:tasks.workunit.client.0.smithi019.stdout:485/600: open fd = 3 2024-04-23T18:39:56.966 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:57.016 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:57.066 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:57.117 INFO:tasks.workunit.client.0.smithi019.stdout:486/600: open fd = 3 2024-04-23T18:39:57.167 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:57.300 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:57.633 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:57.683 INFO:tasks.workunit.client.0.smithi019.stdout:487/600: open fd = 3 2024-04-23T18:39:57.746 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:57.783 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:57.850 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:57.900 INFO:tasks.workunit.client.0.smithi019.stdout:488/600: open fd = 3 2024-04-23T18:39:57.950 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:58.015 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:58.050 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:58.100 INFO:tasks.workunit.client.0.smithi019.stdout:489/600: open fd = 3 2024-04-23T18:39:58.208 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:58.289 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:58.333 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:58.383 INFO:tasks.workunit.client.0.smithi019.stdout:490/600: open fd = 3 2024-04-23T18:39:58.434 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:58.484 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:58.534 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:58.584 INFO:tasks.workunit.client.0.smithi019.stdout:491/600: open fd = 3 2024-04-23T18:39:58.634 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:58.684 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:58.739 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:58.817 INFO:tasks.workunit.client.0.smithi019.stdout:492/600: open fd = 3 2024-04-23T18:39:58.901 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:58.951 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:59.001 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:59.051 INFO:tasks.workunit.client.0.smithi019.stdout:493/600: open fd = 3 2024-04-23T18:39:59.109 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:59.159 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:59.238 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:59.284 INFO:tasks.workunit.client.0.smithi019.stdout:494/600: open fd = 3 2024-04-23T18:39:59.334 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:59.393 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:59.443 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:59.559 INFO:tasks.workunit.client.0.smithi019.stdout:495/600: open fd = 3 2024-04-23T18:39:59.609 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:59.659 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:59.709 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:59.759 INFO:tasks.workunit.client.0.smithi019.stdout:496/600: open fd = 3 2024-04-23T18:39:59.809 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:59.859 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:39:59.909 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:39:59.959 INFO:tasks.workunit.client.0.smithi019.stdout:497/600: open fd = 3 2024-04-23T18:40:00.010 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:40:00.078 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:40:00.168 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:40:00.235 INFO:tasks.workunit.client.0.smithi019.stdout:498/600: open fd = 3 2024-04-23T18:40:00.286 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:40:00.335 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:40:00.385 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:40:00.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:39 smithi019 ceph-mon[29215]: pgmap v390: 97 pgs: 97 active+clean; 3.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:40:00.487 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:33 smithi155 ceph-mon[33468]: pgmap v420: 97 pgs: 97 active+clean; 1.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-23T18:40:00.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:35 smithi155 ceph-mon[33468]: pgmap v421: 97 pgs: 97 active+clean; 1.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 66 KiB/s wr, 26 op/s 2024-04-23T18:40:00.593 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:37 smithi155 ceph-mon[33468]: pgmap v422: 97 pgs: 97 active+clean; 1.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:00.627 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:39 smithi155 ceph-mon[33468]: pgmap v423: 97 pgs: 97 active+clean; 1.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:00.660 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:41 smithi155 ceph-mon[33468]: pgmap v424: 97 pgs: 97 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T18:40:00.694 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:43 smithi155 ceph-mon[33468]: pgmap v425: 97 pgs: 97 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:40:00.774 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:45 smithi155 ceph-mon[33468]: pgmap v426: 97 pgs: 97 active+clean; 2.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:40:00.977 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:47 smithi155 ceph-mon[33468]: pgmap v427: 97 pgs: 97 active+clean; 2.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T18:40:01.016 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:49 smithi155 ceph-mon[33468]: pgmap v428: 97 pgs: 97 active+clean; 2.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T18:40:01.044 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:51 smithi155 ceph-mon[33468]: pgmap v429: 97 pgs: 97 active+clean; 2.5 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T18:40:01.077 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:53 smithi155 ceph-mon[33468]: pgmap v430: 97 pgs: 97 active+clean; 2.5 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:40:01.110 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:55 smithi155 ceph-mon[33468]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:40:01.144 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:57 smithi155 ceph-mon[33468]: pgmap v432: 97 pgs: 97 active+clean; 2.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:01.177 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:39:59 smithi155 ceph-mon[33468]: pgmap v433: 97 pgs: 97 active+clean; 2.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:01.276 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:33 smithi044 ceph-mon[32458]: pgmap v420: 97 pgs: 97 active+clean; 1.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-23T18:40:01.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:35 smithi044 ceph-mon[32458]: pgmap v421: 97 pgs: 97 active+clean; 1.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 66 KiB/s wr, 26 op/s 2024-04-23T18:40:01.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:37 smithi044 ceph-mon[32458]: pgmap v422: 97 pgs: 97 active+clean; 1.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:01.420 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:39 smithi044 ceph-mon[32458]: pgmap v423: 97 pgs: 97 active+clean; 1.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:01.502 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:41 smithi044 ceph-mon[32458]: pgmap v424: 97 pgs: 97 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T18:40:01.536 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:43 smithi044 ceph-mon[32458]: pgmap v425: 97 pgs: 97 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:40:01.569 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:45 smithi044 ceph-mon[32458]: pgmap v426: 97 pgs: 97 active+clean; 2.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:40:01.653 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:47 smithi044 ceph-mon[32458]: pgmap v427: 97 pgs: 97 active+clean; 2.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T18:40:01.686 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:49 smithi044 ceph-mon[32458]: pgmap v428: 97 pgs: 97 active+clean; 2.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T18:40:01.719 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:51 smithi044 ceph-mon[32458]: pgmap v429: 97 pgs: 97 active+clean; 2.5 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T18:40:01.753 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:53 smithi044 ceph-mon[32458]: pgmap v430: 97 pgs: 97 active+clean; 2.5 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:40:01.786 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:55 smithi044 ceph-mon[32458]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:40:01.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:57 smithi044 ceph-mon[32458]: pgmap v432: 97 pgs: 97 active+clean; 2.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:01.869 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:39:59 smithi044 ceph-mon[32458]: pgmap v433: 97 pgs: 97 active+clean; 2.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:01.971 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:00 smithi044 ceph-mon[32458]: overall HEALTH_OK 2024-04-23T18:40:02.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:00 smithi155 ceph-mon[33468]: overall HEALTH_OK 2024-04-23T18:40:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.272+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1351540740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf44101a60 msgr2=0x7fdf44105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.272+0000 7fdf4ba3c640 1 --2- 172.21.15.19:0/1351540740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf44101a60 0x7fdf44105aa0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009900 tx=0x7fdf3402a410 comp rx=0 tx=0).stop 2024-04-23T18:40:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.272+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1351540740 shutdown_connections 2024-04-23T18:40:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.272+0000 7fdf4ba3c640 1 --2- 172.21.15.19:0/1351540740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf44106180 0x7fdf4410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.272+0000 7fdf4ba3c640 1 --2- 172.21.15.19:0/1351540740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf44101a60 0x7fdf44105aa0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.272+0000 7fdf4ba3c640 1 --2- 172.21.15.19:0/1351540740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf441010b0 0x7fdf44101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.272+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1351540740 >> 172.21.15.19:0/1351540740 conn(0x7fdf44078000 msgr2=0x7fdf44078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.272+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1351540740 shutdown_connections 2024-04-23T18:40:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.272+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1351540740 wait complete. 2024-04-23T18:40:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf4ba3c640 1 Processor -- start 2024-04-23T18:40:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf4ba3c640 1 -- start start 2024-04-23T18:40:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf4ba3c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf441010b0 0x7fdf44198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf4ba3c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf44106180 0x7fdf44198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf4ba3c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4419cf00 0x7fdf4419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf4ba3c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441110c0 con 0x7fdf44106180 2024-04-23T18:40:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf4ba3c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44110f40 con 0x7fdf441010b0 2024-04-23T18:40:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf4ba3c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44111240 con 0x7fdf4419cf00 2024-04-23T18:40:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf48fd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf44106180 0x7fdf44198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf48fd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf44106180 0x7fdf44198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42666/0 (socket says 172.21.15.19:42666) 2024-04-23T18:40:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf48fd7640 1 -- 172.21.15.19:0/1613534209 learned_addr learned my addr 172.21.15.19:0/1613534209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.273+0000 7fdf49fd9640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4419cf00 0x7fdf4419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf497d8640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf441010b0 0x7fdf44198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf48fd7640 1 -- 172.21.15.19:0/1613534209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4419cf00 msgr2=0x7fdf4419d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4419cf00 0x7fdf4419d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf48fd7640 1 -- 172.21.15.19:0/1613534209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf441010b0 msgr2=0x7fdf44198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf441010b0 0x7fdf44198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf48fd7640 1 -- 172.21.15.19:0/1613534209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4419d9f0 con 0x7fdf44106180 2024-04-23T18:40:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf497d8640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf441010b0 0x7fdf44198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:40:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf44106180 0x7fdf44198b80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdf34032fd0 tx=0x7fdf34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf3a7fc640 1 -- 172.21.15.19:0/1613534209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3403f030 con 0x7fdf44106180 2024-04-23T18:40:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf3a7fc640 1 -- 172.21.15.19:0/1613534209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf34004490 con 0x7fdf44106180 2024-04-23T18:40:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1613534209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4419dc80 con 0x7fdf44106180 2024-04-23T18:40:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf3a7fc640 1 -- 172.21.15.19:0/1613534209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34035070 con 0x7fdf44106180 2024-04-23T18:40:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.274+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1613534209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf441a5440 con 0x7fdf44106180 2024-04-23T18:40:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.277+0000 7fdf3a7fc640 1 -- 172.21.15.19:0/1613534209 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdf34042030 con 0x7fdf44106180 2024-04-23T18:40:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.277+0000 7fdf4e832740 1 Processor -- start 2024-04-23T18:40:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.277+0000 7fdf4e832740 1 -- start start 2024-04-23T18:40:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.277+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.277+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9e40 0x5631bb7d86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.277+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.277+0000 7fdf4e832740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5631bb72e970 con 0x5631bb7c9520 2024-04-23T18:40:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.277+0000 7fdf4e832740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5631bb6cb4a0 con 0x5631bb7c9e40 2024-04-23T18:40:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.277+0000 7fdf4e832740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5631bb72a930 con 0x5631bb7d8c10 2024-04-23T18:40:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf3a7fc640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdf300a3dc0 0x7fdf300a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42676/0 (socket says 172.21.15.19:42676) 2024-04-23T18:40:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 -- 172.21.15.19:0/2166432546 learned_addr learned my addr 172.21.15.19:0/2166432546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/2166432546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdf300a3dc0 0x7fdf300a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf497d8640 1 --2- 172.21.15.19:0/2166432546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9e40 0x5631bb7d86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf3a7fc640 1 -- 172.21.15.19:0/1613534209 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fdf340f0d10 con 0x7fdf44106180 2024-04-23T18:40:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 -- 172.21.15.19:0/2166432546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 msgr2=0x5631bb7db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 --2- 172.21.15.19:0/2166432546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7db000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 -- 172.21.15.19:0/2166432546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9e40 msgr2=0x5631bb7d86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 --2- 172.21.15.19:0/2166432546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9e40 0x5631bb7d86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 -- 172.21.15.19:0/2166432546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631bb441210 con 0x5631bb7c9520 2024-04-23T18:40:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 --2- 172.21.15.19:0/2166432546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7c9900 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c012d00 tx=0x7fdf3c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d1eb1f06002020 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.278+0000 7fdf49fd9640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdf300a3dc0 0x7fdf300a6280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c010c40 tx=0x7fdf3c00c000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.279+0000 7fdf2f7fe640 1 -- 172.21.15.19:0/2166432546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c024070 con 0x5631bb7c9520 2024-04-23T18:40:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.279+0000 7fdf2f7fe640 1 -- 172.21.15.19:0/2166432546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf3c012520 con 0x5631bb7c9520 2024-04-23T18:40:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.279+0000 7fdf2f7fe640 1 -- 172.21.15.19:0/2166432546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c005260 con 0x5631bb7c9520 2024-04-23T18:40:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.279+0000 7fdf4e832740 1 -- 172.21.15.19:0/2166432546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 msgr2=0x5631bb7c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.279+0000 7fdf4e832740 1 --2- 172.21.15.19:0/2166432546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7c9900 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c012d00 tx=0x7fdf3c010740 comp rx=0 tx=0).stop 2024-04-23T18:40:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.279+0000 7fdf4e832740 1 -- 172.21.15.19:0/2166432546 shutdown_connections 2024-04-23T18:40:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.279+0000 7fdf4e832740 1 --2- 172.21.15.19:0/2166432546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9e40 0x5631bb7d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.279+0000 7fdf4e832740 1 --2- 172.21.15.19:0/2166432546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7db000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.279+0000 7fdf4e832740 1 --2- 172.21.15.19:0/2166432546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7c9900 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.279+0000 7fdf4e832740 1 -- 172.21.15.19:0/2166432546 >> 172.21.15.19:0/2166432546 conn(0x5631bb7c5b30 msgr2=0x5631bb7c5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.280+0000 7fdf4e832740 1 -- 172.21.15.19:0/2166432546 shutdown_connections 2024-04-23T18:40:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.280+0000 7fdf4e832740 1 -- 172.21.15.19:0/2166432546 wait complete. 2024-04-23T18:40:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.280+0000 7fdf4e832740 1 -- start start 2024-04-23T18:40:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.280+0000 7fdf4e832740 1 Processor -- start 2024-04-23T18:40:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.280+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9520 0x5631bb7d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.280+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.280+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7d5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.280+0000 7fdf4e832740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5631bb7db7e0 con 0x5631bb7c9e40 2024-04-23T18:40:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.280+0000 7fdf4e832740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5631bb6c90b0 con 0x5631bb7c9520 2024-04-23T18:40:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf4e832740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5631bb7db960 con 0x5631bb7d8c10 2024-04-23T18:40:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf4e832740 10 client.0 _ll_register_callbacks cb 0x5631bb6b6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf48fd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf48fd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42682/0 (socket says 172.21.15.19:42682) 2024-04-23T18:40:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf48fd7640 1 -- 172.21.15.19:0/763777549 learned_addr learned my addr 172.21.15.19:0/763777549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf497d8640 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9520 0x5631bb7d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf49fd9640 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7d5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf48fd7640 1 -- 172.21.15.19:0/763777549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 msgr2=0x5631bb7d5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7d5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf48fd7640 1 -- 172.21.15.19:0/763777549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9520 msgr2=0x5631bb7d85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9520 0x5631bb7d85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf48fd7640 1 -- 172.21.15.19:0/763777549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631bb50d620 con 0x5631bb7c9e40 2024-04-23T18:40:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d58a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdf340b9110 tx=0x7fdf34031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67444545c93048c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:40:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf2cff9640 1 -- 172.21.15.19:0/763777549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34049040 con 0x5631bb7c9e40 2024-04-23T18:40:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf2cff9640 1 -- 172.21.15.19:0/763777549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fdf340b9320 con 0x5631bb7c9e40 2024-04-23T18:40:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.281+0000 7fdf4e832740 1 -- 172.21.15.19:0/763777549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf3c012520 con 0x5631bb7c9e40 2024-04-23T18:40:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf2cff9640 1 -- 172.21.15.19:0/763777549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf340bc610 con 0x5631bb7c9e40 2024-04-23T18:40:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf4e832740 10 client.17181 fetch_fsmap learned FSMap version 27 2024-04-23T18:40:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf4e832740 1 -- 172.21.15.19:0/763777549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631bb7e77d0 con 0x5631bb7c9e40 2024-04-23T18:40:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf2cff9640 1 -- 172.21.15.19:0/763777549 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf3c012520 con 0x5631bb7c9e40 2024-04-23T18:40:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf2cff9640 1 -- 172.21.15.19:0/763777549 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdf34081ce0 con 0x5631bb7c9e40 2024-04-23T18:40:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf4e832740 10 client.17181 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:40:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf4e832740 10 client.17181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf4e832740 20 client.17181 populate_metadata read hostname 'smithi019' 2024-04-23T18:40:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf4e832740 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631bb7e8c80 0x5631bb7eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf4e832740 4 client.17181 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf4e832740 1 -- 172.21.15.19:0/763777549 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631bb73c390 con 0x5631bb7e8c80 2024-04-23T18:40:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.282+0000 7fdf497d8640 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631bb7e8c80 0x5631bb7eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.283+0000 7fdf2cff9640 10 client.17181 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.283+0000 7fdf497d8640 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631bb7e8c80 0x5631bb7eb0b0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d6a1c3c2d203359 server_cookie=636e1e39174d95b6 in_seq=0 out_seq=0 2024-04-23T18:40:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.283+0000 7fdf2cff9640 1 -- 172.21.15.19:0/763777549 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631bb73c390 con 0x5631bb7e8c80 2024-04-23T18:40:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.283+0000 7fdf2cff9640 10 client.17181 handle_command_reply: tid=0 2024-04-23T18:40:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.283+0000 7fdf4e832740 1 client.17181 shutdown 2024-04-23T18:40:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 1 -- 172.21.15.19:0/763777549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 msgr2=0x5631bb7d58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d58a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdf340b9110 tx=0x7fdf34031a10 comp rx=0 tx=0).stop 2024-04-23T18:40:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 1 -- 172.21.15.19:0/763777549 shutdown_connections 2024-04-23T18:40:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631bb7e8c80 0x5631bb7eb0b0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7d5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d58a0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf2cff9640 0 client.17181 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 1 --2- 172.21.15.19:0/763777549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9520 0x5631bb7d85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 1 -- 172.21.15.19:0/763777549 >> 172.21.15.19:0/763777549 conn(0x5631bb7c5b30 msgr2=0x5631bb7cb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 1 -- 172.21.15.19:0/763777549 shutdown_connections 2024-04-23T18:40:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 1 -- 172.21.15.19:0/763777549 wait complete. 2024-04-23T18:40:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.284+0000 7fdf4e832740 20 client.17181 trim_cache size 0 max 16384 2024-04-23T18:40:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf4e832740 1 Processor -- start 2024-04-23T18:40:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf4e832740 1 -- start start 2024-04-23T18:40:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7c9520 0x5631bb7e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7d8c10 0x5631bb7c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf4e832740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441ab930 con 0x5631bb7c9e40 2024-04-23T18:40:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf4e832740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441ad470 con 0x5631bb7d8c10 2024-04-23T18:40:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf4e832740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441ac000 con 0x5631bb7c9520 2024-04-23T18:40:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf48fd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf48fd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42686/0 (socket says 172.21.15.19:42686) 2024-04-23T18:40:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf48fd7640 1 -- 172.21.15.19:0/3095435670 learned_addr learned my addr 172.21.15.19:0/3095435670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf48fd7640 1 -- 172.21.15.19:0/3095435670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7c9520 msgr2=0x5631bb7e46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/3095435670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7c9520 0x5631bb7e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf48fd7640 1 -- 172.21.15.19:0/3095435670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7d8c10 msgr2=0x5631bb7c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/3095435670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7d8c10 0x5631bb7c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf48fd7640 1 -- 172.21.15.19:0/3095435670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631bb7eb730 con 0x5631bb7c9e40 2024-04-23T18:40:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.299+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/3095435670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d0f00 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fdf340bcc40 tx=0x7fdf34000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a1e71984108b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf2effd640 1 -- 172.21.15.19:0/3095435670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf340bca80 con 0x5631bb7c9e40 2024-04-23T18:40:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf2effd640 1 -- 172.21.15.19:0/3095435670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf340d6210 con 0x5631bb7c9e40 2024-04-23T18:40:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf2effd640 1 -- 172.21.15.19:0/3095435670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf340b7470 con 0x5631bb7c9e40 2024-04-23T18:40:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf4e832740 1 -- 172.21.15.19:0/3095435670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 msgr2=0x5631bb7d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf4e832740 1 --2- 172.21.15.19:0/3095435670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d0f00 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fdf340bcc40 tx=0x7fdf34000bc0 comp rx=0 tx=0).stop 2024-04-23T18:40:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf4e832740 1 -- 172.21.15.19:0/3095435670 shutdown_connections 2024-04-23T18:40:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf4e832740 1 --2- 172.21.15.19:0/3095435670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7d8c10 0x5631bb7c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf4e832740 1 --2- 172.21.15.19:0/3095435670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9e40 0x5631bb7d0f00 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf4e832740 1 --2- 172.21.15.19:0/3095435670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7c9520 0x5631bb7e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf4e832740 1 -- 172.21.15.19:0/3095435670 >> 172.21.15.19:0/3095435670 conn(0x5631bb7c5b30 msgr2=0x5631bb7d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf4e832740 1 -- 172.21.15.19:0/3095435670 shutdown_connections 2024-04-23T18:40:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.300+0000 7fdf4e832740 1 -- 172.21.15.19:0/3095435670 wait complete. 2024-04-23T18:40:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf4e832740 1 -- start start 2024-04-23T18:40:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf4e832740 1 Processor -- start 2024-04-23T18:40:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7e0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9e40 0x5631bb7dc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf4e832740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7dc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf4e832740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441acda0 con 0x5631bb7c9520 2024-04-23T18:40:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf4e832740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441ac6d0 con 0x5631bb7c9e40 2024-04-23T18:40:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf4e832740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441a9950 con 0x5631bb7d8c10 2024-04-23T18:40:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf4e832740 10 client.0 _ll_register_callbacks cb 0x7fdf340095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf497d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7e0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf497d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7e0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42700/0 (socket says 172.21.15.19:42700) 2024-04-23T18:40:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf497d8640 1 -- 172.21.15.19:0/1020293739 learned_addr learned my addr 172.21.15.19:0/1020293739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9e40 0x5631bb7dc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.301+0000 7fdf49fd9640 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7dc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf497d8640 1 -- 172.21.15.19:0/1020293739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 msgr2=0x5631bb7dc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf497d8640 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7dc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf497d8640 1 -- 172.21.15.19:0/1020293739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9e40 msgr2=0x5631bb7dc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf497d8640 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9e40 0x5631bb7dc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf497d8640 1 -- 172.21.15.19:0/1020293739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631bb7dce20 con 0x5631bb7c9520 2024-04-23T18:40:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf497d8640 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7e0e80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x5631bb7eb930 tx=0x7fdf40013990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95a37e1feb03cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:40:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf2cff9640 1 -- 172.21.15.19:0/1020293739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf400130d0 con 0x5631bb7c9520 2024-04-23T18:40:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf2cff9640 1 -- 172.21.15.19:0/1020293739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fdf4001d470 con 0x5631bb7c9520 2024-04-23T18:40:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf4e832740 1 -- 172.21.15.19:0/1020293739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5631bb736200 con 0x5631bb7c9520 2024-04-23T18:40:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf2cff9640 1 -- 172.21.15.19:0/1020293739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf4002b050 con 0x5631bb7c9520 2024-04-23T18:40:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf4e832740 10 client.17187 fetch_fsmap learned FSMap version 27 2024-04-23T18:40:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf4e832740 1 -- 172.21.15.19:0/1020293739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631bb84f1a0 con 0x5631bb7c9520 2024-04-23T18:40:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf2cff9640 1 -- 172.21.15.19:0/1020293739 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5631bb736200 con 0x5631bb7c9520 2024-04-23T18:40:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.302+0000 7fdf2cff9640 1 -- 172.21.15.19:0/1020293739 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdf40003c60 con 0x5631bb7c9520 2024-04-23T18:40:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.303+0000 7fdf4e832740 10 client.17187 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:40:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.303+0000 7fdf4e832740 10 client.17187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.303+0000 7fdf4e832740 20 client.17187 populate_metadata read hostname 'smithi019' 2024-04-23T18:40:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.303+0000 7fdf4e832740 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631bb84f920 0x5631bb851d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.303+0000 7fdf4e832740 4 client.17187 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:40:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.303+0000 7fdf4e832740 1 -- 172.21.15.19:0/1020293739 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631bb6b25f0 con 0x5631bb84f920 2024-04-23T18:40:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.303+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631bb84f920 0x5631bb851d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.303+0000 7fdf48fd7640 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631bb84f920 0x5631bb851d40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7bb6b68843eeda server_cookie=7c515ecc930a1606 in_seq=0 out_seq=0 2024-04-23T18:40:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.303+0000 7fdf2cff9640 10 client.17187 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf2cff9640 1 -- 172.21.15.19:0/1020293739 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5631bb6b25f0 con 0x5631bb84f920 2024-04-23T18:40:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf2cff9640 10 client.17187 handle_command_reply: tid=0 2024-04-23T18:40:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 client.17187 shutdown 2024-04-23T18:40:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 -- 172.21.15.19:0/1020293739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 msgr2=0x5631bb7e0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7e0e80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x5631bb7eb930 tx=0x7fdf40013990 comp rx=0 tx=0).stop 2024-04-23T18:40:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 -- 172.21.15.19:0/1020293739 shutdown_connections 2024-04-23T18:40:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631bb84f920 0x5631bb851d40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631bb7c9e40 0x5631bb7dc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631bb7d8c10 0x5631bb7dc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf2cff9640 0 client.17187 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 --2- 172.21.15.19:0/1020293739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631bb7c9520 0x5631bb7e0e80 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 -- 172.21.15.19:0/1020293739 >> 172.21.15.19:0/1020293739 conn(0x5631bb7c5b30 msgr2=0x5631bb7d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 -- 172.21.15.19:0/1020293739 shutdown_connections 2024-04-23T18:40:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 1 -- 172.21.15.19:0/1020293739 wait complete. 2024-04-23T18:40:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.319+0000 7fdf4e832740 20 client.17187 trim_cache size 0 max 16384 2024-04-23T18:40:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:40:11.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:40:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:40:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1613534209 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdf300a3dc0 msgr2=0x7fdf300a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdf300a3dc0 0x7fdf300a6280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c010c40 tx=0x7fdf3c00c000 comp rx=0 tx=0).stop 2024-04-23T18:40:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1613534209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf44106180 msgr2=0x7fdf44198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf44106180 0x7fdf44198b80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdf34032fd0 tx=0x7fdf34004270 comp rx=0 tx=0).stop 2024-04-23T18:40:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1613534209 shutdown_connections 2024-04-23T18:40:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdf300a3dc0 0x7fdf300a6280 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4419cf00 0x7fdf4419d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf44106180 0x7fdf44198b80 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 --2- 172.21.15.19:0/1613534209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf441010b0 0x7fdf44198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1613534209 >> 172.21.15.19:0/1613534209 conn(0x7fdf44078000 msgr2=0x7fdf4406aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1613534209 shutdown_connections 2024-04-23T18:40:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:40.320+0000 7fdf4ba3c640 1 -- 172.21.15.19:0/1613534209 wait complete. 2024-04-23T18:40:11.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:41 smithi019 ceph-mon[29215]: pgmap v391: 97 pgs: 97 active+clean; 3.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:40:11.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:43 smithi019 ceph-mon[29215]: pgmap v392: 97 pgs: 97 active+clean; 3.5 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-23T18:40:11.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:45 smithi019 ceph-mon[29215]: pgmap v393: 97 pgs: 97 active+clean; 3.8 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:40:11.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:47 smithi019 ceph-mon[29215]: pgmap v394: 97 pgs: 97 active+clean; 3.8 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:11.902 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:49 smithi019 ceph-mon[29215]: pgmap v395: 97 pgs: 97 active+clean; 3.8 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:11.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:51 smithi019 ceph-mon[29215]: pgmap v396: 97 pgs: 97 active+clean; 3.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:40:12.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:01 smithi155 ceph-mon[33468]: pgmap v434: 97 pgs: 97 active+clean; 3.0 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-04-23T18:40:12.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:03 smithi155 ceph-mon[33468]: pgmap v435: 97 pgs: 97 active+clean; 3.0 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:12.102 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:05 smithi155 ceph-mon[33468]: pgmap v436: 97 pgs: 97 active+clean; 3.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-23T18:40:12.135 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:06 smithi155 ceph-mon[33468]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T18:40:12.168 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:06 smithi155 ceph-mon[33468]: pgmap v438: 97 pgs: 97 active+clean; 3.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:40:12.202 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:09 smithi155 ceph-mon[33468]: pgmap v439: 97 pgs: 97 active+clean; 3.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:40:12.235 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:11 smithi155 ceph-mon[33468]: pgmap v440: 97 pgs: 97 active+clean; 3.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-04-23T18:40:12.269 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:01 smithi044 ceph-mon[32458]: pgmap v434: 97 pgs: 97 active+clean; 3.0 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-04-23T18:40:12.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:03 smithi044 ceph-mon[32458]: pgmap v435: 97 pgs: 97 active+clean; 3.0 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:12.419 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:05 smithi044 ceph-mon[32458]: pgmap v436: 97 pgs: 97 active+clean; 3.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-23T18:40:12.477 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:06 smithi044 ceph-mon[32458]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T18:40:12.510 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:06 smithi044 ceph-mon[32458]: pgmap v438: 97 pgs: 97 active+clean; 3.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:40:12.544 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:09 smithi044 ceph-mon[32458]: pgmap v439: 97 pgs: 97 active+clean; 3.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:40:12.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:11 smithi044 ceph-mon[32458]: pgmap v440: 97 pgs: 97 active+clean; 3.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-04-23T18:40:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.419+0000 7fc1905bb640 1 -- 172.21.15.19:0/1214976931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881016a0 msgr2=0x7fc18810e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.419+0000 7fc1905bb640 1 --2- 172.21.15.19:0/1214976931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881016a0 0x7fc18810e3f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fc178009900 tx=0x7fc17802a410 comp rx=0 tx=0).stop 2024-04-23T18:40:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.419+0000 7fc1905bb640 1 -- 172.21.15.19:0/1214976931 shutdown_connections 2024-04-23T18:40:13.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.419+0000 7fc1905bb640 1 --2- 172.21.15.19:0/1214976931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18810e930 0x7fc188110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.419+0000 7fc1905bb640 1 --2- 172.21.15.19:0/1214976931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881016a0 0x7fc18810e3f0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.419+0000 7fc1905bb640 1 --2- 172.21.15.19:0/1214976931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc188100d80 0x7fc188101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.419+0000 7fc1905bb640 1 -- 172.21.15.19:0/1214976931 >> 172.21.15.19:0/1214976931 conn(0x7fc188077fd0 msgr2=0x7fc1880783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.419+0000 7fc1905bb640 1 -- 172.21.15.19:0/1214976931 shutdown_connections 2024-04-23T18:40:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.419+0000 7fc1905bb640 1 -- 172.21.15.19:0/1214976931 wait complete. 2024-04-23T18:40:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc1905bb640 1 Processor -- start 2024-04-23T18:40:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc1905bb640 1 -- start start 2024-04-23T18:40:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc1905bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc188100d80 0x7fc1881a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc1905bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881016a0 0x7fc1881a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc1905bb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18810e930 0x7fc18819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc1905bb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881143e0 con 0x7fc1881016a0 2024-04-23T18:40:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc1905bb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc188114260 con 0x7fc188100d80 2024-04-23T18:40:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc1905bb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc188114560 con 0x7fc18810e930 2024-04-23T18:40:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc18db56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881016a0 0x7fc1881a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc18db56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881016a0 0x7fc1881a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57554/0 (socket says 172.21.15.19:57554) 2024-04-23T18:40:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc18db56640 1 -- 172.21.15.19:0/2832563325 learned_addr learned my addr 172.21.15.19:0/2832563325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc18e357640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc188100d80 0x7fc1881a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc18eb58640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18810e930 0x7fc18819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc18db56640 1 -- 172.21.15.19:0/2832563325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18810e930 msgr2=0x7fc18819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc18db56640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18810e930 0x7fc18819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc18db56640 1 -- 172.21.15.19:0/2832563325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc188100d80 msgr2=0x7fc1881a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc18db56640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc188100d80 0x7fc1881a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.420+0000 7fc18db56640 1 -- 172.21.15.19:0/2832563325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18819afe0 con 0x7fc1881016a0 2024-04-23T18:40:14.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.421+0000 7fc18db56640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881016a0 0x7fc1881a0ba0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc178005940 tx=0x7fc1780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.421+0000 7fc17f7fe640 1 -- 172.21.15.19:0/2832563325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17803f030 con 0x7fc1881016a0 2024-04-23T18:40:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.421+0000 7fc1905bb640 1 -- 172.21.15.19:0/2832563325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18819b270 con 0x7fc1881016a0 2024-04-23T18:40:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.421+0000 7fc17f7fe640 1 -- 172.21.15.19:0/2832563325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc178003cb0 con 0x7fc1881016a0 2024-04-23T18:40:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.421+0000 7fc17f7fe640 1 -- 172.21.15.19:0/2832563325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178035070 con 0x7fc1881016a0 2024-04-23T18:40:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.421+0000 7fc1905bb640 1 -- 172.21.15.19:0/2832563325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1881a7400 con 0x7fc1881016a0 2024-04-23T18:40:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.423+0000 7fc17f7fe640 1 -- 172.21.15.19:0/2832563325 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc178042030 con 0x7fc1881016a0 2024-04-23T18:40:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc1933ae740 1 Processor -- start 2024-04-23T18:40:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc1933ae740 1 -- start start 2024-04-23T18:40:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e13620 0x562e27e13a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 0x562e27e1e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e1e800 0x562e27e20bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc1933ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e27d78970 con 0x562e27e1e800 2024-04-23T18:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc1933ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562e27d154a0 con 0x562e27e13620 2024-04-23T18:40:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc1933ae740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562e27d74930 con 0x562e27e13f40 2024-04-23T18:40:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc17f7fe640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc1740a3dc0 0x7fc1740a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc18db56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e1e800 0x562e27e20bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc18db56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e1e800 0x562e27e20bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57562/0 (socket says 172.21.15.19:57562) 2024-04-23T18:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc18db56640 1 -- 172.21.15.19:0/1784014439 learned_addr learned my addr 172.21.15.19:0/1784014439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.424+0000 7fc18e357640 1 --2- 172.21.15.19:0/1784014439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 0x562e27e1e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc18eb58640 1 --2- 172.21.15.19:0/1784014439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e13620 0x562e27e13a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc18eb58640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc1740a3dc0 0x7fc1740a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc17f7fe640 1 -- 172.21.15.19:0/2832563325 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fc1780f1190 con 0x7fc1881016a0 2024-04-23T18:40:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc18e357640 1 -- 172.21.15.19:0/1784014439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e13620 msgr2=0x562e27e13a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc18e357640 1 --2- 172.21.15.19:0/1784014439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e13620 0x562e27e13a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc18e357640 1 -- 172.21.15.19:0/1784014439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e1e800 msgr2=0x562e27e20bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc18e357640 1 --2- 172.21.15.19:0/1784014439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e1e800 0x562e27e20bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc18e357640 1 -- 172.21.15.19:0/1784014439 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e27a8b210 con 0x562e27e13f40 2024-04-23T18:40:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc18e357640 1 --2- 172.21.15.19:0/1784014439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 0x562e27e1e2c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc184009ee0 tx=0x7fc18400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c185bdc160c5f3c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc18eb58640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc1740a3dc0 0x7fc1740a6280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc1800058c0 tx=0x7fc18000c690 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc17cbfe640 1 -- 172.21.15.19:0/1784014439 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184011640 con 0x562e27e13f40 2024-04-23T18:40:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc17cbfe640 1 -- 172.21.15.19:0/1784014439 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc184011c90 con 0x562e27e13f40 2024-04-23T18:40:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.425+0000 7fc17cbfe640 1 -- 172.21.15.19:0/1784014439 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184010b50 con 0x562e27e13f40 2024-04-23T18:40:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.426+0000 7fc1933ae740 1 -- 172.21.15.19:0/1784014439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 msgr2=0x562e27e1e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.426+0000 7fc1933ae740 1 --2- 172.21.15.19:0/1784014439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 0x562e27e1e2c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc184009ee0 tx=0x7fc18400f730 comp rx=0 tx=0).stop 2024-04-23T18:40:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.426+0000 7fc1933ae740 1 -- 172.21.15.19:0/1784014439 shutdown_connections 2024-04-23T18:40:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.426+0000 7fc1933ae740 1 --2- 172.21.15.19:0/1784014439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e1e800 0x562e27e20bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.426+0000 7fc1933ae740 1 --2- 172.21.15.19:0/1784014439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 0x562e27e1e2c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.426+0000 7fc1933ae740 1 --2- 172.21.15.19:0/1784014439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e13620 0x562e27e13a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.426+0000 7fc1933ae740 1 -- 172.21.15.19:0/1784014439 >> 172.21.15.19:0/1784014439 conn(0x562e27e0fb30 msgr2=0x562e27e0ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.426+0000 7fc1933ae740 1 -- 172.21.15.19:0/1784014439 shutdown_connections 2024-04-23T18:40:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.426+0000 7fc1933ae740 1 -- 172.21.15.19:0/1784014439 wait complete. 2024-04-23T18:40:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc1933ae740 1 -- start start 2024-04-23T18:40:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc1933ae740 1 Processor -- start 2024-04-23T18:40:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13620 0x562e27e1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13f40 0x562e27e1b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e1e800 0x562e27e1b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc1933ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e27e21430 con 0x562e27e13f40 2024-04-23T18:40:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc1933ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562e27d130b0 con 0x562e27e1e800 2024-04-23T18:40:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc1933ae740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562e27e215b0 con 0x562e27e13620 2024-04-23T18:40:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc1933ae740 10 client.0 _ll_register_callbacks cb 0x562e27d00d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18db56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13f40 0x562e27e1b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18db56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13f40 0x562e27e1b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57566/0 (socket says 172.21.15.19:57566) 2024-04-23T18:40:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18db56640 1 -- 172.21.15.19:0/3156749009 learned_addr learned my addr 172.21.15.19:0/3156749009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18e357640 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13620 0x562e27e1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18eb58640 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e1e800 0x562e27e1b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18db56640 1 -- 172.21.15.19:0/3156749009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13620 msgr2=0x562e27e1e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18db56640 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13620 0x562e27e1e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18db56640 1 -- 172.21.15.19:0/3156749009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e1e800 msgr2=0x562e27e1b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18db56640 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e1e800 0x562e27e1b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18db56640 1 -- 172.21.15.19:0/3156749009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e27b57620 con 0x562e27e13f40 2024-04-23T18:40:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.427+0000 7fc18eb58640 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e1e800 0x562e27e1b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:40:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc18db56640 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13f40 0x562e27e1b400 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc178004650 tx=0x7fc17803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbca1c7bf53a029 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc171ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:40:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc171ffb640 1 -- 172.21.15.19:0/3156749009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178049040 con 0x562e27e13f40 2024-04-23T18:40:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc171ffb640 1 -- 172.21.15.19:0/3156749009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fc1780eeb00 con 0x562e27e13f40 2024-04-23T18:40:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc1933ae740 1 -- 172.21.15.19:0/3156749009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc184011c90 con 0x562e27e13f40 2024-04-23T18:40:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc171ffb640 1 -- 172.21.15.19:0/3156749009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc178005d50 con 0x562e27e13f40 2024-04-23T18:40:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc1933ae740 10 client.17199 fetch_fsmap learned FSMap version 27 2024-04-23T18:40:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc1933ae740 1 -- 172.21.15.19:0/3156749009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e27e2d310 con 0x562e27e13f40 2024-04-23T18:40:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc171ffb640 1 -- 172.21.15.19:0/3156749009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc184011c90 con 0x562e27e13f40 2024-04-23T18:40:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc171ffb640 1 -- 172.21.15.19:0/3156749009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc17803bbf0 con 0x562e27e13f40 2024-04-23T18:40:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc1933ae740 10 client.17199 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:40:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc1933ae740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.428+0000 7fc1933ae740 20 client.17199 populate_metadata read hostname 'smithi019' 2024-04-23T18:40:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.429+0000 7fc1933ae740 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e27e2e7c0 0x562e27e30c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.429+0000 7fc1933ae740 4 client.17199 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:40:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.429+0000 7fc1933ae740 1 -- 172.21.15.19:0/3156749009 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e27d86390 con 0x562e27e2e7c0 2024-04-23T18:40:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.429+0000 7fc18e357640 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e27e2e7c0 0x562e27e30c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.429+0000 7fc18e357640 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e27e2e7c0 0x562e27e30c60 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a2e5b1caa1e52d server_cookie=f0b33571482eef in_seq=0 out_seq=0 2024-04-23T18:40:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.429+0000 7fc171ffb640 10 client.17199 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc171ffb640 1 -- 172.21.15.19:0/3156749009 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e27d86390 con 0x562e27e2e7c0 2024-04-23T18:40:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc171ffb640 10 client.17199 handle_command_reply: tid=0 2024-04-23T18:40:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 client.17199 shutdown 2024-04-23T18:40:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 -- 172.21.15.19:0/3156749009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13f40 msgr2=0x562e27e1b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13f40 0x562e27e1b400 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc178004650 tx=0x7fc17803b000 comp rx=0 tx=0).stop 2024-04-23T18:40:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 -- 172.21.15.19:0/3156749009 shutdown_connections 2024-04-23T18:40:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e27e2e7c0 0x562e27e30c60 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e1e800 0x562e27e1b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13f40 0x562e27e1b400 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc171ffb640 0 client.17199 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 --2- 172.21.15.19:0/3156749009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13620 0x562e27e1e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 -- 172.21.15.19:0/3156749009 >> 172.21.15.19:0/3156749009 conn(0x562e27e0fb30 msgr2=0x562e27e15860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 -- 172.21.15.19:0/3156749009 shutdown_connections 2024-04-23T18:40:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.430+0000 7fc1933ae740 1 -- 172.21.15.19:0/3156749009 wait complete. 2024-04-23T18:40:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.431+0000 7fc1933ae740 20 client.17199 trim_cache size 0 max 16384 2024-04-23T18:40:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc1933ae740 1 Processor -- start 2024-04-23T18:40:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc1933ae740 1 -- start start 2024-04-23T18:40:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e0fb30 0x562e27e2a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e0eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 0x562e27e36570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc1933ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881afe20 con 0x562e27e13620 2024-04-23T18:40:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc1933ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881b1960 con 0x562e27e0fb30 2024-04-23T18:40:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc1933ae740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881b04f0 con 0x562e27e13f40 2024-04-23T18:40:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18db56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e0eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18db56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e0eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57578/0 (socket says 172.21.15.19:57578) 2024-04-23T18:40:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18db56640 1 -- 172.21.15.19:0/1123319075 learned_addr learned my addr 172.21.15.19:0/1123319075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18eb58640 1 --2- 172.21.15.19:0/1123319075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 0x562e27e36570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18db56640 1 -- 172.21.15.19:0/1123319075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 msgr2=0x562e27e36570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18db56640 1 --2- 172.21.15.19:0/1123319075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 0x562e27e36570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18db56640 1 -- 172.21.15.19:0/1123319075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e0fb30 msgr2=0x562e27e2a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18e357640 1 --2- 172.21.15.19:0/1123319075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e0fb30 0x562e27e2a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18db56640 1 --2- 172.21.15.19:0/1123319075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e0fb30 0x562e27e2a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18db56640 1 -- 172.21.15.19:0/1123319075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e27e0f660 con 0x562e27e13620 2024-04-23T18:40:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.446+0000 7fc18db56640 1 --2- 172.21.15.19:0/1123319075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e0eb30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc1780bb380 tx=0x7fc178032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cddebd3c8073fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc173fff640 1 -- 172.21.15.19:0/1123319075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178049070 con 0x562e27e13620 2024-04-23T18:40:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc173fff640 1 -- 172.21.15.19:0/1123319075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1780c1260 con 0x562e27e13620 2024-04-23T18:40:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc173fff640 1 -- 172.21.15.19:0/1123319075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178003e60 con 0x562e27e13620 2024-04-23T18:40:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc1933ae740 1 -- 172.21.15.19:0/1123319075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 msgr2=0x562e27e0eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc1933ae740 1 --2- 172.21.15.19:0/1123319075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e0eb30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc1780bb380 tx=0x7fc178032f60 comp rx=0 tx=0).stop 2024-04-23T18:40:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc1933ae740 1 -- 172.21.15.19:0/1123319075 shutdown_connections 2024-04-23T18:40:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc1933ae740 1 --2- 172.21.15.19:0/1123319075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e13f40 0x562e27e36570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc1933ae740 1 --2- 172.21.15.19:0/1123319075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e0eb30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc1933ae740 1 --2- 172.21.15.19:0/1123319075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e0fb30 0x562e27e2a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc1933ae740 1 -- 172.21.15.19:0/1123319075 >> 172.21.15.19:0/1123319075 conn(0x562e27e1baa0 msgr2=0x562e27e0e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc1933ae740 1 -- 172.21.15.19:0/1123319075 shutdown_connections 2024-04-23T18:40:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc1933ae740 1 -- 172.21.15.19:0/1123319075 wait complete. 2024-04-23T18:40:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.447+0000 7fc1933ae740 1 -- start start 2024-04-23T18:40:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc1933ae740 1 Processor -- start 2024-04-23T18:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e0fb30 0x562e27e24420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e24960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc1933ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e13f40 0x562e27e24ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc1933ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881b1290 con 0x562e27e13620 2024-04-23T18:40:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc1933ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881b0bc0 con 0x562e27e13f40 2024-04-23T18:40:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc1933ae740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881ade20 con 0x562e27e0fb30 2024-04-23T18:40:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc1933ae740 10 client.0 _ll_register_callbacks cb 0x7fc178009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:21.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18db56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e24960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18db56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e24960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57586/0 (socket says 172.21.15.19:57586) 2024-04-23T18:40:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18db56640 1 -- 172.21.15.19:0/2179110036 learned_addr learned my addr 172.21.15.19:0/2179110036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18e357640 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e0fb30 0x562e27e24420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18eb58640 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e13f40 0x562e27e24ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18db56640 1 -- 172.21.15.19:0/2179110036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e0fb30 msgr2=0x562e27e24420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18db56640 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e0fb30 0x562e27e24420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18db56640 1 -- 172.21.15.19:0/2179110036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e13f40 msgr2=0x562e27e24ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18db56640 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e13f40 0x562e27e24ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18db56640 1 -- 172.21.15.19:0/2179110036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e27e93ff0 con 0x562e27e13620 2024-04-23T18:40:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc18db56640 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e24960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc178006010 tx=0x7fc1780bb5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2ace4c2bf44a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.448+0000 7fc171ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:40:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc171ffb640 1 -- 172.21.15.19:0/2179110036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178049050 con 0x562e27e13620 2024-04-23T18:40:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc171ffb640 1 -- 172.21.15.19:0/2179110036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fc1780d3cd0 con 0x562e27e13620 2024-04-23T18:40:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc1933ae740 1 -- 172.21.15.19:0/2179110036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e27d80200 con 0x562e27e13620 2024-04-23T18:40:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc171ffb640 1 -- 172.21.15.19:0/2179110036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc1780eed10 con 0x562e27e13620 2024-04-23T18:40:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc1933ae740 10 client.17205 fetch_fsmap learned FSMap version 27 2024-04-23T18:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc1933ae740 1 -- 172.21.15.19:0/2179110036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e27e93ff0 con 0x562e27e13620 2024-04-23T18:40:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc171ffb640 1 -- 172.21.15.19:0/2179110036 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562e27d80200 con 0x562e27e13620 2024-04-23T18:40:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc171ffb640 1 -- 172.21.15.19:0/2179110036 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc1780d1b70 con 0x562e27e13620 2024-04-23T18:40:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc1933ae740 10 client.17205 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:40:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc1933ae740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc1933ae740 20 client.17205 populate_metadata read hostname 'smithi019' 2024-04-23T18:40:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc1933ae740 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e27e1e800 0x562e27e975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc1933ae740 4 client.17205 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:40:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc1933ae740 1 -- 172.21.15.19:0/2179110036 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e27cfc5f0 con 0x562e27e1e800 2024-04-23T18:40:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.449+0000 7fc18e357640 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e27e1e800 0x562e27e975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.450+0000 7fc171ffb640 10 client.17205 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.450+0000 7fc18e357640 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e27e1e800 0x562e27e975a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f5199e32f58032b server_cookie=70643a2346bab44b in_seq=0 out_seq=0 2024-04-23T18:40:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc171ffb640 1 -- 172.21.15.19:0/2179110036 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e27cfc5f0 con 0x562e27e1e800 2024-04-23T18:40:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc171ffb640 10 client.17205 handle_command_reply: tid=0 2024-04-23T18:40:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 client.17205 shutdown 2024-04-23T18:40:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 -- 172.21.15.19:0/2179110036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 msgr2=0x562e27e24960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e24960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc178006010 tx=0x7fc1780bb5e0 comp rx=0 tx=0).stop 2024-04-23T18:40:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 -- 172.21.15.19:0/2179110036 shutdown_connections 2024-04-23T18:40:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e27e1e800 0x562e27e975a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e27e13f40 0x562e27e24ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e27e13620 0x562e27e24960 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc171ffb640 0 client.17205 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 --2- 172.21.15.19:0/2179110036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e27e0fb30 0x562e27e24420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 -- 172.21.15.19:0/2179110036 >> 172.21.15.19:0/2179110036 conn(0x562e27e1baa0 msgr2=0x562e27e0e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 -- 172.21.15.19:0/2179110036 shutdown_connections 2024-04-23T18:40:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 1 -- 172.21.15.19:0/2179110036 wait complete. 2024-04-23T18:40:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.465+0000 7fc1933ae740 20 client.17205 trim_cache size 0 max 16384 2024-04-23T18:40:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:40:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:40:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:40:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:40:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 -- 172.21.15.19:0/2832563325 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc1740a3dc0 msgr2=0x7fc1740a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc1740a3dc0 0x7fc1740a6280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc1800058c0 tx=0x7fc18000c690 comp rx=0 tx=0).stop 2024-04-23T18:40:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 -- 172.21.15.19:0/2832563325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881016a0 msgr2=0x7fc1881a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881016a0 0x7fc1881a0ba0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc178005940 tx=0x7fc1780042f0 comp rx=0 tx=0).stop 2024-04-23T18:40:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 -- 172.21.15.19:0/2832563325 shutdown_connections 2024-04-23T18:40:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc1740a3dc0 0x7fc1740a6280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18810e930 0x7fc18819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1881016a0 0x7fc1881a0ba0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 --2- 172.21.15.19:0/2832563325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc188100d80 0x7fc1881a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 -- 172.21.15.19:0/2832563325 >> 172.21.15.19:0/2832563325 conn(0x7fc188077fd0 msgr2=0x7fc18810fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 -- 172.21.15.19:0/2832563325 shutdown_connections 2024-04-23T18:40:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:38:53.466+0000 7fc1905bb640 1 -- 172.21.15.19:0/2832563325 wait complete. 2024-04-23T18:40:24.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:13 smithi155 ceph-mon[33468]: pgmap v441: 97 pgs: 97 active+clean; 3.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-04-23T18:40:24.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:13 smithi155 ceph-mon[33468]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T18:40:24.355 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:15 smithi155 ceph-mon[33468]: pgmap v443: 97 pgs: 97 active+clean; 3.7 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-04-23T18:40:24.427 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:17 smithi155 ceph-mon[33468]: pgmap v444: 97 pgs: 97 active+clean; 3.7 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:40:24.513 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:19 smithi155 ceph-mon[33468]: pgmap v445: 97 pgs: 97 active+clean; 3.7 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:40:24.546 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:21 smithi155 ceph-mon[33468]: pgmap v446: 97 pgs: 97 active+clean; 3.9 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:40:24.580 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:13 smithi044 ceph-mon[32458]: pgmap v441: 97 pgs: 97 active+clean; 3.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-04-23T18:40:24.648 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:13 smithi044 ceph-mon[32458]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T18:40:24.680 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:15 smithi044 ceph-mon[32458]: pgmap v443: 97 pgs: 97 active+clean; 3.7 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-04-23T18:40:24.739 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:17 smithi044 ceph-mon[32458]: pgmap v444: 97 pgs: 97 active+clean; 3.7 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:40:24.763 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:19 smithi044 ceph-mon[32458]: pgmap v445: 97 pgs: 97 active+clean; 3.7 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:40:24.797 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:21 smithi044 ceph-mon[32458]: pgmap v446: 97 pgs: 97 active+clean; 3.9 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:40:24.832 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:53 smithi019 ceph-mon[29215]: pgmap v397: 97 pgs: 97 active+clean; 3.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T18:40:24.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:55 smithi019 ceph-mon[29215]: pgmap v398: 97 pgs: 97 active+clean; 4.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-23T18:40:24.945 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:57 smithi019 ceph-mon[29215]: pgmap v399: 97 pgs: 97 active+clean; 4.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:25.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:38:59 smithi019 ceph-mon[29215]: pgmap v400: 97 pgs: 97 active+clean; 4.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:25.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:01 smithi019 ceph-mon[29215]: pgmap v401: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:40:25.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:03 smithi019 ceph-mon[29215]: pgmap v402: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:40:25.215 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:05 smithi019 ceph-mon[29215]: pgmap v403: 97 pgs: 97 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T18:40:25.248 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:23 smithi155 ceph-mon[33468]: pgmap v447: 97 pgs: 97 active+clean; 3.9 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:40:25.297 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:23 smithi044 ceph-mon[32458]: pgmap v447: 97 pgs: 97 active+clean; 3.9 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:40:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.564+0000 7fa12c308640 1 -- 172.21.15.19:0/3024560344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1241072a0 msgr2=0x7fa12410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.564+0000 7fa12c308640 1 --2- 172.21.15.19:0/3024560344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1241072a0 0x7fa12410ee70 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009ca0 tx=0x7fa11c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:40:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.564+0000 7fa12c308640 1 -- 172.21.15.19:0/3024560344 shutdown_connections 2024-04-23T18:40:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.564+0000 7fa12c308640 1 --2- 172.21.15.19:0/3024560344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1241072a0 0x7fa12410ee70 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.564+0000 7fa12c308640 1 --2- 172.21.15.19:0/3024560344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa124102b80 0x7fa124106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.564+0000 7fa12c308640 1 --2- 172.21.15.19:0/3024560344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1241021d0 0x7fa1241025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.564+0000 7fa12c308640 1 -- 172.21.15.19:0/3024560344 >> 172.21.15.19:0/3024560344 conn(0x7fa1240793e0 msgr2=0x7fa1240797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.564+0000 7fa12c308640 1 -- 172.21.15.19:0/3024560344 shutdown_connections 2024-04-23T18:40:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.564+0000 7fa12c308640 1 -- 172.21.15.19:0/3024560344 wait complete. 2024-04-23T18:40:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.565+0000 7fa12c308640 1 Processor -- start 2024-04-23T18:40:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.565+0000 7fa12c308640 1 -- start start 2024-04-23T18:40:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.565+0000 7fa12c308640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1241021d0 0x7fa124198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.565+0000 7fa12c308640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1241072a0 0x7fa124198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.565+0000 7fa12c308640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12419ced0 0x7fa12419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.565+0000 7fa12c308640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa124112210 con 0x7fa12419ced0 2024-04-23T18:40:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.565+0000 7fa12c308640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa124112090 con 0x7fa1241021d0 2024-04-23T18:40:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.565+0000 7fa12c308640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa124112390 con 0x7fa1241072a0 2024-04-23T18:40:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12a8a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12419ced0 0x7fa12419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12a8a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12419ced0 0x7fa12419d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60816/0 (socket says 172.21.15.19:60816) 2024-04-23T18:40:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12a8a5640 1 -- 172.21.15.19:0/1535944499 learned_addr learned my addr 172.21.15.19:0/1535944499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12a0a4640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1241021d0 0x7fa124198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa1298a3640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1241072a0 0x7fa124198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12a8a5640 1 -- 172.21.15.19:0/1535944499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1241072a0 msgr2=0x7fa124198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12a8a5640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1241072a0 0x7fa124198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12a8a5640 1 -- 172.21.15.19:0/1535944499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1241021d0 msgr2=0x7fa124198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12a8a5640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1241021d0 0x7fa124198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12a8a5640 1 -- 172.21.15.19:0/1535944499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12419d9c0 con 0x7fa12419ced0 2024-04-23T18:40:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12a8a5640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12419ced0 0x7fa12419d380 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0098d0 tx=0x7fa11c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa11b7fe640 1 -- 172.21.15.19:0/1535944499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c003b70 con 0x7fa12419ced0 2024-04-23T18:40:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa11b7fe640 1 -- 172.21.15.19:0/1535944499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa11c003d10 con 0x7fa12419ced0 2024-04-23T18:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12c308640 1 -- 172.21.15.19:0/1535944499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12419dc50 con 0x7fa12419ced0 2024-04-23T18:40:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa11b7fe640 1 -- 172.21.15.19:0/1535944499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c02f9d0 con 0x7fa12419ced0 2024-04-23T18:40:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.566+0000 7fa12c308640 1 -- 172.21.15.19:0/1535944499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1241a5470 con 0x7fa12419ced0 2024-04-23T18:40:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.569+0000 7fa11b7fe640 1 -- 172.21.15.19:0/1535944499 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa11c02fb70 con 0x7fa12419ced0 2024-04-23T18:40:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.569+0000 7fa12f024740 1 Processor -- start 2024-04-23T18:40:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.569+0000 7fa12f024740 1 -- start start 2024-04-23T18:40:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12f024740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3620 0x5567b4cd3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12f024740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3f40 0x5567b4cde2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12f024740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 0x5567b4ce0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12f024740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567b4c38970 con 0x5567b4cd3f40 2024-04-23T18:40:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12f024740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5567b4bd54a0 con 0x5567b4cd3620 2024-04-23T18:40:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12f024740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567b4c34930 con 0x5567b4cde800 2024-04-23T18:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12a0a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3f40 0x5567b4cde2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12a0a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3f40 0x5567b4cde2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60826/0 (socket says 172.21.15.19:60826) 2024-04-23T18:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12a0a4640 1 -- 172.21.15.19:0/3982195785 learned_addr learned my addr 172.21.15.19:0/3982195785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa1298a3640 1 --2- 172.21.15.19:0/3982195785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3620 0x5567b4cd3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:28.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa1298a3640 1 --2- 172.21.15.19:0/3982195785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 0x5567b4ce0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa11b7fe640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa1100a3dc0 0x7fa1100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12a0a4640 1 -- 172.21.15.19:0/3982195785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 msgr2=0x5567b4ce0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12a0a4640 1 --2- 172.21.15.19:0/3982195785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 0x5567b4ce0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12a0a4640 1 -- 172.21.15.19:0/3982195785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3620 msgr2=0x5567b4cd3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12a0a4640 1 --2- 172.21.15.19:0/3982195785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3620 0x5567b4cd3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12a0a4640 1 -- 172.21.15.19:0/3982195785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b494b210 con 0x5567b4cd3f40 2024-04-23T18:40:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa1298a3640 1 --2- 172.21.15.19:0/3982195785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 0x5567b4ce0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:40:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12a8a5640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa1100a3dc0 0x7fa1100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa11b7fe640 1 -- 172.21.15.19:0/1535944499 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8557+0+0 (secure 0 0 0) 0x7fa11c0f1c90 con 0x7fa12419ced0 2024-04-23T18:40:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.570+0000 7fa12a0a4640 1 --2- 172.21.15.19:0/3982195785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3f40 0x5567b4cde2c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa12000ad00 tx=0x7fa12000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c3165a9ed8316a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa118bfe640 1 -- 172.21.15.19:0/3982195785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12001d070 con 0x5567b4cd3f40 2024-04-23T18:40:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa118bfe640 1 -- 172.21.15.19:0/3982195785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa120018af0 con 0x5567b4cd3f40 2024-04-23T18:40:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa118bfe640 1 -- 172.21.15.19:0/3982195785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120010b30 con 0x5567b4cd3f40 2024-04-23T18:40:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa12f024740 1 -- 172.21.15.19:0/3982195785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3f40 msgr2=0x5567b4cde2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa12a8a5640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa1100a3dc0 0x7fa1100a6280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0ef580 tx=0x7fa11c03e040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa12f024740 1 --2- 172.21.15.19:0/3982195785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3f40 0x5567b4cde2c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa12000ad00 tx=0x7fa12000f790 comp rx=0 tx=0).stop 2024-04-23T18:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa12f024740 1 -- 172.21.15.19:0/3982195785 shutdown_connections 2024-04-23T18:40:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa12f024740 1 --2- 172.21.15.19:0/3982195785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 0x5567b4ce0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa12f024740 1 --2- 172.21.15.19:0/3982195785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3f40 0x5567b4cde2c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa12f024740 1 --2- 172.21.15.19:0/3982195785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3620 0x5567b4cd3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.571+0000 7fa12f024740 1 -- 172.21.15.19:0/3982195785 >> 172.21.15.19:0/3982195785 conn(0x5567b4ccfb30 msgr2=0x5567b4ccff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 1 -- 172.21.15.19:0/3982195785 shutdown_connections 2024-04-23T18:40:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 1 -- 172.21.15.19:0/3982195785 wait complete. 2024-04-23T18:40:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 1 -- start start 2024-04-23T18:40:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 1 Processor -- start 2024-04-23T18:40:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cd3620 0x5567b4cde090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3f40 0x5567b4cdb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cdb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567b4ce1430 con 0x5567b4cde800 2024-04-23T18:40:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5567b4bd30b0 con 0x5567b4cd3f40 2024-04-23T18:40:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567b4ce15b0 con 0x5567b4cd3620 2024-04-23T18:40:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa12f024740 10 client.0 _ll_register_callbacks cb 0x5567b4bc0d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa1298a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cdb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa1298a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cdb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60842/0 (socket says 172.21.15.19:60842) 2024-04-23T18:40:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa1298a3640 1 -- 172.21.15.19:0/169529043 learned_addr learned my addr 172.21.15.19:0/169529043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.572+0000 7fa1298a3640 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cd3620 0x5567b4cde090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa12a0a4640 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3f40 0x5567b4cdb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa1298a3640 1 -- 172.21.15.19:0/169529043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cd3620 msgr2=0x5567b4cde090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa1298a3640 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cd3620 0x5567b4cde090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa1298a3640 1 -- 172.21.15.19:0/169529043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3f40 msgr2=0x5567b4cdb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa1298a3640 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3f40 0x5567b4cdb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa1298a3640 1 -- 172.21.15.19:0/169529043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b4a17620 con 0x5567b4cde800 2024-04-23T18:40:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa1298a3640 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cdb8c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa12406b630 tx=0x7fa114014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ac7aa0469e5b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa10dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:40:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa10dffb640 1 -- 172.21.15.19:0/169529043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11400f970 con 0x5567b4cde800 2024-04-23T18:40:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa10dffb640 1 -- 172.21.15.19:0/169529043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8557+0+0 (secure 0 0 0) 0x7fa11400fb10 con 0x5567b4cde800 2024-04-23T18:40:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa12f024740 1 -- 172.21.15.19:0/169529043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa120018af0 con 0x5567b4cde800 2024-04-23T18:40:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa10dffb640 1 -- 172.21.15.19:0/169529043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa11400fd10 con 0x5567b4cde800 2024-04-23T18:40:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa12f024740 10 client.17217 fetch_fsmap learned FSMap version 27 2024-04-23T18:40:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa12f024740 1 -- 172.21.15.19:0/169529043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567b4ced340 con 0x5567b4cde800 2024-04-23T18:40:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.573+0000 7fa10dffb640 1 -- 172.21.15.19:0/169529043 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa120018af0 con 0x5567b4cde800 2024-04-23T18:40:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.574+0000 7fa10dffb640 1 -- 172.21.15.19:0/169529043 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa114004a70 con 0x5567b4cde800 2024-04-23T18:40:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.574+0000 7fa12f024740 10 client.17217 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:40:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.574+0000 7fa12f024740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.574+0000 7fa12f024740 20 client.17217 populate_metadata read hostname 'smithi019' 2024-04-23T18:40:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.574+0000 7fa12f024740 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567b4cee800 0x5567b4cf0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.574+0000 7fa12f024740 4 client.17217 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:40:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.574+0000 7fa12f024740 1 -- 172.21.15.19:0/169529043 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567b4c46390 con 0x5567b4cee800 2024-04-23T18:40:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.574+0000 7fa12a0a4640 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567b4cee800 0x5567b4cf0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.574+0000 7fa12a0a4640 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567b4cee800 0x5567b4cf0c10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ce4a0ff5879a7a5 server_cookie=421d44e116b65674 in_seq=0 out_seq=0 2024-04-23T18:40:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.574+0000 7fa10dffb640 10 client.17217 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa10dffb640 1 -- 172.21.15.19:0/169529043 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567b4c46390 con 0x5567b4cee800 2024-04-23T18:40:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa10dffb640 10 client.17217 handle_command_reply: tid=0 2024-04-23T18:40:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 client.17217 shutdown 2024-04-23T18:40:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 -- 172.21.15.19:0/169529043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 msgr2=0x5567b4cdb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cdb8c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa12406b630 tx=0x7fa114014040 comp rx=0 tx=0).stop 2024-04-23T18:40:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 -- 172.21.15.19:0/169529043 shutdown_connections 2024-04-23T18:40:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567b4cee800 0x5567b4cf0c10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3f40 0x5567b4cdb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cdb8c0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 --2- 172.21.15.19:0/169529043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cd3620 0x5567b4cde090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa10dffb640 0 client.17217 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 -- 172.21.15.19:0/169529043 >> 172.21.15.19:0/169529043 conn(0x5567b4ccfb30 msgr2=0x5567b4cd58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 -- 172.21.15.19:0/169529043 shutdown_connections 2024-04-23T18:40:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.575+0000 7fa12f024740 1 -- 172.21.15.19:0/169529043 wait complete. 2024-04-23T18:40:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.576+0000 7fa12f024740 20 client.17217 trim_cache size 0 max 16384 2024-04-23T18:40:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa12f024740 1 Processor -- start 2024-04-23T18:40:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa12f024740 1 -- start start 2024-04-23T18:40:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa12f024740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3620 0x5567b4cea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa12f024740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cd3f40 0x5567b4cd5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa12f024740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa12f024740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241adbd0 con 0x5567b4cde800 2024-04-23T18:40:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa12f024740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241af710 con 0x5567b4cd3620 2024-04-23T18:40:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa12f024740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241ae2a0 con 0x5567b4cd3f40 2024-04-23T18:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa1298a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa1298a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cddbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60856/0 (socket says 172.21.15.19:60856) 2024-04-23T18:40:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa1298a3640 1 -- 172.21.15.19:0/814763958 learned_addr learned my addr 172.21.15.19:0/814763958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa1298a3640 1 --2- 172.21.15.19:0/814763958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3620 0x5567b4cea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa1298a3640 1 -- 172.21.15.19:0/814763958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cd3f40 msgr2=0x5567b4cd5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa1298a3640 1 --2- 172.21.15.19:0/814763958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cd3f40 0x5567b4cd5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa1298a3640 1 -- 172.21.15.19:0/814763958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3620 msgr2=0x5567b4cea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa1298a3640 1 --2- 172.21.15.19:0/814763958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3620 0x5567b4cea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa1298a3640 1 -- 172.21.15.19:0/814763958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b4cdbec0 con 0x5567b4cde800 2024-04-23T18:40:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.591+0000 7fa1298a3640 1 --2- 172.21.15.19:0/814763958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cddbc0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa114005fd0 tx=0x7fa114012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d25af12958fa44 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa10ffff640 1 -- 172.21.15.19:0/814763958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11402d040 con 0x5567b4cde800 2024-04-23T18:40:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa10ffff640 1 -- 172.21.15.19:0/814763958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa114021580 con 0x5567b4cde800 2024-04-23T18:40:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa10ffff640 1 -- 172.21.15.19:0/814763958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114021ae0 con 0x5567b4cde800 2024-04-23T18:40:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa12f024740 1 -- 172.21.15.19:0/814763958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 msgr2=0x5567b4cddbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa12f024740 1 --2- 172.21.15.19:0/814763958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cddbc0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa114005fd0 tx=0x7fa114012820 comp rx=0 tx=0).stop 2024-04-23T18:40:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa12f024740 1 -- 172.21.15.19:0/814763958 shutdown_connections 2024-04-23T18:40:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa12f024740 1 --2- 172.21.15.19:0/814763958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cde800 0x5567b4cddbc0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa12f024740 1 --2- 172.21.15.19:0/814763958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cd3f40 0x5567b4cd5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa12f024740 1 --2- 172.21.15.19:0/814763958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3620 0x5567b4cea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa12f024740 1 -- 172.21.15.19:0/814763958 >> 172.21.15.19:0/814763958 conn(0x5567b4ccfb30 msgr2=0x5567b4cd5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa12f024740 1 -- 172.21.15.19:0/814763958 shutdown_connections 2024-04-23T18:40:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.592+0000 7fa12f024740 1 -- 172.21.15.19:0/814763958 wait complete. 2024-04-23T18:40:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa12f024740 1 -- start start 2024-04-23T18:40:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa12f024740 1 Processor -- start 2024-04-23T18:40:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa12f024740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3620 0x5567b4cd8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa12f024740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3f40 0x5567b4cd85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa12f024740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 0x5567b4cd8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa12f024740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241af040 con 0x5567b4cd3620 2024-04-23T18:40:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa12f024740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241ae970 con 0x5567b4cd3f40 2024-04-23T18:40:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa12f024740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241abbd0 con 0x5567b4cde800 2024-04-23T18:40:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa12f024740 10 client.0 _ll_register_callbacks cb 0x7fa114012770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa1298a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3620 0x5567b4cd8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa1298a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3620 0x5567b4cd8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60858/0 (socket says 172.21.15.19:60858) 2024-04-23T18:40:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa1298a3640 1 -- 172.21.15.19:0/471295157 learned_addr learned my addr 172.21.15.19:0/471295157 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa12a0a4640 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3f40 0x5567b4cd85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa1298a3640 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 0x5567b4cd8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa1298a3640 1 -- 172.21.15.19:0/471295157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 msgr2=0x5567b4cd8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa1298a3640 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 0x5567b4cd8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa1298a3640 1 -- 172.21.15.19:0/471295157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3f40 msgr2=0x5567b4cd85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa1298a3640 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3f40 0x5567b4cd85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.593+0000 7fa1298a3640 1 -- 172.21.15.19:0/471295157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b4d53dc0 con 0x5567b4cd3620 2024-04-23T18:40:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa1298a3640 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3620 0x5567b4cd8080 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa1140128f0 tx=0x7fa11400fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0e8a91411dfa0c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa10dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:40:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa10dffb640 1 -- 172.21.15.19:0/471295157 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11402d070 con 0x5567b4cd3620 2024-04-23T18:40:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa10dffb640 1 -- 172.21.15.19:0/471295157 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8557+0+0 (secure 0 0 0) 0x7fa114021880 con 0x5567b4cd3620 2024-04-23T18:40:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa12f024740 1 -- 172.21.15.19:0/471295157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa114021580 con 0x5567b4cd3620 2024-04-23T18:40:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa10dffb640 1 -- 172.21.15.19:0/471295157 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa114021a80 con 0x5567b4cd3620 2024-04-23T18:40:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa12f024740 10 client.17223 fetch_fsmap learned FSMap version 27 2024-04-23T18:40:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa12f024740 1 -- 172.21.15.19:0/471295157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567b4d550d0 con 0x5567b4cd3620 2024-04-23T18:40:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa10dffb640 1 -- 172.21.15.19:0/471295157 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa114021580 con 0x5567b4cd3620 2024-04-23T18:40:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa10dffb640 1 -- 172.21.15.19:0/471295157 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa11401b040 con 0x5567b4cd3620 2024-04-23T18:40:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa12f024740 10 client.17223 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:40:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa12f024740 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa12f024740 20 client.17223 populate_metadata read hostname 'smithi019' 2024-04-23T18:40:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa12f024740 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567b4d557b0 0x5567b4d57bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa12f024740 4 client.17223 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:40:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.594+0000 7fa12f024740 1 -- 172.21.15.19:0/471295157 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567b4c40200 con 0x5567b4d557b0 2024-04-23T18:40:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.595+0000 7fa12a0a4640 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567b4d557b0 0x5567b4d57bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.595+0000 7fa10dffb640 10 client.17223 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.595+0000 7fa12a0a4640 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567b4d557b0 0x5567b4d57bd0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1e0d732b9311f1 server_cookie=c551bb2f5c6c8379 in_seq=0 out_seq=0 2024-04-23T18:40:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa10dffb640 1 -- 172.21.15.19:0/471295157 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567b4c40200 con 0x5567b4d557b0 2024-04-23T18:40:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa10dffb640 10 client.17223 handle_command_reply: tid=0 2024-04-23T18:40:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 client.17223 shutdown 2024-04-23T18:40:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 -- 172.21.15.19:0/471295157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3620 msgr2=0x5567b4cd8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3620 0x5567b4cd8080 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa1140128f0 tx=0x7fa11400fa10 comp rx=0 tx=0).stop 2024-04-23T18:40:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 -- 172.21.15.19:0/471295157 shutdown_connections 2024-04-23T18:40:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567b4d557b0 0x5567b4d57bd0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567b4cd3f40 0x5567b4cd85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567b4cde800 0x5567b4cd8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 --2- 172.21.15.19:0/471295157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b4cd3620 0x5567b4cd8080 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa10dffb640 0 client.17223 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 -- 172.21.15.19:0/471295157 >> 172.21.15.19:0/471295157 conn(0x5567b4ccfb30 msgr2=0x5567b4ccecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 -- 172.21.15.19:0/471295157 shutdown_connections 2024-04-23T18:40:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 1 -- 172.21.15.19:0/471295157 wait complete. 2024-04-23T18:40:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.617+0000 7fa12f024740 20 client.17223 trim_cache size 0 max 16384 2024-04-23T18:40:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:40:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:40:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:40:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:40:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 -- 172.21.15.19:0/1535944499 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa1100a3dc0 msgr2=0x7fa1100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa1100a3dc0 0x7fa1100a6280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0ef580 tx=0x7fa11c03e040 comp rx=0 tx=0).stop 2024-04-23T18:40:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 -- 172.21.15.19:0/1535944499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12419ced0 msgr2=0x7fa12419d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12419ced0 0x7fa12419d380 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0098d0 tx=0x7fa11c0317b0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 -- 172.21.15.19:0/1535944499 shutdown_connections 2024-04-23T18:40:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa1100a3dc0 0x7fa1100a6280 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12419ced0 0x7fa12419d380 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1241072a0 0x7fa124198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 --2- 172.21.15.19:0/1535944499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1241021d0 0x7fa124198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 -- 172.21.15.19:0/1535944499 >> 172.21.15.19:0/1535944499 conn(0x7fa1240793e0 msgr2=0x7fa1240ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 -- 172.21.15.19:0/1535944499 shutdown_connections 2024-04-23T18:40:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:06.618+0000 7fa12c308640 1 -- 172.21.15.19:0/1535944499 wait complete. 2024-04-23T18:40:37.191 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:25 smithi155 ceph-mon[33468]: pgmap v448: 97 pgs: 97 active+clean; 4.0 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-23T18:40:37.239 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:27 smithi155 ceph-mon[33468]: pgmap v449: 97 pgs: 97 active+clean; 4.0 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-23T18:40:37.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:27 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:40:37.339 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:27 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:40:37.389 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:27 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:40:37.440 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:27 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:40:37.490 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:29 smithi155 ceph-mon[33468]: pgmap v450: 97 pgs: 97 active+clean; 4.0 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-23T18:40:37.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:31 smithi155 ceph-mon[33468]: pgmap v451: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-23T18:40:37.609 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:33 smithi155 ceph-mon[33468]: pgmap v452: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-23T18:40:37.723 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:35 smithi155 ceph-mon[33468]: pgmap v453: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-23T18:40:37.773 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:25 smithi044 ceph-mon[32458]: pgmap v448: 97 pgs: 97 active+clean; 4.0 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-23T18:40:37.848 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:27 smithi044 ceph-mon[32458]: pgmap v449: 97 pgs: 97 active+clean; 4.0 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-23T18:40:37.898 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:27 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:40:37.948 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:27 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:40:38.007 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:27 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:40:38.081 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:27 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:40:38.131 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:29 smithi044 ceph-mon[32458]: pgmap v450: 97 pgs: 97 active+clean; 4.0 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-23T18:40:38.181 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:31 smithi044 ceph-mon[32458]: pgmap v451: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-23T18:40:38.232 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:33 smithi044 ceph-mon[32458]: pgmap v452: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-23T18:40:38.310 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:35 smithi044 ceph-mon[32458]: pgmap v453: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-23T18:40:38.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:06 smithi019 ceph-mon[29215]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T18:40:38.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:07 smithi019 ceph-mon[29215]: pgmap v405: 97 pgs: 97 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-04-23T18:40:38.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:09 smithi019 ceph-mon[29215]: pgmap v406: 97 pgs: 97 active+clean; 4.6 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-04-23T18:40:38.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:11 smithi019 ceph-mon[29215]: pgmap v407: 97 pgs: 97 active+clean; 4.9 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:40:38.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:13 smithi019 ceph-mon[29215]: pgmap v408: 97 pgs: 97 active+clean; 4.9 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:40:38.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:13 smithi019 ceph-mon[29215]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T18:40:38.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:15 smithi019 ceph-mon[29215]: pgmap v410: 97 pgs: 97 active+clean; 5.1 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T18:40:38.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:17 smithi019 ceph-mon[29215]: pgmap v411: 97 pgs: 97 active+clean; 5.1 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-23T18:40:38.850 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:37 smithi155 ceph-mon[33468]: pgmap v454: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-23T18:40:38.899 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:37 smithi044 ceph-mon[32458]: pgmap v454: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-23T18:40:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.718+0000 7f750b544640 1 -- 172.21.15.19:0/3867608288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f750406a360 msgr2=0x7f7504106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.718+0000 7f750b544640 1 --2- 172.21.15.19:0/3867608288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f750406a360 0x7f7504106ba0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009900 tx=0x7f74f402a410 comp rx=0 tx=0).stop 2024-04-23T18:40:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.718+0000 7f750b544640 1 -- 172.21.15.19:0/3867608288 shutdown_connections 2024-04-23T18:40:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.718+0000 7f750b544640 1 --2- 172.21.15.19:0/3867608288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75041005c0 0x7f7504100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.718+0000 7f750b544640 1 --2- 172.21.15.19:0/3867608288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f750406a360 0x7f7504106ba0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.718+0000 7f750b544640 1 --2- 172.21.15.19:0/3867608288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504069ef0 0x7f7504106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.718+0000 7f750b544640 1 -- 172.21.15.19:0/3867608288 >> 172.21.15.19:0/3867608288 conn(0x7f7504077390 msgr2=0x7f75040fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.718+0000 7f750b544640 1 -- 172.21.15.19:0/3867608288 shutdown_connections 2024-04-23T18:40:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.718+0000 7f750b544640 1 -- 172.21.15.19:0/3867608288 wait complete. 2024-04-23T18:40:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f750b544640 1 Processor -- start 2024-04-23T18:40:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f750b544640 1 -- start start 2024-04-23T18:40:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f750b544640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504069ef0 0x7f75041986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f750b544640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f750406a360 0x7f7504198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f750b544640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75041005c0 0x7f750419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f750b544640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75041110c0 con 0x7f75041005c0 2024-04-23T18:40:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f750b544640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7504110f40 con 0x7f750406a360 2024-04-23T18:40:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f750b544640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7504111240 con 0x7f7504069ef0 2024-04-23T18:40:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f7509ae1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75041005c0 0x7f750419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f7509ae1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75041005c0 0x7f750419cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55228/0 (socket says 172.21.15.19:55228) 2024-04-23T18:40:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f7509ae1640 1 -- 172.21.15.19:0/3341483645 learned_addr learned my addr 172.21.15.19:0/3341483645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.719+0000 7f75092e0640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504069ef0 0x7f75041986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f7508adf640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f750406a360 0x7f7504198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f7509ae1640 1 -- 172.21.15.19:0/3341483645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504069ef0 msgr2=0x7f75041986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f7509ae1640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504069ef0 0x7f75041986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f7509ae1640 1 -- 172.21.15.19:0/3341483645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f750406a360 msgr2=0x7f7504198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f7509ae1640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f750406a360 0x7f7504198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f7509ae1640 1 -- 172.21.15.19:0/3341483645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750419d6a0 con 0x7f75041005c0 2024-04-23T18:40:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f7508adf640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f750406a360 0x7f7504198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:40:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f7509ae1640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75041005c0 0x7f750419cf70 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fd0 tx=0x7f74fc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f74fa7fc640 1 -- 172.21.15.19:0/3341483645 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc017070 con 0x7f75041005c0 2024-04-23T18:40:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f74fa7fc640 1 -- 172.21.15.19:0/3341483645 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74fc003e20 con 0x7f75041005c0 2024-04-23T18:40:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f750b544640 1 -- 172.21.15.19:0/3341483645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750419d990 con 0x7f75041005c0 2024-04-23T18:40:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f74fa7fc640 1 -- 172.21.15.19:0/3341483645 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc010460 con 0x7f75041005c0 2024-04-23T18:40:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.720+0000 7f750b544640 1 -- 172.21.15.19:0/3341483645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75041a5220 con 0x7f75041005c0 2024-04-23T18:40:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.722+0000 7f74fa7fc640 1 -- 172.21.15.19:0/3341483645 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f74fc003970 con 0x7f75041005c0 2024-04-23T18:40:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.723+0000 7f750e266740 1 Processor -- start 2024-04-23T18:40:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.723+0000 7f750e266740 1 -- start start 2024-04-23T18:40:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.723+0000 7f750e266740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f4389fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.723+0000 7f750e266740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.723+0000 7f750e266740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.723+0000 7f750e266740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561f43804970 con 0x561f4389ff40 2024-04-23T18:40:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.723+0000 7f750e266740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561f437a14a0 con 0x561f438aa800 2024-04-23T18:40:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.723+0000 7f750e266740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561f43800930 con 0x561f4389f620 2024-04-23T18:40:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f75092e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f75092e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438aa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55234/0 (socket says 172.21.15.19:55234) 2024-04-23T18:40:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f7508adf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f75092e0640 1 -- 172.21.15.19:0/2891307673 learned_addr learned my addr 172.21.15.19:0/2891307673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f74fa7fc640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f74f00a3dc0 0x7f74f00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f7508adf640 1 --2- 172.21.15.19:0/2891307673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f4389fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f7509ae1640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f74f00a3dc0 0x7f74f00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f75092e0640 1 -- 172.21.15.19:0/2891307673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 msgr2=0x561f4389fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f75092e0640 1 --2- 172.21.15.19:0/2891307673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f4389fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f75092e0640 1 -- 172.21.15.19:0/2891307673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 msgr2=0x561f438acbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f75092e0640 1 --2- 172.21.15.19:0/2891307673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438acbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f75092e0640 1 -- 172.21.15.19:0/2891307673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f43517210 con 0x561f4389ff40 2024-04-23T18:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f74fa7fc640 1 -- 172.21.15.19:0/3341483645 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f74fc0d05c0 con 0x7f75041005c0 2024-04-23T18:40:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f75092e0640 1 --2- 172.21.15.19:0/2891307673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438aa2c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f750000bcb0 tx=0x7f75000107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4aa3e9df779218 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f74ef7fe640 1 -- 172.21.15.19:0/2891307673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500005020 con 0x561f4389ff40 2024-04-23T18:40:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f7509ae1640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f74f00a3dc0 0x7f74f00a6280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74fc09c260 tx=0x7f74fc087090 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f74ef7fe640 1 -- 172.21.15.19:0/2891307673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75000047a0 con 0x561f4389ff40 2024-04-23T18:40:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.724+0000 7f74ef7fe640 1 -- 172.21.15.19:0/2891307673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75000116a0 con 0x561f4389ff40 2024-04-23T18:40:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.725+0000 7f750e266740 1 -- 172.21.15.19:0/2891307673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 msgr2=0x561f438aa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.725+0000 7f750e266740 1 --2- 172.21.15.19:0/2891307673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438aa2c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f750000bcb0 tx=0x7f75000107a0 comp rx=0 tx=0).stop 2024-04-23T18:40:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.725+0000 7f750e266740 1 -- 172.21.15.19:0/2891307673 shutdown_connections 2024-04-23T18:40:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.725+0000 7f750e266740 1 --2- 172.21.15.19:0/2891307673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.725+0000 7f750e266740 1 --2- 172.21.15.19:0/2891307673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438aa2c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.725+0000 7f750e266740 1 --2- 172.21.15.19:0/2891307673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f4389fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.725+0000 7f750e266740 1 -- 172.21.15.19:0/2891307673 >> 172.21.15.19:0/2891307673 conn(0x561f4389bb30 msgr2=0x561f4389bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.725+0000 7f750e266740 1 -- 172.21.15.19:0/2891307673 shutdown_connections 2024-04-23T18:40:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.725+0000 7f750e266740 1 -- 172.21.15.19:0/2891307673 wait complete. 2024-04-23T18:40:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.726+0000 7f750e266740 1 -- start start 2024-04-23T18:40:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.726+0000 7f750e266740 1 Processor -- start 2024-04-23T18:40:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.726+0000 7f750e266740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f4389f620 0x561f438aa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.726+0000 7f750e266740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438a7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.726+0000 7f750e266740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f438aa800 0x561f438a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.726+0000 7f750e266740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561f438ad430 con 0x561f4389ff40 2024-04-23T18:40:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.726+0000 7f750e266740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561f4379f0b0 con 0x561f4389f620 2024-04-23T18:40:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.726+0000 7f750e266740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561f438ad5b0 con 0x561f438aa800 2024-04-23T18:40:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.726+0000 7f750e266740 10 client.0 _ll_register_callbacks cb 0x561f4378cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.726+0000 7f75092e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438a7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.727+0000 7f75092e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438a7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57822/0 (socket says 172.21.15.19:57822) 2024-04-23T18:40:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.727+0000 7f75092e0640 1 -- 172.21.15.19:0/3195577958 learned_addr learned my addr 172.21.15.19:0/3195577958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.727+0000 7f7508adf640 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f4389f620 0x561f438aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.727+0000 7f7508adf640 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f438aa800 0x561f438a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.727+0000 7f7508adf640 1 -- 172.21.15.19:0/3195577958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f438aa800 msgr2=0x561f438a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.727+0000 7f7508adf640 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f438aa800 0x561f438a7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.727+0000 7f7508adf640 1 -- 172.21.15.19:0/3195577958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 msgr2=0x561f438a7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.727+0000 7f7508adf640 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438a7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.727+0000 7f7508adf640 1 -- 172.21.15.19:0/3195577958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f435e3620 con 0x561f4389f620 2024-04-23T18:40:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.728+0000 7f7508adf640 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f4389f620 0x561f438aa120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f74f4006010 tx=0x7f74f4040040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75d3298f9d503315 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.728+0000 7f74ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-23T18:40:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.728+0000 7f74ecff9640 1 -- 172.21.15.19:0/3195577958 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4035040 con 0x561f4389f620 2024-04-23T18:40:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.728+0000 7f750e266740 1 -- 172.21.15.19:0/3195577958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75000047a0 con 0x561f4389f620 2024-04-23T18:40:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.728+0000 7f74ecff9640 1 -- 172.21.15.19:0/3195577958 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f74f4003a00 con 0x561f4389f620 2024-04-23T18:40:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.728+0000 7f74ecff9640 1 -- 172.21.15.19:0/3195577958 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f74f4003c00 con 0x561f4389f620 2024-04-23T18:40:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.728+0000 7f750e266740 10 client.26462 fetch_fsmap learned FSMap version 27 2024-04-23T18:40:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.728+0000 7f750e266740 1 -- 172.21.15.19:0/3195577958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f438bd770 con 0x561f4389f620 2024-04-23T18:40:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.728+0000 7f74ecff9640 1 -- 172.21.15.19:0/3195577958 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75000047a0 con 0x561f4389f620 2024-04-23T18:40:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.729+0000 7f74ecff9640 1 -- 172.21.15.19:0/3195577958 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f74f403a040 con 0x561f4389f620 2024-04-23T18:40:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.729+0000 7f750e266740 10 client.26462 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:40:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.729+0000 7f750e266740 10 client.26462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.729+0000 7f750e266740 20 client.26462 populate_metadata read hostname 'smithi019' 2024-04-23T18:40:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.729+0000 7f750e266740 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561f438bec30 0x561f438c1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.729+0000 7f750e266740 4 client.26462 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:40:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.729+0000 7f750e266740 1 -- 172.21.15.19:0/3195577958 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f43812390 con 0x561f438bec30 2024-04-23T18:40:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.729+0000 7f75092e0640 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561f438bec30 0x561f438c1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.729+0000 7f74ecff9640 10 client.26462 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.729+0000 7f75092e0640 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561f438bec30 0x561f438c1020 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15bfb89ae4e305c4 server_cookie=935310c5979a4c40 in_seq=0 out_seq=0 2024-04-23T18:40:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.730+0000 7f74ecff9640 1 -- 172.21.15.19:0/3195577958 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f43812390 con 0x561f438bec30 2024-04-23T18:40:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.730+0000 7f74ecff9640 10 client.26462 handle_command_reply: tid=0 2024-04-23T18:40:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.730+0000 7f750e266740 1 client.26462 shutdown 2024-04-23T18:40:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 1 -- 172.21.15.19:0/3195577958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f4389f620 msgr2=0x561f438aa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f4389f620 0x561f438aa120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f74f4006010 tx=0x7f74f4040040 comp rx=0 tx=0).stop 2024-04-23T18:40:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 1 -- 172.21.15.19:0/3195577958 shutdown_connections 2024-04-23T18:40:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561f438bec30 0x561f438c1020 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f438aa800 0x561f438a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438a7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f74ecff9640 0 client.26462 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 1 --2- 172.21.15.19:0/3195577958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f4389f620 0x561f438aa120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 1 -- 172.21.15.19:0/3195577958 >> 172.21.15.19:0/3195577958 conn(0x561f4389bb30 msgr2=0x561f4389acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 1 -- 172.21.15.19:0/3195577958 shutdown_connections 2024-04-23T18:40:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 1 -- 172.21.15.19:0/3195577958 wait complete. 2024-04-23T18:40:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.731+0000 7f750e266740 20 client.26462 trim_cache size 0 max 16384 2024-04-23T18:40:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f750e266740 1 Processor -- start 2024-04-23T18:40:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f750e266740 1 -- start start 2024-04-23T18:40:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f750e266740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f438b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f750e266740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f750e266740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f750e266740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75041ab8a0 con 0x561f4389ff40 2024-04-23T18:40:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f750e266740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f75041ad3e0 con 0x561f438aa800 2024-04-23T18:40:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f750e266740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75041abf70 con 0x561f4389f620 2024-04-23T18:40:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f75092e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f75092e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57838/0 (socket says 172.21.15.19:57838) 2024-04-23T18:40:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f75092e0640 1 -- 172.21.15.19:0/2619621949 learned_addr learned my addr 172.21.15.19:0/2619621949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.747+0000 7f7508adf640 1 --2- 172.21.15.19:0/2619621949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f438b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.748+0000 7f7508adf640 1 --2- 172.21.15.19:0/2619621949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.748+0000 7f75092e0640 1 -- 172.21.15.19:0/2619621949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 msgr2=0x561f438b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.748+0000 7f75092e0640 1 --2- 172.21.15.19:0/2619621949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f438b62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.748+0000 7f75092e0640 1 -- 172.21.15.19:0/2619621949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 msgr2=0x561f438c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.748+0000 7f75092e0640 1 --2- 172.21.15.19:0/2619621949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.748+0000 7f75092e0640 1 -- 172.21.15.19:0/2619621949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f438c1390 con 0x561f4389ff40 2024-04-23T18:40:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.748+0000 7f75092e0640 1 --2- 172.21.15.19:0/2619621949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438c6930 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f7500010ca0 tx=0x7f750000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ff10bea95014b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.748+0000 7f74eeffd640 1 -- 172.21.15.19:0/2619621949 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750002f070 con 0x561f4389ff40 2024-04-23T18:40:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.748+0000 7f74eeffd640 1 -- 172.21.15.19:0/2619621949 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7500004fb0 con 0x561f4389ff40 2024-04-23T18:40:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.748+0000 7f74eeffd640 1 -- 172.21.15.19:0/2619621949 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500004930 con 0x561f4389ff40 2024-04-23T18:40:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 -- 172.21.15.19:0/2619621949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 msgr2=0x561f438c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 --2- 172.21.15.19:0/2619621949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438c6930 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f7500010ca0 tx=0x7f750000b390 comp rx=0 tx=0).stop 2024-04-23T18:40:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 -- 172.21.15.19:0/2619621949 shutdown_connections 2024-04-23T18:40:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 --2- 172.21.15.19:0/2619621949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 --2- 172.21.15.19:0/2619621949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438c6930 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 --2- 172.21.15.19:0/2619621949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f438b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 -- 172.21.15.19:0/2619621949 >> 172.21.15.19:0/2619621949 conn(0x561f4389bb30 msgr2=0x561f438a6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 -- 172.21.15.19:0/2619621949 shutdown_connections 2024-04-23T18:40:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 -- 172.21.15.19:0/2619621949 wait complete. 2024-04-23T18:40:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 -- start start 2024-04-23T18:40:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.749+0000 7f750e266740 1 Processor -- start 2024-04-23T18:40:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f750e266740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f438b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f750e266740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f750e266740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f750e266740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75041acd10 con 0x561f4389ff40 2024-04-23T18:40:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f750e266740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f75041ac640 con 0x561f438aa800 2024-04-23T18:40:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f750e266740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75041a98a0 con 0x561f4389f620 2024-04-23T18:40:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f750e266740 10 client.0 _ll_register_callbacks cb 0x7f74f40040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f75092e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f75092e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57852/0 (socket says 172.21.15.19:57852) 2024-04-23T18:40:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f75092e0640 1 -- 172.21.15.19:0/531014049 learned_addr learned my addr 172.21.15.19:0/531014049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f7508adf640 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f438b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f7508adf640 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f75092e0640 1 -- 172.21.15.19:0/531014049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 msgr2=0x561f438b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f75092e0640 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f438b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f75092e0640 1 -- 172.21.15.19:0/531014049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 msgr2=0x561f438b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f75092e0640 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f75092e0640 1 -- 172.21.15.19:0/531014049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f439241e0 con 0x561f4389ff40 2024-04-23T18:40:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f75092e0640 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438b07b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7500006010 tx=0x7f7500005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d79a50824c3517 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.750+0000 7f74ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:40:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f74ecff9640 1 -- 172.21.15.19:0/531014049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750002f050 con 0x561f4389ff40 2024-04-23T18:40:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f74ecff9640 1 -- 172.21.15.19:0/531014049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f7500029430 con 0x561f4389ff40 2024-04-23T18:40:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f750e266740 1 -- 172.21.15.19:0/531014049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7500004fb0 con 0x561f4389ff40 2024-04-23T18:40:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f74ecff9640 1 -- 172.21.15.19:0/531014049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7500029630 con 0x561f4389ff40 2024-04-23T18:40:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f750e266740 10 client.17241 fetch_fsmap learned FSMap version 27 2024-04-23T18:40:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f750e266740 1 -- 172.21.15.19:0/531014049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f439254d0 con 0x561f4389ff40 2024-04-23T18:40:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f74ecff9640 1 -- 172.21.15.19:0/531014049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7500004fb0 con 0x561f4389ff40 2024-04-23T18:40:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f74ecff9640 1 -- 172.21.15.19:0/531014049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f75000275e0 con 0x561f4389ff40 2024-04-23T18:40:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f750e266740 10 client.17241 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:40:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f750e266740 10 client.17241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f750e266740 20 client.17241 populate_metadata read hostname 'smithi019' 2024-04-23T18:40:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f750e266740 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561f439259a0 0x561f43927dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f750e266740 4 client.17241 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:40:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.751+0000 7f750e266740 1 -- 172.21.15.19:0/531014049 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f4380c200 con 0x561f439259a0 2024-04-23T18:40:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.752+0000 7f7508adf640 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561f439259a0 0x561f43927dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.752+0000 7f7508adf640 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561f439259a0 0x561f43927dc0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20c643026d9a097e server_cookie=ffb0faa4796e7976 in_seq=0 out_seq=0 2024-04-23T18:40:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.752+0000 7f74ecff9640 10 client.17241 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f74ecff9640 1 -- 172.21.15.19:0/531014049 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f4380c200 con 0x561f439259a0 2024-04-23T18:40:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f74ecff9640 10 client.17241 handle_command_reply: tid=0 2024-04-23T18:40:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 client.17241 shutdown 2024-04-23T18:40:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 -- 172.21.15.19:0/531014049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 msgr2=0x561f438b07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438b07b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7500006010 tx=0x7f7500005160 comp rx=0 tx=0).stop 2024-04-23T18:40:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 -- 172.21.15.19:0/531014049 shutdown_connections 2024-04-23T18:40:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561f439259a0 0x561f43927dc0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f438aa800 0x561f438b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f4389ff40 0x561f438b07b0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f74ecff9640 0 client.17241 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 --2- 172.21.15.19:0/531014049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f4389f620 0x561f438b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 -- 172.21.15.19:0/531014049 >> 172.21.15.19:0/531014049 conn(0x561f4389bb30 msgr2=0x561f438a7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 -- 172.21.15.19:0/531014049 shutdown_connections 2024-04-23T18:40:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 1 -- 172.21.15.19:0/531014049 wait complete. 2024-04-23T18:40:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.766+0000 7f750e266740 20 client.17241 trim_cache size 0 max 16384 2024-04-23T18:40:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:40:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:40:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:40:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:40:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 -- 172.21.15.19:0/3341483645 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f74f00a3dc0 msgr2=0x7f74f00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f74f00a3dc0 0x7f74f00a6280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74fc09c260 tx=0x7f74fc087090 comp rx=0 tx=0).stop 2024-04-23T18:40:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 -- 172.21.15.19:0/3341483645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75041005c0 msgr2=0x7f750419cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75041005c0 0x7f750419cf70 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fd0 tx=0x7f74fc00b260 comp rx=0 tx=0).stop 2024-04-23T18:40:50.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 -- 172.21.15.19:0/3341483645 shutdown_connections 2024-04-23T18:40:50.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f74f00a3dc0 0x7f74f00a6280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f750406a360 0x7f7504198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75041005c0 0x7f750419cf70 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 --2- 172.21.15.19:0/3341483645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504069ef0 0x7f75041986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 -- 172.21.15.19:0/3341483645 >> 172.21.15.19:0/3341483645 conn(0x7f7504077390 msgr2=0x7f75040fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 -- 172.21.15.19:0/3341483645 shutdown_connections 2024-04-23T18:40:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:19.767+0000 7f750b544640 1 -- 172.21.15.19:0/3341483645 wait complete. 2024-04-23T18:40:51.058 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:39 smithi155 ceph-mon[33468]: pgmap v455: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-23T18:40:51.123 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:41 smithi155 ceph-mon[33468]: pgmap v456: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-23T18:40:51.155 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:43 smithi155 ceph-mon[33468]: pgmap v457: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:40:51.189 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:45 smithi155 ceph-mon[33468]: pgmap v458: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-23T18:40:51.222 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:47 smithi155 ceph-mon[33468]: pgmap v459: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:40:51.256 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:49 smithi155 ceph-mon[33468]: pgmap v460: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:40:51.297 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:39 smithi044 ceph-mon[32458]: pgmap v455: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-23T18:40:51.377 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:41 smithi044 ceph-mon[32458]: pgmap v456: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-23T18:40:51.454 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:43 smithi044 ceph-mon[32458]: pgmap v457: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:40:51.497 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:45 smithi044 ceph-mon[32458]: pgmap v458: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-23T18:40:51.531 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:47 smithi044 ceph-mon[32458]: pgmap v459: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:40:51.589 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:49 smithi044 ceph-mon[32458]: pgmap v460: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:40:51.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:19 smithi019 ceph-mon[29215]: pgmap v412: 97 pgs: 97 active+clean; 5.1 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-23T18:40:51.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:19 smithi019 ceph-mon[29215]: daemon mds.e restarted 2024-04-23T18:40:51.723 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:20 smithi019 ceph-mon[29215]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T18:40:51.756 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:20 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6834/942982921,v1:172.21.15.44:6835/942982921] up:boot 2024-04-23T18:40:51.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:20 smithi019 ceph-mon[29215]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-23T18:40:51.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:20 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:40:51.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:20 smithi019 ceph-mon[29215]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:40:52.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:21 smithi019 ceph-mon[29215]: pgmap v414: 97 pgs: 97 active+clean; 5.3 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T18:40:52.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:23 smithi019 ceph-mon[29215]: pgmap v415: 97 pgs: 97 active+clean; 5.3 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-04-23T18:40:52.110 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:25 smithi019 ceph-mon[29215]: pgmap v416: 97 pgs: 97 active+clean; 1.5 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-23T18:40:52.173 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:26 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:40:52.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:27 smithi019 ceph-mon[29215]: pgmap v417: 97 pgs: 97 active+clean; 1.5 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-23T18:40:52.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:27 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:40:52.273 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:27 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:40:52.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:27 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:40:52.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:29 smithi019 ceph-mon[29215]: pgmap v418: 97 pgs: 97 active+clean; 1.5 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-23T18:40:52.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:31 smithi019 ceph-mon[29215]: pgmap v419: 97 pgs: 97 active+clean; 1.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-04-23T18:40:52.458 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:51 smithi155 ceph-mon[33468]: pgmap v461: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T18:40:52.590 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:51 smithi044 ceph-mon[32458]: pgmap v461: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T18:40:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.870+0000 7f4efd36f640 1 -- 172.21.15.19:0/4137183342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ef8102b70 msgr2=0x7f4ef8106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.870+0000 7f4efd36f640 1 --2- 172.21.15.19:0/4137183342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ef8102b70 0x7f4ef8106bb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009900 tx=0x7f4ee002a410 comp rx=0 tx=0).stop 2024-04-23T18:40:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.870+0000 7f4efd36f640 1 -- 172.21.15.19:0/4137183342 shutdown_connections 2024-04-23T18:40:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.870+0000 7f4efd36f640 1 --2- 172.21.15.19:0/4137183342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8107290 0x7f4ef810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.870+0000 7f4efd36f640 1 --2- 172.21.15.19:0/4137183342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ef8102b70 0x7f4ef8106bb0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.871+0000 7f4efd36f640 1 --2- 172.21.15.19:0/4137183342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ef81021c0 0x7f4ef81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.871+0000 7f4efd36f640 1 -- 172.21.15.19:0/4137183342 >> 172.21.15.19:0/4137183342 conn(0x7f4ef80fda20 msgr2=0x7f4ef80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.871+0000 7f4efd36f640 1 -- 172.21.15.19:0/4137183342 shutdown_connections 2024-04-23T18:40:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.871+0000 7f4efd36f640 1 -- 172.21.15.19:0/4137183342 wait complete. 2024-04-23T18:40:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.871+0000 7f4efd36f640 1 Processor -- start 2024-04-23T18:40:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.871+0000 7f4efd36f640 1 -- start start 2024-04-23T18:40:53.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4efd36f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef81021c0 0x7f4ef81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4efd36f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ef8107290 0x7f4ef8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4efd36f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ef819cfa0 0x7f4ef819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4efd36f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8112220 con 0x7f4ef819cfa0 2024-04-23T18:40:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4efd36f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81120a0 con 0x7f4ef81021c0 2024-04-23T18:40:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4efd36f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81123a0 con 0x7f4ef8107290 2024-04-23T18:40:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ef819cfa0 0x7f4ef819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ef819cfa0 0x7f4ef819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42374/0 (socket says 172.21.15.19:42374) 2024-04-23T18:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef77fe640 1 -- 172.21.15.19:0/95714378 learned_addr learned my addr 172.21.15.19:0/95714378 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef6ffd640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef81021c0 0x7f4ef81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ef8107290 0x7f4ef8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef77fe640 1 -- 172.21.15.19:0/95714378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ef8107290 msgr2=0x7f4ef8198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef77fe640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ef8107290 0x7f4ef8198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef77fe640 1 -- 172.21.15.19:0/95714378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef81021c0 msgr2=0x7f4ef81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef77fe640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef81021c0 0x7f4ef81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef77fe640 1 -- 172.21.15.19:0/95714378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef819da90 con 0x7f4ef819cfa0 2024-04-23T18:40:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.872+0000 7f4ef77fe640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ef819cfa0 0x7f4ef819d450 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800cad0 tx=0x7f4ee800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.873+0000 7f4eeffff640 1 -- 172.21.15.19:0/95714378 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee800ae20 con 0x7f4ef819cfa0 2024-04-23T18:40:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.873+0000 7f4eeffff640 1 -- 172.21.15.19:0/95714378 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee8005020 con 0x7f4ef819cfa0 2024-04-23T18:40:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.873+0000 7f4efd36f640 1 -- 172.21.15.19:0/95714378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef819dd80 con 0x7f4ef819cfa0 2024-04-23T18:40:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.873+0000 7f4eeffff640 1 -- 172.21.15.19:0/95714378 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee800f6b0 con 0x7f4ef819cfa0 2024-04-23T18:40:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.873+0000 7f4efd36f640 1 -- 172.21.15.19:0/95714378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef81a54f0 con 0x7f4ef819cfa0 2024-04-23T18:40:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.875+0000 7f4eeffff640 1 -- 172.21.15.19:0/95714378 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ee80051d0 con 0x7f4ef819cfa0 2024-04-23T18:40:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.875+0000 7f4f00167740 1 Processor -- start 2024-04-23T18:40:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.875+0000 7f4f00167740 1 -- start start 2024-04-23T18:40:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4f00167740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58812a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4f00167740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e58812f40 0x562e5881d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4f00167740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5881d800 0x562e5881fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4f00167740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e58777970 con 0x562e58812620 2024-04-23T18:40:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4f00167740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562e587144a0 con 0x562e5881d800 2024-04-23T18:40:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4f00167740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562e58773930 con 0x562e58812f40 2024-04-23T18:40:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4eeffff640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4ee40a3dc0 0x7f4ee40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4ef67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58812a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4ef67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58812a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42390/0 (socket says 172.21.15.19:42390) 2024-04-23T18:40:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4ef67fc640 1 -- 172.21.15.19:0/172750056 learned_addr learned my addr 172.21.15.19:0/172750056 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4ef77fe640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4ee40a3dc0 0x7f4ee40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4ef6ffd640 1 --2- 172.21.15.19:0/172750056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e58812f40 0x562e5881d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4eeffff640 1 -- 172.21.15.19:0/95714378 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f4ee80d0890 con 0x7f4ef819cfa0 2024-04-23T18:40:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.876+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/172750056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5881d800 0x562e5881fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.877+0000 7f4ef67fc640 1 -- 172.21.15.19:0/172750056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e58812f40 msgr2=0x562e5881d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.877+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/172750056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e58812f40 0x562e5881d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.877+0000 7f4ef67fc640 1 -- 172.21.15.19:0/172750056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5881d800 msgr2=0x562e5881fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.877+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/172750056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5881d800 0x562e5881fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.877+0000 7f4ef67fc640 1 -- 172.21.15.19:0/172750056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e5848a210 con 0x562e58812620 2024-04-23T18:40:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.877+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/172750056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58812a00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00096c0 tx=0x7f4ee0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf4230f8c851b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.877+0000 7f4ef77fe640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4ee40a3dc0 0x7f4ee40a6280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800a6e0 tx=0x7f4ee8087040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.877+0000 7f4eed7fa640 1 -- 172.21.15.19:0/172750056 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0035070 con 0x562e58812620 2024-04-23T18:40:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.877+0000 7f4eed7fa640 1 -- 172.21.15.19:0/172750056 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee00046c0 con 0x562e58812620 2024-04-23T18:40:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.877+0000 7f4eed7fa640 1 -- 172.21.15.19:0/172750056 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0003850 con 0x562e58812620 2024-04-23T18:40:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 -- 172.21.15.19:0/172750056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 msgr2=0x562e58812a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 --2- 172.21.15.19:0/172750056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58812a00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00096c0 tx=0x7f4ee0009560 comp rx=0 tx=0).stop 2024-04-23T18:40:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 -- 172.21.15.19:0/172750056 shutdown_connections 2024-04-23T18:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 --2- 172.21.15.19:0/172750056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5881d800 0x562e5881fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 --2- 172.21.15.19:0/172750056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e58812f40 0x562e5881d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 --2- 172.21.15.19:0/172750056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58812a00 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 -- 172.21.15.19:0/172750056 >> 172.21.15.19:0/172750056 conn(0x562e5880eb30 msgr2=0x562e5880ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 -- 172.21.15.19:0/172750056 shutdown_connections 2024-04-23T18:40:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 -- 172.21.15.19:0/172750056 wait complete. 2024-04-23T18:40:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 -- start start 2024-04-23T18:40:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.878+0000 7f4f00167740 1 Processor -- start 2024-04-23T18:40:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4f00167740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 0x562e5881d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4f00167740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e58812f40 0x562e5881a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4f00167740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5881d800 0x562e5881a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4f00167740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e588203d0 con 0x562e5881d800 2024-04-23T18:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4f00167740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562e587120b0 con 0x562e58812620 2024-04-23T18:40:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4f00167740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562e58820550 con 0x562e58812f40 2024-04-23T18:40:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4f00167740 10 client.0 _ll_register_callbacks cb 0x562e586ffd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5881d800 0x562e5881a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5881d800 0x562e5881a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42398/0 (socket says 172.21.15.19:42398) 2024-04-23T18:40:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef67fc640 1 -- 172.21.15.19:0/650714161 learned_addr learned my addr 172.21.15.19:0/650714161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:40:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 0x562e5881d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef6ffd640 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e58812f40 0x562e5881a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef67fc640 1 -- 172.21.15.19:0/650714161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e58812f40 msgr2=0x562e5881a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e58812f40 0x562e5881a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef67fc640 1 -- 172.21.15.19:0/650714161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 msgr2=0x562e5881d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 0x562e5881d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef67fc640 1 -- 172.21.15.19:0/650714161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e58556620 con 0x562e5881d800 2024-04-23T18:40:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5881d800 0x562e5881a9b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0004650 tx=0x7f4ee002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f77b48bda0b8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ede7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:40:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ede7fc640 1 -- 172.21.15.19:0/650714161 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0035040 con 0x562e5881d800 2024-04-23T18:40:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.879+0000 7f4ede7fc640 1 -- 172.21.15.19:0/650714161 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f4ee0005480 con 0x562e5881d800 2024-04-23T18:40:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4f00167740 1 -- 172.21.15.19:0/650714161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee00046c0 con 0x562e5881d800 2024-04-23T18:40:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4ede7fc640 1 -- 172.21.15.19:0/650714161 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ee005a050 con 0x562e5881d800 2024-04-23T18:40:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4f00167740 10 client.17259 fetch_fsmap learned FSMap version 29 2024-04-23T18:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4f00167740 1 -- 172.21.15.19:0/650714161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562e5882c300 con 0x562e5881d800 2024-04-23T18:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4ede7fc640 1 -- 172.21.15.19:0/650714161 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee00046c0 con 0x562e5881d800 2024-04-23T18:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4ede7fc640 1 -- 172.21.15.19:0/650714161 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ee003d450 con 0x562e5881d800 2024-04-23T18:40:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4f00167740 10 client.17259 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:40:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4f00167740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4f00167740 20 client.17259 populate_metadata read hostname 'smithi019' 2024-04-23T18:40:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4f00167740 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e5882d650 0x562e5882fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4f00167740 4 client.17259 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:40:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.880+0000 7f4f00167740 1 -- 172.21.15.19:0/650714161 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e58785390 con 0x562e5882d650 2024-04-23T18:40:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.881+0000 7f4ef6ffd640 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e5882d650 0x562e5882fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.881+0000 7f4ede7fc640 10 client.17259 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.881+0000 7f4ef6ffd640 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e5882d650 0x562e5882fa40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c0c57784c55054a server_cookie=962856640be08a79 in_seq=0 out_seq=0 2024-04-23T18:40:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4ede7fc640 1 -- 172.21.15.19:0/650714161 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e58785390 con 0x562e5882d650 2024-04-23T18:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4ede7fc640 10 client.17259 handle_command_reply: tid=0 2024-04-23T18:40:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 client.17259 shutdown 2024-04-23T18:40:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 -- 172.21.15.19:0/650714161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5881d800 msgr2=0x562e5881a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5881d800 0x562e5881a9b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0004650 tx=0x7f4ee002a410 comp rx=0 tx=0).stop 2024-04-23T18:40:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 -- 172.21.15.19:0/650714161 shutdown_connections 2024-04-23T18:40:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e5882d650 0x562e5882fa40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5881d800 0x562e5881a9b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4ede7fc640 0 client.17259 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:40:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e58812f40 0x562e5881a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 --2- 172.21.15.19:0/650714161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 0x562e5881d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 -- 172.21.15.19:0/650714161 >> 172.21.15.19:0/650714161 conn(0x562e5880eb30 msgr2=0x562e588148b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 -- 172.21.15.19:0/650714161 shutdown_connections 2024-04-23T18:40:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 1 -- 172.21.15.19:0/650714161 wait complete. 2024-04-23T18:41:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.882+0000 7f4f00167740 20 client.17259 trim_cache size 0 max 16384 2024-04-23T18:41:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.899+0000 7f4f00167740 1 Processor -- start 2024-04-23T18:41:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.899+0000 7f4f00167740 1 -- start start 2024-04-23T18:41:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4f00167740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58828eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4f00167740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812f40 0x562e5881cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4f00167740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e5881d800 0x562e58832a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4f00167740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81ad990 con 0x562e58812620 2024-04-23T18:41:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4f00167740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81af4d0 con 0x562e58812f40 2024-04-23T18:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4f00167740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81ae060 con 0x562e5881d800 2024-04-23T18:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58828eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58828eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42412/0 (socket says 172.21.15.19:42412) 2024-04-23T18:41:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef67fc640 1 -- 172.21.15.19:0/3748741502 learned_addr learned my addr 172.21.15.19:0/3748741502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef6ffd640 1 --2- 172.21.15.19:0/3748741502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812f40 0x562e5881cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/3748741502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e5881d800 0x562e58832a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef67fc640 1 -- 172.21.15.19:0/3748741502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e5881d800 msgr2=0x562e58832a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/3748741502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e5881d800 0x562e58832a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef67fc640 1 -- 172.21.15.19:0/3748741502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812f40 msgr2=0x562e5881cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/3748741502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812f40 0x562e5881cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef67fc640 1 -- 172.21.15.19:0/3748741502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e5881ad20 con 0x562e58812620 2024-04-23T18:41:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.900+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/3748741502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58828eb0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f4ee003d850 tx=0x7f4ee0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe25c98acd495c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4eecff9640 1 -- 172.21.15.19:0/3748741502 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0035040 con 0x562e58812620 2024-04-23T18:41:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4eecff9640 1 -- 172.21.15.19:0/3748741502 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee00046c0 con 0x562e58812620 2024-04-23T18:41:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4eecff9640 1 -- 172.21.15.19:0/3748741502 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee003d6b0 con 0x562e58812620 2024-04-23T18:41:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4f00167740 1 -- 172.21.15.19:0/3748741502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 msgr2=0x562e58828eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4f00167740 1 --2- 172.21.15.19:0/3748741502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58828eb0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f4ee003d850 tx=0x7f4ee0001e70 comp rx=0 tx=0).stop 2024-04-23T18:41:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4f00167740 1 -- 172.21.15.19:0/3748741502 shutdown_connections 2024-04-23T18:41:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4f00167740 1 --2- 172.21.15.19:0/3748741502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812f40 0x562e5881cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4f00167740 1 --2- 172.21.15.19:0/3748741502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e5881d800 0x562e58832a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4f00167740 1 --2- 172.21.15.19:0/3748741502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812620 0x562e58828eb0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4f00167740 1 -- 172.21.15.19:0/3748741502 >> 172.21.15.19:0/3748741502 conn(0x562e5880eb30 msgr2=0x562e588287a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4f00167740 1 -- 172.21.15.19:0/3748741502 shutdown_connections 2024-04-23T18:41:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.901+0000 7f4f00167740 1 -- 172.21.15.19:0/3748741502 wait complete. 2024-04-23T18:41:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4f00167740 1 -- start start 2024-04-23T18:41:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4f00167740 1 Processor -- start 2024-04-23T18:41:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4f00167740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 0x562e5888abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4f00167740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812f40 0x562e5888b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4f00167740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e5881d800 0x562e5882fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4f00167740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81aee00 con 0x562e58812f40 2024-04-23T18:41:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4f00167740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81ae730 con 0x562e58812620 2024-04-23T18:41:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4f00167740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81ab990 con 0x562e5881d800 2024-04-23T18:41:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4ef6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812f40 0x562e5888b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4f00167740 10 client.0 _ll_register_callbacks cb 0x7f4ee0004cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4ef6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812f40 0x562e5888b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42424/0 (socket says 172.21.15.19:42424) 2024-04-23T18:41:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4ef6ffd640 1 -- 172.21.15.19:0/1280506733 learned_addr learned my addr 172.21.15.19:0/1280506733 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 0x562e5888abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.902+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e5881d800 0x562e5882fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ef6ffd640 1 -- 172.21.15.19:0/1280506733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e5881d800 msgr2=0x562e5882fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ef6ffd640 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e5881d800 0x562e5882fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ef6ffd640 1 -- 172.21.15.19:0/1280506733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 msgr2=0x562e5888abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ef6ffd640 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 0x562e5888abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ef6ffd640 1 -- 172.21.15.19:0/1280506733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e588304b0 con 0x562e58812f40 2024-04-23T18:41:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 0x562e5888abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:41:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ef6ffd640 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812f40 0x562e5888b0f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x562e58820650 tx=0x7f4ef001f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ea86b162b98608 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ede7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:41:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ede7fc640 1 -- 172.21.15.19:0/1280506733 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0003cc0 con 0x562e58812f40 2024-04-23T18:41:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ede7fc640 1 -- 172.21.15.19:0/1280506733 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f4ef0005020 con 0x562e58812f40 2024-04-23T18:41:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4f00167740 1 -- 172.21.15.19:0/1280506733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee00046c0 con 0x562e58812f40 2024-04-23T18:41:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ede7fc640 1 -- 172.21.15.19:0/1280506733 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ef0003e60 con 0x562e58812f40 2024-04-23T18:41:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4f00167740 10 client.17265 fetch_fsmap learned FSMap version 29 2024-04-23T18:41:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4f00167740 1 -- 172.21.15.19:0/1280506733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562e588933e0 con 0x562e58812f40 2024-04-23T18:41:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ede7fc640 1 -- 172.21.15.19:0/1280506733 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee00046c0 con 0x562e58812f40 2024-04-23T18:41:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4ede7fc640 1 -- 172.21.15.19:0/1280506733 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ef0027430 con 0x562e58812f40 2024-04-23T18:41:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.903+0000 7f4f00167740 10 client.17265 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:41:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.904+0000 7f4f00167740 10 client.17265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.904+0000 7f4f00167740 20 client.17265 populate_metadata read hostname 'smithi019' 2024-04-23T18:41:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.904+0000 7f4f00167740 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e58893b90 0x562e58895fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.904+0000 7f4f00167740 4 client.17265 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:41:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.904+0000 7f4f00167740 1 -- 172.21.15.19:0/1280506733 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e5877f200 con 0x562e58893b90 2024-04-23T18:41:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.904+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e58893b90 0x562e58895fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.904+0000 7f4ef67fc640 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e58893b90 0x562e58895fb0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=174ea8f47d324dbc server_cookie=27b100f90caa5efd in_seq=0 out_seq=0 2024-04-23T18:41:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.904+0000 7f4ede7fc640 10 client.17265 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4ede7fc640 1 -- 172.21.15.19:0/1280506733 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e5877f200 con 0x562e58893b90 2024-04-23T18:41:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4ede7fc640 10 client.17265 handle_command_reply: tid=0 2024-04-23T18:41:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 client.17265 shutdown 2024-04-23T18:41:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 -- 172.21.15.19:0/1280506733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812f40 msgr2=0x562e5888b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812f40 0x562e5888b0f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x562e58820650 tx=0x7f4ef001f770 comp rx=0 tx=0).stop 2024-04-23T18:41:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 -- 172.21.15.19:0/1280506733 shutdown_connections 2024-04-23T18:41:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x562e58893b90 0x562e58895fb0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e5881d800 0x562e5882fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e58812f40 0x562e5888b0f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4ede7fc640 0 client.17265 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 --2- 172.21.15.19:0/1280506733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e58812620 0x562e5888abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 -- 172.21.15.19:0/1280506733 >> 172.21.15.19:0/1280506733 conn(0x562e5880eb30 msgr2=0x562e58829030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 -- 172.21.15.19:0/1280506733 shutdown_connections 2024-04-23T18:41:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 1 -- 172.21.15.19:0/1280506733 wait complete. 2024-04-23T18:41:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.918+0000 7f4f00167740 20 client.17265 trim_cache size 0 max 16384 2024-04-23T18:41:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:41:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:41:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:41:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:41:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 -- 172.21.15.19:0/95714378 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4ee40a3dc0 msgr2=0x7f4ee40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4ee40a3dc0 0x7f4ee40a6280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800a6e0 tx=0x7f4ee8087040 comp rx=0 tx=0).stop 2024-04-23T18:41:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 -- 172.21.15.19:0/95714378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ef819cfa0 msgr2=0x7f4ef819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ef819cfa0 0x7f4ef819d450 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800cad0 tx=0x7f4ee800cf90 comp rx=0 tx=0).stop 2024-04-23T18:41:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 -- 172.21.15.19:0/95714378 shutdown_connections 2024-04-23T18:41:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f4ee40a3dc0 0x7f4ee40a6280 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ef819cfa0 0x7f4ef819d450 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ef8107290 0x7f4ef8198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 --2- 172.21.15.19:0/95714378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef81021c0 0x7f4ef81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 -- 172.21.15.19:0/95714378 >> 172.21.15.19:0/95714378 conn(0x7f4ef80fda20 msgr2=0x7f4ef80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 -- 172.21.15.19:0/95714378 shutdown_connections 2024-04-23T18:41:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:32.919+0000 7f4efd36f640 1 -- 172.21.15.19:0/95714378 wait complete. 2024-04-23T18:41:05.448 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:53 smithi155 ceph-mon[33468]: pgmap v462: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-23T18:41:05.497 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:55 smithi155 ceph-mon[33468]: pgmap v463: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-23T18:41:05.547 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:57 smithi155 ceph-mon[33468]: pgmap v464: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-04-23T18:41:05.597 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:40:59 smithi155 ceph-mon[33468]: pgmap v465: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-04-23T18:41:05.647 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:01 smithi155 ceph-mon[33468]: pgmap v466: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-23T18:41:05.697 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:03 smithi155 ceph-mon[33468]: pgmap v467: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-04-23T18:41:05.748 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:53 smithi044 ceph-mon[32458]: pgmap v462: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-23T18:41:05.797 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:55 smithi044 ceph-mon[32458]: pgmap v463: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-23T18:41:05.848 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:57 smithi044 ceph-mon[32458]: pgmap v464: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-04-23T18:41:05.898 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:40:59 smithi044 ceph-mon[32458]: pgmap v465: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-04-23T18:41:05.973 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:01 smithi044 ceph-mon[32458]: pgmap v466: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-23T18:41:06.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:03 smithi044 ceph-mon[32458]: pgmap v467: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-04-23T18:41:06.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:33 smithi019 ceph-mon[29215]: pgmap v420: 97 pgs: 97 active+clean; 1.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-23T18:41:06.135 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:35 smithi019 ceph-mon[29215]: pgmap v421: 97 pgs: 97 active+clean; 1.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 66 KiB/s wr, 26 op/s 2024-04-23T18:41:06.173 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:37 smithi019 ceph-mon[29215]: pgmap v422: 97 pgs: 97 active+clean; 1.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:41:06.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:39 smithi019 ceph-mon[29215]: pgmap v423: 97 pgs: 97 active+clean; 1.9 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:41:06.278 INFO:tasks.workunit.client.0.smithi019.stdout:499/600: open fd = 3 2024-04-23T18:41:06.339 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:06.389 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:06.439 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:06.490 INFO:tasks.workunit.client.0.smithi019.stdout:500/600: open fd = 3 2024-04-23T18:41:06.581 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:06.631 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:06.690 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:06.740 INFO:tasks.workunit.client.0.smithi019.stdout:501/600: open fd = 3 2024-04-23T18:41:06.790 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:06.852 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:06.890 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:06.945 INFO:tasks.workunit.client.0.smithi019.stdout:502/600: open fd = 3 2024-04-23T18:41:06.982 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.032 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.074 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:07.148 INFO:tasks.workunit.client.0.smithi019.stdout:503/600: open fd = 3 2024-04-23T18:41:07.190 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.215 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.248 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:07.290 INFO:tasks.workunit.client.0.smithi019.stdout:504/600: open fd = 3 2024-04-23T18:41:07.323 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.357 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.452 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:07.540 INFO:tasks.workunit.client.0.smithi019.stdout:505/600: open fd = 3 2024-04-23T18:41:07.574 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.626 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.698 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:07.724 INFO:tasks.workunit.client.0.smithi019.stdout:506/600: open fd = 3 2024-04-23T18:41:07.757 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.790 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.824 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:07.857 INFO:tasks.workunit.client.0.smithi019.stdout:507/600: open fd = 3 2024-04-23T18:41:07.899 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.945 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:07.984 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:08.024 INFO:tasks.workunit.client.0.smithi019.stdout:508/600: open fd = 3 2024-04-23T18:41:08.058 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.191 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.224 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:08.257 INFO:tasks.workunit.client.0.smithi019.stdout:509/600: open fd = 3 2024-04-23T18:41:08.307 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.341 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.374 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:08.408 INFO:tasks.workunit.client.0.smithi019.stdout:510/600: open fd = 3 2024-04-23T18:41:08.458 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.497 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.537 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:08.616 INFO:tasks.workunit.client.0.smithi019.stdout:511/600: open fd = 3 2024-04-23T18:41:08.649 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.683 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.716 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:08.749 INFO:tasks.workunit.client.0.smithi019.stdout:512/600: open fd = 3 2024-04-23T18:41:08.783 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.816 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.850 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:08.900 INFO:tasks.workunit.client.0.smithi019.stdout:513/600: open fd = 3 2024-04-23T18:41:08.933 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:08.966 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:09.000 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:09.083 INFO:tasks.workunit.client.0.smithi019.stdout:514/600: open fd = 3 2024-04-23T18:41:09.116 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:09.150 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:09.183 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:09.217 INFO:tasks.workunit.client.0.smithi019.stdout:515/600: open fd = 3 2024-04-23T18:41:09.250 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:09.283 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:09.324 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:09.361 INFO:tasks.workunit.client.0.smithi019.stdout:516/600: open fd = 3 2024-04-23T18:41:09.416 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:09.467 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:09.525 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:09.575 INFO:tasks.workunit.client.0.smithi019.stdout:517/600: open fd = 3 2024-04-23T18:41:09.625 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:09.676 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:09.751 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:09.859 INFO:tasks.workunit.client.0.smithi019.stdout:518/600: open fd = 3 2024-04-23T18:41:09.909 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:09.959 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:10.009 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:10.059 INFO:tasks.workunit.client.0.smithi019.stdout:519/600: open fd = 3 2024-04-23T18:41:10.109 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:10.234 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:10.284 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:10.334 INFO:tasks.workunit.client.0.smithi019.stdout:520/600: open fd = 3 2024-04-23T18:41:10.384 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:10.434 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:10.484 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:10.534 INFO:tasks.workunit.client.0.smithi019.stdout:521/600: open fd = 3 2024-04-23T18:41:10.601 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:10.651 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:10.701 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:10.752 INFO:tasks.workunit.client.0.smithi019.stdout:522/600: open fd = 3 2024-04-23T18:41:10.859 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:10.909 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:10.959 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:11.011 INFO:tasks.workunit.client.0.smithi019.stdout:523/600: open fd = 3 2024-04-23T18:41:11.060 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:11.109 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:11.160 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:11.210 INFO:tasks.workunit.client.0.smithi019.stdout:524/600: open fd = 3 2024-04-23T18:41:11.260 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:11.310 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:11.360 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:11.410 INFO:tasks.workunit.client.0.smithi019.stdout:525/600: open fd = 3 2024-04-23T18:41:11.460 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:11.510 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:11.581 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:11.624 INFO:tasks.workunit.client.0.smithi019.stdout:526/600: open fd = 3 2024-04-23T18:41:11.756 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:11.790 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:11.823 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:11.856 INFO:tasks.workunit.client.0.smithi019.stdout:527/600: open fd = 3 2024-04-23T18:41:11.890 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:11.923 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:11.957 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:11.990 INFO:tasks.workunit.client.0.smithi019.stdout:528/600: open fd = 3 2024-04-23T18:41:12.023 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:12.057 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:12.118 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:12.157 INFO:tasks.workunit.client.0.smithi019.stdout:529/600: open fd = 3 2024-04-23T18:41:12.190 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:12.223 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:12.257 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:12.357 INFO:tasks.workunit.client.0.smithi019.stdout:530/600: open fd = 3 2024-04-23T18:41:12.415 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:12.465 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:12.515 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:12.565 INFO:tasks.workunit.client.0.smithi019.stdout:531/600: open fd = 3 2024-04-23T18:41:12.615 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:12.665 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:12.716 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:12.765 INFO:tasks.workunit.client.0.smithi019.stdout:532/600: open fd = 3 2024-04-23T18:41:12.850 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:12.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:13.024 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:13.074 INFO:tasks.workunit.client.0.smithi019.stdout:533/600: open fd = 3 2024-04-23T18:41:13.124 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:13.179 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:13.224 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:13.274 INFO:tasks.workunit.client.0.smithi019.stdout:534/600: open fd = 3 2024-04-23T18:41:13.324 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:13.376 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:13.424 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:13.474 INFO:tasks.workunit.client.0.smithi019.stdout:535/600: open fd = 3 2024-04-23T18:41:13.525 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:13.674 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:13.724 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:13.774 INFO:tasks.workunit.client.0.smithi019.stdout:536/600: open fd = 3 2024-04-23T18:41:13.827 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:13.875 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:13.949 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:14.000 INFO:tasks.workunit.client.0.smithi019.stdout:537/600: open fd = 3 2024-04-23T18:41:14.058 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:14.099 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:14.150 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:14.200 INFO:tasks.workunit.client.0.smithi019.stdout:538/600: open fd = 3 2024-04-23T18:41:14.350 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:14.400 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:14.450 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:14.563 INFO:tasks.workunit.client.0.smithi019.stdout:539/600: open fd = 3 2024-04-23T18:41:14.600 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:14.650 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:14.700 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:14.750 INFO:tasks.workunit.client.0.smithi019.stdout:540/600: open fd = 3 2024-04-23T18:41:14.800 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:14.850 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:14.900 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:15.025 INFO:tasks.workunit.client.0.smithi019.stdout:541/600: open fd = 3 2024-04-23T18:41:15.075 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:15.135 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:15.225 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:15.276 INFO:tasks.workunit.client.0.smithi019.stdout:542/600: open fd = 3 2024-04-23T18:41:15.325 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:15.376 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:15.450 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:15.501 INFO:tasks.workunit.client.0.smithi019.stdout:543/600: open fd = 3 2024-04-23T18:41:15.568 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:15.659 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:15.718 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:15.768 INFO:tasks.workunit.client.0.smithi019.stdout:544/600: open fd = 3 2024-04-23T18:41:15.818 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:15.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:15.943 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:15.993 INFO:tasks.workunit.client.0.smithi019.stdout:545/600: open fd = 3 2024-04-23T18:41:16.043 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:16.158 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:16.193 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:16.243 INFO:tasks.workunit.client.0.smithi019.stdout:546/600: open fd = 3 2024-04-23T18:41:16.326 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:16.376 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:16.426 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:16.476 INFO:tasks.workunit.client.0.smithi019.stdout:547/600: open fd = 3 2024-04-23T18:41:16.558 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:16.602 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:16.677 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:16.727 INFO:tasks.workunit.client.0.smithi019.stdout:548/600: open fd = 3 2024-04-23T18:41:16.791 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:16.852 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:16.902 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:16.985 INFO:tasks.workunit.client.0.smithi019.stdout:549/600: open fd = 3 2024-04-23T18:41:17.035 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:17.077 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:17.127 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:17.214 INFO:tasks.workunit.client.0.smithi019.stdout:550/600: open fd = 3 2024-04-23T18:41:17.252 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:17.294 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:17.335 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:17.395 INFO:tasks.workunit.client.0.smithi019.stdout:551/600: open fd = 3 2024-04-23T18:41:17.427 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:17.461 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:17.535 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:17.569 INFO:tasks.workunit.client.0.smithi019.stdout:552/600: open fd = 3 2024-04-23T18:41:17.602 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:17.636 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:17.789 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:17.819 INFO:tasks.workunit.client.0.smithi019.stdout:553/600: open fd = 3 2024-04-23T18:41:17.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:17.919 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:17.965 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:17.994 INFO:tasks.workunit.client.0.smithi019.stdout:554/600: open fd = 3 2024-04-23T18:41:18.028 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:18.111 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:18.144 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:18.177 INFO:tasks.workunit.client.0.smithi019.stdout:555/600: open fd = 3 2024-04-23T18:41:18.211 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:18.302 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:18.344 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:18.395 INFO:tasks.workunit.client.0.smithi019.stdout:556/600: open fd = 3 2024-04-23T18:41:18.428 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:18.489 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:18.528 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:18.562 INFO:tasks.workunit.client.0.smithi019.stdout:557/600: open fd = 3 2024-04-23T18:41:18.661 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:18.695 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:18.728 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:18.762 INFO:tasks.workunit.client.0.smithi019.stdout:558/600: open fd = 3 2024-04-23T18:41:18.845 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:18.887 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:18.920 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:18.953 INFO:tasks.workunit.client.0.smithi019.stdout:559/600: open fd = 3 2024-04-23T18:41:19.001 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:19.056 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:19.099 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:19.204 INFO:tasks.workunit.client.0.smithi019.stdout:560/600: open fd = 3 2024-04-23T18:41:19.237 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:19.271 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:19.304 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:19.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:41 smithi019 ceph-mon[29215]: pgmap v424: 97 pgs: 97 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T18:41:19.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:43 smithi019 ceph-mon[29215]: pgmap v425: 97 pgs: 97 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T18:41:19.459 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:05 smithi155 ceph-mon[33468]: pgmap v468: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-04-23T18:41:19.496 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:06 smithi155 ceph-mon[33468]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T18:41:19.529 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:07 smithi155 ceph-mon[33468]: pgmap v470: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:41:19.631 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:09 smithi155 ceph-mon[33468]: pgmap v471: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:41:19.663 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:11 smithi155 ceph-mon[33468]: pgmap v472: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:41:19.738 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:13 smithi155 ceph-mon[33468]: pgmap v473: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:41:19.821 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:13 smithi155 ceph-mon[33468]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T18:41:19.854 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:15 smithi155 ceph-mon[33468]: pgmap v475: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 14 op/s 2024-04-23T18:41:19.896 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:17 smithi155 ceph-mon[33468]: pgmap v476: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-23T18:41:19.946 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:05 smithi044 ceph-mon[32458]: pgmap v468: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-04-23T18:41:19.996 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:06 smithi044 ceph-mon[32458]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T18:41:20.046 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:07 smithi044 ceph-mon[32458]: pgmap v470: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:41:20.100 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:09 smithi044 ceph-mon[32458]: pgmap v471: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:41:20.146 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:11 smithi044 ceph-mon[32458]: pgmap v472: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:41:20.205 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:13 smithi044 ceph-mon[32458]: pgmap v473: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:41:20.246 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:13 smithi044 ceph-mon[32458]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T18:41:20.306 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:15 smithi044 ceph-mon[32458]: pgmap v475: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 14 op/s 2024-04-23T18:41:20.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:17 smithi044 ceph-mon[32458]: pgmap v476: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-23T18:41:20.457 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:19 smithi155 ceph-mon[33468]: pgmap v477: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-23T18:41:20.538 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:19 smithi044 ceph-mon[32458]: pgmap v477: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-23T18:41:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 -- 172.21.15.19:0/750275995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ac10c5d0 msgr2=0x7ff1ac10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 --2- 172.21.15.19:0/750275995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ac10c5d0 0x7ff1ac10ea00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009ca0 tx=0x7ff1a402a950 comp rx=0 tx=0).stop 2024-04-23T18:41:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 -- 172.21.15.19:0/750275995 shutdown_connections 2024-04-23T18:41:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 --2- 172.21.15.19:0/750275995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1ac103b20 0x7ff1ac10c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 --2- 172.21.15.19:0/750275995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ac10c5d0 0x7ff1ac10ea00 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 --2- 172.21.15.19:0/750275995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1ac103200 0x7ff1ac1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 -- 172.21.15.19:0/750275995 >> 172.21.15.19:0/750275995 conn(0x7ff1ac0771d0 msgr2=0x7ff1ac0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 -- 172.21.15.19:0/750275995 shutdown_connections 2024-04-23T18:41:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 -- 172.21.15.19:0/750275995 wait complete. 2024-04-23T18:41:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 Processor -- start 2024-04-23T18:41:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.020+0000 7ff1b362a640 1 -- start start 2024-04-23T18:41:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b362a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1ac103200 0x7ff1ac198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b362a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1ac103b20 0x7ff1ac198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b362a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ac10c5d0 0x7ff1ac19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b362a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1121d0 con 0x7ff1ac10c5d0 2024-04-23T18:41:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b362a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac112050 con 0x7ff1ac103200 2024-04-23T18:41:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b362a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac112350 con 0x7ff1ac103b20 2024-04-23T18:41:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b1bc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ac10c5d0 0x7ff1ac19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b1bc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ac10c5d0 0x7ff1ac19cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44734/0 (socket says 172.21.15.19:44734) 2024-04-23T18:41:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b1bc7640 1 -- 172.21.15.19:0/2916092421 learned_addr learned my addr 172.21.15.19:0/2916092421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b13c6640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1ac103200 0x7ff1ac198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1ac103b20 0x7ff1ac198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b1bc7640 1 -- 172.21.15.19:0/2916092421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1ac103b20 msgr2=0x7ff1ac198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b1bc7640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1ac103b20 0x7ff1ac198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b1bc7640 1 -- 172.21.15.19:0/2916092421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1ac103200 msgr2=0x7ff1ac198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b1bc7640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1ac103200 0x7ff1ac198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:22.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b1bc7640 1 -- 172.21.15.19:0/2916092421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac19d620 con 0x7ff1ac10c5d0 2024-04-23T18:41:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.021+0000 7ff1b1bc7640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ac10c5d0 0x7ff1ac19cef0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4004af0 tx=0x7ff1a402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.022+0000 7ff1a27fc640 1 -- 172.21.15.19:0/2916092421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4034040 con 0x7ff1ac10c5d0 2024-04-23T18:41:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.022+0000 7ff1a27fc640 1 -- 172.21.15.19:0/2916092421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a402fe80 con 0x7ff1ac10c5d0 2024-04-23T18:41:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.022+0000 7ff1b362a640 1 -- 172.21.15.19:0/2916092421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac19d8b0 con 0x7ff1ac10c5d0 2024-04-23T18:41:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.022+0000 7ff1a27fc640 1 -- 172.21.15.19:0/2916092421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4004670 con 0x7ff1ac10c5d0 2024-04-23T18:41:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.022+0000 7ff1b362a640 1 -- 172.21.15.19:0/2916092421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac1a51a0 con 0x7ff1ac10c5d0 2024-04-23T18:41:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.024+0000 7ff1a27fc640 1 -- 172.21.15.19:0/2916092421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff1a4037020 con 0x7ff1ac10c5d0 2024-04-23T18:41:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.024+0000 7ff1b641d740 1 Processor -- start 2024-04-23T18:41:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.024+0000 7ff1b641d740 1 -- start start 2024-04-23T18:41:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bc2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bc2f40 0x558ab4bcd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bcd800 0x558ab4bcfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b641d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ab4b27970 con 0x558ab4bc2620 2024-04-23T18:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b641d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558ab4ac44a0 con 0x558ab4bcd800 2024-04-23T18:41:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b641d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558ab4b23930 con 0x558ab4bc2f40 2024-04-23T18:41:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1a27fc640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3dc0 0x7ff1980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b0bc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bc2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b0bc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bc2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44740/0 (socket says 172.21.15.19:44740) 2024-04-23T18:41:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/2352892073 learned_addr learned my addr 172.21.15.19:0/2352892073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b1bc7640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3dc0 0x7ff1980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/2352892073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bcd800 0x558ab4bcfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1a27fc640 1 -- 172.21.15.19:0/2916092421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7ff1a40f14e0 con 0x7ff1ac10c5d0 2024-04-23T18:41:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b13c6640 1 --2- 172.21.15.19:0/2352892073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bc2f40 0x558ab4bcd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/2352892073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bc2f40 msgr2=0x558ab4bcd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/2352892073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bc2f40 0x558ab4bcd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/2352892073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bcd800 msgr2=0x558ab4bcfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/2352892073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bcd800 0x558ab4bcfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/2352892073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab483a210 con 0x558ab4bc2620 2024-04-23T18:41:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.025+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/2352892073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bc2a00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0058c0 tx=0x7ff19c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25fdaf3fcc3e0df server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1b1bc7640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3dc0 0x7ff1980a6280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40bd260 tx=0x7ff1a408efa0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1977fe640 1 -- 172.21.15.19:0/2352892073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c004050 con 0x558ab4bc2620 2024-04-23T18:41:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1977fe640 1 -- 172.21.15.19:0/2352892073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19c0041f0 con 0x558ab4bc2620 2024-04-23T18:41:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1977fe640 1 -- 172.21.15.19:0/2352892073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00f760 con 0x558ab4bc2620 2024-04-23T18:41:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1b641d740 1 -- 172.21.15.19:0/2352892073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 msgr2=0x558ab4bc2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:24.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1b641d740 1 --2- 172.21.15.19:0/2352892073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bc2a00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0058c0 tx=0x7ff19c005850 comp rx=0 tx=0).stop 2024-04-23T18:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1b641d740 1 -- 172.21.15.19:0/2352892073 shutdown_connections 2024-04-23T18:41:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1b641d740 1 --2- 172.21.15.19:0/2352892073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bcd800 0x558ab4bcfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1b641d740 1 --2- 172.21.15.19:0/2352892073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bc2f40 0x558ab4bcd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1b641d740 1 --2- 172.21.15.19:0/2352892073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bc2a00 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.026+0000 7ff1b641d740 1 -- 172.21.15.19:0/2352892073 >> 172.21.15.19:0/2352892073 conn(0x558ab4bbeb30 msgr2=0x558ab4bbef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 1 -- 172.21.15.19:0/2352892073 shutdown_connections 2024-04-23T18:41:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 1 -- 172.21.15.19:0/2352892073 wait complete. 2024-04-23T18:41:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 1 -- start start 2024-04-23T18:41:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 1 Processor -- start 2024-04-23T18:41:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bcd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 0x558ab4bca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ab4bd03d0 con 0x558ab4bc2620 2024-04-23T18:41:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558ab4ac20b0 con 0x558ab4bc2f40 2024-04-23T18:41:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558ab4bd0550 con 0x558ab4bcd800 2024-04-23T18:41:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b641d740 10 client.0 _ll_register_callbacks cb 0x558ab4aafd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.027+0000 7ff1b0bc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bcd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b0bc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bcd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44752/0 (socket says 172.21.15.19:44752) 2024-04-23T18:41:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/710452302 learned_addr learned my addr 172.21.15.19:0/710452302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b13c6640 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/710452302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 msgr2=0x558ab4bca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 0x558ab4bca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/710452302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 msgr2=0x558ab4bca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/710452302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab4906620 con 0x558ab4bc2620 2024-04-23T18:41:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bcd0d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009e70 tx=0x7ff19c013380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c5a1394aa36f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff194ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:41:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff194ff9640 1 -- 172.21.15.19:0/710452302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c005040 con 0x558ab4bc2620 2024-04-23T18:41:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff194ff9640 1 -- 172.21.15.19:0/710452302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7ff19c0051e0 con 0x558ab4bc2620 2024-04-23T18:41:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b641d740 1 -- 172.21.15.19:0/710452302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff19c0041f0 con 0x558ab4bc2620 2024-04-23T18:41:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff194ff9640 1 -- 172.21.15.19:0/710452302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff19c0053e0 con 0x558ab4bc2620 2024-04-23T18:41:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b641d740 10 client.17277 fetch_fsmap learned FSMap version 29 2024-04-23T18:41:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.028+0000 7ff1b641d740 1 -- 172.21.15.19:0/710452302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ab4bdc2b0 con 0x558ab4bc2620 2024-04-23T18:41:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.029+0000 7ff194ff9640 1 -- 172.21.15.19:0/710452302 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19c0041f0 con 0x558ab4bc2620 2024-04-23T18:41:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.029+0000 7ff194ff9640 1 -- 172.21.15.19:0/710452302 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff19c01e070 con 0x558ab4bc2620 2024-04-23T18:41:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.029+0000 7ff1b641d740 10 client.17277 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:41:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.029+0000 7ff1b641d740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.029+0000 7ff1b641d740 20 client.17277 populate_metadata read hostname 'smithi019' 2024-04-23T18:41:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.029+0000 7ff1b641d740 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ab4bdd6c0 0x558ab4bdfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:26.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.029+0000 7ff1b641d740 4 client.17277 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:41:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.029+0000 7ff1b641d740 1 -- 172.21.15.19:0/710452302 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ab4b35390 con 0x558ab4bdd6c0 2024-04-23T18:41:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.029+0000 7ff1b13c6640 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ab4bdd6c0 0x558ab4bdfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff1b13c6640 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ab4bdd6c0 0x558ab4bdfab0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a1d59bd11a6130 server_cookie=e265814762826de2 in_seq=0 out_seq=0 2024-04-23T18:41:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff194ff9640 10 client.17277 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff194ff9640 1 -- 172.21.15.19:0/710452302 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ab4b35390 con 0x558ab4bdd6c0 2024-04-23T18:41:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff194ff9640 10 client.17277 handle_command_reply: tid=0 2024-04-23T18:41:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff1b641d740 1 client.17277 shutdown 2024-04-23T18:41:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff1b641d740 1 -- 172.21.15.19:0/710452302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 msgr2=0x558ab4bcd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff1b641d740 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bcd0d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009e70 tx=0x7ff19c013380 comp rx=0 tx=0).stop 2024-04-23T18:41:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff1b641d740 1 -- 172.21.15.19:0/710452302 shutdown_connections 2024-04-23T18:41:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff1b641d740 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ab4bdd6c0 0x558ab4bdfab0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff1b641d740 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff1b641d740 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 0x558ab4bca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff194ff9640 0 client.17277 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff1b641d740 1 --2- 172.21.15.19:0/710452302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bcd0d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.030+0000 7ff1b641d740 1 -- 172.21.15.19:0/710452302 >> 172.21.15.19:0/710452302 conn(0x558ab4bbeb30 msgr2=0x558ab4bc4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.031+0000 7ff1b641d740 1 -- 172.21.15.19:0/710452302 shutdown_connections 2024-04-23T18:41:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.031+0000 7ff1b641d740 1 -- 172.21.15.19:0/710452302 wait complete. 2024-04-23T18:41:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.031+0000 7ff1b641d740 20 client.17277 trim_cache size 0 max 16384 2024-04-23T18:41:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b641d740 1 Processor -- start 2024-04-23T18:41:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b641d740 1 -- start start 2024-04-23T18:41:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bbd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 0x558ab4bccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b641d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1ab7d0 con 0x558ab4bc2620 2024-04-23T18:41:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b641d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1ad310 con 0x558ab4bc2f40 2024-04-23T18:41:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b641d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1abea0 con 0x558ab4bcd800 2024-04-23T18:41:28.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b0bc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b0bc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44758/0 (socket says 172.21.15.19:44758) 2024-04-23T18:41:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/3124054017 learned_addr learned my addr 172.21.15.19:0/3124054017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/3124054017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 msgr2=0x558ab4bccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/3124054017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 0x558ab4bccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/3124054017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 msgr2=0x558ab4bbd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b13c6640 1 --2- 172.21.15.19:0/3124054017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bbd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/3124054017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bbd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/3124054017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab4bc9370 con 0x558ab4bc2620 2024-04-23T18:41:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b13c6640 1 --2- 172.21.15.19:0/3124054017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bbd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:41:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.046+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/3124054017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd9290 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff19c013550 tx=0x7ff19c009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80845564de1c668 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff196ffd640 1 -- 172.21.15.19:0/3124054017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00f840 con 0x558ab4bc2620 2024-04-23T18:41:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff196ffd640 1 -- 172.21.15.19:0/3124054017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19c038020 con 0x558ab4bc2620 2024-04-23T18:41:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff196ffd640 1 -- 172.21.15.19:0/3124054017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c003ce0 con 0x558ab4bc2620 2024-04-23T18:41:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff1b641d740 1 -- 172.21.15.19:0/3124054017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 msgr2=0x558ab4bd9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff1b641d740 1 --2- 172.21.15.19:0/3124054017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd9290 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff19c013550 tx=0x7ff19c009b40 comp rx=0 tx=0).stop 2024-04-23T18:41:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff1b641d740 1 -- 172.21.15.19:0/3124054017 shutdown_connections 2024-04-23T18:41:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff1b641d740 1 --2- 172.21.15.19:0/3124054017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bbd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff1b641d740 1 --2- 172.21.15.19:0/3124054017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 0x558ab4bccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff1b641d740 1 --2- 172.21.15.19:0/3124054017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd9290 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff1b641d740 1 -- 172.21.15.19:0/3124054017 >> 172.21.15.19:0/3124054017 conn(0x558ab4bbeb30 msgr2=0x558ab4bc59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff1b641d740 1 -- 172.21.15.19:0/3124054017 shutdown_connections 2024-04-23T18:41:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.047+0000 7ff1b641d740 1 -- 172.21.15.19:0/3124054017 wait complete. 2024-04-23T18:41:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b641d740 1 -- start start 2024-04-23T18:41:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b641d740 1 Processor -- start 2024-04-23T18:41:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bd0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b641d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 0x558ab4bd13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b641d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1acc40 con 0x558ab4bc2620 2024-04-23T18:41:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b641d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1ac570 con 0x558ab4bc2f40 2024-04-23T18:41:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b641d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1a97d0 con 0x558ab4bcd800 2024-04-23T18:41:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b641d740 10 client.0 _ll_register_callbacks cb 0x7ff19c013240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b0bc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b0bc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44772/0 (socket says 172.21.15.19:44772) 2024-04-23T18:41:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/3727801799 learned_addr learned my addr 172.21.15.19:0/3727801799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 0x558ab4bd13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.048+0000 7ff1b13c6640 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bd0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/3727801799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 msgr2=0x558ab4bd13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 0x558ab4bd13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/3727801799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 msgr2=0x558ab4bd0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bd0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff1b0bc5640 1 -- 172.21.15.19:0/3727801799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab4bd1b90 con 0x558ab4bc2620 2024-04-23T18:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff1b0bc5640 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd5ae0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009380 tx=0x7ff19c005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3933c99a0c229a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff194ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:41:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff194ff9640 1 -- 172.21.15.19:0/3727801799 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c009c40 con 0x558ab4bc2620 2024-04-23T18:41:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff194ff9640 1 -- 172.21.15.19:0/3727801799 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7ff19c003980 con 0x558ab4bc2620 2024-04-23T18:41:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff1b641d740 1 -- 172.21.15.19:0/3727801799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff19c038020 con 0x558ab4bc2620 2024-04-23T18:41:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff194ff9640 1 -- 172.21.15.19:0/3727801799 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff19c046050 con 0x558ab4bc2620 2024-04-23T18:41:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff1b641d740 10 client.17283 fetch_fsmap learned FSMap version 29 2024-04-23T18:41:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff1b641d740 1 -- 172.21.15.19:0/3727801799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ab4c439b0 con 0x558ab4bc2620 2024-04-23T18:41:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.049+0000 7ff194ff9640 1 -- 172.21.15.19:0/3727801799 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19c038020 con 0x558ab4bc2620 2024-04-23T18:41:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.050+0000 7ff194ff9640 1 -- 172.21.15.19:0/3727801799 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff19c01e040 con 0x558ab4bc2620 2024-04-23T18:41:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.050+0000 7ff1b641d740 10 client.17283 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:41:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.050+0000 7ff1b641d740 10 client.17283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.050+0000 7ff1b641d740 20 client.17283 populate_metadata read hostname 'smithi019' 2024-04-23T18:41:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.050+0000 7ff1b641d740 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ab4c43f40 0x558ab4c46360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.050+0000 7ff1b641d740 4 client.17283 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:41:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.050+0000 7ff1b641d740 1 -- 172.21.15.19:0/3727801799 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ab4b2f200 con 0x558ab4c43f40 2024-04-23T18:41:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.050+0000 7ff1b13c6640 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ab4c43f40 0x558ab4c46360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.050+0000 7ff1b13c6640 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ab4c43f40 0x558ab4c46360 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c195b5b992751d server_cookie=a3aa7fa50e9d09c4 in_seq=0 out_seq=0 2024-04-23T18:41:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.050+0000 7ff194ff9640 10 client.17283 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff194ff9640 1 -- 172.21.15.19:0/3727801799 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ab4b2f200 con 0x558ab4c43f40 2024-04-23T18:41:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff194ff9640 10 client.17283 handle_command_reply: tid=0 2024-04-23T18:41:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 client.17283 shutdown 2024-04-23T18:41:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 -- 172.21.15.19:0/3727801799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 msgr2=0x558ab4bd5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd5ae0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009380 tx=0x7ff19c005b90 comp rx=0 tx=0).stop 2024-04-23T18:41:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 -- 172.21.15.19:0/3727801799 shutdown_connections 2024-04-23T18:41:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x558ab4c43f40 0x558ab4c46360 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ab4bc2f40 0x558ab4bd0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff194ff9640 0 client.17283 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ab4bcd800 0x558ab4bd13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 --2- 172.21.15.19:0/3727801799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ab4bc2620 0x558ab4bd5ae0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 -- 172.21.15.19:0/3727801799 >> 172.21.15.19:0/3727801799 conn(0x558ab4bbeb30 msgr2=0x558ab4bc3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 -- 172.21.15.19:0/3727801799 shutdown_connections 2024-04-23T18:41:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.064+0000 7ff1b641d740 1 -- 172.21.15.19:0/3727801799 wait complete. 2024-04-23T18:41:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b641d740 20 client.17283 trim_cache size 0 max 16384 2024-04-23T18:41:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:41:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:41:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:41:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:41:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 -- 172.21.15.19:0/2916092421 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3dc0 msgr2=0x7ff1980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3dc0 0x7ff1980a6280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40bd260 tx=0x7ff1a408efa0 comp rx=0 tx=0).stop 2024-04-23T18:41:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 -- 172.21.15.19:0/2916092421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ac10c5d0 msgr2=0x7ff1ac19cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ac10c5d0 0x7ff1ac19cef0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4004af0 tx=0x7ff1a402f5d0 comp rx=0 tx=0).stop 2024-04-23T18:41:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 -- 172.21.15.19:0/2916092421 shutdown_connections 2024-04-23T18:41:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3dc0 0x7ff1980a6280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ac10c5d0 0x7ff1ac19cef0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1ac103b20 0x7ff1ac198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 --2- 172.21.15.19:0/2916092421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1ac103200 0x7ff1ac198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 -- 172.21.15.19:0/2916092421 >> 172.21.15.19:0/2916092421 conn(0x7ff1ac0771d0 msgr2=0x7ff1ac0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 -- 172.21.15.19:0/2916092421 shutdown_connections 2024-04-23T18:41:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:46.065+0000 7ff1b362a640 1 -- 172.21.15.19:0/2916092421 wait complete. 2024-04-23T18:41:33.557 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:45 smithi019 ceph-mon[29215]: pgmap v426: 97 pgs: 97 active+clean; 2.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T18:41:33.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:47 smithi019 ceph-mon[29215]: pgmap v427: 97 pgs: 97 active+clean; 2.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T18:41:33.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:49 smithi019 ceph-mon[29215]: pgmap v428: 97 pgs: 97 active+clean; 2.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T18:41:33.723 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:51 smithi019 ceph-mon[29215]: pgmap v429: 97 pgs: 97 active+clean; 2.5 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T18:41:33.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:53 smithi019 ceph-mon[29215]: pgmap v430: 97 pgs: 97 active+clean; 2.5 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T18:41:33.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:55 smithi019 ceph-mon[29215]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T18:41:33.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:57 smithi019 ceph-mon[29215]: pgmap v432: 97 pgs: 97 active+clean; 2.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:41:33.933 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:21 smithi155 ceph-mon[33468]: pgmap v478: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-23T18:41:33.964 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:23 smithi155 ceph-mon[33468]: pgmap v479: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-23T18:41:34.014 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:25 smithi155 ceph-mon[33468]: pgmap v480: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-23T18:41:34.061 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:27 smithi155 ceph-mon[33468]: pgmap v481: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-23T18:41:34.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:28 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:41:34.131 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:28 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:41:34.203 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:28 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:41:34.248 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:28 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:41:34.281 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:29 smithi155 ceph-mon[33468]: pgmap v482: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-23T18:41:34.314 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:31 smithi155 ceph-mon[33468]: pgmap v483: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-23T18:41:34.348 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:21 smithi044 ceph-mon[32458]: pgmap v478: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-23T18:41:34.414 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:23 smithi044 ceph-mon[32458]: pgmap v479: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-23T18:41:34.448 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:25 smithi044 ceph-mon[32458]: pgmap v480: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-23T18:41:34.481 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:27 smithi044 ceph-mon[32458]: pgmap v481: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-23T18:41:34.565 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:28 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:41:34.598 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:28 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:41:34.645 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:28 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:41:34.681 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:28 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:41:34.732 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:29 smithi044 ceph-mon[32458]: pgmap v482: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-23T18:41:34.793 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:31 smithi044 ceph-mon[32458]: pgmap v483: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-23T18:41:34.833 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:33 smithi155 ceph-mon[33468]: pgmap v484: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:41:34.865 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:33 smithi044 ceph-mon[32458]: pgmap v484: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:41:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.163+0000 7fd50c920640 1 -- 172.21.15.19:0/1551926505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd50810c5d0 msgr2=0x7fd50810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.163+0000 7fd50c920640 1 --2- 172.21.15.19:0/1551926505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd50810c5d0 0x7fd50810ea00 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd500009ca0 tx=0x7fd50002a950 comp rx=0 tx=0).stop 2024-04-23T18:41:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.164+0000 7fd50c920640 1 -- 172.21.15.19:0/1551926505 shutdown_connections 2024-04-23T18:41:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.164+0000 7fd50c920640 1 --2- 172.21.15.19:0/1551926505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd50810c5d0 0x7fd50810ea00 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.164+0000 7fd50c920640 1 --2- 172.21.15.19:0/1551926505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd508103b20 0x7fd50810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.164+0000 7fd50c920640 1 --2- 172.21.15.19:0/1551926505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508103200 0x7fd5081035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.164+0000 7fd50c920640 1 -- 172.21.15.19:0/1551926505 >> 172.21.15.19:0/1551926505 conn(0x7fd5080771d0 msgr2=0x7fd5080775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.164+0000 7fd50c920640 1 -- 172.21.15.19:0/1551926505 shutdown_connections 2024-04-23T18:41:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.164+0000 7fd50c920640 1 -- 172.21.15.19:0/1551926505 wait complete. 2024-04-23T18:41:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.164+0000 7fd50c920640 1 Processor -- start 2024-04-23T18:41:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.164+0000 7fd50c920640 1 -- start start 2024-04-23T18:41:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd50c920640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508103200 0x7fd5081986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd50c920640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd508103b20 0x7fd508198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd50c920640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd50810c5d0 0x7fd50819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd50c920640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081121d0 con 0x7fd50810c5d0 2024-04-23T18:41:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd50c920640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd508112050 con 0x7fd508103200 2024-04-23T18:41:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd50c920640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd508112350 con 0x7fd508103b20 2024-04-23T18:41:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd506d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd50810c5d0 0x7fd50819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd506d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd50810c5d0 0x7fd50819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34552/0 (socket says 172.21.15.19:34552) 2024-04-23T18:41:36.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd506d9d640 1 -- 172.21.15.19:0/2218541630 learned_addr learned my addr 172.21.15.19:0/2218541630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd50659c640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508103200 0x7fd5081986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd4fffff640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd508103b20 0x7fd508198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd506d9d640 1 -- 172.21.15.19:0/2218541630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd508103b20 msgr2=0x7fd508198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd506d9d640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd508103b20 0x7fd508198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd506d9d640 1 -- 172.21.15.19:0/2218541630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508103200 msgr2=0x7fd5081986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd506d9d640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508103200 0x7fd5081986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd506d9d640 1 -- 172.21.15.19:0/2218541630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50819d6a0 con 0x7fd50810c5d0 2024-04-23T18:41:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.165+0000 7fd506d9d640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd50810c5d0 0x7fd50819cf70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd500004af0 tx=0x7fd50002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.166+0000 7fd4ff7fe640 1 -- 172.21.15.19:0/2218541630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500034040 con 0x7fd50810c5d0 2024-04-23T18:41:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.166+0000 7fd50c920640 1 -- 172.21.15.19:0/2218541630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50819d930 con 0x7fd50810c5d0 2024-04-23T18:41:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.166+0000 7fd4ff7fe640 1 -- 172.21.15.19:0/2218541630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd50002fe80 con 0x7fd50810c5d0 2024-04-23T18:41:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.166+0000 7fd4ff7fe640 1 -- 172.21.15.19:0/2218541630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500004670 con 0x7fd50810c5d0 2024-04-23T18:41:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.166+0000 7fd50c920640 1 -- 172.21.15.19:0/2218541630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5081a5220 con 0x7fd50810c5d0 2024-04-23T18:41:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.168+0000 7fd4ff7fe640 1 -- 172.21.15.19:0/2218541630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd500037020 con 0x7fd50810c5d0 2024-04-23T18:41:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.168+0000 7fd50f63a740 1 Processor -- start 2024-04-23T18:41:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50f63a740 1 -- start start 2024-04-23T18:41:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438d620 0x55ce4438da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce443982c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce44398800 0x55ce4439abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50f63a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ce442f2970 con 0x55ce4438df40 2024-04-23T18:41:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50f63a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ce4428f4a0 con 0x55ce44398800 2024-04-23T18:41:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50f63a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ce442ee930 con 0x55ce4438d620 2024-04-23T18:41:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce443982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce443982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34556/0 (socket says 172.21.15.19:34556) 2024-04-23T18:41:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50659c640 1 -- 172.21.15.19:0/1043015878 learned_addr learned my addr 172.21.15.19:0/1043015878 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd4ff7fe640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4f40a3dc0 0x7fd4f40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd4fffff640 1 --2- 172.21.15.19:0/1043015878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce44398800 0x55ce4439abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd4fffff640 1 --2- 172.21.15.19:0/1043015878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438d620 0x55ce4438da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd506d9d640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4f40a3dc0 0x7fd4f40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50659c640 1 -- 172.21.15.19:0/1043015878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438d620 msgr2=0x55ce4438da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50659c640 1 --2- 172.21.15.19:0/1043015878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438d620 0x55ce4438da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50659c640 1 -- 172.21.15.19:0/1043015878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce44398800 msgr2=0x55ce4439abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50659c640 1 --2- 172.21.15.19:0/1043015878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce44398800 0x55ce4439abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd50659c640 1 -- 172.21.15.19:0/1043015878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce44005210 con 0x55ce4438df40 2024-04-23T18:41:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.169+0000 7fd4fffff640 1 --2- 172.21.15.19:0/1043015878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce44398800 0x55ce4439abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:41:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.170+0000 7fd4ff7fe640 1 -- 172.21.15.19:0/2218541630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7fd5000f1660 con 0x7fd50810c5d0 2024-04-23T18:41:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.170+0000 7fd50659c640 1 --2- 172.21.15.19:0/1043015878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce443982c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009c80 tx=0x7fd4f80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3337e29d671ed1dd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.170+0000 7fd4fcff9640 1 -- 172.21.15.19:0/1043015878 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8012450 con 0x55ce4438df40 2024-04-23T18:41:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.170+0000 7fd506d9d640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4f40a3dc0 0x7fd4f40a6280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5000bd670 tx=0x7fd50008f120 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.170+0000 7fd4fcff9640 1 -- 172.21.15.19:0/1043015878 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f8012aa0 con 0x55ce4438df40 2024-04-23T18:41:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.170+0000 7fd4fcff9640 1 -- 172.21.15.19:0/1043015878 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800fb20 con 0x55ce4438df40 2024-04-23T18:41:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 -- 172.21.15.19:0/1043015878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 msgr2=0x55ce443982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 --2- 172.21.15.19:0/1043015878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce443982c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009c80 tx=0x7fd4f80117b0 comp rx=0 tx=0).stop 2024-04-23T18:41:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 -- 172.21.15.19:0/1043015878 shutdown_connections 2024-04-23T18:41:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 --2- 172.21.15.19:0/1043015878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce44398800 0x55ce4439abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 --2- 172.21.15.19:0/1043015878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce443982c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 --2- 172.21.15.19:0/1043015878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438d620 0x55ce4438da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 -- 172.21.15.19:0/1043015878 >> 172.21.15.19:0/1043015878 conn(0x55ce44389b30 msgr2=0x55ce44389f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 -- 172.21.15.19:0/1043015878 shutdown_connections 2024-04-23T18:41:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 -- 172.21.15.19:0/1043015878 wait complete. 2024-04-23T18:41:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 -- start start 2024-04-23T18:41:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.171+0000 7fd50f63a740 1 Processor -- start 2024-04-23T18:41:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438d620 0x55ce44398080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438df40 0x55ce44395370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce44398800 0x55ce443958b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd50f63a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ce4439b430 con 0x55ce44398800 2024-04-23T18:41:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd50f63a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ce4428d0b0 con 0x55ce4438df40 2024-04-23T18:41:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd50f63a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ce4439b5b0 con 0x55ce4438d620 2024-04-23T18:41:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd50f63a740 10 client.0 _ll_register_callbacks cb 0x55ce4427ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce44398800 0x55ce443958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce44398800 0x55ce443958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34570/0 (socket says 172.21.15.19:34570) 2024-04-23T18:41:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4fffff640 1 -- 172.21.15.19:0/2232341096 learned_addr learned my addr 172.21.15.19:0/2232341096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd50659c640 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438df40 0x55ce44395370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4fffff640 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438d620 0x55ce44398080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4fffff640 1 -- 172.21.15.19:0/2232341096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438d620 msgr2=0x55ce44398080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4fffff640 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438d620 0x55ce44398080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4fffff640 1 -- 172.21.15.19:0/2232341096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438df40 msgr2=0x55ce44395370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4fffff640 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438df40 0x55ce44395370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4fffff640 1 -- 172.21.15.19:0/2232341096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce440d1620 con 0x55ce44398800 2024-04-23T18:41:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd50659c640 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438df40 0x55ce44395370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:41:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4fffff640 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce44398800 0x55ce443958b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0013820 tx=0x7fd4f0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f754113783428d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4edffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4edffb640 1 -- 172.21.15.19:0/2232341096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0013e60 con 0x55ce44398800 2024-04-23T18:41:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4edffb640 1 -- 172.21.15.19:0/2232341096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7fd4f0011b70 con 0x55ce44398800 2024-04-23T18:41:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd50f63a740 1 -- 172.21.15.19:0/2232341096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4f8012aa0 con 0x55ce44398800 2024-04-23T18:41:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.172+0000 7fd4edffb640 1 -- 172.21.15.19:0/2232341096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd4f0011d70 con 0x55ce44398800 2024-04-23T18:41:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd50f63a740 10 client.17295 fetch_fsmap learned FSMap version 29 2024-04-23T18:41:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd50f63a740 1 -- 172.21.15.19:0/2232341096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce443a7320 con 0x55ce44398800 2024-04-23T18:41:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd4edffb640 1 -- 172.21.15.19:0/2232341096 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f8012aa0 con 0x55ce44398800 2024-04-23T18:41:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd4edffb640 1 -- 172.21.15.19:0/2232341096 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd4f000f040 con 0x55ce44398800 2024-04-23T18:41:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd50f63a740 10 client.17295 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:41:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd50f63a740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd50f63a740 20 client.17295 populate_metadata read hostname 'smithi019' 2024-04-23T18:41:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd50f63a740 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ce443a8830 0x55ce443aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd50f63a740 4 client.17295 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:41:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd50f63a740 1 -- 172.21.15.19:0/2232341096 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce44300390 con 0x55ce443a8830 2024-04-23T18:41:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.173+0000 7fd50659c640 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ce443a8830 0x55ce443aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.174+0000 7fd4edffb640 10 client.17295 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.174+0000 7fd50659c640 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ce443a8830 0x55ce443aac40 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca27f60ce27c3a7 server_cookie=66c57cc2c3ef4962 in_seq=0 out_seq=0 2024-04-23T18:41:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.174+0000 7fd4edffb640 1 -- 172.21.15.19:0/2232341096 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce44300390 con 0x55ce443a8830 2024-04-23T18:41:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd4edffb640 10 client.17295 handle_command_reply: tid=0 2024-04-23T18:41:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 client.17295 shutdown 2024-04-23T18:41:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 -- 172.21.15.19:0/2232341096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce44398800 msgr2=0x55ce443958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce44398800 0x55ce443958b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0013820 tx=0x7fd4f0015040 comp rx=0 tx=0).stop 2024-04-23T18:41:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 -- 172.21.15.19:0/2232341096 shutdown_connections 2024-04-23T18:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ce443a8830 0x55ce443aac40 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438df40 0x55ce44395370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce44398800 0x55ce443958b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd4edffb640 0 client.17295 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 --2- 172.21.15.19:0/2232341096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438d620 0x55ce44398080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 -- 172.21.15.19:0/2232341096 >> 172.21.15.19:0/2232341096 conn(0x55ce44389b30 msgr2=0x55ce4438f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 -- 172.21.15.19:0/2232341096 shutdown_connections 2024-04-23T18:41:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 1 -- 172.21.15.19:0/2232341096 wait complete. 2024-04-23T18:41:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.175+0000 7fd50f63a740 20 client.17295 trim_cache size 0 max 16384 2024-04-23T18:41:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd50f63a740 1 Processor -- start 2024-04-23T18:41:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd50f63a740 1 -- start start 2024-04-23T18:41:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438d620 0x55ce443a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438df40 0x55ce443944d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce44398800 0x55ce44388b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd50f63a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081ada90 con 0x55ce4438d620 2024-04-23T18:41:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd50f63a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081af5d0 con 0x55ce44398800 2024-04-23T18:41:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd50f63a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081ae160 con 0x55ce4438df40 2024-04-23T18:41:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd4fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438d620 0x55ce443a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd4fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438d620 0x55ce443a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34582/0 (socket says 172.21.15.19:34582) 2024-04-23T18:41:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.190+0000 7fd4fffff640 1 -- 172.21.15.19:0/3299051101 learned_addr learned my addr 172.21.15.19:0/3299051101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd50659c640 1 --2- 172.21.15.19:0/3299051101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438df40 0x55ce443944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd4fffff640 1 -- 172.21.15.19:0/3299051101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438df40 msgr2=0x55ce443944d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd4fffff640 1 --2- 172.21.15.19:0/3299051101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438df40 0x55ce443944d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd4fffff640 1 -- 172.21.15.19:0/3299051101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce44398800 msgr2=0x55ce44388b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd4fffff640 1 --2- 172.21.15.19:0/3299051101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce44398800 0x55ce44388b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd4fffff640 1 -- 172.21.15.19:0/3299051101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce443938c0 con 0x55ce4438d620 2024-04-23T18:41:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd50659c640 1 --2- 172.21.15.19:0/3299051101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438df40 0x55ce443944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:41:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd4fffff640 1 --2- 172.21.15.19:0/3299051101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438d620 0x55ce443a42f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f0011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fbc4e77a89baa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd4effff640 1 -- 172.21.15.19:0/3299051101 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0034070 con 0x55ce4438d620 2024-04-23T18:41:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd4effff640 1 -- 172.21.15.19:0/3299051101 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f0005850 con 0x55ce4438d620 2024-04-23T18:41:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.191+0000 7fd4effff640 1 -- 172.21.15.19:0/3299051101 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0013e60 con 0x55ce4438d620 2024-04-23T18:41:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 -- 172.21.15.19:0/3299051101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438d620 msgr2=0x55ce443a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 --2- 172.21.15.19:0/3299051101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438d620 0x55ce443a42f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f0011fd0 comp rx=0 tx=0).stop 2024-04-23T18:41:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 -- 172.21.15.19:0/3299051101 shutdown_connections 2024-04-23T18:41:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 --2- 172.21.15.19:0/3299051101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce44398800 0x55ce44388b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 --2- 172.21.15.19:0/3299051101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce4438df40 0x55ce443944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 --2- 172.21.15.19:0/3299051101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438d620 0x55ce443a42f0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 -- 172.21.15.19:0/3299051101 >> 172.21.15.19:0/3299051101 conn(0x55ce44389b30 msgr2=0x55ce44388420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 -- 172.21.15.19:0/3299051101 shutdown_connections 2024-04-23T18:41:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 -- 172.21.15.19:0/3299051101 wait complete. 2024-04-23T18:41:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 -- start start 2024-04-23T18:41:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 Processor -- start 2024-04-23T18:41:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438d620 0x55ce443a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce4439dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce44398800 0x55ce4439e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081aef00 con 0x55ce4438df40 2024-04-23T18:41:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081ae830 con 0x55ce4438d620 2024-04-23T18:41:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081aba70 con 0x55ce44398800 2024-04-23T18:41:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50f63a740 10 client.0 _ll_register_callbacks cb 0x7fd4f000ee20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.192+0000 7fd50659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce4439dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce4439dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34592/0 (socket says 172.21.15.19:34592) 2024-04-23T18:41:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50659c640 1 -- 172.21.15.19:0/1712243613 learned_addr learned my addr 172.21.15.19:0/1712243613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd4fffff640 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438d620 0x55ce443a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd4fffff640 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce44398800 0x55ce4439e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50659c640 1 -- 172.21.15.19:0/1712243613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce44398800 msgr2=0x55ce4439e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50659c640 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce44398800 0x55ce4439e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50659c640 1 -- 172.21.15.19:0/1712243613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438d620 msgr2=0x55ce443a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50659c640 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438d620 0x55ce443a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50659c640 1 -- 172.21.15.19:0/1712243613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce4439e880 con 0x55ce4438df40 2024-04-23T18:41:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50659c640 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce4439dd50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8000c00 tx=0x7fd4f8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a879c43a829f626a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd4edffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:41:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd4edffb640 1 -- 172.21.15.19:0/1712243613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8030060 con 0x55ce4438df40 2024-04-23T18:41:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd4edffb640 1 -- 172.21.15.19:0/1712243613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7fd4f80052d0 con 0x55ce4438df40 2024-04-23T18:41:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50f63a740 1 -- 172.21.15.19:0/1712243613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4f0005850 con 0x55ce4438df40 2024-04-23T18:41:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd4edffb640 1 -- 172.21.15.19:0/1712243613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd4f80054d0 con 0x55ce4438df40 2024-04-23T18:41:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50f63a740 10 client.17301 fetch_fsmap learned FSMap version 29 2024-04-23T18:41:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd50f63a740 1 -- 172.21.15.19:0/1712243613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce4440eb80 con 0x55ce4438df40 2024-04-23T18:41:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.193+0000 7fd4edffb640 1 -- 172.21.15.19:0/1712243613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f0005850 con 0x55ce4438df40 2024-04-23T18:41:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.194+0000 7fd4edffb640 1 -- 172.21.15.19:0/1712243613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd4f8003980 con 0x55ce4438df40 2024-04-23T18:41:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.194+0000 7fd50f63a740 10 client.17301 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:41:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.194+0000 7fd50f63a740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.194+0000 7fd50f63a740 20 client.17301 populate_metadata read hostname 'smithi019' 2024-04-23T18:41:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.194+0000 7fd50f63a740 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ce4440f1a0 0x55ce444115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.194+0000 7fd50f63a740 4 client.17301 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:41:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.194+0000 7fd50f63a740 1 -- 172.21.15.19:0/1712243613 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce442fa200 con 0x55ce4440f1a0 2024-04-23T18:41:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.194+0000 7fd4fffff640 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ce4440f1a0 0x55ce444115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.194+0000 7fd4fffff640 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ce4440f1a0 0x55ce444115c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd7b367b728734d server_cookie=ebb5b5aa1e80450e in_seq=0 out_seq=0 2024-04-23T18:41:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.194+0000 7fd4edffb640 10 client.17301 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd4edffb640 1 -- 172.21.15.19:0/1712243613 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce442fa200 con 0x55ce4440f1a0 2024-04-23T18:41:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd4edffb640 10 client.17301 handle_command_reply: tid=0 2024-04-23T18:41:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 client.17301 shutdown 2024-04-23T18:41:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 -- 172.21.15.19:0/1712243613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 msgr2=0x55ce4439dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce4439dd50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8000c00 tx=0x7fd4f8009560 comp rx=0 tx=0).stop 2024-04-23T18:41:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 -- 172.21.15.19:0/1712243613 shutdown_connections 2024-04-23T18:41:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55ce4440f1a0 0x55ce444115c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce44398800 0x55ce4439e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce4438df40 0x55ce4439dd50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 --2- 172.21.15.19:0/1712243613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ce4438d620 0x55ce443a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd4edffb640 0 client.17301 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 -- 172.21.15.19:0/1712243613 >> 172.21.15.19:0/1712243613 conn(0x55ce44389b30 msgr2=0x55ce44394350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 -- 172.21.15.19:0/1712243613 shutdown_connections 2024-04-23T18:41:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 1 -- 172.21.15.19:0/1712243613 wait complete. 2024-04-23T18:41:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.208+0000 7fd50f63a740 20 client.17301 trim_cache size 0 max 16384 2024-04-23T18:41:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:41:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:41:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:41:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:41:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 -- 172.21.15.19:0/2218541630 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4f40a3dc0 msgr2=0x7fd4f40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4f40a3dc0 0x7fd4f40a6280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5000bd670 tx=0x7fd50008f120 comp rx=0 tx=0).stop 2024-04-23T18:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 -- 172.21.15.19:0/2218541630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd50810c5d0 msgr2=0x7fd50819cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd50810c5d0 0x7fd50819cf70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd500004af0 tx=0x7fd50002f5d0 comp rx=0 tx=0).stop 2024-04-23T18:41:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 -- 172.21.15.19:0/2218541630 shutdown_connections 2024-04-23T18:41:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd4f40a3dc0 0x7fd4f40a6280 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd50810c5d0 0x7fd50819cf70 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd508103b20 0x7fd508198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 --2- 172.21.15.19:0/2218541630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508103200 0x7fd5081986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 -- 172.21.15.19:0/2218541630 >> 172.21.15.19:0/2218541630 conn(0x7fd5080771d0 msgr2=0x7fd5080fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 -- 172.21.15.19:0/2218541630 shutdown_connections 2024-04-23T18:41:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:39:59.209+0000 7fd50c920640 1 -- 172.21.15.19:0/2218541630 wait complete. 2024-04-23T18:41:45.579 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:35 smithi155 ceph-mon[33468]: pgmap v485: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:41:45.610 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:37 smithi155 ceph-mon[33468]: pgmap v486: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:41:45.651 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:39 smithi155 ceph-mon[33468]: pgmap v487: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:41:45.685 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:41 smithi155 ceph-mon[33468]: pgmap v488: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:41:45.723 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:43 smithi155 ceph-mon[33468]: pgmap v489: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:41:45.792 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:35 smithi044 ceph-mon[32458]: pgmap v485: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:41:45.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:37 smithi044 ceph-mon[32458]: pgmap v486: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:41:45.870 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:39 smithi044 ceph-mon[32458]: pgmap v487: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:41:45.893 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:41 smithi044 ceph-mon[32458]: pgmap v488: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:41:45.927 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:43 smithi044 ceph-mon[32458]: pgmap v489: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:41:45.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:39:59 smithi019 ceph-mon[29215]: pgmap v433: 97 pgs: 97 active+clean; 2.7 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:41:46.028 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:00 smithi019 ceph-mon[29215]: overall HEALTH_OK 2024-04-23T18:41:46.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:01 smithi019 ceph-mon[29215]: pgmap v434: 97 pgs: 97 active+clean; 3.0 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-04-23T18:41:46.177 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:03 smithi019 ceph-mon[29215]: pgmap v435: 97 pgs: 97 active+clean; 3.0 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T18:41:46.211 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:05 smithi019 ceph-mon[29215]: pgmap v436: 97 pgs: 97 active+clean; 3.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-23T18:41:46.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:06 smithi019 ceph-mon[29215]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T18:41:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:06 smithi019 ceph-mon[29215]: pgmap v438: 97 pgs: 97 active+clean; 3.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:41:46.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:09 smithi019 ceph-mon[29215]: pgmap v439: 97 pgs: 97 active+clean; 3.2 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 109 KiB/s wr, 39 op/s 2024-04-23T18:41:46.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:11 smithi019 ceph-mon[29215]: pgmap v440: 97 pgs: 97 active+clean; 3.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-04-23T18:41:46.412 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:45 smithi155 ceph-mon[33468]: pgmap v490: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:41:46.470 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:45 smithi044 ceph-mon[32458]: pgmap v490: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:41:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.308+0000 7fc340795640 1 -- 172.21.15.19:0/4220240601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc338103200 msgr2=0x7fc3381035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.308+0000 7fc340795640 1 --2- 172.21.15.19:0/4220240601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc338103200 0x7fc3381035e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc334009900 tx=0x7fc33402a480 comp rx=0 tx=0).stop 2024-04-23T18:41:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.309+0000 7fc340795640 1 -- 172.21.15.19:0/4220240601 shutdown_connections 2024-04-23T18:41:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.309+0000 7fc340795640 1 --2- 172.21.15.19:0/4220240601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc338103b20 0x7fc33810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.309+0000 7fc340795640 1 --2- 172.21.15.19:0/4220240601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc33810c5d0 0x7fc33810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.309+0000 7fc340795640 1 --2- 172.21.15.19:0/4220240601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc338103200 0x7fc3381035e0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.309+0000 7fc340795640 1 -- 172.21.15.19:0/4220240601 >> 172.21.15.19:0/4220240601 conn(0x7fc3380771d0 msgr2=0x7fc3380775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.309+0000 7fc340795640 1 -- 172.21.15.19:0/4220240601 shutdown_connections 2024-04-23T18:41:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.309+0000 7fc340795640 1 -- 172.21.15.19:0/4220240601 wait complete. 2024-04-23T18:41:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.309+0000 7fc340795640 1 Processor -- start 2024-04-23T18:41:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.309+0000 7fc340795640 1 -- start start 2024-04-23T18:41:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc340795640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc338103200 0x7fc338100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc340795640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc338103b20 0x7fc338101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc340795640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc33810c5d0 0x7fc3381018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc340795640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc338112220 con 0x7fc33810c5d0 2024-04-23T18:41:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc340795640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381120a0 con 0x7fc338103200 2024-04-23T18:41:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc340795640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381123a0 con 0x7fc338103b20 2024-04-23T18:41:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33ed32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc33810c5d0 0x7fc3381018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33ed32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc33810c5d0 0x7fc3381018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53430/0 (socket says 172.21.15.19:53430) 2024-04-23T18:41:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33ed32640 1 -- 172.21.15.19:0/2178242247 learned_addr learned my addr 172.21.15.19:0/2178242247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33e531640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc338103200 0x7fc338100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33dd30640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc338103b20 0x7fc338101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33ed32640 1 -- 172.21.15.19:0/2178242247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc338103b20 msgr2=0x7fc338101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33ed32640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc338103b20 0x7fc338101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33ed32640 1 -- 172.21.15.19:0/2178242247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc338103200 msgr2=0x7fc338100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33ed32640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc338103200 0x7fc338100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33ed32640 1 -- 172.21.15.19:0/2178242247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3381a6b80 con 0x7fc33810c5d0 2024-04-23T18:41:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.310+0000 7fc33ed32640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc33810c5d0 0x7fc3381018d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc33000e9d0 tx=0x7fc33000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.311+0000 7fc32f7fe640 1 -- 172.21.15.19:0/2178242247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33000cd80 con 0x7fc33810c5d0 2024-04-23T18:41:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.311+0000 7fc32f7fe640 1 -- 172.21.15.19:0/2178242247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc330005020 con 0x7fc33810c5d0 2024-04-23T18:41:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.311+0000 7fc32f7fe640 1 -- 172.21.15.19:0/2178242247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3300116d0 con 0x7fc33810c5d0 2024-04-23T18:41:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.311+0000 7fc340795640 1 -- 172.21.15.19:0/2178242247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3381a6e70 con 0x7fc33810c5d0 2024-04-23T18:41:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.311+0000 7fc340795640 1 -- 172.21.15.19:0/2178242247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3381a7340 con 0x7fc33810c5d0 2024-04-23T18:41:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.313+0000 7fc32f7fe640 1 -- 172.21.15.19:0/2178242247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3300051d0 con 0x7fc33810c5d0 2024-04-23T18:41:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.313+0000 7fc343583740 1 Processor -- start 2024-04-23T18:41:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc343583740 1 -- start start 2024-04-23T18:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc343583740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103620 0x56289c103a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc343583740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 0x56289c10e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc343583740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 0x56289c110bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc343583740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56289c068970 con 0x56289c103620 2024-04-23T18:41:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc343583740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56289c0054a0 con 0x56289c10e800 2024-04-23T18:41:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc343583740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56289c064930 con 0x56289c103f40 2024-04-23T18:41:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc32f7fe640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc3240a3e70 0x7fc3240a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc33dd30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103620 0x56289c103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc33dd30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103620 0x56289c103a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53446/0 (socket says 172.21.15.19:53446) 2024-04-23T18:41:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc33dd30640 1 -- 172.21.15.19:0/1115369841 learned_addr learned my addr 172.21.15.19:0/1115369841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc33ed32640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc3240a3e70 0x7fc3240a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc32f7fe640 1 -- 172.21.15.19:0/2178242247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8557+0+0 (secure 0 0 0) 0x7fc3300d0820 con 0x7fc33810c5d0 2024-04-23T18:41:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc33e531640 1 --2- 172.21.15.19:0/1115369841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 0x56289c10e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc33dd30640 1 -- 172.21.15.19:0/1115369841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 msgr2=0x56289c10e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc33dd30640 1 --2- 172.21.15.19:0/1115369841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 0x56289c10e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc33dd30640 1 -- 172.21.15.19:0/1115369841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 msgr2=0x56289c110bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc33dd30640 1 --2- 172.21.15.19:0/1115369841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 0x56289c110bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.314+0000 7fc33dd30640 1 -- 172.21.15.19:0/1115369841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56289bd7b210 con 0x56289c103620 2024-04-23T18:41:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.315+0000 7fc33dd30640 1 --2- 172.21.15.19:0/1115369841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103620 0x56289c103a00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc3280058a0 tx=0x7fc3280127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e527f2a5affd2c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.315+0000 7fc33ed32640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc3240a3e70 0x7fc3240a6330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3300ce000 tx=0x7fc330087040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.315+0000 7fc32cff9640 1 -- 172.21.15.19:0/1115369841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32801b470 con 0x56289c103620 2024-04-23T18:41:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.315+0000 7fc32cff9640 1 -- 172.21.15.19:0/1115369841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc328016070 con 0x56289c103620 2024-04-23T18:41:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.315+0000 7fc32cff9640 1 -- 172.21.15.19:0/1115369841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32800f620 con 0x56289c103620 2024-04-23T18:41:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.315+0000 7fc343583740 1 -- 172.21.15.19:0/1115369841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103620 msgr2=0x56289c103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.315+0000 7fc343583740 1 --2- 172.21.15.19:0/1115369841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103620 0x56289c103a00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc3280058a0 tx=0x7fc3280127c0 comp rx=0 tx=0).stop 2024-04-23T18:41:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 -- 172.21.15.19:0/1115369841 shutdown_connections 2024-04-23T18:41:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 --2- 172.21.15.19:0/1115369841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 0x56289c110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 --2- 172.21.15.19:0/1115369841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 0x56289c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 --2- 172.21.15.19:0/1115369841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103620 0x56289c103a00 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 -- 172.21.15.19:0/1115369841 >> 172.21.15.19:0/1115369841 conn(0x56289c0ffb30 msgr2=0x56289c0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 -- 172.21.15.19:0/1115369841 shutdown_connections 2024-04-23T18:41:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 -- 172.21.15.19:0/1115369841 wait complete. 2024-04-23T18:41:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 -- start start 2024-04-23T18:41:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 Processor -- start 2024-04-23T18:41:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c103620 0x56289c10e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 0x56289c10b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c10e800 0x56289c10b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56289c111430 con 0x56289c10e800 2024-04-23T18:41:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56289c0030b0 con 0x56289c103620 2024-04-23T18:41:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56289c1115b0 con 0x56289c103f40 2024-04-23T18:41:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.316+0000 7fc343583740 10 client.0 _ll_register_callbacks cb 0x56289bff0d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33dd30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c10e800 0x56289c10b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33dd30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c10e800 0x56289c10b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53450/0 (socket says 172.21.15.19:53450) 2024-04-23T18:41:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33dd30640 1 -- 172.21.15.19:0/2737998890 learned_addr learned my addr 172.21.15.19:0/2737998890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33dd30640 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c103620 0x56289c10e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:49.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33e531640 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 0x56289c10b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33dd30640 1 -- 172.21.15.19:0/2737998890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 msgr2=0x56289c10b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33dd30640 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 0x56289c10b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33dd30640 1 -- 172.21.15.19:0/2737998890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c103620 msgr2=0x56289c10e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33dd30640 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c103620 0x56289c10e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33dd30640 1 -- 172.21.15.19:0/2737998890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56289be47620 con 0x56289c10e800 2024-04-23T18:41:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33e531640 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 0x56289c10b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:41:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc33dd30640 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c10e800 0x56289c10b960 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc328005870 tx=0x7fc328010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67ee1bd6e34a5c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc321ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:41:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc321ffb640 1 -- 172.21.15.19:0/2737998890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328024040 con 0x56289c10e800 2024-04-23T18:41:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc321ffb640 1 -- 172.21.15.19:0/2737998890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8557+0+0 (secure 0 0 0) 0x7fc328004050 con 0x56289c10e800 2024-04-23T18:41:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc343583740 1 -- 172.21.15.19:0/2737998890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc328016070 con 0x56289c10e800 2024-04-23T18:41:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.317+0000 7fc321ffb640 1 -- 172.21.15.19:0/2737998890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc32802e050 con 0x56289c10e800 2024-04-23T18:41:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc343583740 10 client.17313 fetch_fsmap learned FSMap version 29 2024-04-23T18:41:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc343583740 1 -- 172.21.15.19:0/2737998890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56289c11d380 con 0x56289c10e800 2024-04-23T18:41:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc321ffb640 1 -- 172.21.15.19:0/2737998890 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc328016070 con 0x56289c10e800 2024-04-23T18:41:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc321ffb640 1 -- 172.21.15.19:0/2737998890 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc3280046c0 con 0x56289c10e800 2024-04-23T18:41:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc343583740 10 client.17313 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:41:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc343583740 10 client.17313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc343583740 20 client.17313 populate_metadata read hostname 'smithi019' 2024-04-23T18:41:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc343583740 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56289c11e730 0x56289c120b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc343583740 4 client.17313 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:41:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc343583740 1 -- 172.21.15.19:0/2737998890 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56289c076390 con 0x56289c11e730 2024-04-23T18:41:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.318+0000 7fc33e531640 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56289c11e730 0x56289c120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.319+0000 7fc33e531640 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56289c11e730 0x56289c120b20 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a94ef34e577ae6d server_cookie=bfb5a9f96080a075 in_seq=0 out_seq=0 2024-04-23T18:41:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.319+0000 7fc321ffb640 10 client.17313 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.319+0000 7fc321ffb640 1 -- 172.21.15.19:0/2737998890 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56289c076390 con 0x56289c11e730 2024-04-23T18:41:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.319+0000 7fc321ffb640 10 client.17313 handle_command_reply: tid=0 2024-04-23T18:41:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.319+0000 7fc343583740 1 client.17313 shutdown 2024-04-23T18:41:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 1 -- 172.21.15.19:0/2737998890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c10e800 msgr2=0x56289c10b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c10e800 0x56289c10b960 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc328005870 tx=0x7fc328010d30 comp rx=0 tx=0).stop 2024-04-23T18:41:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 1 -- 172.21.15.19:0/2737998890 shutdown_connections 2024-04-23T18:41:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56289c11e730 0x56289c120b20 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c10e800 0x56289c10b960 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103f40 0x56289c10b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 1 --2- 172.21.15.19:0/2737998890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c103620 0x56289c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc321ffb640 0 client.17313 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 1 -- 172.21.15.19:0/2737998890 >> 172.21.15.19:0/2737998890 conn(0x56289c0ffb30 msgr2=0x56289c1058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 1 -- 172.21.15.19:0/2737998890 shutdown_connections 2024-04-23T18:41:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 1 -- 172.21.15.19:0/2737998890 wait complete. 2024-04-23T18:41:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.320+0000 7fc343583740 20 client.17313 trim_cache size 0 max 16384 2024-04-23T18:41:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc343583740 1 Processor -- start 2024-04-23T18:41:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc343583740 1 -- start start 2024-04-23T18:41:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc343583740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 0x56289c10ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc343583740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c106140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc343583740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 0x56289c10dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc343583740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381afe70 con 0x56289c103f40 2024-04-23T18:41:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc343583740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381b19b0 con 0x56289c10e800 2024-04-23T18:41:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc343583740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381b0540 con 0x56289c103620 2024-04-23T18:41:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc33e531640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c106140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc33e531640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c106140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53460/0 (socket says 172.21.15.19:53460) 2024-04-23T18:41:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.335+0000 7fc33e531640 1 -- 172.21.15.19:0/1867492216 learned_addr learned my addr 172.21.15.19:0/1867492216 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:53.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc33dd30640 1 --2- 172.21.15.19:0/1867492216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 0x56289c10ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc33e531640 1 -- 172.21.15.19:0/1867492216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 msgr2=0x56289c10ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc33e531640 1 --2- 172.21.15.19:0/1867492216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 0x56289c10ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc33e531640 1 -- 172.21.15.19:0/1867492216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 msgr2=0x56289c10dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc33e531640 1 --2- 172.21.15.19:0/1867492216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 0x56289c10dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc33e531640 1 -- 172.21.15.19:0/1867492216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56289c110bf0 con 0x56289c103f40 2024-04-23T18:41:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc33dd30640 1 --2- 172.21.15.19:0/1867492216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 0x56289c10ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:41:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc33e531640 1 --2- 172.21.15.19:0/1867492216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c106140 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc334031440 tx=0x7fc3340095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433cdca894834dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc323fff640 1 -- 172.21.15.19:0/1867492216 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334049070 con 0x56289c103f40 2024-04-23T18:41:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc323fff640 1 -- 172.21.15.19:0/1867492216 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc334044dc0 con 0x56289c103f40 2024-04-23T18:41:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc323fff640 1 -- 172.21.15.19:0/1867492216 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334035040 con 0x56289c103f40 2024-04-23T18:41:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc343583740 1 -- 172.21.15.19:0/1867492216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 msgr2=0x56289c106140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc343583740 1 --2- 172.21.15.19:0/1867492216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c106140 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc334031440 tx=0x7fc3340095c0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc343583740 1 -- 172.21.15.19:0/1867492216 shutdown_connections 2024-04-23T18:41:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc343583740 1 --2- 172.21.15.19:0/1867492216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 0x56289c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc343583740 1 --2- 172.21.15.19:0/1867492216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c106140 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc343583740 1 --2- 172.21.15.19:0/1867492216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 0x56289c10ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.336+0000 7fc343583740 1 -- 172.21.15.19:0/1867492216 >> 172.21.15.19:0/1867492216 conn(0x56289c0ffb30 msgr2=0x56289c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 1 -- 172.21.15.19:0/1867492216 shutdown_connections 2024-04-23T18:41:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 1 -- 172.21.15.19:0/1867492216 wait complete. 2024-04-23T18:41:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 1 -- start start 2024-04-23T18:41:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 1 Processor -- start 2024-04-23T18:41:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 0x56289c116720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c116c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 0x56289c111c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381b12e0 con 0x56289c103f40 2024-04-23T18:41:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381b0c10 con 0x56289c10e800 2024-04-23T18:41:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381ade70 con 0x56289c103620 2024-04-23T18:41:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc343583740 10 client.0 _ll_register_callbacks cb 0x7fc32800b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc33e531640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c116c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc33e531640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c116c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53472/0 (socket says 172.21.15.19:53472) 2024-04-23T18:41:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc33e531640 1 -- 172.21.15.19:0/1054781681 learned_addr learned my addr 172.21.15.19:0/1054781681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:41:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.337+0000 7fc33dd30640 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 0x56289c116720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc33e531640 1 -- 172.21.15.19:0/1054781681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 msgr2=0x56289c116720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc33e531640 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 0x56289c116720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc33dd30640 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 0x56289c111c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc33e531640 1 -- 172.21.15.19:0/1054781681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 msgr2=0x56289c111c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc33e531640 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 0x56289c111c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc33e531640 1 -- 172.21.15.19:0/1054781681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56289c112440 con 0x56289c103f40 2024-04-23T18:41:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc33e531640 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c116c60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc334032d10 tx=0x7fc334005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734ea40d12afd147 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc321ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:41:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc321ffb640 1 -- 172.21.15.19:0/1054781681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334049040 con 0x56289c103f40 2024-04-23T18:41:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc321ffb640 1 -- 172.21.15.19:0/1054781681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8557+0+0 (secure 0 0 0) 0x7fc334004440 con 0x56289c103f40 2024-04-23T18:41:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc343583740 1 -- 172.21.15.19:0/1054781681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc334044dc0 con 0x56289c103f40 2024-04-23T18:41:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc321ffb640 1 -- 172.21.15.19:0/1054781681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc334035040 con 0x56289c103f40 2024-04-23T18:41:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc343583740 10 client.17319 fetch_fsmap learned FSMap version 29 2024-04-23T18:41:54.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc343583740 1 -- 172.21.15.19:0/1054781681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56289c1845b0 con 0x56289c103f40 2024-04-23T18:41:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.338+0000 7fc321ffb640 1 -- 172.21.15.19:0/1054781681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc334044dc0 con 0x56289c103f40 2024-04-23T18:41:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.339+0000 7fc321ffb640 1 -- 172.21.15.19:0/1054781681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc33404c070 con 0x56289c103f40 2024-04-23T18:41:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.339+0000 7fc343583740 10 client.17319 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:41:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.339+0000 7fc343583740 10 client.17319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.339+0000 7fc343583740 20 client.17319 populate_metadata read hostname 'smithi019' 2024-04-23T18:41:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.339+0000 7fc343583740 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56289c184cc0 0x56289c1870e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.339+0000 7fc343583740 4 client.17319 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:41:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.339+0000 7fc343583740 1 -- 172.21.15.19:0/1054781681 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56289c070200 con 0x56289c184cc0 2024-04-23T18:41:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.339+0000 7fc33dd30640 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56289c184cc0 0x56289c1870e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.339+0000 7fc33dd30640 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56289c184cc0 0x56289c1870e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee63ee1c0bf4875 server_cookie=f44541f38cc87795 in_seq=0 out_seq=0 2024-04-23T18:41:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.339+0000 7fc321ffb640 10 client.17319 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc321ffb640 1 -- 172.21.15.19:0/1054781681 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56289c070200 con 0x56289c184cc0 2024-04-23T18:41:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc321ffb640 10 client.17319 handle_command_reply: tid=0 2024-04-23T18:41:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 client.17319 shutdown 2024-04-23T18:41:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 -- 172.21.15.19:0/1054781681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 msgr2=0x56289c116c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c116c60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc334032d10 tx=0x7fc334005640 comp rx=0 tx=0).stop 2024-04-23T18:41:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 -- 172.21.15.19:0/1054781681 shutdown_connections 2024-04-23T18:41:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56289c184cc0 0x56289c1870e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56289c10e800 0x56289c111c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56289c103f40 0x56289c116c60 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 --2- 172.21.15.19:0/1054781681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56289c103620 0x56289c116720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc321ffb640 0 client.17319 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:41:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 -- 172.21.15.19:0/1054781681 >> 172.21.15.19:0/1054781681 conn(0x56289c0ffb30 msgr2=0x56289c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 -- 172.21.15.19:0/1054781681 shutdown_connections 2024-04-23T18:41:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 1 -- 172.21.15.19:0/1054781681 wait complete. 2024-04-23T18:41:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.354+0000 7fc343583740 20 client.17319 trim_cache size 0 max 16384 2024-04-23T18:41:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:41:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:41:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:41:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:41:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 -- 172.21.15.19:0/2178242247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc3240a3e70 msgr2=0x7fc3240a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc3240a3e70 0x7fc3240a6330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3300ce000 tx=0x7fc330087040 comp rx=0 tx=0).stop 2024-04-23T18:41:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 -- 172.21.15.19:0/2178242247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc33810c5d0 msgr2=0x7fc3381018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc33810c5d0 0x7fc3381018d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc33000e9d0 tx=0x7fc33000ee90 comp rx=0 tx=0).stop 2024-04-23T18:41:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 -- 172.21.15.19:0/2178242247 shutdown_connections 2024-04-23T18:41:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc3240a3e70 0x7fc3240a6330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc33810c5d0 0x7fc3381018d0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc338103b20 0x7fc338101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 --2- 172.21.15.19:0/2178242247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc338103200 0x7fc338100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 -- 172.21.15.19:0/2178242247 >> 172.21.15.19:0/2178242247 conn(0x7fc3380771d0 msgr2=0x7fc3380fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 -- 172.21.15.19:0/2178242247 shutdown_connections 2024-04-23T18:41:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:12.355+0000 7fc340795640 1 -- 172.21.15.19:0/2178242247 wait complete. 2024-04-23T18:41:56.723 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:13 smithi019 ceph-mon[29215]: pgmap v441: 97 pgs: 97 active+clean; 3.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-04-23T18:41:56.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:13 smithi019 ceph-mon[29215]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T18:41:56.788 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:15 smithi019 ceph-mon[29215]: pgmap v443: 97 pgs: 97 active+clean; 3.7 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 22 op/s 2024-04-23T18:41:56.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:17 smithi019 ceph-mon[29215]: pgmap v444: 97 pgs: 97 active+clean; 3.7 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:41:56.855 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:19 smithi019 ceph-mon[29215]: pgmap v445: 97 pgs: 97 active+clean; 3.7 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-04-23T18:41:56.889 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:21 smithi019 ceph-mon[29215]: pgmap v446: 97 pgs: 97 active+clean; 3.9 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:41:56.946 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:47 smithi155 ceph-mon[33468]: pgmap v491: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:41:57.022 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:49 smithi155 ceph-mon[33468]: pgmap v492: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:41:57.055 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:51 smithi155 ceph-mon[33468]: pgmap v493: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:41:57.088 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:53 smithi155 ceph-mon[33468]: pgmap v494: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:41:57.122 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:55 smithi155 ceph-mon[33468]: pgmap v495: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:41:57.155 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:47 smithi044 ceph-mon[32458]: pgmap v491: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:41:57.189 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:49 smithi044 ceph-mon[32458]: pgmap v492: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:41:57.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:51 smithi044 ceph-mon[32458]: pgmap v493: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:41:57.255 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:53 smithi044 ceph-mon[32458]: pgmap v494: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:41:57.316 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:55 smithi044 ceph-mon[32458]: pgmap v495: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:41:57.355 INFO:tasks.workunit.client.0.smithi019.stdout:561/600: open fd = 3 2024-04-23T18:41:57.430 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:57.455 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:57.489 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:57.522 INFO:tasks.workunit.client.0.smithi019.stdout:562/600: open fd = 3 2024-04-23T18:41:57.555 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:57.597 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:57.631 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:57.664 INFO:tasks.workunit.client.0.smithi019.stdout:563/600: open fd = 3 2024-04-23T18:41:57.697 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:57.731 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:57.764 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:57.814 INFO:tasks.workunit.client.0.smithi019.stdout:564/600: open fd = 3 2024-04-23T18:41:57.906 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:57.939 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:57.973 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:58.006 INFO:tasks.workunit.client.0.smithi019.stdout:565/600: open fd = 3 2024-04-23T18:41:58.039 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.073 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.106 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:58.139 INFO:tasks.workunit.client.0.smithi019.stdout:566/600: open fd = 3 2024-04-23T18:41:58.173 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.206 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.240 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:58.316 INFO:tasks.workunit.client.0.smithi019.stdout:567/600: open fd = 3 2024-04-23T18:41:58.340 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.373 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.406 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:58.440 INFO:tasks.workunit.client.0.smithi019.stdout:568/600: open fd = 3 2024-04-23T18:41:58.473 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.506 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.556 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:58.590 INFO:tasks.workunit.client.0.smithi019.stdout:569/600: open fd = 3 2024-04-23T18:41:58.623 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.656 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.690 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:58.727 INFO:tasks.workunit.client.0.smithi019.stdout:570/600: open fd = 3 2024-04-23T18:41:58.757 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.790 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:58.849 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:58.882 INFO:tasks.workunit.client.0.smithi019.stdout:571/600: open fd = 3 2024-04-23T18:41:58.932 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:59.090 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:59.140 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:59.190 INFO:tasks.workunit.client.0.smithi019.stdout:572/600: open fd = 3 2024-04-23T18:41:59.240 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:59.290 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:59.401 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:59.441 INFO:tasks.workunit.client.0.smithi019.stdout:573/600: open fd = 3 2024-04-23T18:41:59.490 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:59.541 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:59.590 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:59.642 INFO:tasks.workunit.client.0.smithi019.stdout:574/600: open fd = 3 2024-04-23T18:41:59.691 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:59.833 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:41:59.890 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:41:59.979 INFO:tasks.workunit.client.0.smithi019.stdout:575/600: open fd = 3 2024-04-23T18:42:00.016 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:00.066 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:00.116 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:00.177 INFO:tasks.workunit.client.0.smithi019.stdout:576/600: open fd = 3 2024-04-23T18:42:00.267 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:00.316 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:00.366 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:00.416 INFO:tasks.workunit.client.0.smithi019.stdout:577/600: open fd = 3 2024-04-23T18:42:00.508 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:00.558 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:00.644 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:00.683 INFO:tasks.workunit.client.0.smithi019.stdout:578/600: open fd = 3 2024-04-23T18:42:00.733 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:00.791 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:00.841 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:00.924 INFO:tasks.workunit.client.0.smithi019.stdout:579/600: open fd = 3 2024-04-23T18:42:00.975 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:01.043 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:01.074 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:01.125 INFO:tasks.workunit.client.0.smithi019.stdout:580/600: open fd = 3 2024-04-23T18:42:01.175 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:01.225 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:01.275 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:01.361 INFO:tasks.workunit.client.0.smithi019.stdout:581/600: open fd = 3 2024-04-23T18:42:01.400 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:01.525 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:01.575 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:01.625 INFO:tasks.workunit.client.0.smithi019.stdout:582/600: open fd = 3 2024-04-23T18:42:01.675 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:01.725 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:01.775 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:01.825 INFO:tasks.workunit.client.0.smithi019.stdout:583/600: open fd = 3 2024-04-23T18:42:01.898 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:01.950 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:02.009 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:02.072 INFO:tasks.workunit.client.0.smithi019.stdout:584/600: open fd = 3 2024-04-23T18:42:02.150 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:02.235 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:02.284 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:02.334 INFO:tasks.workunit.client.0.smithi019.stdout:585/600: open fd = 3 2024-04-23T18:42:02.384 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:02.434 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:02.484 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:02.534 INFO:tasks.workunit.client.0.smithi019.stdout:586/600: open fd = 3 2024-04-23T18:42:02.584 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:02.634 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:02.684 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:02.784 INFO:tasks.workunit.client.0.smithi019.stdout:587/600: open fd = 3 2024-04-23T18:42:02.860 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:02.929 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:02.984 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:03.051 INFO:tasks.workunit.client.0.smithi019.stdout:588/600: open fd = 3 2024-04-23T18:42:03.093 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:03.134 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:03.223 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:03.259 INFO:tasks.workunit.client.0.smithi019.stdout:589/600: open fd = 3 2024-04-23T18:42:03.293 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:03.368 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:03.401 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:03.459 INFO:tasks.workunit.client.0.smithi019.stdout:590/600: open fd = 3 2024-04-23T18:42:03.485 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:03.518 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:03.551 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:03.585 INFO:tasks.workunit.client.0.smithi019.stdout:591/600: open fd = 3 2024-04-23T18:42:03.626 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:03.660 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:03.703 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:03.727 INFO:tasks.workunit.client.0.smithi019.stdout:592/600: open fd = 3 2024-04-23T18:42:03.819 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:03.888 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:03.919 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:03.962 INFO:tasks.workunit.client.0.smithi019.stdout:593/600: open fd = 3 2024-04-23T18:42:03.985 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.019 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.060 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:04.094 INFO:tasks.workunit.client.0.smithi019.stdout:594/600: open fd = 3 2024-04-23T18:42:04.127 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.161 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.207 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:04.329 INFO:tasks.workunit.client.0.smithi019.stdout:595/600: open fd = 3 2024-04-23T18:42:04.369 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.419 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.452 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:04.508 INFO:tasks.workunit.client.0.smithi019.stdout:596/600: open fd = 3 2024-04-23T18:42:04.536 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.569 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.603 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:04.636 INFO:tasks.workunit.client.0.smithi019.stdout:597/600: open fd = 3 2024-04-23T18:42:04.669 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.703 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.788 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:04.820 INFO:tasks.workunit.client.0.smithi019.stdout:598/600: open fd = 3 2024-04-23T18:42:04.877 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.911 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:04.948 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:04.978 INFO:tasks.workunit.client.0.smithi019.stdout:599/600: open fd = 3 2024-04-23T18:42:05.011 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:05.120 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:05.153 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:05.186 INFO:tasks.workunit.client.0.smithi019.stdout:600/600: open fd = 3 2024-04-23T18:42:05.254 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:05.287 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-04-23T18:42:05.370 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-04-23T18:42:05.425 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo OK 2024-04-23T18:42:05.463 INFO:tasks.workunit.client.0.smithi019.stdout:OK 2024-04-23T18:42:05.521 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T18:42:05.570 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-23T18:42:05.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:23 smithi019 ceph-mon[29215]: pgmap v447: 97 pgs: 97 active+clean; 3.9 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T18:42:05.671 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:57 smithi155 ceph-mon[33468]: pgmap v496: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:05.720 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:41:59 smithi155 ceph-mon[33468]: pgmap v497: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:05.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:01 smithi155 ceph-mon[33468]: pgmap v498: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-23T18:42:05.845 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:03 smithi155 ceph-mon[33468]: pgmap v499: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:05.896 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:57 smithi044 ceph-mon[32458]: pgmap v496: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:05.995 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:41:59 smithi044 ceph-mon[32458]: pgmap v497: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:06.055 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:01 smithi044 ceph-mon[32458]: pgmap v498: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-23T18:42:06.095 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:03 smithi044 ceph-mon[32458]: pgmap v499: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.455+0000 7f9a53a9a640 1 -- 172.21.15.19:0/3780385173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4c106170 msgr2=0x7f9a4c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.455+0000 7f9a53a9a640 1 --2- 172.21.15.19:0/3780385173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4c106170 0x7f9a4c10dd40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009900 tx=0x7f9a4402a410 comp rx=0 tx=0).stop 2024-04-23T18:42:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.455+0000 7f9a53a9a640 1 -- 172.21.15.19:0/3780385173 shutdown_connections 2024-04-23T18:42:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.455+0000 7f9a53a9a640 1 --2- 172.21.15.19:0/3780385173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4c106170 0x7f9a4c10dd40 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.455+0000 7f9a53a9a640 1 --2- 172.21.15.19:0/3780385173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c101aa0 0x7f9a4c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.455+0000 7f9a53a9a640 1 --2- 172.21.15.19:0/3780385173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a4c1010f0 0x7f9a4c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.455+0000 7f9a53a9a640 1 -- 172.21.15.19:0/3780385173 >> 172.21.15.19:0/3780385173 conn(0x7f9a4c077fd0 msgr2=0x7f9a4c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.455+0000 7f9a53a9a640 1 -- 172.21.15.19:0/3780385173 shutdown_connections 2024-04-23T18:42:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.455+0000 7f9a53a9a640 1 -- 172.21.15.19:0/3780385173 wait complete. 2024-04-23T18:42:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a53a9a640 1 Processor -- start 2024-04-23T18:42:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a53a9a640 1 -- start start 2024-04-23T18:42:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a53a9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4c1010f0 0x7f9a4c1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a53a9a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c106170 0x7f9a4c1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a53a9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a4c19a6f0 0x7f9a4c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a53a9a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1110e0 con 0x7f9a4c1010f0 2024-04-23T18:42:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a53a9a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c110f60 con 0x7f9a4c19a6f0 2024-04-23T18:42:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a53a9a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c111260 con 0x7f9a4c106170 2024-04-23T18:42:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a51836640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4c1010f0 0x7f9a4c1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a51836640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4c1010f0 0x7f9a4c1a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38356/0 (socket says 172.21.15.19:38356) 2024-04-23T18:42:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a51836640 1 -- 172.21.15.19:0/712774725 learned_addr learned my addr 172.21.15.19:0/712774725 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a52037640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a4c19a6f0 0x7f9a4c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a51035640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c106170 0x7f9a4c1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.456+0000 7f9a51836640 1 -- 172.21.15.19:0/712774725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c106170 msgr2=0x7f9a4c1a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a51836640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c106170 0x7f9a4c1a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a51836640 1 -- 172.21.15.19:0/712774725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a4c19a6f0 msgr2=0x7f9a4c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a51836640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a4c19a6f0 0x7f9a4c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a51836640 1 -- 172.21.15.19:0/712774725 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c19b2d0 con 0x7f9a4c1010f0 2024-04-23T18:42:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a51035640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c106170 0x7f9a4c1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:42:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a51836640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4c1010f0 0x7f9a4c1a05a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800cce0 tx=0x7f9a4800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a42ffd640 1 -- 172.21.15.19:0/712774725 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4800f650 con 0x7f9a4c1010f0 2024-04-23T18:42:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a53a9a640 1 -- 172.21.15.19:0/712774725 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c1a73b0 con 0x7f9a4c1010f0 2024-04-23T18:42:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a42ffd640 1 -- 172.21.15.19:0/712774725 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a4800fca0 con 0x7f9a4c1010f0 2024-04-23T18:42:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a42ffd640 1 -- 172.21.15.19:0/712774725 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48018b40 con 0x7f9a4c1010f0 2024-04-23T18:42:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.457+0000 7f9a53a9a640 1 -- 172.21.15.19:0/712774725 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c1a77b0 con 0x7f9a4c1010f0 2024-04-23T18:42:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.459+0000 7f9a42ffd640 1 -- 172.21.15.19:0/712774725 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a4800f7f0 con 0x7f9a4c1010f0 2024-04-23T18:42:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a567b4740 1 Processor -- start 2024-04-23T18:42:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a567b4740 1 -- start start 2024-04-23T18:42:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad62a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad62f40 0x55f64ad6d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a567b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f64acc7970 con 0x55f64ad6d800 2024-04-23T18:42:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a567b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f64ac644a0 con 0x55f64ad62620 2024-04-23T18:42:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a567b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f64acc3930 con 0x55f64ad62f40 2024-04-23T18:42:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a42ffd640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a380a3dc0 0x7f9a380a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a51035640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.460+0000 7f9a51035640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38358/0 (socket says 172.21.15.19:38358) 2024-04-23T18:42:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a51035640 1 -- 172.21.15.19:0/838462273 learned_addr learned my addr 172.21.15.19:0/838462273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a51836640 1 --2- 172.21.15.19:0/838462273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad62f40 0x55f64ad6d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a52037640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a380a3dc0 0x7f9a380a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a52037640 1 --2- 172.21.15.19:0/838462273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad62a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a42ffd640 1 -- 172.21.15.19:0/712774725 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7f9a480d0c80 con 0x7f9a4c1010f0 2024-04-23T18:42:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a51035640 1 -- 172.21.15.19:0/838462273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad62f40 msgr2=0x55f64ad6d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a51035640 1 --2- 172.21.15.19:0/838462273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad62f40 0x55f64ad6d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a51035640 1 -- 172.21.15.19:0/838462273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 msgr2=0x55f64ad62a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a51035640 1 --2- 172.21.15.19:0/838462273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad62a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a51035640 1 -- 172.21.15.19:0/838462273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64a9da210 con 0x55f64ad6d800 2024-04-23T18:42:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a51035640 1 --2- 172.21.15.19:0/838462273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6fbf0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ba60 tx=0x7f9a3c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b00f551ec0a83b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a52037640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a380a3dc0 0x7f9a380a6280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a44006010 tx=0x7f9a44031400 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a37fff640 1 -- 172.21.15.19:0/838462273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c017420 con 0x55f64ad6d800 2024-04-23T18:42:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a37fff640 1 -- 172.21.15.19:0/838462273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3c017a70 con 0x55f64ad6d800 2024-04-23T18:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.461+0000 7f9a37fff640 1 -- 172.21.15.19:0/838462273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00fa10 con 0x55f64ad6d800 2024-04-23T18:42:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.462+0000 7f9a567b4740 1 -- 172.21.15.19:0/838462273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 msgr2=0x55f64ad6fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.462+0000 7f9a567b4740 1 --2- 172.21.15.19:0/838462273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6fbf0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ba60 tx=0x7f9a3c00bf20 comp rx=0 tx=0).stop 2024-04-23T18:42:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.462+0000 7f9a567b4740 1 -- 172.21.15.19:0/838462273 shutdown_connections 2024-04-23T18:42:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.462+0000 7f9a567b4740 1 --2- 172.21.15.19:0/838462273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6fbf0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.462+0000 7f9a567b4740 1 --2- 172.21.15.19:0/838462273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad62f40 0x55f64ad6d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.462+0000 7f9a567b4740 1 --2- 172.21.15.19:0/838462273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad62a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.462+0000 7f9a567b4740 1 -- 172.21.15.19:0/838462273 >> 172.21.15.19:0/838462273 conn(0x55f64ad5eb30 msgr2=0x55f64ad5ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.462+0000 7f9a567b4740 1 -- 172.21.15.19:0/838462273 shutdown_connections 2024-04-23T18:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.462+0000 7f9a567b4740 1 -- 172.21.15.19:0/838462273 wait complete. 2024-04-23T18:42:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a567b4740 1 -- start start 2024-04-23T18:42:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a567b4740 1 Processor -- start 2024-04-23T18:42:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad6d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad62f40 0x55f64ad6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a567b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f64ad70430 con 0x55f64ad6d800 2024-04-23T18:42:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a567b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f64ac620b0 con 0x55f64ad62620 2024-04-23T18:42:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a567b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f64ad705b0 con 0x55f64ad62f40 2024-04-23T18:42:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a567b4740 10 client.0 _ll_register_callbacks cb 0x55f64ac4fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a52037640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a52037640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38374/0 (socket says 172.21.15.19:38374) 2024-04-23T18:42:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a52037640 1 -- 172.21.15.19:0/3590010574 learned_addr learned my addr 172.21.15.19:0/3590010574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.463+0000 7f9a51035640 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad62f40 0x55f64ad6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a51836640 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad6d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a52037640 1 -- 172.21.15.19:0/3590010574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad62f40 msgr2=0x55f64ad6a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a52037640 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad62f40 0x55f64ad6a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a52037640 1 -- 172.21.15.19:0/3590010574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 msgr2=0x55f64ad6d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a52037640 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad6d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a52037640 1 -- 172.21.15.19:0/3590010574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64aaa6620 con 0x55f64ad6d800 2024-04-23T18:42:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a51836640 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad6d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:42:11.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a52037640 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6a900 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f9a44033000 tx=0x7f9a44009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef5957e5e80c9ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a357fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:42:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a357fa640 1 -- 172.21.15.19:0/3590010574 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44035070 con 0x55f64ad6d800 2024-04-23T18:42:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a357fa640 1 -- 172.21.15.19:0/3590010574 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7f9a44004320 con 0x55f64ad6d800 2024-04-23T18:42:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a567b4740 1 -- 172.21.15.19:0/3590010574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a3c017a70 con 0x55f64ad6d800 2024-04-23T18:42:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a357fa640 1 -- 172.21.15.19:0/3590010574 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a44058050 con 0x55f64ad6d800 2024-04-23T18:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a567b4740 10 client.17331 fetch_fsmap learned FSMap version 29 2024-04-23T18:42:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.464+0000 7f9a567b4740 1 -- 172.21.15.19:0/3590010574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f64ad7c2d0 con 0x55f64ad6d800 2024-04-23T18:42:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.465+0000 7f9a357fa640 1 -- 172.21.15.19:0/3590010574 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3c017a70 con 0x55f64ad6d800 2024-04-23T18:42:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.465+0000 7f9a357fa640 1 -- 172.21.15.19:0/3590010574 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9a44004b60 con 0x55f64ad6d800 2024-04-23T18:42:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.465+0000 7f9a567b4740 10 client.17331 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:42:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.465+0000 7f9a567b4740 10 client.17331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.465+0000 7f9a567b4740 20 client.17331 populate_metadata read hostname 'smithi019' 2024-04-23T18:42:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.465+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f64ad7d7e0 0x55f64ad7fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.465+0000 7f9a567b4740 4 client.17331 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:42:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.465+0000 7f9a567b4740 1 -- 172.21.15.19:0/3590010574 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f64acd5390 con 0x55f64ad7d7e0 2024-04-23T18:42:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.465+0000 7f9a51836640 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f64ad7d7e0 0x55f64ad7fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.466+0000 7f9a51836640 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f64ad7d7e0 0x55f64ad7fbf0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e4830d9b1df1453 server_cookie=54212b32cab2f2a3 in_seq=0 out_seq=0 2024-04-23T18:42:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.466+0000 7f9a357fa640 10 client.17331 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.466+0000 7f9a357fa640 1 -- 172.21.15.19:0/3590010574 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f64acd5390 con 0x55f64ad7d7e0 2024-04-23T18:42:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.466+0000 7f9a357fa640 10 client.17331 handle_command_reply: tid=0 2024-04-23T18:42:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.466+0000 7f9a567b4740 1 client.17331 shutdown 2024-04-23T18:42:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.466+0000 7f9a567b4740 1 -- 172.21.15.19:0/3590010574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 msgr2=0x55f64ad6a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.466+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6a900 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f9a44033000 tx=0x7f9a44009260 comp rx=0 tx=0).stop 2024-04-23T18:42:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.467+0000 7f9a567b4740 1 -- 172.21.15.19:0/3590010574 shutdown_connections 2024-04-23T18:42:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.467+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f64ad7d7e0 0x55f64ad7fbf0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.467+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad6d800 0x55f64ad6a900 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.467+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad62f40 0x55f64ad6a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.467+0000 7f9a357fa640 0 client.17331 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.467+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3590010574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad6d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.467+0000 7f9a567b4740 1 -- 172.21.15.19:0/3590010574 >> 172.21.15.19:0/3590010574 conn(0x55f64ad5eb30 msgr2=0x55f64ad64860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.467+0000 7f9a567b4740 1 -- 172.21.15.19:0/3590010574 shutdown_connections 2024-04-23T18:42:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.467+0000 7f9a567b4740 1 -- 172.21.15.19:0/3590010574 wait complete. 2024-04-23T18:42:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.467+0000 7f9a567b4740 20 client.17331 trim_cache size 0 max 16384 2024-04-23T18:42:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a567b4740 1 Processor -- start 2024-04-23T18:42:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a567b4740 1 -- start start 2024-04-23T18:42:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad5eb30 0x55f64ad792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad85500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad85a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a567b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1afbe0 con 0x55f64ad62f40 2024-04-23T18:42:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a567b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1b1720 con 0x55f64ad62620 2024-04-23T18:42:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a567b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1b02b0 con 0x55f64ad5eb30 2024-04-23T18:42:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a52037640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad85a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a52037640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad85a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38384/0 (socket says 172.21.15.19:38384) 2024-04-23T18:42:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a52037640 1 -- 172.21.15.19:0/871886098 learned_addr learned my addr 172.21.15.19:0/871886098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a51836640 1 --2- 172.21.15.19:0/871886098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad5eb30 0x55f64ad792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a51035640 1 --2- 172.21.15.19:0/871886098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad85500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a52037640 1 -- 172.21.15.19:0/871886098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad5eb30 msgr2=0x55f64ad792f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a52037640 1 --2- 172.21.15.19:0/871886098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad5eb30 0x55f64ad792f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a52037640 1 -- 172.21.15.19:0/871886098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 msgr2=0x55f64ad85500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a52037640 1 --2- 172.21.15.19:0/871886098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad85500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a52037640 1 -- 172.21.15.19:0/871886098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64ad5f700 con 0x55f64ad62f40 2024-04-23T18:42:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a51035640 1 --2- 172.21.15.19:0/871886098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad85500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:42:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.482+0000 7f9a52037640 1 --2- 172.21.15.19:0/871886098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad85a40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9a44004f80 tx=0x7f9a44005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1534ab8c18b203 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a377fe640 1 -- 172.21.15.19:0/871886098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44061070 con 0x55f64ad62f40 2024-04-23T18:42:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a377fe640 1 -- 172.21.15.19:0/871886098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a44004c50 con 0x55f64ad62f40 2024-04-23T18:42:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a377fe640 1 -- 172.21.15.19:0/871886098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44003df0 con 0x55f64ad62f40 2024-04-23T18:42:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a567b4740 1 -- 172.21.15.19:0/871886098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 msgr2=0x55f64ad85a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a567b4740 1 --2- 172.21.15.19:0/871886098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad85a40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9a44004f80 tx=0x7f9a44005940 comp rx=0 tx=0).stop 2024-04-23T18:42:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a567b4740 1 -- 172.21.15.19:0/871886098 shutdown_connections 2024-04-23T18:42:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a567b4740 1 --2- 172.21.15.19:0/871886098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad85500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a567b4740 1 --2- 172.21.15.19:0/871886098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad85a40 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a567b4740 1 --2- 172.21.15.19:0/871886098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad5eb30 0x55f64ad792f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a567b4740 1 -- 172.21.15.19:0/871886098 >> 172.21.15.19:0/871886098 conn(0x55f64ad6aa60 msgr2=0x55f64ad5d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:14.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a567b4740 1 -- 172.21.15.19:0/871886098 shutdown_connections 2024-04-23T18:42:14.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.483+0000 7f9a567b4740 1 -- 172.21.15.19:0/871886098 wait complete. 2024-04-23T18:42:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a567b4740 1 -- start start 2024-04-23T18:42:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a567b4740 1 Processor -- start 2024-04-23T18:42:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad5eb30 0x55f64ad75980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad72d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a567b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad73250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a567b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1b1050 con 0x55f64ad62f40 2024-04-23T18:42:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a567b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1b0980 con 0x55f64ad62620 2024-04-23T18:42:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a567b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1adbe0 con 0x55f64ad5eb30 2024-04-23T18:42:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a567b4740 10 client.0 _ll_register_callbacks cb 0x7f9a440319c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a52037640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad73250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a52037640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad73250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38386/0 (socket says 172.21.15.19:38386) 2024-04-23T18:42:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a52037640 1 -- 172.21.15.19:0/3093680181 learned_addr learned my addr 172.21.15.19:0/3093680181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a51836640 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad5eb30 0x55f64ad75980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.484+0000 7f9a51035640 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad72d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a52037640 1 -- 172.21.15.19:0/3093680181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad5eb30 msgr2=0x55f64ad75980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a52037640 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad5eb30 0x55f64ad75980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a52037640 1 -- 172.21.15.19:0/3093680181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 msgr2=0x55f64ad72d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a52037640 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad72d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a52037640 1 -- 172.21.15.19:0/3093680181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64ad73810 con 0x55f64ad62f40 2024-04-23T18:42:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a52037640 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad73250 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009850 tx=0x7f9a440058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fd23480e3c60d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a357fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:42:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a357fa640 1 -- 172.21.15.19:0/3093680181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44061070 con 0x55f64ad62f40 2024-04-23T18:42:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a357fa640 1 -- 172.21.15.19:0/3093680181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7f9a440049b0 con 0x55f64ad62f40 2024-04-23T18:42:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a567b4740 1 -- 172.21.15.19:0/3093680181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a44004c50 con 0x55f64ad62f40 2024-04-23T18:42:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a357fa640 1 -- 172.21.15.19:0/3093680181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a44035070 con 0x55f64ad62f40 2024-04-23T18:42:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a567b4740 10 client.17337 fetch_fsmap learned FSMap version 29 2024-04-23T18:42:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a567b4740 1 -- 172.21.15.19:0/3093680181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f64ade3ab0 con 0x55f64ad62f40 2024-04-23T18:42:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.485+0000 7f9a357fa640 1 -- 172.21.15.19:0/3093680181 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a44004c50 con 0x55f64ad62f40 2024-04-23T18:42:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.486+0000 7f9a357fa640 1 -- 172.21.15.19:0/3093680181 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9a44064030 con 0x55f64ad62f40 2024-04-23T18:42:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.486+0000 7f9a567b4740 10 client.17337 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:42:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.486+0000 7f9a567b4740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.486+0000 7f9a567b4740 20 client.17337 populate_metadata read hostname 'smithi019' 2024-04-23T18:42:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.486+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f64ad6d800 0x55f64ade60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.486+0000 7f9a567b4740 4 client.17337 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:42:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.486+0000 7f9a567b4740 1 -- 172.21.15.19:0/3093680181 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f64accf200 con 0x55f64ad6d800 2024-04-23T18:42:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.486+0000 7f9a51836640 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f64ad6d800 0x55f64ade60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.486+0000 7f9a51836640 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f64ad6d800 0x55f64ade60c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4849cde8afb53 server_cookie=db99d55532a5dcf6 in_seq=0 out_seq=0 2024-04-23T18:42:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.486+0000 7f9a357fa640 10 client.17337 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.501+0000 7f9a357fa640 1 -- 172.21.15.19:0/3093680181 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f64accf200 con 0x55f64ad6d800 2024-04-23T18:42:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.501+0000 7f9a357fa640 10 client.17337 handle_command_reply: tid=0 2024-04-23T18:42:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.501+0000 7f9a567b4740 1 client.17337 shutdown 2024-04-23T18:42:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 1 -- 172.21.15.19:0/3093680181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 msgr2=0x55f64ad73250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad73250 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009850 tx=0x7f9a440058d0 comp rx=0 tx=0).stop 2024-04-23T18:42:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 1 -- 172.21.15.19:0/3093680181 shutdown_connections 2024-04-23T18:42:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55f64ad6d800 0x55f64ade60c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f64ad62620 0x55f64ad72d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ad62f40 0x55f64ad73250 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a357fa640 0 client.17337 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 1 --2- 172.21.15.19:0/3093680181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f64ad5eb30 0x55f64ad75980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 1 -- 172.21.15.19:0/3093680181 >> 172.21.15.19:0/3093680181 conn(0x55f64ad6aa60 msgr2=0x55f64ad5dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 1 -- 172.21.15.19:0/3093680181 shutdown_connections 2024-04-23T18:42:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 1 -- 172.21.15.19:0/3093680181 wait complete. 2024-04-23T18:42:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a567b4740 20 client.17337 trim_cache size 0 max 16384 2024-04-23T18:42:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:42:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:42:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:42:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:42:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a53a9a640 1 -- 172.21.15.19:0/712774725 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a380a3dc0 msgr2=0x7f9a380a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a53a9a640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a380a3dc0 0x7f9a380a6280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a44006010 tx=0x7f9a44031400 comp rx=0 tx=0).stop 2024-04-23T18:42:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a53a9a640 1 -- 172.21.15.19:0/712774725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4c1010f0 msgr2=0x7f9a4c1a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.502+0000 7f9a53a9a640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4c1010f0 0x7f9a4c1a05a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800cce0 tx=0x7f9a4800a620 comp rx=0 tx=0).stop 2024-04-23T18:42:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.503+0000 7f9a53a9a640 1 -- 172.21.15.19:0/712774725 shutdown_connections 2024-04-23T18:42:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.503+0000 7f9a53a9a640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f9a380a3dc0 0x7f9a380a6280 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.503+0000 7f9a53a9a640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a4c19a6f0 0x7f9a4c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.503+0000 7f9a53a9a640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4c106170 0x7f9a4c1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.503+0000 7f9a53a9a640 1 --2- 172.21.15.19:0/712774725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4c1010f0 0x7f9a4c1a05a0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.503+0000 7f9a53a9a640 1 -- 172.21.15.19:0/712774725 >> 172.21.15.19:0/712774725 conn(0x7f9a4c077fd0 msgr2=0x7f9a4c069a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.503+0000 7f9a53a9a640 1 -- 172.21.15.19:0/712774725 shutdown_connections 2024-04-23T18:42:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:25.503+0000 7f9a53a9a640 1 -- 172.21.15.19:0/712774725 wait complete. 2024-04-23T18:42:18.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:25 smithi019 ceph-mon[29215]: pgmap v448: 97 pgs: 97 active+clean; 4.0 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-23T18:42:18.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:27 smithi019 ceph-mon[29215]: pgmap v449: 97 pgs: 97 active+clean; 4.0 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-23T18:42:18.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:27 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:42:18.890 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:27 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:42:18.921 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:27 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:42:19.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:27 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:42:19.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:29 smithi019 ceph-mon[29215]: pgmap v450: 97 pgs: 97 active+clean; 4.0 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-23T18:42:19.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:31 smithi019 ceph-mon[29215]: pgmap v451: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-23T18:42:19.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:33 smithi019 ceph-mon[29215]: pgmap v452: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-23T18:42:19.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:35 smithi019 ceph-mon[29215]: pgmap v453: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-23T18:42:19.197 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:37 smithi019 ceph-mon[29215]: pgmap v454: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-23T18:42:19.232 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:05 smithi155 ceph-mon[33468]: pgmap v500: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-23T18:42:19.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:06 smithi155 ceph-mon[33468]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T18:42:19.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:07 smithi155 ceph-mon[33468]: pgmap v502: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:42:19.359 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:09 smithi155 ceph-mon[33468]: pgmap v503: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:42:19.454 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:11 smithi155 ceph-mon[33468]: pgmap v504: 97 pgs: 97 active+clean; 4.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:42:19.488 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:13 smithi155 ceph-mon[33468]: pgmap v505: 97 pgs: 97 active+clean; 4.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:42:19.521 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:13 smithi155 ceph-mon[33468]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T18:42:19.554 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:15 smithi155 ceph-mon[33468]: pgmap v507: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T18:42:19.588 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:17 smithi155 ceph-mon[33468]: pgmap v508: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-23T18:42:19.677 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:05 smithi044 ceph-mon[32458]: pgmap v500: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-23T18:42:19.705 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:06 smithi044 ceph-mon[32458]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T18:42:19.738 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:07 smithi044 ceph-mon[32458]: pgmap v502: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:42:19.772 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:09 smithi044 ceph-mon[32458]: pgmap v503: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:42:19.805 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:11 smithi044 ceph-mon[32458]: pgmap v504: 97 pgs: 97 active+clean; 4.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:42:19.866 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:13 smithi044 ceph-mon[32458]: pgmap v505: 97 pgs: 97 active+clean; 4.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:42:19.908 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:13 smithi044 ceph-mon[32458]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T18:42:19.939 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:15 smithi044 ceph-mon[32458]: pgmap v507: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T18:42:20.088 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:17 smithi044 ceph-mon[32458]: pgmap v508: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-23T18:42:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.602+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1911028964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1b0101aa0 msgr2=0x7ff1b0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.602+0000 7ff1b57f7640 1 --2- 172.21.15.19:0/1911028964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1b0101aa0 0x7ff1b0105a90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009900 tx=0x7ff1a402a410 comp rx=0 tx=0).stop 2024-04-23T18:42:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.602+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1911028964 shutdown_connections 2024-04-23T18:42:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.602+0000 7ff1b57f7640 1 --2- 172.21.15.19:0/1911028964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b0106170 0x7ff1b010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.602+0000 7ff1b57f7640 1 --2- 172.21.15.19:0/1911028964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1b0101aa0 0x7ff1b0105a90 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.602+0000 7ff1b57f7640 1 --2- 172.21.15.19:0/1911028964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b01010f0 0x7ff1b01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.602+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1911028964 >> 172.21.15.19:0/1911028964 conn(0x7ff1b0077fd0 msgr2=0x7ff1b00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.602+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1911028964 shutdown_connections 2024-04-23T18:42:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.602+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1911028964 wait complete. 2024-04-23T18:42:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.602+0000 7ff1b57f7640 1 Processor -- start 2024-04-23T18:42:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1b57f7640 1 -- start start 2024-04-23T18:42:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1b57f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b01010f0 0x7ff1b0102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1b57f7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b0106170 0x7ff1b0103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1b57f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1b01035b0 0x7ff1b01a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1b57f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b0111090 con 0x7ff1b01035b0 2024-04-23T18:42:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1b57f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b0110f10 con 0x7ff1b01010f0 2024-04-23T18:42:20.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1b57f7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b0111210 con 0x7ff1b0106170 2024-04-23T18:42:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1b01035b0 0x7ff1b01a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1b01035b0 0x7ff1b01a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51290/0 (socket says 172.21.15.19:51290) 2024-04-23T18:42:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1af7fe640 1 -- 172.21.15.19:0/1670792322 learned_addr learned my addr 172.21.15.19:0/1670792322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1aeffd640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b01010f0 0x7ff1b0102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b0106170 0x7ff1b0103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1af7fe640 1 -- 172.21.15.19:0/1670792322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b0106170 msgr2=0x7ff1b0103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1af7fe640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b0106170 0x7ff1b0103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1af7fe640 1 -- 172.21.15.19:0/1670792322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b01010f0 msgr2=0x7ff1b0102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1af7fe640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b01010f0 0x7ff1b0102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1af7fe640 1 -- 172.21.15.19:0/1670792322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b01a6fa0 con 0x7ff1b01035b0 2024-04-23T18:42:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.603+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b0106170 0x7ff1b0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:42:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.604+0000 7ff1af7fe640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1b01035b0 0x7ff1b01a68a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800eaa0 tx=0x7ff1a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.604+0000 7ff197fff640 1 -- 172.21.15.19:0/1670792322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800cdb0 con 0x7ff1b01035b0 2024-04-23T18:42:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.604+0000 7ff197fff640 1 -- 172.21.15.19:0/1670792322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a8005020 con 0x7ff1b01035b0 2024-04-23T18:42:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.604+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1670792322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b01a7290 con 0x7ff1b01035b0 2024-04-23T18:42:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.604+0000 7ff197fff640 1 -- 172.21.15.19:0/1670792322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80116d0 con 0x7ff1b01035b0 2024-04-23T18:42:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.604+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1670792322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b01a7760 con 0x7ff1b01035b0 2024-04-23T18:42:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.606+0000 7ff197fff640 1 -- 172.21.15.19:0/1670792322 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff1a80051d0 con 0x7ff1b01035b0 2024-04-23T18:42:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.606+0000 7ff1b85e9740 1 Processor -- start 2024-04-23T18:42:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1b85e9740 1 -- start start 2024-04-23T18:42:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960620 0x5631f7960a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f796b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f796b800 0x5631f796dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5631f78c5970 con 0x5631f7960f40 2024-04-23T18:42:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5631f78624a0 con 0x5631f7960620 2024-04-23T18:42:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5631f78c1930 con 0x5631f796b800 2024-04-23T18:42:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1aeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f796b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:22.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1aeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f796b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51302/0 (socket says 172.21.15.19:51302) 2024-04-23T18:42:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1aeffd640 1 -- 172.21.15.19:0/3390755880 learned_addr learned my addr 172.21.15.19:0/3390755880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff197fff640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3df0 0x7ff1980a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/3390755880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960620 0x5631f7960a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1af7fe640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3df0 0x7ff1980a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1aeffd640 1 -- 172.21.15.19:0/3390755880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f796b800 msgr2=0x5631f796dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1aeffd640 1 --2- 172.21.15.19:0/3390755880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f796b800 0x5631f796dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1aeffd640 1 -- 172.21.15.19:0/3390755880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960620 msgr2=0x5631f7960a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1aeffd640 1 --2- 172.21.15.19:0/3390755880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960620 0x5631f7960a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1aeffd640 1 -- 172.21.15.19:0/3390755880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631f75d8210 con 0x5631f7960f40 2024-04-23T18:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff197fff640 1 -- 172.21.15.19:0/1670792322 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7ff1a80d09b0 con 0x7ff1b01035b0 2024-04-23T18:42:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.607+0000 7ff1aeffd640 1 --2- 172.21.15.19:0/3390755880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f796b2c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000ac20 tx=0x7ff1a000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf69a84705aa5ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1957fa640 1 -- 172.21.15.19:0/3390755880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a001d070 con 0x5631f7960f40 2024-04-23T18:42:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1af7fe640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3df0 0x7ff1980a62b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff1a809c880 tx=0x7ff1a80bd040 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1957fa640 1 -- 172.21.15.19:0/3390755880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a0018af0 con 0x5631f7960f40 2024-04-23T18:42:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1957fa640 1 -- 172.21.15.19:0/3390755880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0010ab0 con 0x5631f7960f40 2024-04-23T18:42:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1b85e9740 1 -- 172.21.15.19:0/3390755880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 msgr2=0x5631f796b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/3390755880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f796b2c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000ac20 tx=0x7ff1a000f790 comp rx=0 tx=0).stop 2024-04-23T18:42:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1b85e9740 1 -- 172.21.15.19:0/3390755880 shutdown_connections 2024-04-23T18:42:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/3390755880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f796b800 0x5631f796dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/3390755880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f796b2c0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/3390755880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960620 0x5631f7960a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1b85e9740 1 -- 172.21.15.19:0/3390755880 >> 172.21.15.19:0/3390755880 conn(0x5631f795cb30 msgr2=0x5631f795cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1b85e9740 1 -- 172.21.15.19:0/3390755880 shutdown_connections 2024-04-23T18:42:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.608+0000 7ff1b85e9740 1 -- 172.21.15.19:0/3390755880 wait complete. 2024-04-23T18:42:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1b85e9740 1 -- start start 2024-04-23T18:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1b85e9740 1 Processor -- start 2024-04-23T18:42:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960620 0x5631f796b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f7968470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f796b800 0x5631f79689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5631f796e430 con 0x5631f7960f40 2024-04-23T18:42:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5631f78600b0 con 0x5631f796b800 2024-04-23T18:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5631f796e5b0 con 0x5631f7960620 2024-04-23T18:42:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1b85e9740 10 client.0 _ll_register_callbacks cb 0x5631f784dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1aeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f7968470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1aeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f7968470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51314/0 (socket says 172.21.15.19:51314) 2024-04-23T18:42:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1aeffd640 1 -- 172.21.15.19:0/4038224635 learned_addr learned my addr 172.21.15.19:0/4038224635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960620 0x5631f796b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.609+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f796b800 0x5631f79689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff1aeffd640 1 -- 172.21.15.19:0/4038224635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960620 msgr2=0x5631f796b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff1aeffd640 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960620 0x5631f796b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff1aeffd640 1 -- 172.21.15.19:0/4038224635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f796b800 msgr2=0x5631f79689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff1aeffd640 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f796b800 0x5631f79689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff1aeffd640 1 -- 172.21.15.19:0/4038224635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631f76a4620 con 0x5631f7960f40 2024-04-23T18:42:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960620 0x5631f796b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:42:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff1aeffd640 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f7968470 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00105f0 tx=0x7ff1a00186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff6cfaaa99cfb48 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff192ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:42:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff192ff5640 1 -- 172.21.15.19:0/4038224635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a001d050 con 0x5631f7960f40 2024-04-23T18:42:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff192ff5640 1 -- 172.21.15.19:0/4038224635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7ff1a0005450 con 0x5631f7960f40 2024-04-23T18:42:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff1b85e9740 1 -- 172.21.15.19:0/4038224635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a0018af0 con 0x5631f7960f40 2024-04-23T18:42:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff192ff5640 1 -- 172.21.15.19:0/4038224635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff1a0032050 con 0x5631f7960f40 2024-04-23T18:42:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff1b85e9740 10 client.17349 fetch_fsmap learned FSMap version 29 2024-04-23T18:42:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff1b85e9740 1 -- 172.21.15.19:0/4038224635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631f797a380 con 0x5631f7960f40 2024-04-23T18:42:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.610+0000 7ff192ff5640 1 -- 172.21.15.19:0/4038224635 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a0018af0 con 0x5631f7960f40 2024-04-23T18:42:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.611+0000 7ff192ff5640 1 -- 172.21.15.19:0/4038224635 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff1a00046c0 con 0x5631f7960f40 2024-04-23T18:42:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.611+0000 7ff1b85e9740 10 client.17349 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:42:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.611+0000 7ff1b85e9740 10 client.17349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.611+0000 7ff1b85e9740 20 client.17349 populate_metadata read hostname 'smithi019' 2024-04-23T18:42:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.611+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631f797b780 0x5631f797db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.611+0000 7ff1b85e9740 4 client.17349 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:42:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.611+0000 7ff1b85e9740 1 -- 172.21.15.19:0/4038224635 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631f78d3390 con 0x5631f797b780 2024-04-23T18:42:25.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.611+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631f797b780 0x5631f797db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.611+0000 7ff192ff5640 10 client.17349 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.611+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631f797b780 0x5631f797db70 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5591ee01e8fb30f3 server_cookie=b23d766a795f7e4e in_seq=0 out_seq=0 2024-04-23T18:42:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff192ff5640 1 -- 172.21.15.19:0/4038224635 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631f78d3390 con 0x5631f797b780 2024-04-23T18:42:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff192ff5640 10 client.17349 handle_command_reply: tid=0 2024-04-23T18:42:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 client.17349 shutdown 2024-04-23T18:42:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 -- 172.21.15.19:0/4038224635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 msgr2=0x5631f7968470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f7968470 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00105f0 tx=0x7ff1a00186f0 comp rx=0 tx=0).stop 2024-04-23T18:42:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 -- 172.21.15.19:0/4038224635 shutdown_connections 2024-04-23T18:42:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631f797b780 0x5631f797db70 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f796b800 0x5631f79689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff192ff5640 0 client.17349 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960f40 0x5631f7968470 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/4038224635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960620 0x5631f796b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 -- 172.21.15.19:0/4038224635 >> 172.21.15.19:0/4038224635 conn(0x5631f795cb30 msgr2=0x5631f79628b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 -- 172.21.15.19:0/4038224635 shutdown_connections 2024-04-23T18:42:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.612+0000 7ff1b85e9740 1 -- 172.21.15.19:0/4038224635 wait complete. 2024-04-23T18:42:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.613+0000 7ff1b85e9740 20 client.17349 trim_cache size 0 max 16384 2024-04-23T18:42:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1b85e9740 1 Processor -- start 2024-04-23T18:42:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1b85e9740 1 -- start start 2024-04-23T18:42:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960620 0x5631f79772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960f40 0x5631f796acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f796b800 0x5631f7983170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01afde0 con 0x5631f7960620 2024-04-23T18:42:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01b1920 con 0x5631f7960f40 2024-04-23T18:42:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01b04b0 con 0x5631f796b800 2024-04-23T18:42:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1ae7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960620 0x5631f79772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1ae7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960620 0x5631f79772f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51324/0 (socket says 172.21.15.19:51324) 2024-04-23T18:42:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1ae7fc640 1 -- 172.21.15.19:0/243603907 learned_addr learned my addr 172.21.15.19:0/243603907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1ae7fc640 1 -- 172.21.15.19:0/243603907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f796b800 msgr2=0x5631f7983170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/243603907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f796b800 0x5631f7983170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1ae7fc640 1 -- 172.21.15.19:0/243603907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960f40 msgr2=0x5631f796acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/243603907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960f40 0x5631f796acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1ae7fc640 1 -- 172.21.15.19:0/243603907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631f7961af0 con 0x5631f7960620 2024-04-23T18:42:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.628+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/243603907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960620 0x5631f79772f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4000c00 tx=0x7ff1a40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7585dc46f86bc251 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff194ff9640 1 -- 172.21.15.19:0/243603907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4035040 con 0x5631f7960620 2024-04-23T18:42:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff194ff9640 1 -- 172.21.15.19:0/243603907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a404a070 con 0x5631f7960620 2024-04-23T18:42:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff194ff9640 1 -- 172.21.15.19:0/243603907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4003850 con 0x5631f7960620 2024-04-23T18:42:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff1b85e9740 1 -- 172.21.15.19:0/243603907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960620 msgr2=0x5631f79772f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/243603907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960620 0x5631f79772f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4000c00 tx=0x7ff1a40056d0 comp rx=0 tx=0).stop 2024-04-23T18:42:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff1b85e9740 1 -- 172.21.15.19:0/243603907 shutdown_connections 2024-04-23T18:42:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/243603907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960f40 0x5631f796acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/243603907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f796b800 0x5631f7983170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/243603907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f7960620 0x5631f79772f0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff1b85e9740 1 -- 172.21.15.19:0/243603907 >> 172.21.15.19:0/243603907 conn(0x5631f795cb30 msgr2=0x5631f7963af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff1b85e9740 1 -- 172.21.15.19:0/243603907 shutdown_connections 2024-04-23T18:42:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.629+0000 7ff1b85e9740 1 -- 172.21.15.19:0/243603907 wait complete. 2024-04-23T18:42:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1b85e9740 1 -- start start 2024-04-23T18:42:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1b85e9740 1 Processor -- start 2024-04-23T18:42:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960620 0x5631f7973800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960f40 0x5631f7973d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1b85e9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f796b800 0x5631f79646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01b1250 con 0x5631f796b800 2024-04-23T18:42:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01b0b80 con 0x5631f7960620 2024-04-23T18:42:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1b85e9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01adde0 con 0x5631f7960f40 2024-04-23T18:42:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1b85e9740 10 client.0 _ll_register_callbacks cb 0x7ff1a000af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1ae7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f796b800 0x5631f79646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1ae7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f796b800 0x5631f79646e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51336/0 (socket says 172.21.15.19:51336) 2024-04-23T18:42:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1ae7fc640 1 -- 172.21.15.19:0/1892144823 learned_addr learned my addr 172.21.15.19:0/1892144823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960620 0x5631f7973800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1ae7fc640 1 -- 172.21.15.19:0/1892144823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960f40 msgr2=0x5631f7973d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1aeffd640 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960f40 0x5631f7973d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960f40 0x5631f7973d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:28.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1ae7fc640 1 -- 172.21.15.19:0/1892144823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960620 msgr2=0x5631f7973800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960620 0x5631f7973800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.630+0000 7ff1ae7fc640 1 -- 172.21.15.19:0/1892144823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631f7964c50 con 0x5631f796b800 2024-04-23T18:42:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff1ae7fc640 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f796b800 0x5631f79646e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4004450 tx=0x7ff1a4042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9f43bf7bb5f86f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff192ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:42:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff192ff5640 1 -- 172.21.15.19:0/1892144823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4054070 con 0x5631f796b800 2024-04-23T18:42:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff192ff5640 1 -- 172.21.15.19:0/1892144823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7ff1a40319f0 con 0x5631f796b800 2024-04-23T18:42:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff1b85e9740 1 -- 172.21.15.19:0/1892144823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a404a070 con 0x5631f796b800 2024-04-23T18:42:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff192ff5640 1 -- 172.21.15.19:0/1892144823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff1a4060050 con 0x5631f796b800 2024-04-23T18:42:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff1b85e9740 10 client.17355 fetch_fsmap learned FSMap version 29 2024-04-23T18:42:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff1b85e9740 1 -- 172.21.15.19:0/1892144823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631f79e1230 con 0x5631f796b800 2024-04-23T18:42:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff192ff5640 1 -- 172.21.15.19:0/1892144823 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a404a070 con 0x5631f796b800 2024-04-23T18:42:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff192ff5640 1 -- 172.21.15.19:0/1892144823 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff1a4044420 con 0x5631f796b800 2024-04-23T18:42:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff1b85e9740 10 client.17355 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:42:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff1b85e9740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff1b85e9740 20 client.17355 populate_metadata read hostname 'smithi019' 2024-04-23T18:42:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631f79e18d0 0x5631f79e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff1b85e9740 4 client.17355 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:42:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.631+0000 7ff1b85e9740 1 -- 172.21.15.19:0/1892144823 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631f78cd200 con 0x5631f79e18d0 2024-04-23T18:42:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.632+0000 7ff1aeffd640 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631f79e18d0 0x5631f79e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.632+0000 7ff1aeffd640 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631f79e18d0 0x5631f79e3cf0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e393b1bb5ca7cca0 server_cookie=ab185713699ae2d4 in_seq=0 out_seq=0 2024-04-23T18:42:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.632+0000 7ff192ff5640 10 client.17355 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff192ff5640 1 -- 172.21.15.19:0/1892144823 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5631f78cd200 con 0x5631f79e18d0 2024-04-23T18:42:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff192ff5640 10 client.17355 handle_command_reply: tid=0 2024-04-23T18:42:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 client.17355 shutdown 2024-04-23T18:42:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 -- 172.21.15.19:0/1892144823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f796b800 msgr2=0x5631f79646e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f796b800 0x5631f79646e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4004450 tx=0x7ff1a4042040 comp rx=0 tx=0).stop 2024-04-23T18:42:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 -- 172.21.15.19:0/1892144823 shutdown_connections 2024-04-23T18:42:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5631f79e18d0 0x5631f79e3cf0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631f796b800 0x5631f79646e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff192ff5640 0 client.17355 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631f7960f40 0x5631f7973d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 --2- 172.21.15.19:0/1892144823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f7960620 0x5631f7973800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 -- 172.21.15.19:0/1892144823 >> 172.21.15.19:0/1892144823 conn(0x5631f795cb30 msgr2=0x5631f7966c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 -- 172.21.15.19:0/1892144823 shutdown_connections 2024-04-23T18:42:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 1 -- 172.21.15.19:0/1892144823 wait complete. 2024-04-23T18:42:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b85e9740 20 client.17355 trim_cache size 0 max 16384 2024-04-23T18:42:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:42:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:42:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:42:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:42:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1670792322 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3df0 msgr2=0x7ff1980a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3df0 0x7ff1980a62b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff1a809c880 tx=0x7ff1a80bd040 comp rx=0 tx=0).stop 2024-04-23T18:42:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1670792322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1b01035b0 msgr2=0x7ff1b01a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1b01035b0 0x7ff1b01a68a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800eaa0 tx=0x7ff1a800ef60 comp rx=0 tx=0).stop 2024-04-23T18:42:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1670792322 shutdown_connections 2024-04-23T18:42:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff1980a3df0 0x7ff1980a62b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1b01035b0 0x7ff1b01a68a0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b0106170 0x7ff1b0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 --2- 172.21.15.19:0/1670792322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b01010f0 0x7ff1b0102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1670792322 >> 172.21.15.19:0/1670792322 conn(0x7ff1b0077fd0 msgr2=0x7ff1b0069940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1670792322 shutdown_connections 2024-04-23T18:42:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:38.646+0000 7ff1b57f7640 1 -- 172.21.15.19:0/1670792322 wait complete. 2024-04-23T18:42:32.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:19 smithi155 ceph-mon[33468]: pgmap v509: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-23T18:42:32.869 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:21 smithi155 ceph-mon[33468]: pgmap v510: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-23T18:42:32.903 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:23 smithi155 ceph-mon[33468]: pgmap v511: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-23T18:42:32.936 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:25 smithi155 ceph-mon[33468]: pgmap v512: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-23T18:42:32.970 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:27 smithi155 ceph-mon[33468]: pgmap v513: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:33.003 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:28 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:42:33.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:29 smithi155 ceph-mon[33468]: pgmap v514: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:33.070 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:29 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:42:33.142 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:29 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:42:33.170 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:29 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:42:33.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:31 smithi155 ceph-mon[33468]: pgmap v515: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:42:33.345 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:19 smithi044 ceph-mon[32458]: pgmap v509: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-23T18:42:33.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:21 smithi044 ceph-mon[32458]: pgmap v510: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-23T18:42:33.438 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:23 smithi044 ceph-mon[32458]: pgmap v511: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-23T18:42:33.462 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:25 smithi044 ceph-mon[32458]: pgmap v512: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-23T18:42:33.497 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:27 smithi044 ceph-mon[32458]: pgmap v513: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:33.529 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:28 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:42:33.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:29 smithi044 ceph-mon[32458]: pgmap v514: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:33.595 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:29 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:42:33.629 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:29 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:42:33.662 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:29 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:42:33.745 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:31 smithi044 ceph-mon[32458]: pgmap v515: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:42:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 -- 172.21.15.19:0/251969018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa10810c520 msgr2=0x7fa10810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:33.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 --2- 172.21.15.19:0/251969018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa10810c520 0x7fa10810e9c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa100009ca0 tx=0x7fa10002a9e0 comp rx=0 tx=0).stop 2024-04-23T18:42:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 -- 172.21.15.19:0/251969018 shutdown_connections 2024-04-23T18:42:33.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 --2- 172.21.15.19:0/251969018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa10810c520 0x7fa10810e9c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:33.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 --2- 172.21.15.19:0/251969018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa108103b00 0x7fa10810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:34.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 --2- 172.21.15.19:0/251969018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1081031e0 0x7fa1081035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 -- 172.21.15.19:0/251969018 >> 172.21.15.19:0/251969018 conn(0x7fa108077f50 msgr2=0x7fa108078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 -- 172.21.15.19:0/251969018 shutdown_connections 2024-04-23T18:42:34.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 -- 172.21.15.19:0/251969018 wait complete. 2024-04-23T18:42:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 Processor -- start 2024-04-23T18:42:34.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 -- start start 2024-04-23T18:42:34.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1081031e0 0x7fa108100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa108103b00 0x7fa108101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa10810c520 0x7fa1081018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:34.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa108112210 con 0x7fa1081031e0 2024-04-23T18:42:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa108112090 con 0x7fa10810c520 2024-04-23T18:42:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10ebbf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa108112390 con 0x7fa108103b00 2024-04-23T18:42:34.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10c95b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1081031e0 0x7fa108100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:34.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10c95b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1081031e0 0x7fa108100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51350/0 (socket says 172.21.15.19:51350) 2024-04-23T18:42:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.973+0000 7fa10c95b640 1 -- 172.21.15.19:0/3211399012 learned_addr learned my addr 172.21.15.19:0/3211399012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa10d15c640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa10810c520 0x7fa1081018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:34.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa107fff640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa108103b00 0x7fa108101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:34.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa10c95b640 1 -- 172.21.15.19:0/3211399012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa108103b00 msgr2=0x7fa108101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:35.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa10c95b640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa108103b00 0x7fa108101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa10c95b640 1 -- 172.21.15.19:0/3211399012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa10810c520 msgr2=0x7fa1081018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:35.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa10c95b640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa10810c520 0x7fa1081018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa10c95b640 1 -- 172.21.15.19:0/3211399012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1081a6c10 con 0x7fa1081031e0 2024-04-23T18:42:35.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa10d15c640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa10810c520 0x7fa1081018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:42:35.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa10c95b640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1081031e0 0x7fa108100e30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800b4a0 tx=0x7fa0f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa105ffb640 1 -- 172.21.15.19:0/3211399012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8014070 con 0x7fa1081031e0 2024-04-23T18:42:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa105ffb640 1 -- 172.21.15.19:0/3211399012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0f8010ac0 con 0x7fa1081031e0 2024-04-23T18:42:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa10ebbf640 1 -- 172.21.15.19:0/3211399012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1081a6f00 con 0x7fa1081031e0 2024-04-23T18:42:35.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa105ffb640 1 -- 172.21.15.19:0/3211399012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8017c30 con 0x7fa1081031e0 2024-04-23T18:42:35.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.974+0000 7fa10ebbf640 1 -- 172.21.15.19:0/3211399012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081a7410 con 0x7fa1081031e0 2024-04-23T18:42:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa105ffb640 1 -- 172.21.15.19:0/3211399012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0f8017070 con 0x7fa1081031e0 2024-04-23T18:42:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa1119b2740 1 Processor -- start 2024-04-23T18:42:35.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa1119b2740 1 -- start start 2024-04-23T18:42:35.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68bc0a0 0x55e6a68bc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 0x55e6a68c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:36.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68c7090 0x55e6a68c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa1119b2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e6a6821370 con 0x55e6a68c7090 2024-04-23T18:42:36.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa105ffb640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa0f00a3da0 0x7fa0f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa1119b2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e6a67be2e0 con 0x55e6a68bc9c0 2024-04-23T18:42:36.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa1119b2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e6a681d330 con 0x55e6a68bc0a0 2024-04-23T18:42:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa107fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68c7090 0x55e6a68c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa10d15c640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa0f00a3da0 0x7fa0f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:36.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa107fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68c7090 0x55e6a68c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51360/0 (socket says 172.21.15.19:51360) 2024-04-23T18:42:36.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa107fff640 1 -- 172.21.15.19:0/2550277399 learned_addr learned my addr 172.21.15.19:0/2550277399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa10d15c640 1 --2- 172.21.15.19:0/2550277399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68bc0a0 0x55e6a68bc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:36.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.977+0000 7fa105ffb640 1 -- 172.21.15.19:0/3211399012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fa0f80cfe00 con 0x7fa1081031e0 2024-04-23T18:42:36.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa10c95b640 1 --2- 172.21.15.19:0/2550277399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 0x55e6a68c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:36.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa107fff640 1 -- 172.21.15.19:0/2550277399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68bc0a0 msgr2=0x55e6a68bc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:36.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa107fff640 1 --2- 172.21.15.19:0/2550277399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68bc0a0 0x55e6a68bc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa107fff640 1 -- 172.21.15.19:0/2550277399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 msgr2=0x55e6a68c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa107fff640 1 --2- 172.21.15.19:0/2550277399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 0x55e6a68c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa107fff640 1 -- 172.21.15.19:0/2550277399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6a65ffd80 con 0x55e6a68c7090 2024-04-23T18:42:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa107fff640 1 --2- 172.21.15.19:0/2550277399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68c7090 0x55e6a68c9480 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00cbc0 tx=0x7fa0fc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdffcf54d30e443f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:37.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa10d15c640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa0f00a3da0 0x7fa0f00a6260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa100006010 tx=0x7fa10002eb40 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:37.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa0eeffd640 1 -- 172.21.15.19:0/2550277399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc010630 con 0x55e6a68c7090 2024-04-23T18:42:37.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa0eeffd640 1 -- 172.21.15.19:0/2550277399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0fc010c80 con 0x55e6a68c7090 2024-04-23T18:42:37.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.978+0000 7fa0eeffd640 1 -- 172.21.15.19:0/2550277399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc01ab80 con 0x55e6a68c7090 2024-04-23T18:42:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 -- 172.21.15.19:0/2550277399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68c7090 msgr2=0x55e6a68c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:37.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 --2- 172.21.15.19:0/2550277399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68c7090 0x55e6a68c9480 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00cbc0 tx=0x7fa0fc00e720 comp rx=0 tx=0).stop 2024-04-23T18:42:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 -- 172.21.15.19:0/2550277399 shutdown_connections 2024-04-23T18:42:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 --2- 172.21.15.19:0/2550277399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 0x55e6a68c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 --2- 172.21.15.19:0/2550277399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68c7090 0x55e6a68c9480 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:37.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 --2- 172.21.15.19:0/2550277399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68bc0a0 0x55e6a68bc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:37.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 -- 172.21.15.19:0/2550277399 >> 172.21.15.19:0/2550277399 conn(0x55e6a68b83c0 msgr2=0x55e6a68b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:37.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 -- 172.21.15.19:0/2550277399 shutdown_connections 2024-04-23T18:42:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 -- 172.21.15.19:0/2550277399 wait complete. 2024-04-23T18:42:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 -- start start 2024-04-23T18:42:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.979+0000 7fa1119b2740 1 Processor -- start 2024-04-23T18:42:37.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc0a0 0x55e6a68c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:37.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 0x55e6a68c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:37.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:37.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa1119b2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e6a68c9e80 con 0x55e6a68bc0a0 2024-04-23T18:42:38.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa1119b2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e6a67bbfb0 con 0x55e6a68bc9c0 2024-04-23T18:42:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa1119b2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e6a68ca000 con 0x55e6a68c7090 2024-04-23T18:42:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa1119b2740 10 client.0 _ll_register_callbacks cb 0x55e6a67bb9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa10c95b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc0a0 0x55e6a68c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:38.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa10c95b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc0a0 0x55e6a68c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51368/0 (socket says 172.21.15.19:51368) 2024-04-23T18:42:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa10c95b640 1 -- 172.21.15.19:0/3106222929 learned_addr learned my addr 172.21.15.19:0/3106222929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:38.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa10d15c640 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:38.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa107fff640 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 0x55e6a68c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:38.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa10c95b640 1 -- 172.21.15.19:0/3106222929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 msgr2=0x55e6a68c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:38.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa10c95b640 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:38.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa10c95b640 1 -- 172.21.15.19:0/3106222929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 msgr2=0x55e6a68c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa10c95b640 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 0x55e6a68c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:38.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa10c95b640 1 -- 172.21.15.19:0/3106222929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6a6660510 con 0x55e6a68bc0a0 2024-04-23T18:42:38.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa107fff640 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 0x55e6a68c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:42:38.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa10c95b640 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc0a0 0x55e6a68c3fc0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa0f809bbb0 tx=0x7fa0f80a13f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f154aa6295ad71a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa0e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:42:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa0e7fff640 1 -- 172.21.15.19:0/3106222929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f801e040 con 0x55e6a68bc0a0 2024-04-23T18:42:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa0e7fff640 1 -- 172.21.15.19:0/3106222929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fa0f80b52b0 con 0x55e6a68bc0a0 2024-04-23T18:42:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa1119b2740 1 -- 172.21.15.19:0/3106222929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0fc010c80 con 0x55e6a68bc0a0 2024-04-23T18:42:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa0e7fff640 1 -- 172.21.15.19:0/3106222929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0f809beb0 con 0x55e6a68bc0a0 2024-04-23T18:42:38.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa1119b2740 10 client.17367 fetch_fsmap learned FSMap version 29 2024-04-23T18:42:38.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.980+0000 7fa1119b2740 1 -- 172.21.15.19:0/3106222929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e6a64f3670 con 0x55e6a68bc0a0 2024-04-23T18:42:38.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.981+0000 7fa0e7fff640 1 -- 172.21.15.19:0/3106222929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0fc010c80 con 0x55e6a68bc0a0 2024-04-23T18:42:38.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.981+0000 7fa0e7fff640 1 -- 172.21.15.19:0/3106222929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa0f80a1630 con 0x55e6a68bc0a0 2024-04-23T18:42:39.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.981+0000 7fa1119b2740 10 client.17367 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:42:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.981+0000 7fa1119b2740 10 client.17367 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:42:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.981+0000 7fa1119b2740 20 client.17367 populate_metadata read hostname 'smithi019' 2024-04-23T18:42:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.981+0000 7fa1119b2740 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e6a68d7310 0x55e6a68d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.981+0000 7fa1119b2740 4 client.17367 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:42:39.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.981+0000 7fa1119b2740 1 -- 172.21.15.19:0/3106222929 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6a682ed90 con 0x55e6a68d7310 2024-04-23T18:42:39.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.981+0000 7fa107fff640 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e6a68d7310 0x55e6a68d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa107fff640 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e6a68d7310 0x55e6a68d97d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c567ee19424196 server_cookie=f539e20e016c4fb9 in_seq=0 out_seq=0 2024-04-23T18:42:39.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa0e7fff640 10 client.17367 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:39.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa0e7fff640 1 -- 172.21.15.19:0/3106222929 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6a682ed90 con 0x55e6a68d7310 2024-04-23T18:42:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa0e7fff640 10 client.17367 handle_command_reply: tid=0 2024-04-23T18:42:39.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa1119b2740 1 client.17367 shutdown 2024-04-23T18:42:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa1119b2740 1 -- 172.21.15.19:0/3106222929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc0a0 msgr2=0x55e6a68c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa1119b2740 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc0a0 0x55e6a68c3fc0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa0f809bbb0 tx=0x7fa0f80a13f0 comp rx=0 tx=0).stop 2024-04-23T18:42:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa1119b2740 1 -- 172.21.15.19:0/3106222929 shutdown_connections 2024-04-23T18:42:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa1119b2740 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e6a68d7310 0x55e6a68d97d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:39.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa1119b2740 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc9c0 0x55e6a68c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:39.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa1119b2740 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa1119b2740 1 --2- 172.21.15.19:0/3106222929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc0a0 0x55e6a68c3fc0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:39.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa0e7fff640 0 client.17367 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.982+0000 7fa1119b2740 1 -- 172.21.15.19:0/3106222929 >> 172.21.15.19:0/3106222929 conn(0x55e6a68b83c0 msgr2=0x55e6a68be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.983+0000 7fa1119b2740 1 -- 172.21.15.19:0/3106222929 shutdown_connections 2024-04-23T18:42:39.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.983+0000 7fa1119b2740 1 -- 172.21.15.19:0/3106222929 wait complete. 2024-04-23T18:42:40.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:38.983+0000 7fa1119b2740 20 client.17367 trim_cache size 0 max 16384 2024-04-23T18:42:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa1119b2740 1 Processor -- start 2024-04-23T18:42:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa1119b2740 1 -- start start 2024-04-23T18:42:40.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 0x55e6a68d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:40.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68be910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:40.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa1119b2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081afdd0 con 0x55e6a68bc9c0 2024-04-23T18:42:40.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa1119b2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081b1910 con 0x55e6a68bc0a0 2024-04-23T18:42:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa1119b2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081b04a0 con 0x55e6a68c7090 2024-04-23T18:42:40.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa107fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa107fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68be910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51382/0 (socket says 172.21.15.19:51382) 2024-04-23T18:42:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa107fff640 1 -- 172.21.15.19:0/3838428527 learned_addr learned my addr 172.21.15.19:0/3838428527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa10d15c640 1 --2- 172.21.15.19:0/3838428527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:40.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa107fff640 1 -- 172.21.15.19:0/3838428527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 msgr2=0x55e6a68c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:40.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa107fff640 1 --2- 172.21.15.19:0/3838428527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:40.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa107fff640 1 -- 172.21.15.19:0/3838428527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 msgr2=0x55e6a68d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:40.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa10c95b640 1 --2- 172.21.15.19:0/3838428527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 0x55e6a68d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:40.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa107fff640 1 --2- 172.21.15.19:0/3838428527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 0x55e6a68d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa107fff640 1 -- 172.21.15.19:0/3838428527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6a68b8f90 con 0x55e6a68bc9c0 2024-04-23T18:42:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.000+0000 7fa107fff640 1 --2- 172.21.15.19:0/3838428527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68be910 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc000c00 tx=0x7fa0fc01a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5776d2eb9648dae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:41.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa0ee7fc640 1 -- 172.21.15.19:0/3838428527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc004220 con 0x55e6a68bc9c0 2024-04-23T18:42:41.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa0ee7fc640 1 -- 172.21.15.19:0/3838428527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0fc0043c0 con 0x55e6a68bc9c0 2024-04-23T18:42:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa0ee7fc640 1 -- 172.21.15.19:0/3838428527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc01aac0 con 0x55e6a68bc9c0 2024-04-23T18:42:41.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa1119b2740 1 -- 172.21.15.19:0/3838428527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 msgr2=0x55e6a68be910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:41.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa1119b2740 1 --2- 172.21.15.19:0/3838428527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68be910 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc000c00 tx=0x7fa0fc01a760 comp rx=0 tx=0).stop 2024-04-23T18:42:41.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa1119b2740 1 -- 172.21.15.19:0/3838428527 shutdown_connections 2024-04-23T18:42:41.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa1119b2740 1 --2- 172.21.15.19:0/3838428527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:41.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa1119b2740 1 --2- 172.21.15.19:0/3838428527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68be910 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:41.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa1119b2740 1 --2- 172.21.15.19:0/3838428527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 0x55e6a68d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:41.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa1119b2740 1 -- 172.21.15.19:0/3838428527 >> 172.21.15.19:0/3838428527 conn(0x55e6a68b83c0 msgr2=0x55e6a68c3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:41.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa1119b2740 1 -- 172.21.15.19:0/3838428527 shutdown_connections 2024-04-23T18:42:41.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa1119b2740 1 -- 172.21.15.19:0/3838428527 wait complete. 2024-04-23T18:42:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.001+0000 7fa1119b2740 1 -- start start 2024-04-23T18:42:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa1119b2740 1 Processor -- start 2024-04-23T18:42:41.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 0x55e6a68caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:41.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa1119b2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa1119b2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081b1240 con 0x55e6a68bc9c0 2024-04-23T18:42:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa1119b2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081b0b70 con 0x55e6a68bc0a0 2024-04-23T18:42:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa1119b2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081addd0 con 0x55e6a68c7090 2024-04-23T18:42:42.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa1119b2740 10 client.0 _ll_register_callbacks cb 0x7fa0f809b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:42.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa107fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:42.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa107fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51394/0 (socket says 172.21.15.19:51394) 2024-04-23T18:42:42.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa107fff640 1 -- 172.21.15.19:0/1191387974 learned_addr learned my addr 172.21.15.19:0/1191387974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa10d15c640 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa10c95b640 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 0x55e6a68caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:42.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa107fff640 1 -- 172.21.15.19:0/1191387974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 msgr2=0x55e6a68cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa107fff640 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:42.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa107fff640 1 -- 172.21.15.19:0/1191387974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 msgr2=0x55e6a68caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:42.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa107fff640 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 0x55e6a68caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa107fff640 1 -- 172.21.15.19:0/1191387974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6a693cac0 con 0x55e6a68bc9c0 2024-04-23T18:42:42.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa10d15c640 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:42:42.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.002+0000 7fa10c95b640 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 0x55e6a68caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:42:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa107fff640 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68cb440 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc005940 tx=0x7fa0fc01a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e823783f40ccbf76 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa0e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:42:43.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa0e7fff640 1 -- 172.21.15.19:0/1191387974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc004220 con 0x55e6a68bc9c0 2024-04-23T18:42:43.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa0e7fff640 1 -- 172.21.15.19:0/1191387974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fa0fc005050 con 0x55e6a68bc9c0 2024-04-23T18:42:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa1119b2740 1 -- 172.21.15.19:0/1191387974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0fc0043c0 con 0x55e6a68bc9c0 2024-04-23T18:42:43.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa0e7fff640 1 -- 172.21.15.19:0/1191387974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0fc040050 con 0x55e6a68bc9c0 2024-04-23T18:42:43.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa1119b2740 10 client.17373 fetch_fsmap learned FSMap version 29 2024-04-23T18:42:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa1119b2740 1 -- 172.21.15.19:0/1191387974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e6a693dd80 con 0x55e6a68bc9c0 2024-04-23T18:42:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa0e7fff640 1 -- 172.21.15.19:0/1191387974 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0fc0043c0 con 0x55e6a68bc9c0 2024-04-23T18:42:43.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa0e7fff640 1 -- 172.21.15.19:0/1191387974 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa0fc004660 con 0x55e6a68bc9c0 2024-04-23T18:42:43.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa1119b2740 10 client.17373 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:42:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa1119b2740 10 client.17373 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:42:43.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa1119b2740 20 client.17373 populate_metadata read hostname 'smithi019' 2024-04-23T18:42:43.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa1119b2740 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e6a693e400 0x55e6a6940820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:43.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa1119b2740 4 client.17373 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 91.5, "await_for": 42.0}] 2024-04-23T18:42:43.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.003+0000 7fa1119b2740 1 -- 172.21.15.19:0/1191387974 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 91.5, "await_for": 42.0}) -- 0x55e6a6828c00 con 0x55e6a693e400 2024-04-23T18:42:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.004+0000 7fa10c95b640 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e6a693e400 0x55e6a6940820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:43.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.004+0000 7fa0e7fff640 10 client.17373 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.004+0000 7fa10c95b640 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e6a693e400 0x55e6a6940820 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9f41ede2c609a58 server_cookie=673189fb2def05a6 in_seq=0 out_seq=0 2024-04-23T18:42:44.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.019+0000 7fa0e7fff640 1 -- 172.21.15.19:0/1191387974 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55e6a6828c00 con 0x55e6a693e400 2024-04-23T18:42:44.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa0e7fff640 10 client.17373 handle_command_reply: tid=0 2024-04-23T18:42:44.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 client.17373 shutdown 2024-04-23T18:42:44.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 -- 172.21.15.19:0/1191387974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 msgr2=0x55e6a68cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68cb440 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc005940 tx=0x7fa0fc01a6a0 comp rx=0 tx=0).stop 2024-04-23T18:42:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 -- 172.21.15.19:0/1191387974 shutdown_connections 2024-04-23T18:42:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e6a693e400 0x55e6a6940820 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6a68c7090 0x55e6a68cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:44.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6a68bc9c0 0x55e6a68cb440 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:44.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 --2- 172.21.15.19:0/1191387974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6a68bc0a0 0x55e6a68caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:44.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa0e7fff640 0 client.17373 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:44.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 -- 172.21.15.19:0/1191387974 >> 172.21.15.19:0/1191387974 conn(0x55e6a68b83c0 msgr2=0x55e6a68c4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:44.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 -- 172.21.15.19:0/1191387974 shutdown_connections 2024-04-23T18:42:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 1 -- 172.21.15.19:0/1191387974 wait complete. 2024-04-23T18:42:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa1119b2740 20 client.17373 trim_cache size 0 max 16384 2024-04-23T18:42:44.778 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:42:44.827 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 29, 2024-04-23T18:42:44.877 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:42:44.928 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 46, 2024-04-23T18:42:44.977 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:42:45.028 INFO:teuthology.orchestra.run.smithi019.stdout: "46eff5eb": { 2024-04-23T18:42:45.136 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 46, 2024-04-23T18:42:45.186 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-04-23T18:42:45.236 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:42:45.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:42:45.336 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:42:45.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:42:45.491 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:42:45.603 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 91.5, 2024-04-23T18:42:45.653 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:42:45.711 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:42:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:42:45.812 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:42:45.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:42:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:42:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:42:46.078 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:42:46.128 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:42:46.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:42:46.253 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:42:46.303 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:42:46.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa10ebbf640 1 -- 172.21.15.19:0/3211399012 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa0f00a3da0 msgr2=0x7fa0f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa10ebbf640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa0f00a3da0 0x7fa0f00a6260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa100006010 tx=0x7fa10002eb40 comp rx=0 tx=0).stop 2024-04-23T18:42:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa10ebbf640 1 -- 172.21.15.19:0/3211399012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1081031e0 msgr2=0x7fa108100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:46.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.020+0000 7fa10ebbf640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1081031e0 0x7fa108100e30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800b4a0 tx=0x7fa0f800b960 comp rx=0 tx=0).stop 2024-04-23T18:42:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.021+0000 7fa10ebbf640 1 -- 172.21.15.19:0/3211399012 shutdown_connections 2024-04-23T18:42:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.021+0000 7fa10ebbf640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa0f00a3da0 0x7fa0f00a6260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.021+0000 7fa10ebbf640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa10810c520 0x7fa1081018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:46.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.021+0000 7fa10ebbf640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa108103b00 0x7fa108101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:46.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.021+0000 7fa10ebbf640 1 --2- 172.21.15.19:0/3211399012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1081031e0 0x7fa108100e30 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.021+0000 7fa10ebbf640 1 -- 172.21.15.19:0/3211399012 >> 172.21.15.19:0/3211399012 conn(0x7fa108077f50 msgr2=0x7fa1080fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.021+0000 7fa10ebbf640 1 -- 172.21.15.19:0/3211399012 shutdown_connections 2024-04-23T18:42:47.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:40:39.021+0000 7fa10ebbf640 1 -- 172.21.15.19:0/3211399012 wait complete. 2024-04-23T18:42:47.074 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '46eff5eb', quiesce took 331.2 seconds. Will release after: 31.5 2024-04-23T18:42:47.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:39 smithi019 ceph-mon[29215]: pgmap v455: 97 pgs: 97 active+clean; 4.1 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-23T18:42:47.231 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:41 smithi019 ceph-mon[29215]: pgmap v456: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-23T18:42:47.281 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:43 smithi019 ceph-mon[29215]: pgmap v457: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:42:47.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:45 smithi019 ceph-mon[29215]: pgmap v458: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-23T18:42:47.409 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:47 smithi019 ceph-mon[29215]: pgmap v459: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:42:47.456 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:49 smithi019 ceph-mon[29215]: pgmap v460: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:42:47.508 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:33 smithi155 ceph-mon[33468]: pgmap v516: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:47.579 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:35 smithi155 ceph-mon[33468]: pgmap v517: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:42:47.631 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:37 smithi155 ceph-mon[33468]: pgmap v518: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:42:47.681 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:39 smithi155 ceph-mon[33468]: pgmap v519: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:42:47.731 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:41 smithi155 ceph-mon[33468]: pgmap v520: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:42:47.781 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:43 smithi155 ceph-mon[33468]: pgmap v521: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:47.881 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:45 smithi155 ceph-mon[33468]: pgmap v522: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:42:47.931 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:33 smithi044 ceph-mon[32458]: pgmap v516: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:48.058 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:35 smithi044 ceph-mon[32458]: pgmap v517: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:42:48.089 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:37 smithi044 ceph-mon[32458]: pgmap v518: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:42:48.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:39 smithi044 ceph-mon[32458]: pgmap v519: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:42:48.190 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:41 smithi044 ceph-mon[32458]: pgmap v520: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:42:48.277 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:43 smithi044 ceph-mon[32458]: pgmap v521: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:48.315 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:45 smithi044 ceph-mon[32458]: pgmap v522: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:42:48.366 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:47 smithi155 ceph-mon[33468]: pgmap v523: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:48.415 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:47 smithi044 ceph-mon[32458]: pgmap v523: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:42:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.746+0000 7fcbce530640 1 -- 172.21.15.19:0/1314527641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8101a90 msgr2=0x7fcbc8105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.746+0000 7fcbce530640 1 --2- 172.21.15.19:0/1314527641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8101a90 0x7fcbc8105ad0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009900 tx=0x7fcbbc02a410 comp rx=0 tx=0).stop 2024-04-23T18:42:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.746+0000 7fcbce530640 1 -- 172.21.15.19:0/1314527641 shutdown_connections 2024-04-23T18:42:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.746+0000 7fcbce530640 1 --2- 172.21.15.19:0/1314527641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbc81061b0 0x7fcbc810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.746+0000 7fcbce530640 1 --2- 172.21.15.19:0/1314527641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8101a90 0x7fcbc8105ad0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.746+0000 7fcbce530640 1 --2- 172.21.15.19:0/1314527641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc81010e0 0x7fcbc81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.746+0000 7fcbce530640 1 -- 172.21.15.19:0/1314527641 >> 172.21.15.19:0/1314527641 conn(0x7fcbc80771d0 msgr2=0x7fcbc80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.746+0000 7fcbce530640 1 -- 172.21.15.19:0/1314527641 shutdown_connections 2024-04-23T18:42:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.746+0000 7fcbce530640 1 -- 172.21.15.19:0/1314527641 wait complete. 2024-04-23T18:42:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbce530640 1 Processor -- start 2024-04-23T18:42:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbce530640 1 -- start start 2024-04-23T18:42:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbce530640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc81010e0 0x7fcbc81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbce530640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc81061b0 0x7fcbc8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbce530640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbc819cf60 0x7fcbc819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbce530640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8112260 con 0x7fcbc81010e0 2024-04-23T18:42:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbce530640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81120e0 con 0x7fcbc819cf60 2024-04-23T18:42:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbce530640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81123e0 con 0x7fcbc81061b0 2024-04-23T18:42:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc81010e0 0x7fcbc81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc81010e0 0x7fcbc81986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45898/0 (socket says 172.21.15.19:45898) 2024-04-23T18:42:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1940751057 learned_addr learned my addr 172.21.15.19:0/1940751057 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbc77fe640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc81061b0 0x7fcbc8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.747+0000 7fcbccacd640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbc819cf60 0x7fcbc819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1940751057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc81061b0 msgr2=0x7fcbc8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc81061b0 0x7fcbc8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1940751057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbc819cf60 msgr2=0x7fcbc819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:49.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbc819cf60 0x7fcbc819d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1940751057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc819da50 con 0x7fcbc81010e0 2024-04-23T18:42:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc81010e0 0x7fcbc81986a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800d950 tx=0x7fcbb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbc57fa640 1 -- 172.21.15.19:0/1940751057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8017070 con 0x7fcbc81010e0 2024-04-23T18:42:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbc57fa640 1 -- 172.21.15.19:0/1940751057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb8009c30 con 0x7fcbc81010e0 2024-04-23T18:42:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbc57fa640 1 -- 172.21.15.19:0/1940751057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800f620 con 0x7fcbc81010e0 2024-04-23T18:42:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbce530640 1 -- 172.21.15.19:0/1940751057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc819dd40 con 0x7fcbc81010e0 2024-04-23T18:42:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.748+0000 7fcbce530640 1 -- 172.21.15.19:0/1940751057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc81a5550 con 0x7fcbc81010e0 2024-04-23T18:42:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbc57fa640 1 -- 172.21.15.19:0/1940751057 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcbb8010460 con 0x7fcbc81010e0 2024-04-23T18:42:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbd1248740 1 Processor -- start 2024-04-23T18:42:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbd1248740 1 -- start start 2024-04-23T18:42:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbce620 0x5577dfbcea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbcef40 0x5577dfbd92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbd9800 0x5577dfbdbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbd1248740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577dfb33970 con 0x5577dfbcef40 2024-04-23T18:42:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbd1248740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5577dfad04a0 con 0x5577dfbce620 2024-04-23T18:42:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbd1248740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5577dfb2f930 con 0x5577dfbd9800 2024-04-23T18:42:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbcef40 0x5577dfbd92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbcef40 0x5577dfbd92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45902/0 (socket says 172.21.15.19:45902) 2024-04-23T18:42:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1405944403 learned_addr learned my addr 172.21.15.19:0/1405944403 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbc57fa640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcbb00a3e40 0x7fcbb00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbc77fe640 1 --2- 172.21.15.19:0/1405944403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbd9800 0x5577dfbdbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbccacd640 1 --2- 172.21.15.19:0/1405944403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbce620 0x5577dfbcea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.751+0000 7fcbccacd640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcbb00a3e40 0x7fcbb00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbc57fa640 1 -- 172.21.15.19:0/1940751057 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fcbb80d0310 con 0x7fcbc81010e0 2024-04-23T18:42:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1405944403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbd9800 msgr2=0x5577dfbdbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/1405944403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbd9800 0x5577dfbdbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1405944403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbce620 msgr2=0x5577dfbcea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/1405944403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbce620 0x5577dfbcea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1405944403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577df846210 con 0x5577dfbcef40 2024-04-23T18:42:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/1405944403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbcef40 0x5577dfbd92c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80cda00 tx=0x7fcbb8010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880f9aa1d0a5ae4a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbccacd640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcbb00a3e40 0x7fcbb00a6300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009fd0 tx=0x7fcbc0009600 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbae7fc640 1 -- 172.21.15.19:0/1405944403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8020030 con 0x5577dfbcef40 2024-04-23T18:42:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbae7fc640 1 -- 172.21.15.19:0/1405944403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb8009de0 con 0x5577dfbcef40 2024-04-23T18:42:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.752+0000 7fcbae7fc640 1 -- 172.21.15.19:0/1405944403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80989d0 con 0x5577dfbcef40 2024-04-23T18:42:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 -- 172.21.15.19:0/1405944403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbcef40 msgr2=0x5577dfbd92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1405944403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbcef40 0x5577dfbd92c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80cda00 tx=0x7fcbb8010cb0 comp rx=0 tx=0).stop 2024-04-23T18:42:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 -- 172.21.15.19:0/1405944403 shutdown_connections 2024-04-23T18:42:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1405944403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbd9800 0x5577dfbdbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1405944403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbcef40 0x5577dfbd92c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1405944403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbce620 0x5577dfbcea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 -- 172.21.15.19:0/1405944403 >> 172.21.15.19:0/1405944403 conn(0x5577dfbcab30 msgr2=0x5577dfbcaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 -- 172.21.15.19:0/1405944403 shutdown_connections 2024-04-23T18:42:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 -- 172.21.15.19:0/1405944403 wait complete. 2024-04-23T18:42:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 -- start start 2024-04-23T18:42:52.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.753+0000 7fcbd1248740 1 Processor -- start 2024-04-23T18:42:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbce620 0x5577dfbd91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbcef40 0x5577dfbd64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbd9800 0x5577dfbd6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbd1248740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577dfbdc3d0 con 0x5577dfbce620 2024-04-23T18:42:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbd1248740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5577dface0b0 con 0x5577dfbd9800 2024-04-23T18:42:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbd1248740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5577dfbdc550 con 0x5577dfbcef40 2024-04-23T18:42:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbd1248740 10 client.0 _ll_register_callbacks cb 0x5577dfabbd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbce620 0x5577dfbd91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbce620 0x5577dfbd91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45914/0 (socket says 172.21.15.19:45914) 2024-04-23T18:42:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1705637716 learned_addr learned my addr 172.21.15.19:0/1705637716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbccacd640 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbd9800 0x5577dfbd6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbc77fe640 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbcef40 0x5577dfbd64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1705637716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbcef40 msgr2=0x5577dfbd64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbcef40 0x5577dfbd64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1705637716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbd9800 msgr2=0x5577dfbd6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbd9800 0x5577dfbd6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbc7fff640 1 -- 172.21.15.19:0/1705637716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577df912620 con 0x5577dfbce620 2024-04-23T18:42:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbce620 0x5577dfbd91e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80cdb10 tx=0x7fcbb80d39e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe22f21ed82c354 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:42:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.754+0000 7fcba77fe640 1 -- 172.21.15.19:0/1705637716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8020070 con 0x5577dfbce620 2024-04-23T18:42:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcba77fe640 1 -- 172.21.15.19:0/1705637716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fcbb80a0170 con 0x5577dfbce620 2024-04-23T18:42:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcbd1248740 1 -- 172.21.15.19:0/1705637716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb8009de0 con 0x5577dfbce620 2024-04-23T18:42:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcba77fe640 1 -- 172.21.15.19:0/1705637716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcbb809c360 con 0x5577dfbce620 2024-04-23T18:42:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcbd1248740 10 client.17385 fetch_fsmap learned FSMap version 29 2024-04-23T18:42:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcbd1248740 1 -- 172.21.15.19:0/1705637716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577dfbe8380 con 0x5577dfbce620 2024-04-23T18:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcba77fe640 1 -- 172.21.15.19:0/1705637716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb8009de0 con 0x5577dfbce620 2024-04-23T18:42:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcba77fe640 1 -- 172.21.15.19:0/1705637716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcbb8098240 con 0x5577dfbce620 2024-04-23T18:42:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcbd1248740 10 client.17385 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:42:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcbd1248740 10 client.17385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcbd1248740 20 client.17385 populate_metadata read hostname 'smithi019' 2024-04-23T18:42:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5577dfbe96d0 0x5577dfbebac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcbd1248740 4 client.17385 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:42:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcbd1248740 1 -- 172.21.15.19:0/1705637716 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577dfb41390 con 0x5577dfbe96d0 2024-04-23T18:42:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.755+0000 7fcbc77fe640 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5577dfbe96d0 0x5577dfbebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.756+0000 7fcba77fe640 10 client.17385 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.756+0000 7fcbc77fe640 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5577dfbe96d0 0x5577dfbebac0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed37f45c3451483 server_cookie=83e40588cec0c5c9 in_seq=0 out_seq=0 2024-04-23T18:42:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.756+0000 7fcba77fe640 1 -- 172.21.15.19:0/1705637716 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577dfb41390 con 0x5577dfbe96d0 2024-04-23T18:42:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.756+0000 7fcba77fe640 10 client.17385 handle_command_reply: tid=0 2024-04-23T18:42:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 client.17385 shutdown 2024-04-23T18:42:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 -- 172.21.15.19:0/1705637716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbce620 msgr2=0x5577dfbd91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbce620 0x5577dfbd91e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80cdb10 tx=0x7fcbb80d39e0 comp rx=0 tx=0).stop 2024-04-23T18:42:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 -- 172.21.15.19:0/1705637716 shutdown_connections 2024-04-23T18:42:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5577dfbe96d0 0x5577dfbebac0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbd9800 0x5577dfbd6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbcef40 0x5577dfbd64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1705637716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbce620 0x5577dfbd91e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcba77fe640 0 client.17385 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 -- 172.21.15.19:0/1705637716 >> 172.21.15.19:0/1705637716 conn(0x5577dfbcab30 msgr2=0x5577dfbd08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 -- 172.21.15.19:0/1705637716 shutdown_connections 2024-04-23T18:42:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 1 -- 172.21.15.19:0/1705637716 wait complete. 2024-04-23T18:42:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.757+0000 7fcbd1248740 20 client.17385 trim_cache size 0 max 16384 2024-04-23T18:42:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbd1248740 1 Processor -- start 2024-04-23T18:42:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbd1248740 1 -- start start 2024-04-23T18:42:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbce620 0x5577dfbc5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbcef40 0x5577dfbd8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbeeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbd1248740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81adb00 con 0x5577dfbd9800 2024-04-23T18:42:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbd1248740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81af640 con 0x5577dfbcef40 2024-04-23T18:42:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbd1248740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81ae1d0 con 0x5577dfbce620 2024-04-23T18:42:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbccacd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbeeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbccacd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbeeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45924/0 (socket says 172.21.15.19:45924) 2024-04-23T18:42:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbccacd640 1 -- 172.21.15.19:0/1410775058 learned_addr learned my addr 172.21.15.19:0/1410775058 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/1410775058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbce620 0x5577dfbc5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbccacd640 1 -- 172.21.15.19:0/1410775058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbce620 msgr2=0x5577dfbc5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbccacd640 1 --2- 172.21.15.19:0/1410775058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbce620 0x5577dfbc5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbccacd640 1 -- 172.21.15.19:0/1410775058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbcef40 msgr2=0x5577dfbd8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbccacd640 1 --2- 172.21.15.19:0/1410775058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbcef40 0x5577dfbd8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.772+0000 7fcbccacd640 1 -- 172.21.15.19:0/1410775058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577dfbdbf60 con 0x5577dfbd9800 2024-04-23T18:42:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.773+0000 7fcbccacd640 1 --2- 172.21.15.19:0/1410775058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbeeaf0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00129d0 tx=0x7fcbc0009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636a722bb532f14b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.773+0000 7fcbadffb640 1 -- 172.21.15.19:0/1410775058 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc002c070 con 0x5577dfbd9800 2024-04-23T18:42:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.773+0000 7fcbadffb640 1 -- 172.21.15.19:0/1410775058 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc000ae50 con 0x5577dfbd9800 2024-04-23T18:42:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.773+0000 7fcbadffb640 1 -- 172.21.15.19:0/1410775058 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc00279a0 con 0x5577dfbd9800 2024-04-23T18:42:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.773+0000 7fcbd1248740 1 -- 172.21.15.19:0/1410775058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 msgr2=0x5577dfbeeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.773+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1410775058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbeeaf0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00129d0 tx=0x7fcbc0009960 comp rx=0 tx=0).stop 2024-04-23T18:42:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.773+0000 7fcbd1248740 1 -- 172.21.15.19:0/1410775058 shutdown_connections 2024-04-23T18:42:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.773+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1410775058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbcef40 0x5577dfbd8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1410775058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbeeaf0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 --2- 172.21.15.19:0/1410775058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbce620 0x5577dfbc5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 -- 172.21.15.19:0/1410775058 >> 172.21.15.19:0/1410775058 conn(0x5577dfbcab30 msgr2=0x5577dfbe5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 -- 172.21.15.19:0/1410775058 shutdown_connections 2024-04-23T18:42:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 -- 172.21.15.19:0/1410775058 wait complete. 2024-04-23T18:42:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 -- start start 2024-04-23T18:42:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 Processor -- start 2024-04-23T18:42:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbce620 0x5577dfc48290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbcef40 0x5577dfc487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81aef70 con 0x5577dfbd9800 2024-04-23T18:42:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81ae8a0 con 0x5577dfbcef40 2024-04-23T18:42:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81abb00 con 0x5577dfbce620 2024-04-23T18:42:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.774+0000 7fcbd1248740 10 client.0 _ll_register_callbacks cb 0x7fcbb800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbccacd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbccacd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbee420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45928/0 (socket says 172.21.15.19:45928) 2024-04-23T18:42:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbccacd640 1 -- 172.21.15.19:0/2003920395 learned_addr learned my addr 172.21.15.19:0/2003920395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:42:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbce620 0x5577dfc48290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbc77fe640 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbcef40 0x5577dfc487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbccacd640 1 -- 172.21.15.19:0/2003920395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbce620 msgr2=0x5577dfc48290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbccacd640 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbce620 0x5577dfc48290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbccacd640 1 -- 172.21.15.19:0/2003920395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbcef40 msgr2=0x5577dfc487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbccacd640 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbcef40 0x5577dfc487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbccacd640 1 -- 172.21.15.19:0/2003920395 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577dfbeeb10 con 0x5577dfbd9800 2024-04-23T18:42:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbccacd640 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbee420 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009cf0 tx=0x7fcbc0004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3cfbcf147e6343 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:42:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcba77fe640 1 -- 172.21.15.19:0/2003920395 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc002c070 con 0x5577dfbd9800 2024-04-23T18:42:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcba77fe640 1 -- 172.21.15.19:0/2003920395 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fcbc001ec80 con 0x5577dfbd9800 2024-04-23T18:42:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbd1248740 1 -- 172.21.15.19:0/2003920395 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbc000ae50 con 0x5577dfbd9800 2024-04-23T18:42:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcba77fe640 1 -- 172.21.15.19:0/2003920395 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcbc0027740 con 0x5577dfbd9800 2024-04-23T18:42:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbd1248740 10 client.17391 fetch_fsmap learned FSMap version 29 2024-04-23T18:42:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.775+0000 7fcbd1248740 1 -- 172.21.15.19:0/2003920395 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577dfc4faa0 con 0x5577dfbd9800 2024-04-23T18:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcba77fe640 1 -- 172.21.15.19:0/2003920395 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc000ae50 con 0x5577dfbd9800 2024-04-23T18:42:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcba77fe640 1 -- 172.21.15.19:0/2003920395 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcbc0026070 con 0x5577dfbd9800 2024-04-23T18:42:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcbd1248740 10 client.17391 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:42:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcbd1248740 10 client.17391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcbd1248740 20 client.17391 populate_metadata read hostname 'smithi019' 2024-04-23T18:42:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcbd1248740 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5577dfc50190 0x5577dfc525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcbd1248740 4 client.17391 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:42:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcbd1248740 1 -- 172.21.15.19:0/2003920395 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577dfb3b200 con 0x5577dfc50190 2024-04-23T18:42:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5577dfc50190 0x5577dfc525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcbc7fff640 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5577dfc50190 0x5577dfc525b0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd61e05df41e5855 server_cookie=1abf09eefd4e892c in_seq=0 out_seq=0 2024-04-23T18:42:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.776+0000 7fcba77fe640 10 client.17391 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcba77fe640 1 -- 172.21.15.19:0/2003920395 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577dfb3b200 con 0x5577dfc50190 2024-04-23T18:42:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcba77fe640 10 client.17391 handle_command_reply: tid=0 2024-04-23T18:42:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 client.17391 shutdown 2024-04-23T18:42:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 -- 172.21.15.19:0/2003920395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 msgr2=0x5577dfbee420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbee420 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009cf0 tx=0x7fcbc0004c70 comp rx=0 tx=0).stop 2024-04-23T18:42:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 -- 172.21.15.19:0/2003920395 shutdown_connections 2024-04-23T18:42:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5577dfc50190 0x5577dfc525b0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5577dfbcef40 0x5577dfc487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577dfbd9800 0x5577dfbee420 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcba77fe640 0 client.17391 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:42:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 --2- 172.21.15.19:0/2003920395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfbce620 0x5577dfc48290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 -- 172.21.15.19:0/2003920395 >> 172.21.15.19:0/2003920395 conn(0x5577dfbcab30 msgr2=0x5577dfbd7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 -- 172.21.15.19:0/2003920395 shutdown_connections 2024-04-23T18:42:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.788+0000 7fcbd1248740 1 -- 172.21.15.19:0/2003920395 wait complete. 2024-04-23T18:42:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbd1248740 20 client.17391 trim_cache size 0 max 16384 2024-04-23T18:42:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:43:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:43:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:43:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:43:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 -- 172.21.15.19:0/1940751057 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcbb00a3e40 msgr2=0x7fcbb00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcbb00a3e40 0x7fcbb00a6300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009fd0 tx=0x7fcbc0009600 comp rx=0 tx=0).stop 2024-04-23T18:43:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 -- 172.21.15.19:0/1940751057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc81010e0 msgr2=0x7fcbc81986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc81010e0 0x7fcbc81986a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800d950 tx=0x7fcbb800de10 comp rx=0 tx=0).stop 2024-04-23T18:43:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 -- 172.21.15.19:0/1940751057 shutdown_connections 2024-04-23T18:43:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcbb00a3e40 0x7fcbb00a6300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbc819cf60 0x7fcbc819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc81061b0 0x7fcbc8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 --2- 172.21.15.19:0/1940751057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc81010e0 0x7fcbc81986a0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 -- 172.21.15.19:0/1940751057 >> 172.21.15.19:0/1940751057 conn(0x7fcbc80771d0 msgr2=0x7fcbc80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 -- 172.21.15.19:0/1940751057 shutdown_connections 2024-04-23T18:43:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:40:51.789+0000 7fcbce530640 1 -- 172.21.15.19:0/1940751057 wait complete. 2024-04-23T18:43:00.892 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:51 smithi019 ceph-mon[29215]: pgmap v461: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T18:43:00.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:53 smithi019 ceph-mon[29215]: pgmap v462: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-23T18:43:01.030 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:55 smithi019 ceph-mon[29215]: pgmap v463: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-23T18:43:01.107 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:57 smithi019 ceph-mon[29215]: pgmap v464: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-04-23T18:43:01.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:40:59 smithi019 ceph-mon[29215]: pgmap v465: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-04-23T18:43:01.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:01 smithi019 ceph-mon[29215]: pgmap v466: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-23T18:43:01.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:03 smithi019 ceph-mon[29215]: pgmap v467: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-04-23T18:43:01.373 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:49 smithi155 ceph-mon[33468]: pgmap v524: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:01.423 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:51 smithi155 ceph-mon[33468]: pgmap v525: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:01.473 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:53 smithi155 ceph-mon[33468]: pgmap v526: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:01.523 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:55 smithi155 ceph-mon[33468]: pgmap v527: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:01.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:57 smithi155 ceph-mon[33468]: pgmap v528: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:01.623 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:42:59 smithi155 ceph-mon[33468]: pgmap v529: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:01.756 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:49 smithi044 ceph-mon[32458]: pgmap v524: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:01.798 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:51 smithi044 ceph-mon[32458]: pgmap v525: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:01.848 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:53 smithi044 ceph-mon[32458]: pgmap v526: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:01.948 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:55 smithi044 ceph-mon[32458]: pgmap v527: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:02.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:57 smithi044 ceph-mon[32458]: pgmap v528: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:02.073 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:42:59 smithi044 ceph-mon[32458]: pgmap v529: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.888+0000 7f765527c640 1 -- 172.21.15.19:0/1336809995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7650103b20 msgr2=0x7f765010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.888+0000 7f765527c640 1 --2- 172.21.15.19:0/1336809995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7650103b20 0x7f765010c000 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7644009900 tx=0x7f764402a480 comp rx=0 tx=0).stop 2024-04-23T18:43:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.888+0000 7f765527c640 1 -- 172.21.15.19:0/1336809995 shutdown_connections 2024-04-23T18:43:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.889+0000 7f765527c640 1 --2- 172.21.15.19:0/1336809995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765010c5d0 0x7f765010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.889+0000 7f765527c640 1 --2- 172.21.15.19:0/1336809995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7650103b20 0x7f765010c000 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.889+0000 7f765527c640 1 --2- 172.21.15.19:0/1336809995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7650103200 0x7f76501035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.889+0000 7f765527c640 1 -- 172.21.15.19:0/1336809995 >> 172.21.15.19:0/1336809995 conn(0x7f76500771d0 msgr2=0x7f76500775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.889+0000 7f765527c640 1 -- 172.21.15.19:0/1336809995 shutdown_connections 2024-04-23T18:43:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.889+0000 7f765527c640 1 -- 172.21.15.19:0/1336809995 wait complete. 2024-04-23T18:43:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.889+0000 7f765527c640 1 Processor -- start 2024-04-23T18:43:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.889+0000 7f765527c640 1 -- start start 2024-04-23T18:43:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f765527c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7650103200 0x7f76501986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f765527c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7650103b20 0x7f7650198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f765527c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765010c5d0 0x7f765019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f765527c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7650112210 con 0x7f7650103200 2024-04-23T18:43:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f765527c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7650112090 con 0x7f7650103b20 2024-04-23T18:43:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f765527c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7650112390 con 0x7f765010c5d0 2024-04-23T18:43:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7650103200 0x7f76501986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7650103200 0x7f76501986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34372/0 (socket says 172.21.15.19:34372) 2024-04-23T18:43:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764effd640 1 -- 172.21.15.19:0/349280853 learned_addr learned my addr 172.21.15.19:0/349280853 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764f7fe640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765010c5d0 0x7f765019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764e7fc640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7650103b20 0x7f7650198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764effd640 1 -- 172.21.15.19:0/349280853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765010c5d0 msgr2=0x7f765019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764effd640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765010c5d0 0x7f765019cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764effd640 1 -- 172.21.15.19:0/349280853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7650103b20 msgr2=0x7f7650198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:03.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764effd640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7650103b20 0x7f7650198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764effd640 1 -- 172.21.15.19:0/349280853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765019d690 con 0x7f7650103200 2024-04-23T18:43:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764f7fe640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765010c5d0 0x7f765019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:43:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f764effd640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7650103200 0x7f76501986f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f764000e9f0 tx=0x7f764000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f7637fff640 1 -- 172.21.15.19:0/349280853 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764000cd80 con 0x7f7650103200 2024-04-23T18:43:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.890+0000 7f7637fff640 1 -- 172.21.15.19:0/349280853 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76400108e0 con 0x7f7650103200 2024-04-23T18:43:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.891+0000 7f765527c640 1 -- 172.21.15.19:0/349280853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765019d980 con 0x7f7650103200 2024-04-23T18:43:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.891+0000 7f7637fff640 1 -- 172.21.15.19:0/349280853 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640018980 con 0x7f7650103200 2024-04-23T18:43:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.891+0000 7f765527c640 1 -- 172.21.15.19:0/349280853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76501a5210 con 0x7f7650103200 2024-04-23T18:43:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.893+0000 7f7637fff640 1 -- 172.21.15.19:0/349280853 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7640010430 con 0x7f7650103200 2024-04-23T18:43:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.893+0000 7f7657f94740 1 Processor -- start 2024-04-23T18:43:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.893+0000 7f7657f94740 1 -- start start 2024-04-23T18:43:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.893+0000 7f7657f94740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 0x556e6d236a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.893+0000 7f7657f94740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d2412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.893+0000 7f7657f94740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d243bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.893+0000 7f7657f94740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556e6d19b970 con 0x556e6d241800 2024-04-23T18:43:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.893+0000 7f7657f94740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556e6d1384a0 con 0x556e6d236620 2024-04-23T18:43:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.893+0000 7f7657f94740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556e6d197930 con 0x556e6d236f40 2024-04-23T18:43:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d243bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d243bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34378/0 (socket says 172.21.15.19:34378) 2024-04-23T18:43:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764e7fc640 1 -- 172.21.15.19:0/2680970025 learned_addr learned my addr 172.21.15.19:0/2680970025 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764f7fe640 1 --2- 172.21.15.19:0/2680970025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 0x556e6d236a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f7637fff640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76380a3dc0 0x7f76380a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764effd640 1 --2- 172.21.15.19:0/2680970025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d2412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764e7fc640 1 -- 172.21.15.19:0/2680970025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 msgr2=0x556e6d2412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764e7fc640 1 --2- 172.21.15.19:0/2680970025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d2412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764e7fc640 1 -- 172.21.15.19:0/2680970025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 msgr2=0x556e6d236a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764e7fc640 1 --2- 172.21.15.19:0/2680970025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 0x556e6d236a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764e7fc640 1 -- 172.21.15.19:0/2680970025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6ceae210 con 0x556e6d241800 2024-04-23T18:43:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764f7fe640 1 --2- 172.21.15.19:0/2680970025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 0x556e6d236a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:43:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764f7fe640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76380a3dc0 0x7f76380a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f7637fff640 1 -- 172.21.15.19:0/349280853 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7f76400d0c20 con 0x7f7650103200 2024-04-23T18:43:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764e7fc640 1 --2- 172.21.15.19:0/2680970025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d243bf0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7644006010 tx=0x7f7644032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefe5db07f7bcb55 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f76357fa640 1 -- 172.21.15.19:0/2680970025 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764403f070 con 0x556e6d241800 2024-04-23T18:43:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.894+0000 7f764f7fe640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76380a3dc0 0x7f76380a6280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f765006b860 tx=0x7f7648004a40 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f76357fa640 1 -- 172.21.15.19:0/2680970025 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7644004030 con 0x556e6d241800 2024-04-23T18:43:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f76357fa640 1 -- 172.21.15.19:0/2680970025 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644035070 con 0x556e6d241800 2024-04-23T18:43:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f7657f94740 1 -- 172.21.15.19:0/2680970025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 msgr2=0x556e6d243bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f7657f94740 1 --2- 172.21.15.19:0/2680970025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d243bf0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7644006010 tx=0x7f7644032b20 comp rx=0 tx=0).stop 2024-04-23T18:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f7657f94740 1 -- 172.21.15.19:0/2680970025 shutdown_connections 2024-04-23T18:43:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f7657f94740 1 --2- 172.21.15.19:0/2680970025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d243bf0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f7657f94740 1 --2- 172.21.15.19:0/2680970025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d2412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f7657f94740 1 --2- 172.21.15.19:0/2680970025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 0x556e6d236a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f7657f94740 1 -- 172.21.15.19:0/2680970025 >> 172.21.15.19:0/2680970025 conn(0x556e6d232b30 msgr2=0x556e6d232f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f7657f94740 1 -- 172.21.15.19:0/2680970025 shutdown_connections 2024-04-23T18:43:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.895+0000 7f7657f94740 1 -- 172.21.15.19:0/2680970025 wait complete. 2024-04-23T18:43:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f7657f94740 1 -- start start 2024-04-23T18:43:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f7657f94740 1 Processor -- start 2024-04-23T18:43:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f7657f94740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 0x556e6d241130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f7657f94740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d23e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f7657f94740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d23e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f7657f94740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556e6d2443d0 con 0x556e6d241800 2024-04-23T18:43:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f7657f94740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556e6d1360b0 con 0x556e6d236620 2024-04-23T18:43:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f7657f94740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556e6d244550 con 0x556e6d236f40 2024-04-23T18:43:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f7657f94740 10 client.0 _ll_register_callbacks cb 0x556e6d123d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d23e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d23e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34384/0 (socket says 172.21.15.19:34384) 2024-04-23T18:43:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764f7fe640 1 -- 172.21.15.19:0/480376518 learned_addr learned my addr 172.21.15.19:0/480376518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764effd640 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 0x556e6d241130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764e7fc640 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d23e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764f7fe640 1 -- 172.21.15.19:0/480376518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 msgr2=0x556e6d23e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764f7fe640 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d23e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764f7fe640 1 -- 172.21.15.19:0/480376518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 msgr2=0x556e6d241130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764f7fe640 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 0x556e6d241130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764f7fe640 1 -- 172.21.15.19:0/480376518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6cf7a620 con 0x556e6d241800 2024-04-23T18:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.896+0000 7f764e7fc640 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d23e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:43:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f764f7fe640 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d23e960 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7648009bb0 tx=0x7f7648004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad71f755f4853ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7632ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:43:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7632ff5640 1 -- 172.21.15.19:0/480376518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648022070 con 0x556e6d241800 2024-04-23T18:43:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7632ff5640 1 -- 172.21.15.19:0/480376518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7f7648012de0 con 0x556e6d241800 2024-04-23T18:43:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7657f94740 1 -- 172.21.15.19:0/480376518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7644004030 con 0x556e6d241800 2024-04-23T18:43:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7632ff5640 1 -- 172.21.15.19:0/480376518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f764800fe60 con 0x556e6d241800 2024-04-23T18:43:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7657f94740 10 client.17403 fetch_fsmap learned FSMap version 29 2024-04-23T18:43:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7657f94740 1 -- 172.21.15.19:0/480376518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e6d250340 con 0x556e6d241800 2024-04-23T18:43:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7632ff5640 1 -- 172.21.15.19:0/480376518 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7644004030 con 0x556e6d241800 2024-04-23T18:43:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7632ff5640 1 -- 172.21.15.19:0/480376518 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f76480276a0 con 0x556e6d241800 2024-04-23T18:43:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7657f94740 10 client.17403 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:43:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7657f94740 10 client.17403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.897+0000 7f7657f94740 20 client.17403 populate_metadata read hostname 'smithi019' 2024-04-23T18:43:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.898+0000 7f7657f94740 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556e6d2517b0 0x556e6d253ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.898+0000 7f7657f94740 4 client.17403 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:43:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.898+0000 7f7657f94740 1 -- 172.21.15.19:0/480376518 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e6d1a9390 con 0x556e6d2517b0 2024-04-23T18:43:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.898+0000 7f764effd640 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556e6d2517b0 0x556e6d253ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.898+0000 7f7632ff5640 10 client.17403 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.898+0000 7f764effd640 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556e6d2517b0 0x556e6d253ba0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f2a8e8b89478fb server_cookie=f834a0d8ad895a76 in_seq=0 out_seq=0 2024-04-23T18:43:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7632ff5640 1 -- 172.21.15.19:0/480376518 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e6d1a9390 con 0x556e6d2517b0 2024-04-23T18:43:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7632ff5640 10 client.17403 handle_command_reply: tid=0 2024-04-23T18:43:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 client.17403 shutdown 2024-04-23T18:43:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 -- 172.21.15.19:0/480376518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 msgr2=0x556e6d23e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d23e960 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7648009bb0 tx=0x7f7648004ab0 comp rx=0 tx=0).stop 2024-04-23T18:43:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 -- 172.21.15.19:0/480376518 shutdown_connections 2024-04-23T18:43:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556e6d2517b0 0x556e6d253ba0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d241800 0x556e6d23e960 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d23e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7632ff5640 0 client.17403 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 --2- 172.21.15.19:0/480376518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d236620 0x556e6d241130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 -- 172.21.15.19:0/480376518 >> 172.21.15.19:0/480376518 conn(0x556e6d232b30 msgr2=0x556e6d238860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 -- 172.21.15.19:0/480376518 shutdown_connections 2024-04-23T18:43:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.899+0000 7f7657f94740 1 -- 172.21.15.19:0/480376518 wait complete. 2024-04-23T18:43:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.900+0000 7f7657f94740 20 client.17403 trim_cache size 0 max 16384 2024-04-23T18:43:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.914+0000 7f7657f94740 1 Processor -- start 2024-04-23T18:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.914+0000 7f7657f94740 1 -- start start 2024-04-23T18:43:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f7657f94740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d24d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f7657f94740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d231b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f7657f94740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d241800 0x556e6d2594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f7657f94740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76501adb80 con 0x556e6d236620 2024-04-23T18:43:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f7657f94740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76501af6c0 con 0x556e6d241800 2024-04-23T18:43:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f7657f94740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76501ae250 con 0x556e6d236f40 2024-04-23T18:43:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f764effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d24d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f764effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d24d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34392/0 (socket says 172.21.15.19:34392) 2024-04-23T18:43:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f764effd640 1 -- 172.21.15.19:0/3941596706 learned_addr learned my addr 172.21.15.19:0/3941596706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f764e7fc640 1 --2- 172.21.15.19:0/3941596706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d231b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f764effd640 1 -- 172.21.15.19:0/3941596706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 msgr2=0x556e6d231b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f764effd640 1 --2- 172.21.15.19:0/3941596706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d231b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f764effd640 1 -- 172.21.15.19:0/3941596706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d241800 msgr2=0x556e6d2594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f764effd640 1 --2- 172.21.15.19:0/3941596706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d241800 0x556e6d2594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f764effd640 1 -- 172.21.15.19:0/3941596706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6d236d70 con 0x556e6d236620 2024-04-23T18:43:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f764effd640 1 --2- 172.21.15.19:0/3941596706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d24d290 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f76400ce970 tx=0x7f76400b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a305ffa53533f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f7634ff9640 1 -- 172.21.15.19:0/3941596706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764009aec0 con 0x556e6d236620 2024-04-23T18:43:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f7634ff9640 1 -- 172.21.15.19:0/3941596706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7640010a90 con 0x556e6d236620 2024-04-23T18:43:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.915+0000 7f7634ff9640 1 -- 172.21.15.19:0/3941596706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76400adbc0 con 0x556e6d236620 2024-04-23T18:43:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 -- 172.21.15.19:0/3941596706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 msgr2=0x556e6d24d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 --2- 172.21.15.19:0/3941596706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d24d290 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f76400ce970 tx=0x7f76400b1040 comp rx=0 tx=0).stop 2024-04-23T18:43:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 -- 172.21.15.19:0/3941596706 shutdown_connections 2024-04-23T18:43:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 --2- 172.21.15.19:0/3941596706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d241800 0x556e6d2594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 --2- 172.21.15.19:0/3941596706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d231b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 --2- 172.21.15.19:0/3941596706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d24d290 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 -- 172.21.15.19:0/3941596706 >> 172.21.15.19:0/3941596706 conn(0x556e6d232b30 msgr2=0x556e6d231420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 -- 172.21.15.19:0/3941596706 shutdown_connections 2024-04-23T18:43:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 -- 172.21.15.19:0/3941596706 wait complete. 2024-04-23T18:43:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 -- start start 2024-04-23T18:43:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.916+0000 7f7657f94740 1 Processor -- start 2024-04-23T18:43:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7657f94740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d247540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7657f94740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d247a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7657f94740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d241800 0x556e6d2b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7657f94740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76501aeff0 con 0x556e6d236620 2024-04-23T18:43:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7657f94740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76501ae920 con 0x556e6d241800 2024-04-23T18:43:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7657f94740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76501abb80 con 0x556e6d236f40 2024-04-23T18:43:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7657f94740 10 client.0 _ll_register_callbacks cb 0x7f764800f5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d247540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d247540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34404/0 (socket says 172.21.15.19:34404) 2024-04-23T18:43:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764effd640 1 -- 172.21.15.19:0/122444379 learned_addr learned my addr 172.21.15.19:0/122444379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764e7fc640 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d247a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764f7fe640 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d241800 0x556e6d2b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764effd640 1 -- 172.21.15.19:0/122444379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 msgr2=0x556e6d247a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764effd640 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d247a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764effd640 1 -- 172.21.15.19:0/122444379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d241800 msgr2=0x556e6d2b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764effd640 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d241800 0x556e6d2b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764effd640 1 -- 172.21.15.19:0/122444379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6d248350 con 0x556e6d236620 2024-04-23T18:43:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f764effd640 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d247540 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f764009c5f0 tx=0x7f7640018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c02b4843b29c340 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7632ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7632ff5640 1 -- 172.21.15.19:0/122444379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76400b5020 con 0x556e6d236620 2024-04-23T18:43:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7632ff5640 1 -- 172.21.15.19:0/122444379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7f76400bd0b0 con 0x556e6d236620 2024-04-23T18:43:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.917+0000 7f7657f94740 1 -- 172.21.15.19:0/122444379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7640010a90 con 0x556e6d236620 2024-04-23T18:43:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7632ff5640 1 -- 172.21.15.19:0/122444379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7640066050 con 0x556e6d236620 2024-04-23T18:43:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7657f94740 10 client.17409 fetch_fsmap learned FSMap version 29 2024-04-23T18:43:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7657f94740 1 -- 172.21.15.19:0/122444379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e6d2b8380 con 0x556e6d236620 2024-04-23T18:43:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7632ff5640 1 -- 172.21.15.19:0/122444379 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7640010a90 con 0x556e6d236620 2024-04-23T18:43:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7632ff5640 1 -- 172.21.15.19:0/122444379 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7640018c40 con 0x556e6d236620 2024-04-23T18:43:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7657f94740 10 client.17409 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:43:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7657f94740 10 client.17409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7657f94740 20 client.17409 populate_metadata read hostname 'smithi019' 2024-04-23T18:43:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7657f94740 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556e6d2b89e0 0x556e6d2badd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7657f94740 4 client.17409 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:43:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f7657f94740 1 -- 172.21.15.19:0/122444379 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e6d23ecf0 con 0x556e6d2b89e0 2024-04-23T18:43:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.918+0000 7f764e7fc640 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556e6d2b89e0 0x556e6d2badd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.919+0000 7f7632ff5640 10 client.17409 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.919+0000 7f764e7fc640 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556e6d2b89e0 0x556e6d2badd0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b9ae512be01df0 server_cookie=6b4afda8e37c8427 in_seq=0 out_seq=0 2024-04-23T18:43:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7632ff5640 1 -- 172.21.15.19:0/122444379 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e6d23ecf0 con 0x556e6d2b89e0 2024-04-23T18:43:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7632ff5640 10 client.17409 handle_command_reply: tid=0 2024-04-23T18:43:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 client.17409 shutdown 2024-04-23T18:43:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 -- 172.21.15.19:0/122444379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 msgr2=0x556e6d247540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d247540 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f764009c5f0 tx=0x7f7640018420 comp rx=0 tx=0).stop 2024-04-23T18:43:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 -- 172.21.15.19:0/122444379 shutdown_connections 2024-04-23T18:43:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556e6d2b89e0 0x556e6d2badd0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e6d241800 0x556e6d2b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7632ff5640 0 client.17409 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e6d236f40 0x556e6d247a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:13.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 --2- 172.21.15.19:0/122444379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e6d236620 0x556e6d247540 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 -- 172.21.15.19:0/122444379 >> 172.21.15.19:0/122444379 conn(0x556e6d232b30 msgr2=0x556e6d231420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 -- 172.21.15.19:0/122444379 shutdown_connections 2024-04-23T18:43:13.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 1 -- 172.21.15.19:0/122444379 wait complete. 2024-04-23T18:43:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.929+0000 7f7657f94740 20 client.17409 trim_cache size 0 max 16384 2024-04-23T18:43:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:43:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:43:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:43:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:43:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 -- 172.21.15.19:0/349280853 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76380a3dc0 msgr2=0x7f76380a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76380a3dc0 0x7f76380a6280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f765006b860 tx=0x7f7648004a40 comp rx=0 tx=0).stop 2024-04-23T18:43:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 -- 172.21.15.19:0/349280853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7650103200 msgr2=0x7f76501986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7650103200 0x7f76501986f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f764000e9f0 tx=0x7f764000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:43:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 -- 172.21.15.19:0/349280853 shutdown_connections 2024-04-23T18:43:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f76380a3dc0 0x7f76380a6280 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7650103b20 0x7f7650198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f765010c5d0 0x7f765019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 --2- 172.21.15.19:0/349280853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7650103200 0x7f76501986f0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 -- 172.21.15.19:0/349280853 >> 172.21.15.19:0/349280853 conn(0x7f76500771d0 msgr2=0x7f76500fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 -- 172.21.15.19:0/349280853 shutdown_connections 2024-04-23T18:43:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:04.930+0000 7f765527c640 1 -- 172.21.15.19:0/349280853 wait complete. 2024-04-23T18:43:13.958 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:01 smithi044 ceph-mon[32458]: pgmap v530: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:13.989 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:03 smithi044 ceph-mon[32458]: pgmap v531: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:14.024 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:05 smithi019 ceph-mon[29215]: pgmap v468: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-04-23T18:43:14.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:06 smithi019 ceph-mon[29215]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T18:43:14.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:07 smithi019 ceph-mon[29215]: pgmap v470: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:43:14.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:09 smithi019 ceph-mon[29215]: pgmap v471: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:43:14.215 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:11 smithi019 ceph-mon[29215]: pgmap v472: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:43:14.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:13 smithi019 ceph-mon[29215]: pgmap v473: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:43:14.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:13 smithi019 ceph-mon[29215]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T18:43:14.341 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:15 smithi019 ceph-mon[29215]: pgmap v475: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 14 op/s 2024-04-23T18:43:14.402 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:43:14.440 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: in thread 7f50d97ee700 thread_name:md_log_replay 2024-04-23T18:43:14.473 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:43:14.506 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ea791d20] 2024-04-23T18:43:14.540 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2: /usr/bin/ceph-mds(+0x54938f) [0x556e58a5238f] 2024-04-23T18:43:14.573 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556e58a55b23] 2024-04-23T18:43:14.606 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 4: (MDLog::_replay_thread()+0x617) [0x556e58ab0647] 2024-04-23T18:43:14.640 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556e58ab9d31] 2024-04-23T18:43:14.673 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f50ea7871ca] 2024-04-23T18:43:14.724 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 7: clone() 2024-04-23T18:43:14.823 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2024-04-23T18:43:05.349+0000 7f50d97ee700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:43:14.857 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: in thread 7f50d97ee700 thread_name:md_log_replay 2024-04-23T18:43:14.923 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2024-04-23T18:43:14.957 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:43:15.031 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ea791d20] 2024-04-23T18:43:15.066 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2: /usr/bin/ceph-mds(+0x54938f) [0x556e58a5238f] 2024-04-23T18:43:15.106 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556e58a55b23] 2024-04-23T18:43:15.140 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 4: (MDLog::_replay_thread()+0x617) [0x556e58ab0647] 2024-04-23T18:43:15.182 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556e58ab9d31] 2024-04-23T18:43:15.223 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f50ea7871ca] 2024-04-23T18:43:15.249 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 7: clone() 2024-04-23T18:43:15.307 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:43:15.332 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2024-04-23T18:43:15.349 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 0> 2024-04-23T18:43:05.349+0000 7f50d97ee700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:43:15.374 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: in thread 7f50d97ee700 thread_name:md_log_replay 2024-04-23T18:43:15.471 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2024-04-23T18:43:15.507 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:43:15.540 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ea791d20] 2024-04-23T18:43:15.649 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2: /usr/bin/ceph-mds(+0x54938f) [0x556e58a5238f] 2024-04-23T18:43:15.682 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556e58a55b23] 2024-04-23T18:43:15.723 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 4: (MDLog::_replay_thread()+0x617) [0x556e58ab0647] 2024-04-23T18:43:15.749 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556e58ab9d31] 2024-04-23T18:43:15.832 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f50ea7871ca] 2024-04-23T18:43:15.866 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 7: clone() 2024-04-23T18:43:15.907 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:43:15.941 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2024-04-23T18:43:16.020 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 0> 2024-04-23T18:43:05.349+0000 7f50d97ee700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:43:16.057 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: in thread 7f50d97ee700 thread_name:md_log_replay 2024-04-23T18:43:16.091 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2024-04-23T18:43:16.124 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:43:16.157 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ea791d20] 2024-04-23T18:43:16.191 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2: /usr/bin/ceph-mds(+0x54938f) [0x556e58a5238f] 2024-04-23T18:43:16.224 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556e58a55b23] 2024-04-23T18:43:16.316 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 4: (MDLog::_replay_thread()+0x617) [0x556e58ab0647] 2024-04-23T18:43:16.407 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556e58ab9d31] 2024-04-23T18:43:16.441 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f50ea7871ca] 2024-04-23T18:43:16.474 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 7: clone() 2024-04-23T18:43:16.541 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:43:16.584 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[62201]: 2024-04-23T18:43:16.624 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:05 smithi044 podman[69016]: 2024-04-23 18:43:05.411980181 +0000 UTC m=+0.022537616 container died 17343ea90f62c8fe4d22203ccac4e11fe324f24ba3a227ce13700048d9305ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-23T18:43:16.658 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:05 smithi044 ceph-mon[32458]: pgmap v532: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:43:16.692 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:01 smithi155 ceph-mon[33468]: pgmap v530: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:16.724 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:03 smithi155 ceph-mon[33468]: pgmap v531: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:16.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:05 smithi155 ceph-mon[33468]: pgmap v532: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:43:16.816 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:06 smithi155 ceph-mon[33468]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T18:43:16.849 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:07 smithi155 ceph-mon[33468]: pgmap v534: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:43:16.883 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:09 smithi155 ceph-mon[33468]: pgmap v535: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:43:16.916 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:10 smithi155 ceph-mon[33468]: pgmap v536: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:43:16.959 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:13 smithi155 ceph-mon[33468]: pgmap v537: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:43:17.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:13 smithi155 ceph-mon[33468]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T18:43:17.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:15 smithi155 ceph-mon[33468]: pgmap v539: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 606 B/s wr, 10 op/s 2024-04-23T18:43:17.108 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:06 smithi044 podman[69016]: 2024-04-23 18:43:06.229078013 +0000 UTC m=+0.839635453 container remove 17343ea90f62c8fe4d22203ccac4e11fe324f24ba3a227ce13700048d9305ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T18:43:17.141 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:06 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:43:17.175 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:06 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-23T18:43:17.258 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:06 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e.service: Consumed 3.436s CPU time. 2024-04-23T18:43:17.293 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:06 smithi044 ceph-mon[32458]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T18:43:17.325 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:07 smithi044 ceph-mon[32458]: pgmap v534: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:43:17.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:09 smithi044 ceph-mon[32458]: pgmap v535: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:43:17.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:10 smithi044 ceph-mon[32458]: pgmap v536: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:43:17.425 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:13 smithi044 ceph-mon[32458]: pgmap v537: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:43:17.522 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:13 smithi044 ceph-mon[32458]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T18:43:17.583 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:15 smithi044 ceph-mon[32458]: pgmap v539: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 606 B/s wr, 10 op/s 2024-04-23T18:43:17.617 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:16 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 1. 2024-04-23T18:43:17.650 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:16 smithi044 systemd[1]: Stopped Ceph mds.e for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:43:17.683 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:16 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e.service: Consumed 3.436s CPU time. 2024-04-23T18:43:17.717 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:16 smithi044 systemd[1]: Starting Ceph mds.e for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:43:17.750 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:16 smithi044 podman[69111]: 2024-04-23 18:43:16.715042718 +0000 UTC m=+0.019282490 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:43:17.783 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:16 smithi044 podman[69111]: 2024-04-23 18:43:16.843200593 +0000 UTC m=+0.147440355 container create 8cecf12f4df369b9fc1a5ec8789b3e7b969a724587c458ca082f2d4eb9537c00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-23T18:43:17.829 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:17 smithi044 podman[69111]: 2024-04-23 18:43:17.112996707 +0000 UTC m=+0.417236470 container init 8cecf12f4df369b9fc1a5ec8789b3e7b969a724587c458ca082f2d4eb9537c00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:43:17.942 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:17 smithi044 podman[69111]: 2024-04-23 18:43:17.116138111 +0000 UTC m=+0.420377880 container start 8cecf12f4df369b9fc1a5ec8789b3e7b969a724587c458ca082f2d4eb9537c00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-23T18:43:17.975 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:17 smithi044 ceph-mds[69130]: -- 172.21.15.44:0/1072233974 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55580b773a40 con 0x55580b747000 2024-04-23T18:43:18.008 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: starting mds.e at 2024-04-23T18:43:18.042 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:17 smithi044 bash[69111]: 8cecf12f4df369b9fc1a5ec8789b3e7b969a724587c458ca082f2d4eb9537c00 2024-04-23T18:43:18.075 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:43:17 smithi044 systemd[1]: Started Ceph mds.e for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:43:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.030+0000 7fdf152cf640 1 -- 172.21.15.19:0/3612980088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf1010c520 msgr2=0x7fdf1010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.030+0000 7fdf152cf640 1 --2- 172.21.15.19:0/3612980088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf1010c520 0x7fdf1010e9c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009ca0 tx=0x7fdf0802a9e0 comp rx=0 tx=0).stop 2024-04-23T18:43:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.030+0000 7fdf152cf640 1 -- 172.21.15.19:0/3612980088 shutdown_connections 2024-04-23T18:43:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.030+0000 7fdf152cf640 1 --2- 172.21.15.19:0/3612980088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf10103b00 0x7fdf1010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.030+0000 7fdf152cf640 1 --2- 172.21.15.19:0/3612980088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf1010c520 0x7fdf1010e9c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.030+0000 7fdf152cf640 1 --2- 172.21.15.19:0/3612980088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf101031e0 0x7fdf101035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.030+0000 7fdf152cf640 1 -- 172.21.15.19:0/3612980088 >> 172.21.15.19:0/3612980088 conn(0x7fdf10077fd0 msgr2=0x7fdf100783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.030+0000 7fdf152cf640 1 -- 172.21.15.19:0/3612980088 shutdown_connections 2024-04-23T18:43:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.030+0000 7fdf152cf640 1 -- 172.21.15.19:0/3612980088 wait complete. 2024-04-23T18:43:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.030+0000 7fdf152cf640 1 Processor -- start 2024-04-23T18:43:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf152cf640 1 -- start start 2024-04-23T18:43:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf152cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf101031e0 0x7fdf10198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf152cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf10103b00 0x7fdf10198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf152cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1010c520 0x7fdf1019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf152cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101121b0 con 0x7fdf101031e0 2024-04-23T18:43:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf152cf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf10112030 con 0x7fdf1010c520 2024-04-23T18:43:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf152cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf10112330 con 0x7fdf10103b00 2024-04-23T18:43:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf101031e0 0x7fdf10198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf101031e0 0x7fdf10198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55816/0 (socket says 172.21.15.19:55816) 2024-04-23T18:43:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf0effd640 1 -- 172.21.15.19:0/2980265669 learned_addr learned my addr 172.21.15.19:0/2980265669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf0e7fc640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf10103b00 0x7fdf10198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf0f7fe640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1010c520 0x7fdf1019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf0effd640 1 -- 172.21.15.19:0/2980265669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf10103b00 msgr2=0x7fdf10198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf0effd640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf10103b00 0x7fdf10198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf0effd640 1 -- 172.21.15.19:0/2980265669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1010c520 msgr2=0x7fdf1019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf0effd640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1010c520 0x7fdf1019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.031+0000 7fdf0effd640 1 -- 172.21.15.19:0/2980265669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1019d680 con 0x7fdf101031e0 2024-04-23T18:43:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.032+0000 7fdf0effd640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf101031e0 0x7fdf10198690 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000e9f0 tx=0x7fdf0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.032+0000 7fdef7fff640 1 -- 172.21.15.19:0/2980265669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000cd80 con 0x7fdf101031e0 2024-04-23T18:43:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.032+0000 7fdf152cf640 1 -- 172.21.15.19:0/2980265669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1019d970 con 0x7fdf101031e0 2024-04-23T18:43:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.032+0000 7fdef7fff640 1 -- 172.21.15.19:0/2980265669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf000054d0 con 0x7fdf101031e0 2024-04-23T18:43:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.032+0000 7fdf152cf640 1 -- 172.21.15.19:0/2980265669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf101a5200 con 0x7fdf101031e0 2024-04-23T18:43:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.032+0000 7fdef7fff640 1 -- 172.21.15.19:0/2980265669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00018630 con 0x7fdf101031e0 2024-04-23T18:43:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.034+0000 7fdef7fff640 1 -- 172.21.15.19:0/2980265669 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdf00004e70 con 0x7fdf101031e0 2024-04-23T18:43:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf17fe5740 1 Processor -- start 2024-04-23T18:43:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf17fe5740 1 -- start start 2024-04-23T18:43:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229a9620 0x557e229a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229b4800 0x557e229b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e2290e970 con 0x557e229a9f40 2024-04-23T18:43:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557e228ab4a0 con 0x557e229b4800 2024-04-23T18:43:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557e2290a930 con 0x557e229a9620 2024-04-23T18:43:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdef7fff640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdef80a3dc0 0x7fdef80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229b42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55832/0 (socket says 172.21.15.19:55832) 2024-04-23T18:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf0effd640 1 -- 172.21.15.19:0/612639227 learned_addr learned my addr 172.21.15.19:0/612639227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf0f7fe640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdef80a3dc0 0x7fdef80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf0f7fe640 1 --2- 172.21.15.19:0/612639227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229a9620 0x557e229a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.035+0000 7fdf0e7fc640 1 --2- 172.21.15.19:0/612639227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229b4800 0x557e229b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdef7fff640 1 -- 172.21.15.19:0/2980265669 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7fdf000d0e90 con 0x7fdf101031e0 2024-04-23T18:43:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdf0effd640 1 -- 172.21.15.19:0/612639227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229a9620 msgr2=0x557e229a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdf0effd640 1 --2- 172.21.15.19:0/612639227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229a9620 0x557e229a9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdf0effd640 1 -- 172.21.15.19:0/612639227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229b4800 msgr2=0x557e229b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdf0effd640 1 --2- 172.21.15.19:0/612639227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229b4800 0x557e229b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdf0effd640 1 -- 172.21.15.19:0/612639227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e22621210 con 0x557e229a9f40 2024-04-23T18:43:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdf0e7fc640 1 --2- 172.21.15.19:0/612639227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229b4800 0x557e229b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:43:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdf0effd640 1 --2- 172.21.15.19:0/612639227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229b42c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fdf000ce520 tx=0x7fdf0009cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef584fb06052222a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdf0f7fe640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdef80a3dc0 0x7fdef80a6280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006010 tx=0x7fdf08009800 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdef57fa640 1 -- 172.21.15.19:0/612639227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0009ce30 con 0x557e229a9f40 2024-04-23T18:43:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdef57fa640 1 -- 172.21.15.19:0/612639227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf000a11e0 con 0x557e229a9f40 2024-04-23T18:43:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.036+0000 7fdef57fa640 1 -- 172.21.15.19:0/612639227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00005160 con 0x557e229a9f40 2024-04-23T18:43:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 -- 172.21.15.19:0/612639227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 msgr2=0x557e229b42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/612639227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229b42c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fdf000ce520 tx=0x7fdf0009cd30 comp rx=0 tx=0).stop 2024-04-23T18:43:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 -- 172.21.15.19:0/612639227 shutdown_connections 2024-04-23T18:43:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/612639227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229b4800 0x557e229b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/612639227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229b42c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/612639227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229a9620 0x557e229a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 -- 172.21.15.19:0/612639227 >> 172.21.15.19:0/612639227 conn(0x557e229a5b30 msgr2=0x557e229a5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 -- 172.21.15.19:0/612639227 shutdown_connections 2024-04-23T18:43:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 -- 172.21.15.19:0/612639227 wait complete. 2024-04-23T18:43:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 -- start start 2024-04-23T18:43:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.037+0000 7fdf17fe5740 1 Processor -- start 2024-04-23T18:43:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229a9f40 0x557e229b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229b4800 0x557e229b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e229b7370 con 0x557e229b4800 2024-04-23T18:43:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557e228a90b0 con 0x557e229a9620 2024-04-23T18:43:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557e229b74f0 con 0x557e229a9f40 2024-04-23T18:43:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf17fe5740 10 client.0 _ll_register_callbacks cb 0x557e22896d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229b4800 0x557e229b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229b4800 0x557e229b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55834/0 (socket says 172.21.15.19:55834) 2024-04-23T18:43:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0f7fe640 1 -- 172.21.15.19:0/1468410482 learned_addr learned my addr 172.21.15.19:0/1468410482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0effd640 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0e7fc640 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229a9f40 0x557e229b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0f7fe640 1 -- 172.21.15.19:0/1468410482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229a9f40 msgr2=0x557e229b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0f7fe640 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229a9f40 0x557e229b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0f7fe640 1 -- 172.21.15.19:0/1468410482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 msgr2=0x557e229b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0f7fe640 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0f7fe640 1 -- 172.21.15.19:0/1468410482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e226ed620 con 0x557e229b4800 2024-04-23T18:43:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf0f7fe640 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229b4800 0x557e229b1ae0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802ea40 tx=0x7fdf0802a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f258ea568da1cb3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdef2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:43:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdef2ff5640 1 -- 172.21.15.19:0/1468410482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08056060 con 0x557e229b4800 2024-04-23T18:43:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdef2ff5640 1 -- 172.21.15.19:0/1468410482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7fdf08051420 con 0x557e229b4800 2024-04-23T18:43:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.038+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1468410482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf000a11e0 con 0x557e229b4800 2024-04-23T18:43:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdef2ff5640 1 -- 172.21.15.19:0/1468410482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf08041040 con 0x557e229b4800 2024-04-23T18:43:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdf17fe5740 10 client.17421 fetch_fsmap learned FSMap version 29 2024-04-23T18:43:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1468410482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e229c3390 con 0x557e229b4800 2024-04-23T18:43:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdef2ff5640 1 -- 172.21.15.19:0/1468410482 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf000a11e0 con 0x557e229b4800 2024-04-23T18:43:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdef2ff5640 1 -- 172.21.15.19:0/1468410482 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdf08038450 con 0x557e229b4800 2024-04-23T18:43:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdf17fe5740 10 client.17421 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:43:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdf17fe5740 10 client.17421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdf17fe5740 20 client.17421 populate_metadata read hostname 'smithi019' 2024-04-23T18:43:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557e229c48c0 0x557e229c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdf17fe5740 4 client.17421 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:43:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1468410482 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e2291c390 con 0x557e229c48c0 2024-04-23T18:43:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.039+0000 7fdf0effd640 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557e229c48c0 0x557e229c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.040+0000 7fdf0effd640 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557e229c48c0 0x557e229c6cf0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b618cff356492f server_cookie=4baaf4b75aa5f7c2 in_seq=0 out_seq=0 2024-04-23T18:43:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.040+0000 7fdef2ff5640 10 client.17421 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdef2ff5640 1 -- 172.21.15.19:0/1468410482 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e2291c390 con 0x557e229c48c0 2024-04-23T18:43:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdef2ff5640 10 client.17421 handle_command_reply: tid=0 2024-04-23T18:43:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 client.17421 shutdown 2024-04-23T18:43:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1468410482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229b4800 msgr2=0x557e229b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229b4800 0x557e229b1ae0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802ea40 tx=0x7fdf0802a9e0 comp rx=0 tx=0).stop 2024-04-23T18:43:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1468410482 shutdown_connections 2024-04-23T18:43:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557e229c48c0 0x557e229c6cf0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229b4800 0x557e229b1ae0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229a9f40 0x557e229b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdef2ff5640 0 client.17421 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/1468410482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1468410482 >> 172.21.15.19:0/1468410482 conn(0x557e229a5b30 msgr2=0x557e229ab8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1468410482 shutdown_connections 2024-04-23T18:43:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1468410482 wait complete. 2024-04-23T18:43:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.041+0000 7fdf17fe5740 20 client.17421 trim_cache size 0 max 16384 2024-04-23T18:43:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf17fe5740 1 Processor -- start 2024-04-23T18:43:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf17fe5740 1 -- start start 2024-04-23T18:43:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229b4800 0x557e229ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101adc00 con 0x557e229a9f40 2024-04-23T18:43:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101af740 con 0x557e229a9620 2024-04-23T18:43:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101ae2d0 con 0x557e229b4800 2024-04-23T18:43:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf0e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf0e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55842/0 (socket says 172.21.15.19:55842) 2024-04-23T18:43:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.056+0000 7fdf0e7fc640 1 -- 172.21.15.19:0/1739721220 learned_addr learned my addr 172.21.15.19:0/1739721220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdf0effd640 1 --2- 172.21.15.19:0/1739721220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdf0f7fe640 1 --2- 172.21.15.19:0/1739721220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229b4800 0x557e229ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdf0e7fc640 1 -- 172.21.15.19:0/1739721220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229b4800 msgr2=0x557e229ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdf0e7fc640 1 --2- 172.21.15.19:0/1739721220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229b4800 0x557e229ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdf0e7fc640 1 -- 172.21.15.19:0/1739721220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 msgr2=0x557e229c0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdf0e7fc640 1 --2- 172.21.15.19:0/1739721220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229c0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdf0e7fc640 1 -- 172.21.15.19:0/1739721220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e229a1500 con 0x557e229a9f40 2024-04-23T18:43:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdf0e7fc640 1 --2- 172.21.15.19:0/1739721220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229cc650 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdf1010e290 tx=0x7fdf04013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a91b63dfe87e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdef4ff9640 1 -- 172.21.15.19:0/1739721220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04004020 con 0x557e229a9f40 2024-04-23T18:43:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdef4ff9640 1 -- 172.21.15.19:0/1739721220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf040054d0 con 0x557e229a9f40 2024-04-23T18:43:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.057+0000 7fdef4ff9640 1 -- 172.21.15.19:0/1739721220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04004dd0 con 0x557e229a9f40 2024-04-23T18:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1739721220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 msgr2=0x557e229cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/1739721220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229cc650 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdf1010e290 tx=0x7fdf04013830 comp rx=0 tx=0).stop 2024-04-23T18:43:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1739721220 shutdown_connections 2024-04-23T18:43:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/1739721220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229b4800 0x557e229ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/1739721220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229cc650 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/1739721220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229c0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1739721220 >> 172.21.15.19:0/1739721220 conn(0x557e229a5b30 msgr2=0x557e229a4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1739721220 shutdown_connections 2024-04-23T18:43:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 -- 172.21.15.19:0/1739721220 wait complete. 2024-04-23T18:43:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 -- start start 2024-04-23T18:43:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 Processor -- start 2024-04-23T18:43:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229b4800 0x557e22a29ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101af070 con 0x557e229a9f40 2024-04-23T18:43:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101ae9a0 con 0x557e229a9620 2024-04-23T18:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101abc00 con 0x557e229b4800 2024-04-23T18:43:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf17fe5740 10 client.0 _ll_register_callbacks cb 0x7fdf0802e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf0e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf0e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229bad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55844/0 (socket says 172.21.15.19:55844) 2024-04-23T18:43:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf0e7fc640 1 -- 172.21.15.19:0/310884985 learned_addr learned my addr 172.21.15.19:0/310884985 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.058+0000 7fdf0effd640 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdf0f7fe640 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229b4800 0x557e22a29ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdf0e7fc640 1 -- 172.21.15.19:0/310884985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229b4800 msgr2=0x557e22a29ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdf0e7fc640 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229b4800 0x557e22a29ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdf0e7fc640 1 -- 172.21.15.19:0/310884985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 msgr2=0x557e229ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdf0e7fc640 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdf0e7fc640 1 -- 172.21.15.19:0/310884985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e22a2a410 con 0x557e229a9f40 2024-04-23T18:43:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdf0e7fc640 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229bad40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400f470 tx=0x7fdf0400f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4a52fc6bf39022 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdef2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:43:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdef2ff5640 1 -- 172.21.15.19:0/310884985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04029020 con 0x557e229a9f40 2024-04-23T18:43:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdef2ff5640 1 -- 172.21.15.19:0/310884985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7fdf040035f0 con 0x557e229a9f40 2024-04-23T18:43:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdf17fe5740 1 -- 172.21.15.19:0/310884985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf040054d0 con 0x557e229a9f40 2024-04-23T18:43:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdef2ff5640 1 -- 172.21.15.19:0/310884985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf04034050 con 0x557e229a9f40 2024-04-23T18:43:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdf17fe5740 10 client.17427 fetch_fsmap learned FSMap version 29 2024-04-23T18:43:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.059+0000 7fdf17fe5740 1 -- 172.21.15.19:0/310884985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e22a2b650 con 0x557e229a9f40 2024-04-23T18:43:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdef2ff5640 1 -- 172.21.15.19:0/310884985 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf040054d0 con 0x557e229a9f40 2024-04-23T18:43:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdef2ff5640 1 -- 172.21.15.19:0/310884985 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdf0401a070 con 0x557e229a9f40 2024-04-23T18:43:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdf17fe5740 10 client.17427 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:43:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdf17fe5740 10 client.17427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdf17fe5740 20 client.17427 populate_metadata read hostname 'smithi019' 2024-04-23T18:43:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557e22a2bd70 0x557e22a2e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdf17fe5740 4 client.17427 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:43:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdf17fe5740 1 -- 172.21.15.19:0/310884985 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e22916200 con 0x557e22a2bd70 2024-04-23T18:43:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdf0effd640 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557e22a2bd70 0x557e22a2e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdf0effd640 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557e22a2bd70 0x557e22a2e190 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76418f173b2e24d7 server_cookie=fa1d5f59544c2f10 in_seq=0 out_seq=0 2024-04-23T18:43:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.060+0000 7fdef2ff5640 10 client.17427 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdef2ff5640 1 -- 172.21.15.19:0/310884985 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e22916200 con 0x557e22a2bd70 2024-04-23T18:43:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdef2ff5640 10 client.17427 handle_command_reply: tid=0 2024-04-23T18:43:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 client.17427 shutdown 2024-04-23T18:43:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 -- 172.21.15.19:0/310884985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 msgr2=0x557e229bad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229bad40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400f470 tx=0x7fdf0400f8d0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 -- 172.21.15.19:0/310884985 shutdown_connections 2024-04-23T18:43:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557e22a2bd70 0x557e22a2e190 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e229b4800 0x557e22a29ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdef2ff5640 0 client.17427 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e229a9f40 0x557e229bad40 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 --2- 172.21.15.19:0/310884985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e229a9620 0x557e229ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 -- 172.21.15.19:0/310884985 >> 172.21.15.19:0/310884985 conn(0x557e229a5b30 msgr2=0x557e229bf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 -- 172.21.15.19:0/310884985 shutdown_connections 2024-04-23T18:43:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 1 -- 172.21.15.19:0/310884985 wait complete. 2024-04-23T18:43:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.068+0000 7fdf17fe5740 20 client.17427 trim_cache size 0 max 16384 2024-04-23T18:43:27.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:43:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:43:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:43:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:43:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:43:28.009 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:17 smithi155 ceph-mon[33468]: pgmap v540: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:43:28.107 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:17 smithi155 ceph-mon[33468]: daemon mds.e restarted 2024-04-23T18:43:28.155 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:18 smithi155 ceph-mon[33468]: osdmap e113: 12 total, 12 up, 12 in 2024-04-23T18:43:28.261 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:18 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6834/3131205149,v1:172.21.15.44:6835/3131205149] up:boot 2024-04-23T18:43:28.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:18 smithi155 ceph-mon[33468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:43:28.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:18 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:43:28.357 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:19 smithi155 ceph-mon[33468]: pgmap v542: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-23T18:43:28.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:21 smithi155 ceph-mon[33468]: pgmap v543: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 639 B/s wr, 12 op/s 2024-04-23T18:43:28.424 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:23 smithi155 ceph-mon[33468]: pgmap v544: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 537 B/s wr, 10 op/s 2024-04-23T18:43:28.457 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:25 smithi155 ceph-mon[33468]: pgmap v545: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:43:28.505 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:17 smithi044 ceph-mon[32458]: pgmap v540: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:43:28.601 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:17 smithi044 ceph-mon[32458]: daemon mds.e restarted 2024-04-23T18:43:28.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:18 smithi044 ceph-mon[32458]: osdmap e113: 12 total, 12 up, 12 in 2024-04-23T18:43:28.674 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:18 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6834/3131205149,v1:172.21.15.44:6835/3131205149] up:boot 2024-04-23T18:43:28.788 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:18 smithi044 ceph-mon[32458]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:43:28.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:18 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:43:28.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:19 smithi044 ceph-mon[32458]: pgmap v542: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-23T18:43:28.933 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:21 smithi044 ceph-mon[32458]: pgmap v543: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 639 B/s wr, 12 op/s 2024-04-23T18:43:28.966 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:23 smithi044 ceph-mon[32458]: pgmap v544: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 537 B/s wr, 10 op/s 2024-04-23T18:43:28.999 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:25 smithi044 ceph-mon[32458]: pgmap v545: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:43:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 -- 172.21.15.19:0/2980265669 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdef80a3dc0 msgr2=0x7fdef80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdef80a3dc0 0x7fdef80a6280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006010 tx=0x7fdf08009800 comp rx=0 tx=0).stop 2024-04-23T18:43:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 -- 172.21.15.19:0/2980265669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf101031e0 msgr2=0x7fdf10198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf101031e0 0x7fdf10198690 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000e9f0 tx=0x7fdf0000edf0 comp rx=0 tx=0).stop 2024-04-23T18:43:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 -- 172.21.15.19:0/2980265669 shutdown_connections 2024-04-23T18:43:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdef80a3dc0 0x7fdef80a6280 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1010c520 0x7fdf1019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf10103b00 0x7fdf10198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 --2- 172.21.15.19:0/2980265669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf101031e0 0x7fdf10198690 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 -- 172.21.15.19:0/2980265669 >> 172.21.15.19:0/2980265669 conn(0x7fdf10077fd0 msgr2=0x7fdf100fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 -- 172.21.15.19:0/2980265669 shutdown_connections 2024-04-23T18:43:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:18.069+0000 7fdf152cf640 1 -- 172.21.15.19:0/2980265669 wait complete. 2024-04-23T18:43:29.493 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:27 smithi155 ceph-mon[33468]: pgmap v546: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:43:29.592 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:27 smithi044 ceph-mon[32458]: pgmap v546: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:43:29.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:17 smithi019 ceph-mon[29215]: pgmap v476: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-23T18:43:29.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:19 smithi019 ceph-mon[29215]: pgmap v477: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-23T18:43:29.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:21 smithi019 ceph-mon[29215]: pgmap v478: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-23T18:43:29.725 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:23 smithi019 ceph-mon[29215]: pgmap v479: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-23T18:43:29.808 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:25 smithi019 ceph-mon[29215]: pgmap v480: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-23T18:43:29.906 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:27 smithi019 ceph-mon[29215]: pgmap v481: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-23T18:43:29.950 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:28 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:43:29.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:28 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:43:30.025 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:28 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:43:30.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:28 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:43:30.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:29 smithi019 ceph-mon[29215]: pgmap v482: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-23T18:43:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.168+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3730292661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d54103aa0 msgr2=0x7f0d5410bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.168+0000 7f0d5bc91640 1 --2- 172.21.15.19:0/3730292661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d54103aa0 0x7f0d5410bf80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009900 tx=0x7f0d4402a480 comp rx=0 tx=0).stop 2024-04-23T18:43:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.168+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3730292661 shutdown_connections 2024-04-23T18:43:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.168+0000 7f0d5bc91640 1 --2- 172.21.15.19:0/3730292661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d5410c550 0x7f0d5410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.168+0000 7f0d5bc91640 1 --2- 172.21.15.19:0/3730292661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d54103aa0 0x7f0d5410bf80 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.168+0000 7f0d5bc91640 1 --2- 172.21.15.19:0/3730292661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54103180 0x7f0d54103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.168+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3730292661 >> 172.21.15.19:0/3730292661 conn(0x7f0d54077fd0 msgr2=0x7f0d540783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.168+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3730292661 shutdown_connections 2024-04-23T18:43:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.168+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3730292661 wait complete. 2024-04-23T18:43:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5bc91640 1 Processor -- start 2024-04-23T18:43:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5bc91640 1 -- start start 2024-04-23T18:43:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5bc91640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54103180 0x7f0d541986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5bc91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d54103aa0 0x7f0d54198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5bc91640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d5410c550 0x7f0d5419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5bc91640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541121d0 con 0x7f0d54103aa0 2024-04-23T18:43:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5bc91640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d54112050 con 0x7f0d54103180 2024-04-23T18:43:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5bc91640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d54112350 con 0x7f0d5410c550 2024-04-23T18:43:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5922c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d54103aa0 0x7f0d54198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5922c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d54103aa0 0x7f0d54198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52206/0 (socket says 172.21.15.19:52206) 2024-04-23T18:43:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5922c640 1 -- 172.21.15.19:0/3754555013 learned_addr learned my addr 172.21.15.19:0/3754555013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.169+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d5410c550 0x7f0d5419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d59a2d640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54103180 0x7f0d541986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d5922c640 1 -- 172.21.15.19:0/3754555013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d5410c550 msgr2=0x7f0d5419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d5922c640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d5410c550 0x7f0d5419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d5922c640 1 -- 172.21.15.19:0/3754555013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54103180 msgr2=0x7f0d541986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:31.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d5922c640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54103180 0x7f0d541986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d5922c640 1 -- 172.21.15.19:0/3754555013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5419d6a0 con 0x7f0d54103aa0 2024-04-23T18:43:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d59a2d640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54103180 0x7f0d541986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:43:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d5922c640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d54103aa0 0x7f0d54198bf0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005940 tx=0x7f0d44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d4affd640 1 -- 172.21.15.19:0/3754555013 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4403f030 con 0x7f0d54103aa0 2024-04-23T18:43:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d4affd640 1 -- 172.21.15.19:0/3754555013 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d44003aa0 con 0x7f0d54103aa0 2024-04-23T18:43:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3754555013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5419d930 con 0x7f0d54103aa0 2024-04-23T18:43:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d4affd640 1 -- 172.21.15.19:0/3754555013 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44035070 con 0x7f0d54103aa0 2024-04-23T18:43:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.170+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3754555013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d541a5220 con 0x7f0d54103aa0 2024-04-23T18:43:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.173+0000 7f0d4affd640 1 -- 172.21.15.19:0/3754555013 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d44042030 con 0x7f0d54103aa0 2024-04-23T18:43:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.173+0000 7f0d5e9ab740 1 Processor -- start 2024-04-23T18:43:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.173+0000 7f0d5e9ab740 1 -- start start 2024-04-23T18:43:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.173+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 0x56498d61ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.173+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d6292c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:32.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.173+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d62bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.173+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56498d583970 con 0x56498d629800 2024-04-23T18:43:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.173+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56498d5204a0 con 0x56498d61ef40 2024-04-23T18:43:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.173+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56498d57f930 con 0x56498d61e620 2024-04-23T18:43:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5922c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d62bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 0x56498d61ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 0x56498d61ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:42552/0 (socket says 172.21.15.19:42552) 2024-04-23T18:43:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 -- 172.21.15.19:0/3186079823 learned_addr learned my addr 172.21.15.19:0/3186079823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d4affd640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d400a3dc0 0x7f0d400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d59a2d640 1 --2- 172.21.15.19:0/3186079823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d6292c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d400a3dc0 0x7f0d400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d4affd640 1 -- 172.21.15.19:0/3754555013 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7f0d440f0ef0 con 0x7f0d54103aa0 2024-04-23T18:43:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 -- 172.21.15.19:0/3186079823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 msgr2=0x56498d6292c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/3186079823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d6292c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 -- 172.21.15.19:0/3186079823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 msgr2=0x56498d62bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/3186079823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d62bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 -- 172.21.15.19:0/3186079823 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498d296210 con 0x56498d61e620 2024-04-23T18:43:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5922c640 1 --2- 172.21.15.19:0/3186079823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d62bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:43:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/3186079823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 0x56498d61ea00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c005c40 tx=0x7f0d4c013f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceaea5049cf0654b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.174+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d400a3dc0 0x7f0d400a6280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c01d000 tx=0x7f0d4c01f000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d3ffff640 1 -- 172.21.15.19:0/3186079823 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00fde0 con 0x56498d61e620 2024-04-23T18:43:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d3ffff640 1 -- 172.21.15.19:0/3186079823 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d4c022020 con 0x56498d61e620 2024-04-23T18:43:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d3ffff640 1 -- 172.21.15.19:0/3186079823 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c010690 con 0x56498d61e620 2024-04-23T18:43:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/3186079823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 msgr2=0x56498d61ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/3186079823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 0x56498d61ea00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c005c40 tx=0x7f0d4c013f50 comp rx=0 tx=0).stop 2024-04-23T18:43:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/3186079823 shutdown_connections 2024-04-23T18:43:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/3186079823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d6292c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/3186079823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d62bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/3186079823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 0x56498d61ea00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/3186079823 >> 172.21.15.19:0/3186079823 conn(0x56498d61ab30 msgr2=0x56498d61af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/3186079823 shutdown_connections 2024-04-23T18:43:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.175+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/3186079823 wait complete. 2024-04-23T18:43:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.176+0000 7f0d5e9ab740 1 -- start start 2024-04-23T18:43:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.176+0000 7f0d5e9ab740 1 Processor -- start 2024-04-23T18:43:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.176+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61e620 0x56498d629210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.176+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61ef40 0x56498d626500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.176+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d626b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.176+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56498d62c3d0 con 0x56498d629800 2024-04-23T18:43:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.176+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56498d51e0b0 con 0x56498d61e620 2024-04-23T18:43:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.176+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56498d62c550 con 0x56498d61ef40 2024-04-23T18:43:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.176+0000 7f0d5e9ab740 10 client.0 _ll_register_callbacks cb 0x56498d50bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.176+0000 7f0d5a22e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d626b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d59a2d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61e620 0x56498d629210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d5a22e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d626b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52226/0 (socket says 172.21.15.19:52226) 2024-04-23T18:43:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d5a22e640 1 -- 172.21.15.19:0/1036922203 learned_addr learned my addr 172.21.15.19:0/1036922203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d5922c640 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61ef40 0x56498d626500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d5a22e640 1 -- 172.21.15.19:0/1036922203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61ef40 msgr2=0x56498d626500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61ef40 0x56498d626500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d5a22e640 1 -- 172.21.15.19:0/1036922203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61e620 msgr2=0x56498d629210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61e620 0x56498d629210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d5a22e640 1 -- 172.21.15.19:0/1036922203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498d362620 con 0x56498d629800 2024-04-23T18:43:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d59a2d640 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61e620 0x56498d629210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:43:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d626b10 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00fca0 tx=0x7f0d4c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46754bf863c92c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:43:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1036922203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c034060 con 0x56498d629800 2024-04-23T18:43:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1036922203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7f0d4c004bd0 con 0x56498d629800 2024-04-23T18:43:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1036922203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d4c022020 con 0x56498d629800 2024-04-23T18:43:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.177+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1036922203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d4c00fde0 con 0x56498d629800 2024-04-23T18:43:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d5e9ab740 10 client.17439 fetch_fsmap learned FSMap version 29 2024-04-23T18:43:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1036922203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56498d6383c0 con 0x56498d629800 2024-04-23T18:43:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1036922203 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d4c022020 con 0x56498d629800 2024-04-23T18:43:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1036922203 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0d4c010b70 con 0x56498d629800 2024-04-23T18:43:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d5e9ab740 10 client.17439 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:43:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d5e9ab740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d5e9ab740 20 client.17439 populate_metadata read hostname 'smithi019' 2024-04-23T18:43:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56498d639820 0x56498d63bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d5e9ab740 4 client.17439 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1036922203 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56498d591390 con 0x56498d639820 2024-04-23T18:43:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.178+0000 7f0d59a2d640 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56498d639820 0x56498d63bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.179+0000 7f0d59a2d640 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56498d639820 0x56498d63bc10 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17b243051200ac3 server_cookie=c8df9ac6fcd9f10f in_seq=0 out_seq=0 2024-04-23T18:43:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.179+0000 7f0d3d7fa640 10 client.17439 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.179+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1036922203 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56498d591390 con 0x56498d639820 2024-04-23T18:43:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.179+0000 7f0d3d7fa640 10 client.17439 handle_command_reply: tid=0 2024-04-23T18:43:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.179+0000 7f0d5e9ab740 1 client.17439 shutdown 2024-04-23T18:43:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1036922203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 msgr2=0x56498d626b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d626b10 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00fca0 tx=0x7f0d4c009f80 comp rx=0 tx=0).stop 2024-04-23T18:43:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1036922203 shutdown_connections 2024-04-23T18:43:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56498d639820 0x56498d63bc10 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d626b10 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61ef40 0x56498d626500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1036922203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61e620 0x56498d629210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d3d7fa640 0 client.17439 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1036922203 >> 172.21.15.19:0/1036922203 conn(0x56498d61ab30 msgr2=0x56498d6208b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1036922203 shutdown_connections 2024-04-23T18:43:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1036922203 wait complete. 2024-04-23T18:43:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.180+0000 7f0d5e9ab740 20 client.17439 trim_cache size 0 max 16384 2024-04-23T18:43:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5e9ab740 1 Processor -- start 2024-04-23T18:43:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5e9ab740 1 -- start start 2024-04-23T18:43:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 0x56498d628d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d641330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d641870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541adb20 con 0x56498d629800 2024-04-23T18:43:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541af660 con 0x56498d61ef40 2024-04-23T18:43:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541ae1f0 con 0x56498d61e620 2024-04-23T18:43:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5a22e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d641870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5a22e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d641870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52238/0 (socket says 172.21.15.19:52238) 2024-04-23T18:43:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.195+0000 7f0d5a22e640 1 -- 172.21.15.19:0/1670287035 learned_addr learned my addr 172.21.15.19:0/1670287035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d59a2d640 1 --2- 172.21.15.19:0/1670287035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 0x56498d628d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d5922c640 1 --2- 172.21.15.19:0/1670287035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d641330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d5a22e640 1 -- 172.21.15.19:0/1670287035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 msgr2=0x56498d628d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/1670287035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 0x56498d628d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d5a22e640 1 -- 172.21.15.19:0/1670287035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 msgr2=0x56498d641330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/1670287035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d641330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d5a22e640 1 -- 172.21.15.19:0/1670287035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498d61f8b0 con 0x56498d629800 2024-04-23T18:43:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/1670287035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d641870 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c012420 tx=0x7f0d4c010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa840e6cdd90ae9b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d3f7fe640 1 -- 172.21.15.19:0/1670287035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c034040 con 0x56498d629800 2024-04-23T18:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d3f7fe640 1 -- 172.21.15.19:0/1670287035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d4c022030 con 0x56498d629800 2024-04-23T18:43:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.196+0000 7f0d3f7fe640 1 -- 172.21.15.19:0/1670287035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c004c30 con 0x56498d629800 2024-04-23T18:43:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1670287035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 msgr2=0x56498d641870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1670287035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d641870 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c012420 tx=0x7f0d4c010f60 comp rx=0 tx=0).stop 2024-04-23T18:43:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1670287035 shutdown_connections 2024-04-23T18:43:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1670287035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d641330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1670287035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d629800 0x56498d641870 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1670287035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d61e620 0x56498d628d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1670287035 >> 172.21.15.19:0/1670287035 conn(0x56498d61ab30 msgr2=0x56498d61af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1670287035 shutdown_connections 2024-04-23T18:43:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1670287035 wait complete. 2024-04-23T18:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 -- start start 2024-04-23T18:43:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.197+0000 7f0d5e9ab740 1 Processor -- start 2024-04-23T18:43:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d61e620 0x56498d631470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d6319b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d5e9ab740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d629800 0x56498d62eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541aef90 con 0x56498d61e620 2024-04-23T18:43:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541ae8c0 con 0x56498d61ef40 2024-04-23T18:43:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d5e9ab740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541abb20 con 0x56498d629800 2024-04-23T18:43:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d5e9ab740 10 client.0 _ll_register_callbacks cb 0x7f0d4c0138c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d59a2d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d61e620 0x56498d631470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d59a2d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d61e620 0x56498d631470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52252/0 (socket says 172.21.15.19:52252) 2024-04-23T18:43:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d59a2d640 1 -- 172.21.15.19:0/1971447858 learned_addr learned my addr 172.21.15.19:0/1971447858 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d5922c640 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d6319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d5a22e640 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d629800 0x56498d62eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d59a2d640 1 -- 172.21.15.19:0/1971447858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d629800 msgr2=0x56498d62eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d59a2d640 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d629800 0x56498d62eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d59a2d640 1 -- 172.21.15.19:0/1971447858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 msgr2=0x56498d6319b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d59a2d640 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d6319b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d59a2d640 1 -- 172.21.15.19:0/1971447858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498d62f230 con 0x56498d61e620 2024-04-23T18:43:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d59a2d640 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d61e620 0x56498d631470 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0d50005e00 tx=0x7f0d5000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83206f0c00358f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:43:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1971447858 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5001d3b0 con 0x56498d61e620 2024-04-23T18:43:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1971447858 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7f0d5001d550 con 0x56498d61e620 2024-04-23T18:43:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.198+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1971447858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d4c022030 con 0x56498d61e620 2024-04-23T18:43:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1971447858 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d5001d750 con 0x56498d61e620 2024-04-23T18:43:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d5e9ab740 10 client.17445 fetch_fsmap learned FSMap version 29 2024-04-23T18:43:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1971447858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56498d6a3af0 con 0x56498d61e620 2024-04-23T18:43:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1971447858 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d4c022030 con 0x56498d61e620 2024-04-23T18:43:40.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1971447858 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0d50011440 con 0x56498d61e620 2024-04-23T18:43:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d5e9ab740 10 client.17445 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:43:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d5e9ab740 10 client.17445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d5e9ab740 20 client.17445 populate_metadata read hostname 'smithi019' 2024-04-23T18:43:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56498d6a41c0 0x56498d6a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d5e9ab740 4 client.17445 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:43:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1971447858 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56498d58b200 con 0x56498d6a41c0 2024-04-23T18:43:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.199+0000 7f0d5922c640 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56498d6a41c0 0x56498d6a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.200+0000 7f0d5922c640 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56498d6a41c0 0x56498d6a65e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472c34cab1cc31a3 server_cookie=12739f667ffe5f1 in_seq=0 out_seq=0 2024-04-23T18:43:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.200+0000 7f0d3d7fa640 10 client.17445 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.204+0000 7f0d3d7fa640 1 -- 172.21.15.19:0/1971447858 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56498d58b200 con 0x56498d6a41c0 2024-04-23T18:43:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.204+0000 7f0d3d7fa640 10 client.17445 handle_command_reply: tid=0 2024-04-23T18:43:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.204+0000 7f0d5e9ab740 1 client.17445 shutdown 2024-04-23T18:43:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.204+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1971447858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d61e620 msgr2=0x56498d631470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.204+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d61e620 0x56498d631470 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0d50005e00 tx=0x7f0d5000bda0 comp rx=0 tx=0).stop 2024-04-23T18:43:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1971447858 shutdown_connections 2024-04-23T18:43:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56498d6a41c0 0x56498d6a65e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56498d61ef40 0x56498d6319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d3d7fa640 0 client.17445 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56498d629800 0x56498d62eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5e9ab740 1 --2- 172.21.15.19:0/1971447858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56498d61e620 0x56498d631470 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1971447858 >> 172.21.15.19:0/1971447858 conn(0x56498d61ab30 msgr2=0x56498d619420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1971447858 shutdown_connections 2024-04-23T18:43:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5e9ab740 1 -- 172.21.15.19:0/1971447858 wait complete. 2024-04-23T18:43:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5e9ab740 20 client.17445 trim_cache size 0 max 16384 2024-04-23T18:43:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:43:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:43:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:43:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:43:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3754555013 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d400a3dc0 msgr2=0x7f0d400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d400a3dc0 0x7f0d400a6280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c01d000 tx=0x7f0d4c01f000 comp rx=0 tx=0).stop 2024-04-23T18:43:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3754555013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d54103aa0 msgr2=0x7f0d54198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d54103aa0 0x7f0d54198bf0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005940 tx=0x7f0d44004270 comp rx=0 tx=0).stop 2024-04-23T18:43:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3754555013 shutdown_connections 2024-04-23T18:43:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d400a3dc0 0x7f0d400a6280 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d5410c550 0x7f0d5419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d54103aa0 0x7f0d54198bf0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 --2- 172.21.15.19:0/3754555013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54103180 0x7f0d541986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3754555013 >> 172.21.15.19:0/3754555013 conn(0x7f0d54077fd0 msgr2=0x7f0d540fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3754555013 shutdown_connections 2024-04-23T18:43:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:31.205+0000 7f0d5bc91640 1 -- 172.21.15.19:0/3754555013 wait complete. 2024-04-23T18:43:42.705 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:29 smithi155 ceph-mon[33468]: pgmap v547: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 388 B/s wr, 10 op/s 2024-04-23T18:43:42.737 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:29 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:43:42.770 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:30 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:43:42.804 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:30 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:43:42.837 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:30 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:43:42.871 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:31 smithi155 ceph-mon[33468]: pgmap v548: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T18:43:42.954 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:33 smithi155 ceph-mon[33468]: pgmap v549: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:42.996 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:35 smithi155 ceph-mon[33468]: pgmap v550: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:43.029 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:37 smithi155 ceph-mon[33468]: pgmap v551: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:43.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:39 smithi155 ceph-mon[33468]: pgmap v552: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:43.154 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:41 smithi155 ceph-mon[33468]: pgmap v553: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:43.188 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:29 smithi044 ceph-mon[32458]: pgmap v547: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 388 B/s wr, 10 op/s 2024-04-23T18:43:43.221 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:29 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:43:43.254 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:30 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:43:43.296 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:30 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:43:43.346 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:30 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:43:43.396 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:31 smithi044 ceph-mon[32458]: pgmap v548: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T18:43:43.490 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:33 smithi044 ceph-mon[32458]: pgmap v549: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:43.538 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:35 smithi044 ceph-mon[32458]: pgmap v550: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:43.579 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:37 smithi044 ceph-mon[32458]: pgmap v551: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:43.621 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:39 smithi044 ceph-mon[32458]: pgmap v552: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:43.672 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:41 smithi044 ceph-mon[32458]: pgmap v553: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:43.763 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:31 smithi019 ceph-mon[29215]: pgmap v483: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-23T18:43:43.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:33 smithi019 ceph-mon[29215]: pgmap v484: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:43:43.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:35 smithi019 ceph-mon[29215]: pgmap v485: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:43.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:37 smithi019 ceph-mon[29215]: pgmap v486: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:43.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:39 smithi019 ceph-mon[29215]: pgmap v487: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:44.030 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:41 smithi019 ceph-mon[29215]: pgmap v488: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:44.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:43 smithi019 ceph-mon[29215]: pgmap v489: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.306+0000 7f2915720640 1 -- 172.21.15.19:0/4181180315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29101021c0 msgr2=0x7f29101025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.307+0000 7f2915720640 1 --2- 172.21.15.19:0/4181180315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29101021c0 0x7f29101025a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2900009900 tx=0x7f290002a410 comp rx=0 tx=0).stop 2024-04-23T18:43:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.307+0000 7f2915720640 1 -- 172.21.15.19:0/4181180315 shutdown_connections 2024-04-23T18:43:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.307+0000 7f2915720640 1 --2- 172.21.15.19:0/4181180315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2910107290 0x7f291010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.307+0000 7f2915720640 1 --2- 172.21.15.19:0/4181180315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2910102b70 0x7f2910106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.307+0000 7f2915720640 1 --2- 172.21.15.19:0/4181180315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29101021c0 0x7f29101025a0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.307+0000 7f2915720640 1 -- 172.21.15.19:0/4181180315 >> 172.21.15.19:0/4181180315 conn(0x7f29100fda20 msgr2=0x7f29100ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.307+0000 7f2915720640 1 -- 172.21.15.19:0/4181180315 shutdown_connections 2024-04-23T18:43:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.307+0000 7f2915720640 1 -- 172.21.15.19:0/4181180315 wait complete. 2024-04-23T18:43:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.307+0000 7f2915720640 1 Processor -- start 2024-04-23T18:43:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f2915720640 1 -- start start 2024-04-23T18:43:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f2915720640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29101021c0 0x7f2910198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f2915720640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2910107290 0x7f2910198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f2915720640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291019cfe0 0x7f291019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f2915720640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29101121d0 con 0x7f29101021c0 2024-04-23T18:43:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f2915720640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2910112050 con 0x7f2910107290 2024-04-23T18:43:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f2915720640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2910112350 con 0x7f291019cfe0 2024-04-23T18:43:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29101021c0 0x7f2910198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29101021c0 0x7f2910198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37844/0 (socket says 172.21.15.19:37844) 2024-04-23T18:43:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290effd640 1 -- 172.21.15.19:0/56399245 learned_addr learned my addr 172.21.15.19:0/56399245 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290e7fc640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2910107290 0x7f2910198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290f7fe640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291019cfe0 0x7f291019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290effd640 1 -- 172.21.15.19:0/56399245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291019cfe0 msgr2=0x7f291019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290effd640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291019cfe0 0x7f291019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290effd640 1 -- 172.21.15.19:0/56399245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2910107290 msgr2=0x7f2910198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290effd640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2910107290 0x7f2910198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290effd640 1 -- 172.21.15.19:0/56399245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291019dad0 con 0x7f29101021c0 2024-04-23T18:43:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.308+0000 7f290f7fe640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291019cfe0 0x7f291019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:43:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.309+0000 7f290effd640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29101021c0 0x7f2910198770 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2900005940 tx=0x7f29000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.309+0000 7f28f7fff640 1 -- 172.21.15.19:0/56399245 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290003f030 con 0x7f29101021c0 2024-04-23T18:43:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.309+0000 7f28f7fff640 1 -- 172.21.15.19:0/56399245 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2900003cb0 con 0x7f29101021c0 2024-04-23T18:43:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.309+0000 7f2915720640 1 -- 172.21.15.19:0/56399245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291019dd60 con 0x7f29101021c0 2024-04-23T18:43:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.309+0000 7f28f7fff640 1 -- 172.21.15.19:0/56399245 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900035070 con 0x7f29101021c0 2024-04-23T18:43:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.309+0000 7f2915720640 1 -- 172.21.15.19:0/56399245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29101a5520 con 0x7f29101021c0 2024-04-23T18:43:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f28f7fff640 1 -- 172.21.15.19:0/56399245 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2900042030 con 0x7f29101021c0 2024-04-23T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f2918434740 1 Processor -- start 2024-04-23T18:43:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f2918434740 1 -- start start 2024-04-23T18:43:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f2918434740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d66f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f2918434740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7e20 0x5627d67066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f2918434740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d6709000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f2918434740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5627d665c970 con 0x5627d66f7500 2024-04-23T18:43:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f2918434740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5627d65f94a0 con 0x5627d66f7e20 2024-04-23T18:43:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f2918434740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5627d6658930 con 0x5627d6706c10 2024-04-23T18:43:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d66f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d66f78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37860/0 (socket says 172.21.15.19:37860) 2024-04-23T18:43:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.312+0000 7f290f7fe640 1 -- 172.21.15.19:0/503146496 learned_addr learned my addr 172.21.15.19:0/503146496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f28f7fff640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28f80a3dc0 0x7f28f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f290e7fc640 1 --2- 172.21.15.19:0/503146496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d6709000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f290effd640 1 --2- 172.21.15.19:0/503146496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7e20 0x5627d67066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f290f7fe640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28f80a3dc0 0x7f28f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f28f7fff640 1 -- 172.21.15.19:0/56399245 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7f29000f1150 con 0x7f29101021c0 2024-04-23T18:43:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f290f7fe640 1 -- 172.21.15.19:0/503146496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 msgr2=0x5627d6709000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f290f7fe640 1 --2- 172.21.15.19:0/503146496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d6709000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f290f7fe640 1 -- 172.21.15.19:0/503146496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7e20 msgr2=0x5627d67066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f290f7fe640 1 --2- 172.21.15.19:0/503146496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7e20 0x5627d67066d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f290f7fe640 1 -- 172.21.15.19:0/503146496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d636f210 con 0x5627d66f7500 2024-04-23T18:43:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f290f7fe640 1 --2- 172.21.15.19:0/503146496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d66f78e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2908014a40 tx=0x7f2908014f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bd9a15f6aa359a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f28f57fa640 1 -- 172.21.15.19:0/503146496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080144b0 con 0x5627d66f7500 2024-04-23T18:43:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f28f57fa640 1 -- 172.21.15.19:0/503146496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2908023070 con 0x5627d66f7500 2024-04-23T18:43:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.313+0000 7f28f57fa640 1 -- 172.21.15.19:0/503146496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908013a40 con 0x5627d66f7500 2024-04-23T18:43:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.314+0000 7f2918434740 1 -- 172.21.15.19:0/503146496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 msgr2=0x5627d66f78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.314+0000 7f290f7fe640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28f80a3dc0 0x7f28f80a6280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2908022000 tx=0x7f2908020000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.314+0000 7f2918434740 1 --2- 172.21.15.19:0/503146496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d66f78e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2908014a40 tx=0x7f2908014f00 comp rx=0 tx=0).stop 2024-04-23T18:43:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.314+0000 7f2918434740 1 -- 172.21.15.19:0/503146496 shutdown_connections 2024-04-23T18:43:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.314+0000 7f2918434740 1 --2- 172.21.15.19:0/503146496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7e20 0x5627d67066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.314+0000 7f2918434740 1 --2- 172.21.15.19:0/503146496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d6709000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.314+0000 7f2918434740 1 --2- 172.21.15.19:0/503146496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d66f78e0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.314+0000 7f2918434740 1 -- 172.21.15.19:0/503146496 >> 172.21.15.19:0/503146496 conn(0x5627d66f3b30 msgr2=0x5627d66f3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.314+0000 7f2918434740 1 -- 172.21.15.19:0/503146496 shutdown_connections 2024-04-23T18:43:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.314+0000 7f2918434740 1 -- 172.21.15.19:0/503146496 wait complete. 2024-04-23T18:43:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f2918434740 1 -- start start 2024-04-23T18:43:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f2918434740 1 Processor -- start 2024-04-23T18:43:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f2918434740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d6706560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f2918434740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7e20 0x5627d6703850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f2918434740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d6703db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f2918434740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5627d6709840 con 0x5627d66f7500 2024-04-23T18:43:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f2918434740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5627d65f70b0 con 0x5627d66f7e20 2024-04-23T18:43:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f2918434740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5627d67099c0 con 0x5627d6706c10 2024-04-23T18:43:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f2918434740 10 client.0 _ll_register_callbacks cb 0x5627d65e4d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f290effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d6706560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f290effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d6706560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37868/0 (socket says 172.21.15.19:37868) 2024-04-23T18:43:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f290effd640 1 -- 172.21.15.19:0/1905466889 learned_addr learned my addr 172.21.15.19:0/1905466889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.315+0000 7f290e7fc640 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7e20 0x5627d6703850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f290f7fe640 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d6703db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f290effd640 1 -- 172.21.15.19:0/1905466889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 msgr2=0x5627d6703db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f290effd640 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d6703db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f290effd640 1 -- 172.21.15.19:0/1905466889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7e20 msgr2=0x5627d6703850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f290effd640 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7e20 0x5627d6703850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f290effd640 1 -- 172.21.15.19:0/1905466889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d643b620 con 0x5627d66f7500 2024-04-23T18:43:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f290effd640 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d6706560 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f29000bd020 tx=0x7f290002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08e3a8ad114cf79 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f28f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f28f2ff5640 1 -- 172.21.15.19:0/1905466889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900049040 con 0x5627d66f7500 2024-04-23T18:43:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f28f2ff5640 1 -- 172.21.15.19:0/1905466889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7f29000c1080 con 0x5627d66f7500 2024-04-23T18:43:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f2918434740 1 -- 172.21.15.19:0/1905466889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2908023070 con 0x5627d66f7500 2024-04-23T18:43:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f28f2ff5640 1 -- 172.21.15.19:0/1905466889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2900003e60 con 0x5627d66f7500 2024-04-23T18:43:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f2918434740 10 client.17457 fetch_fsmap learned FSMap version 29 2024-04-23T18:43:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f2918434740 1 -- 172.21.15.19:0/1905466889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627d6715820 con 0x5627d66f7500 2024-04-23T18:43:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.316+0000 7f28f2ff5640 1 -- 172.21.15.19:0/1905466889 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2908023070 con 0x5627d66f7500 2024-04-23T18:43:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.317+0000 7f28f2ff5640 1 -- 172.21.15.19:0/1905466889 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f29000bd2f0 con 0x5627d66f7500 2024-04-23T18:43:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.317+0000 7f2918434740 10 client.17457 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:43:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.317+0000 7f2918434740 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.317+0000 7f2918434740 20 client.17457 populate_metadata read hostname 'smithi019' 2024-04-23T18:43:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.317+0000 7f2918434740 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5627d6716c80 0x5627d67190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.317+0000 7f2918434740 4 client.17457 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:43:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.317+0000 7f2918434740 1 -- 172.21.15.19:0/1905466889 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627d666a390 con 0x5627d6716c80 2024-04-23T18:43:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.317+0000 7f290e7fc640 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5627d6716c80 0x5627d67190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.317+0000 7f28f2ff5640 10 client.17457 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.317+0000 7f290e7fc640 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5627d6716c80 0x5627d67190d0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72fac13d8d9c978 server_cookie=7546530b62d4a45d in_seq=0 out_seq=0 2024-04-23T18:43:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f28f2ff5640 1 -- 172.21.15.19:0/1905466889 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5627d666a390 con 0x5627d6716c80 2024-04-23T18:43:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f28f2ff5640 10 client.17457 handle_command_reply: tid=0 2024-04-23T18:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f2918434740 1 client.17457 shutdown 2024-04-23T18:43:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f2918434740 1 -- 172.21.15.19:0/1905466889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 msgr2=0x5627d6706560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f2918434740 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d6706560 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f29000bd020 tx=0x7f290002a410 comp rx=0 tx=0).stop 2024-04-23T18:43:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f2918434740 1 -- 172.21.15.19:0/1905466889 shutdown_connections 2024-04-23T18:43:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f2918434740 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5627d6716c80 0x5627d67190d0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f2918434740 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7e20 0x5627d6703850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f2918434740 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d6703db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f28f2ff5640 0 client.17457 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f2918434740 1 --2- 172.21.15.19:0/1905466889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7500 0x5627d6706560 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.318+0000 7f2918434740 1 -- 172.21.15.19:0/1905466889 >> 172.21.15.19:0/1905466889 conn(0x5627d66f3b30 msgr2=0x5627d66f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.319+0000 7f2918434740 1 -- 172.21.15.19:0/1905466889 shutdown_connections 2024-04-23T18:43:50.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.319+0000 7f2918434740 1 -- 172.21.15.19:0/1905466889 wait complete. 2024-04-23T18:43:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.319+0000 7f2918434740 20 client.17457 trim_cache size 0 max 16384 2024-04-23T18:43:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f2918434740 1 Processor -- start 2024-04-23T18:43:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f2918434740 1 -- start start 2024-04-23T18:43:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f2918434740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 0x5627d66f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f2918434740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d66f7e20 0x5627d6706090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f2918434740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d6706c10 0x5627d671e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f2918434740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29101ab800 con 0x5627d6706c10 2024-04-23T18:43:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f2918434740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29101ad340 con 0x5627d66f7500 2024-04-23T18:43:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f2918434740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29101abed0 con 0x5627d66f7e20 2024-04-23T18:43:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d6706c10 0x5627d671e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d6706c10 0x5627d671e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37870/0 (socket says 172.21.15.19:37870) 2024-04-23T18:43:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290f7fe640 1 -- 172.21.15.19:0/2327017525 learned_addr learned my addr 172.21.15.19:0/2327017525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290effd640 1 --2- 172.21.15.19:0/2327017525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 0x5627d66f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290e7fc640 1 --2- 172.21.15.19:0/2327017525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d66f7e20 0x5627d6706090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290f7fe640 1 -- 172.21.15.19:0/2327017525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d66f7e20 msgr2=0x5627d6706090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290f7fe640 1 --2- 172.21.15.19:0/2327017525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d66f7e20 0x5627d6706090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290f7fe640 1 -- 172.21.15.19:0/2327017525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 msgr2=0x5627d66f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290f7fe640 1 --2- 172.21.15.19:0/2327017525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 0x5627d66f2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290f7fe640 1 -- 172.21.15.19:0/2327017525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d6704120 con 0x5627d6706c10 2024-04-23T18:43:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290e7fc640 1 --2- 172.21.15.19:0/2327017525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d66f7e20 0x5627d6706090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:43:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.334+0000 7f290f7fe640 1 --2- 172.21.15.19:0/2327017525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d6706c10 0x5627d671e990 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f29080096d0 tx=0x7f2908010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99cd04525eb344c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f28f4ff9640 1 -- 172.21.15.19:0/2327017525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290801eb40 con 0x5627d6706c10 2024-04-23T18:43:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f28f4ff9640 1 -- 172.21.15.19:0/2327017525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2908037070 con 0x5627d6706c10 2024-04-23T18:43:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f28f4ff9640 1 -- 172.21.15.19:0/2327017525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080138b0 con 0x5627d6706c10 2024-04-23T18:43:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f2918434740 1 -- 172.21.15.19:0/2327017525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d6706c10 msgr2=0x5627d671e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f2918434740 1 --2- 172.21.15.19:0/2327017525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d6706c10 0x5627d671e990 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f29080096d0 tx=0x7f2908010000 comp rx=0 tx=0).stop 2024-04-23T18:43:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f2918434740 1 -- 172.21.15.19:0/2327017525 shutdown_connections 2024-04-23T18:43:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f2918434740 1 --2- 172.21.15.19:0/2327017525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d6706c10 0x5627d671e990 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f2918434740 1 --2- 172.21.15.19:0/2327017525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d66f7e20 0x5627d6706090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f2918434740 1 --2- 172.21.15.19:0/2327017525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 0x5627d66f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f2918434740 1 -- 172.21.15.19:0/2327017525 >> 172.21.15.19:0/2327017525 conn(0x5627d66f3b30 msgr2=0x5627d66f2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f2918434740 1 -- 172.21.15.19:0/2327017525 shutdown_connections 2024-04-23T18:43:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.335+0000 7f2918434740 1 -- 172.21.15.19:0/2327017525 wait complete. 2024-04-23T18:43:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f2918434740 1 -- start start 2024-04-23T18:43:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f2918434740 1 Processor -- start 2024-04-23T18:43:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f2918434740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 0x5627d6700770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f2918434740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7e20 0x5627d6700cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f2918434740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d67011f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f2918434740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29101acc70 con 0x5627d66f7e20 2024-04-23T18:43:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f2918434740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29101ac5a0 con 0x5627d66f7500 2024-04-23T18:43:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f2918434740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29101a9820 con 0x5627d6706c10 2024-04-23T18:43:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f2918434740 10 client.0 _ll_register_callbacks cb 0x7f2900005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7e20 0x5627d6700cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7e20 0x5627d6700cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37880/0 (socket says 172.21.15.19:37880) 2024-04-23T18:43:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290e7fc640 1 -- 172.21.15.19:0/2475274004 learned_addr learned my addr 172.21.15.19:0/2475274004 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290effd640 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 0x5627d6700770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290f7fe640 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d67011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290e7fc640 1 -- 172.21.15.19:0/2475274004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 msgr2=0x5627d67011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290e7fc640 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d67011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290e7fc640 1 -- 172.21.15.19:0/2475274004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 msgr2=0x5627d6700770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290e7fc640 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 0x5627d6700770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290e7fc640 1 -- 172.21.15.19:0/2475274004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d677c450 con 0x5627d66f7e20 2024-04-23T18:43:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.336+0000 7f290effd640 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 0x5627d6700770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:43:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f290e7fc640 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7e20 0x5627d6700cb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2904005d00 tx=0x7f2904011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e84d5498b8c4f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f28f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:43:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f28f2ff5640 1 -- 172.21.15.19:0/2475274004 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904004220 con 0x5627d66f7e20 2024-04-23T18:43:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f28f2ff5640 1 -- 172.21.15.19:0/2475274004 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7f29040043c0 con 0x5627d66f7e20 2024-04-23T18:43:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f2918434740 1 -- 172.21.15.19:0/2475274004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2908037070 con 0x5627d66f7e20 2024-04-23T18:43:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f28f2ff5640 1 -- 172.21.15.19:0/2475274004 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f290402e050 con 0x5627d66f7e20 2024-04-23T18:43:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f2918434740 10 client.17463 fetch_fsmap learned FSMap version 29 2024-04-23T18:43:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f2918434740 1 -- 172.21.15.19:0/2475274004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627d677d6f0 con 0x5627d66f7e20 2024-04-23T18:43:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f28f2ff5640 1 -- 172.21.15.19:0/2475274004 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2908037070 con 0x5627d66f7e20 2024-04-23T18:43:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f28f2ff5640 1 -- 172.21.15.19:0/2475274004 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f29040047a0 con 0x5627d66f7e20 2024-04-23T18:43:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f2918434740 10 client.17463 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:43:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f2918434740 10 client.17463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f2918434740 20 client.17463 populate_metadata read hostname 'smithi019' 2024-04-23T18:43:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f2918434740 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5627d677de10 0x5627d6780230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f2918434740 4 client.17463 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:43:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.337+0000 7f2918434740 1 -- 172.21.15.19:0/2475274004 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627d6664200 con 0x5627d677de10 2024-04-23T18:43:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.338+0000 7f290effd640 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5627d677de10 0x5627d6780230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.338+0000 7f290effd640 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5627d677de10 0x5627d6780230 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f988bcdd9a300445 server_cookie=e008d60a4818b4d4 in_seq=0 out_seq=0 2024-04-23T18:43:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.338+0000 7f28f2ff5640 10 client.17463 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f28f2ff5640 1 -- 172.21.15.19:0/2475274004 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627d6664200 con 0x5627d677de10 2024-04-23T18:43:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f28f2ff5640 10 client.17463 handle_command_reply: tid=0 2024-04-23T18:43:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 client.17463 shutdown 2024-04-23T18:43:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 -- 172.21.15.19:0/2475274004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7e20 msgr2=0x5627d6700cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7e20 0x5627d6700cb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2904005d00 tx=0x7f2904011960 comp rx=0 tx=0).stop 2024-04-23T18:43:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 -- 172.21.15.19:0/2475274004 shutdown_connections 2024-04-23T18:43:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5627d677de10 0x5627d6780230 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627d6706c10 0x5627d67011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f28f2ff5640 0 client.17463 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:43:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627d66f7e20 0x5627d6700cb0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 --2- 172.21.15.19:0/2475274004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627d66f7500 0x5627d6700770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 -- 172.21.15.19:0/2475274004 >> 172.21.15.19:0/2475274004 conn(0x5627d66f3b30 msgr2=0x5627d66f2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 -- 172.21.15.19:0/2475274004 shutdown_connections 2024-04-23T18:43:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 1 -- 172.21.15.19:0/2475274004 wait complete. 2024-04-23T18:43:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.343+0000 7f2918434740 20 client.17463 trim_cache size 0 max 16384 2024-04-23T18:43:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:43:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:43:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:43:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:43:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 -- 172.21.15.19:0/56399245 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28f80a3dc0 msgr2=0x7f28f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28f80a3dc0 0x7f28f80a6280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2908022000 tx=0x7f2908020000 comp rx=0 tx=0).stop 2024-04-23T18:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 -- 172.21.15.19:0/56399245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29101021c0 msgr2=0x7f2910198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29101021c0 0x7f2910198770 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2900005940 tx=0x7f29000042f0 comp rx=0 tx=0).stop 2024-04-23T18:43:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 -- 172.21.15.19:0/56399245 shutdown_connections 2024-04-23T18:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f28f80a3dc0 0x7f28f80a6280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2910107290 0x7f2910198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291019cfe0 0x7f291019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 --2- 172.21.15.19:0/56399245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29101021c0 0x7f2910198770 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 -- 172.21.15.19:0/56399245 >> 172.21.15.19:0/56399245 conn(0x7f29100fda20 msgr2=0x7f29100ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 -- 172.21.15.19:0/56399245 shutdown_connections 2024-04-23T18:43:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:44.344+0000 7f2915720640 1 -- 172.21.15.19:0/56399245 wait complete. 2024-04-23T18:43:56.364 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:43 smithi155 ceph-mon[33468]: pgmap v554: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:56.430 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:45 smithi155 ceph-mon[33468]: pgmap v555: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:56.480 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:47 smithi155 ceph-mon[33468]: pgmap v556: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:56.530 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:49 smithi155 ceph-mon[33468]: pgmap v557: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:56.580 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:51 smithi155 ceph-mon[33468]: pgmap v558: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:56.638 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:53 smithi155 ceph-mon[33468]: pgmap v559: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:56.680 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:43 smithi044 ceph-mon[32458]: pgmap v554: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:56.713 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:45 smithi044 ceph-mon[32458]: pgmap v555: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:56.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:47 smithi044 ceph-mon[32458]: pgmap v556: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:56.855 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:49 smithi044 ceph-mon[32458]: pgmap v557: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:56.893 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:51 smithi044 ceph-mon[32458]: pgmap v558: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:56.922 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:53 smithi044 ceph-mon[32458]: pgmap v559: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:56.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:45 smithi019 ceph-mon[29215]: pgmap v490: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:56.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:47 smithi019 ceph-mon[29215]: pgmap v491: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:43:57.022 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:49 smithi019 ceph-mon[29215]: pgmap v492: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:43:57.055 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:51 smithi019 ceph-mon[29215]: pgmap v493: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:57.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:53 smithi019 ceph-mon[29215]: pgmap v494: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:43:57.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:55 smithi019 ceph-mon[29215]: pgmap v495: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:57.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:55 smithi155 ceph-mon[33468]: pgmap v560: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:57.230 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:55 smithi044 ceph-mon[32458]: pgmap v560: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:43:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.443+0000 7fa3db0b9640 1 -- 172.21.15.19:0/364204645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3d410c5d0 msgr2=0x7fa3d410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.443+0000 7fa3db0b9640 1 --2- 172.21.15.19:0/364204645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3d410c5d0 0x7fa3d410ea00 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009ca0 tx=0x7fa3cc02a950 comp rx=0 tx=0).stop 2024-04-23T18:43:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.443+0000 7fa3db0b9640 1 -- 172.21.15.19:0/364204645 shutdown_connections 2024-04-23T18:43:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.443+0000 7fa3db0b9640 1 --2- 172.21.15.19:0/364204645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4103b20 0x7fa3d410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.443+0000 7fa3db0b9640 1 --2- 172.21.15.19:0/364204645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3d410c5d0 0x7fa3d410ea00 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.443+0000 7fa3db0b9640 1 --2- 172.21.15.19:0/364204645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d4103200 0x7fa3d41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.443+0000 7fa3db0b9640 1 -- 172.21.15.19:0/364204645 >> 172.21.15.19:0/364204645 conn(0x7fa3d40771d0 msgr2=0x7fa3d40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.443+0000 7fa3db0b9640 1 -- 172.21.15.19:0/364204645 shutdown_connections 2024-04-23T18:43:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.443+0000 7fa3db0b9640 1 -- 172.21.15.19:0/364204645 wait complete. 2024-04-23T18:43:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3db0b9640 1 Processor -- start 2024-04-23T18:43:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3db0b9640 1 -- start start 2024-04-23T18:43:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3db0b9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3d4103200 0x7fa3d4100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3db0b9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4103b20 0x7fa3d4101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3db0b9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d410c5d0 0x7fa3d41018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3db0b9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4112220 con 0x7fa3d4103200 2024-04-23T18:43:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3db0b9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41120a0 con 0x7fa3d4103b20 2024-04-23T18:43:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3db0b9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41123a0 con 0x7fa3d410c5d0 2024-04-23T18:43:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3d8e55640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3d4103200 0x7fa3d4100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3d8e55640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3d4103200 0x7fa3d4100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47148/0 (socket says 172.21.15.19:47148) 2024-04-23T18:43:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3d8e55640 1 -- 172.21.15.19:0/3380352892 learned_addr learned my addr 172.21.15.19:0/3380352892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4103b20 0x7fa3d4101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3d8e55640 1 -- 172.21.15.19:0/3380352892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d410c5d0 msgr2=0x7fa3d41018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d410c5d0 0x7fa3d41018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3d8e55640 1 -- 172.21.15.19:0/3380352892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4103b20 msgr2=0x7fa3d4101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4103b20 0x7fa3d4101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3d8e55640 1 -- 172.21.15.19:0/3380352892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d41a6b00 con 0x7fa3d4103200 2024-04-23T18:43:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.444+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4103b20 0x7fa3d4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:43:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.445+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3d4103200 0x7fa3d4100e50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00e9f0 tx=0x7fa3bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.445+0000 7fa3d1ffb640 1 -- 172.21.15.19:0/3380352892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00cd80 con 0x7fa3d4103200 2024-04-23T18:43:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.445+0000 7fa3d1ffb640 1 -- 172.21.15.19:0/3380352892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3bc0108e0 con 0x7fa3d4103200 2024-04-23T18:43:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.445+0000 7fa3d1ffb640 1 -- 172.21.15.19:0/3380352892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc018980 con 0x7fa3d4103200 2024-04-23T18:43:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.445+0000 7fa3db0b9640 1 -- 172.21.15.19:0/3380352892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d41a6df0 con 0x7fa3d4103200 2024-04-23T18:43:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.445+0000 7fa3db0b9640 1 -- 172.21.15.19:0/3380352892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d41a72b0 con 0x7fa3d4103200 2024-04-23T18:43:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3d1ffb640 1 -- 172.21.15.19:0/3380352892 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa3bc010430 con 0x7fa3d4103200 2024-04-23T18:43:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3dddcd740 1 Processor -- start 2024-04-23T18:43:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3dddcd740 1 -- start start 2024-04-23T18:43:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7e620 0x55b1c6a7ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a89800 0x55b1c6a8bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b1c69e3970 con 0x55b1c6a7e620 2024-04-23T18:43:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b1c69804a0 con 0x55b1c6a7ef40 2024-04-23T18:43:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b1c69df930 con 0x55b1c6a89800 2024-04-23T18:43:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3d9656640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7e620 0x55b1c6a7ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3d9656640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7e620 0x55b1c6a7ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47156/0 (socket says 172.21.15.19:47156) 2024-04-23T18:43:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3d9656640 1 -- 172.21.15.19:0/3163101040 learned_addr learned my addr 172.21.15.19:0/3163101040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3d1ffb640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa3c00a3df0 0x7fa3c00a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.448+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/3163101040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/3163101040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a89800 0x55b1c6a8bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3d9656640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa3c00a3df0 0x7fa3c00a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3d1ffb640 1 -- 172.21.15.19:0/3380352892 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7fa3bc0d0c20 con 0x7fa3d4103200 2024-04-23T18:43:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3d9656640 1 -- 172.21.15.19:0/3163101040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a89800 msgr2=0x55b1c6a8bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3d9656640 1 --2- 172.21.15.19:0/3163101040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a89800 0x55b1c6a8bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3d9656640 1 -- 172.21.15.19:0/3163101040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7ef40 msgr2=0x55b1c6a892c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3d9656640 1 --2- 172.21.15.19:0/3163101040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a892c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3d9656640 1 -- 172.21.15.19:0/3163101040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c66f6210 con 0x55b1c6a7e620 2024-04-23T18:43:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3d9656640 1 --2- 172.21.15.19:0/3163101040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7e620 0x55b1c6a7ea00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009430 tx=0x7fa3cc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa2ae15121bc01a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3caffd640 1 -- 172.21.15.19:0/3163101040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc034040 con 0x55b1c6a7e620 2024-04-23T18:44:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3caffd640 1 -- 172.21.15.19:0/3163101040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc04c070 con 0x55b1c6a7e620 2024-04-23T18:44:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.449+0000 7fa3caffd640 1 -- 172.21.15.19:0/3163101040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc03b470 con 0x55b1c6a7e620 2024-04-23T18:44:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3dddcd740 1 -- 172.21.15.19:0/3163101040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7e620 msgr2=0x55b1c6a7ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3d9656640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa3c00a3df0 0x7fa3c00a62b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc04b000 tx=0x7fa3cc04c000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/3163101040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7e620 0x55b1c6a7ea00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009430 tx=0x7fa3cc003970 comp rx=0 tx=0).stop 2024-04-23T18:44:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3dddcd740 1 -- 172.21.15.19:0/3163101040 shutdown_connections 2024-04-23T18:44:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/3163101040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a892c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/3163101040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a89800 0x55b1c6a8bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/3163101040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7e620 0x55b1c6a7ea00 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3dddcd740 1 -- 172.21.15.19:0/3163101040 >> 172.21.15.19:0/3163101040 conn(0x55b1c6a7ab30 msgr2=0x55b1c6a7af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3dddcd740 1 -- 172.21.15.19:0/3163101040 shutdown_connections 2024-04-23T18:44:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3dddcd740 1 -- 172.21.15.19:0/3163101040 wait complete. 2024-04-23T18:44:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.450+0000 7fa3dddcd740 1 -- start start 2024-04-23T18:44:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3dddcd740 1 Processor -- start 2024-04-23T18:44:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 0x55b1c6a89120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a86410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a86950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b1c6a8c430 con 0x55b1c6a89800 2024-04-23T18:44:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b1c697e0b0 con 0x55b1c6a7e620 2024-04-23T18:44:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b1c6a8c5b0 con 0x55b1c6a7ef40 2024-04-23T18:44:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3dddcd740 10 client.0 _ll_register_callbacks cb 0x55b1c696bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d9656640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a86950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d9656640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a86950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47168/0 (socket says 172.21.15.19:47168) 2024-04-23T18:44:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d9656640 1 -- 172.21.15.19:0/2427869532 learned_addr learned my addr 172.21.15.19:0/2427869532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 0x55b1c6a89120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a86410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d9656640 1 -- 172.21.15.19:0/2427869532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 msgr2=0x55b1c6a86410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d9656640 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a86410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d9656640 1 -- 172.21.15.19:0/2427869532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 msgr2=0x55b1c6a89120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d9656640 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 0x55b1c6a89120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d9656640 1 -- 172.21.15.19:0/2427869532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c67c2620 con 0x55b1c6a89800 2024-04-23T18:44:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.451+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 0x55b1c6a89120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:44:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3d9656640 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a86950 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc003cb0 tx=0x7fa3cc003a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd309bbdcfc2b24c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2427869532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc058030 con 0x55b1c6a89800 2024-04-23T18:44:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2427869532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7fa3cc02fc40 con 0x55b1c6a89800 2024-04-23T18:44:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2427869532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3cc04c070 con 0x55b1c6a89800 2024-04-23T18:44:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2427869532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc034040 con 0x55b1c6a89800 2024-04-23T18:44:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3dddcd740 10 client.17475 fetch_fsmap learned FSMap version 29 2024-04-23T18:44:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2427869532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b1c6a98310 con 0x55b1c6a89800 2024-04-23T18:44:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2427869532 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc04c070 con 0x55b1c6a89800 2024-04-23T18:44:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2427869532 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa3cc03bab0 con 0x55b1c6a89800 2024-04-23T18:44:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3dddcd740 10 client.17475 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3dddcd740 10 client.17475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.452+0000 7fa3dddcd740 20 client.17475 populate_metadata read hostname 'smithi019' 2024-04-23T18:44:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.453+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b1c6a99660 0x55b1c6a9ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.453+0000 7fa3dddcd740 4 client.17475 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:44:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.453+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2427869532 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1c69f1390 con 0x55b1c6a99660 2024-04-23T18:44:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.453+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b1c6a99660 0x55b1c6a9ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.453+0000 7fa3b3fff640 10 client.17475 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.453+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b1c6a99660 0x55b1c6a9ba50 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a88c547607cb8630 server_cookie=d5ac78d385b2934d in_seq=0 out_seq=0 2024-04-23T18:44:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2427869532 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1c69f1390 con 0x55b1c6a99660 2024-04-23T18:44:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3b3fff640 10 client.17475 handle_command_reply: tid=0 2024-04-23T18:44:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 client.17475 shutdown 2024-04-23T18:44:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2427869532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 msgr2=0x55b1c6a86950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a86950 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc003cb0 tx=0x7fa3cc003a60 comp rx=0 tx=0).stop 2024-04-23T18:44:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2427869532 shutdown_connections 2024-04-23T18:44:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b1c6a99660 0x55b1c6a9ba50 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a86950 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a86410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3b3fff640 0 client.17475 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2427869532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 0x55b1c6a89120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2427869532 >> 172.21.15.19:0/2427869532 conn(0x55b1c6a7ab30 msgr2=0x55b1c6a808b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2427869532 shutdown_connections 2024-04-23T18:44:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2427869532 wait complete. 2024-04-23T18:44:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.454+0000 7fa3dddcd740 20 client.17475 trim_cache size 0 max 16384 2024-04-23T18:44:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.469+0000 7fa3dddcd740 1 Processor -- start 2024-04-23T18:44:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.469+0000 7fa3dddcd740 1 -- start start 2024-04-23T18:44:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.469+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7e620 0x55b1c6a952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7ef40 0x55b1c6aa1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a89800 0x55b1c6aa15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41afd00 con 0x55b1c6a7ef40 2024-04-23T18:44:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41b1840 con 0x55b1c6a89800 2024-04-23T18:44:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41b03d0 con 0x55b1c6a7e620 2024-04-23T18:44:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7ef40 0x55b1c6aa1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7ef40 0x55b1c6aa1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47184/0 (socket says 172.21.15.19:47184) 2024-04-23T18:44:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d3fff640 1 -- 172.21.15.19:0/1637487346 learned_addr learned my addr 172.21.15.19:0/1637487346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/1637487346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7e620 0x55b1c6a952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d3fff640 1 -- 172.21.15.19:0/1637487346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7e620 msgr2=0x55b1c6a952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/1637487346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7e620 0x55b1c6a952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d9656640 1 --2- 172.21.15.19:0/1637487346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a89800 0x55b1c6aa15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:04.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d3fff640 1 -- 172.21.15.19:0/1637487346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a89800 msgr2=0x55b1c6aa15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/1637487346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a89800 0x55b1c6aa15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d3fff640 1 -- 172.21.15.19:0/1637487346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c6a81fe0 con 0x55b1c6a7ef40 2024-04-23T18:44:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/1637487346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7ef40 0x55b1c6aa1070 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009c50 tx=0x7fa3c4013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaa368d6321a03d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3ca7fc640 1 -- 172.21.15.19:0/1637487346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4005020 con 0x55b1c6a7ef40 2024-04-23T18:44:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3ca7fc640 1 -- 172.21.15.19:0/1637487346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c40047a0 con 0x55b1c6a7ef40 2024-04-23T18:44:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.470+0000 7fa3ca7fc640 1 -- 172.21.15.19:0/1637487346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40116b0 con 0x55b1c6a7ef40 2024-04-23T18:44:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 -- 172.21.15.19:0/1637487346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7ef40 msgr2=0x55b1c6aa1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/1637487346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7ef40 0x55b1c6aa1070 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009c50 tx=0x7fa3c4013830 comp rx=0 tx=0).stop 2024-04-23T18:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 -- 172.21.15.19:0/1637487346 shutdown_connections 2024-04-23T18:44:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/1637487346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a89800 0x55b1c6aa15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/1637487346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a7ef40 0x55b1c6aa1070 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/1637487346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7e620 0x55b1c6a952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 -- 172.21.15.19:0/1637487346 >> 172.21.15.19:0/1637487346 conn(0x55b1c6a7ab30 msgr2=0x55b1c6a79cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 -- 172.21.15.19:0/1637487346 shutdown_connections 2024-04-23T18:44:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 -- 172.21.15.19:0/1637487346 wait complete. 2024-04-23T18:44:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 -- start start 2024-04-23T18:44:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.471+0000 7fa3dddcd740 1 Processor -- start 2024-04-23T18:44:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 0x55b1c6a8f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a8fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3dddcd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a8ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41b1170 con 0x55b1c6a89800 2024-04-23T18:44:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41b0aa0 con 0x55b1c6a7e620 2024-04-23T18:44:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3dddcd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d41add00 con 0x55b1c6a7ef40 2024-04-23T18:44:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3dddcd740 10 client.0 _ll_register_callbacks cb 0x7fa3cc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d9656640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a8ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d9656640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a8ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47192/0 (socket says 172.21.15.19:47192) 2024-04-23T18:44:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d9656640 1 -- 172.21.15.19:0/2206546263 learned_addr learned my addr 172.21.15.19:0/2206546263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 0x55b1c6a8f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a8fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:06.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d9656640 1 -- 172.21.15.19:0/2206546263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 msgr2=0x55b1c6a8fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d9656640 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a8fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d9656640 1 -- 172.21.15.19:0/2206546263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 msgr2=0x55b1c6a8f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d9656640 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 0x55b1c6a8f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d9656640 1 -- 172.21.15.19:0/2206546263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c6afec10 con 0x55b1c6a89800 2024-04-23T18:44:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d3fff640 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a8fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3d9656640 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a8ffe0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc003cb0 tx=0x7fa3cc0042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9e38be06c80e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.472+0000 7fa3b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:44:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2206546263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc058070 con 0x55b1c6a89800 2024-04-23T18:44:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2206546263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8513+0+0 (secure 0 0 0) 0x7fa3cc03b6d0 con 0x55b1c6a89800 2024-04-23T18:44:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2206546263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c40047a0 con 0x55b1c6a89800 2024-04-23T18:44:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2206546263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc02afd0 con 0x55b1c6a89800 2024-04-23T18:44:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3dddcd740 10 client.17481 fetch_fsmap learned FSMap version 29 2024-04-23T18:44:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2206546263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b1c6affec0 con 0x55b1c6a89800 2024-04-23T18:44:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2206546263 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c40047a0 con 0x55b1c6a89800 2024-04-23T18:44:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2206546263 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa3cc05f8c0 con 0x55b1c6a89800 2024-04-23T18:44:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3dddcd740 10 client.17481 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:44:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3dddcd740 10 client.17481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3dddcd740 20 client.17481 populate_metadata read hostname 'smithi019' 2024-04-23T18:44:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b1c6b00580 0x55b1c6b02970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3dddcd740 4 client.17481 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:44:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2206546263 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1c6a86ce0 con 0x55b1c6b00580 2024-04-23T18:44:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.473+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b1c6b00580 0x55b1c6b02970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.474+0000 7fa3d8e55640 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b1c6b00580 0x55b1c6b02970 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe89034d304f7421 server_cookie=b49b5d4135a43fe2 in_seq=0 out_seq=0 2024-04-23T18:44:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.474+0000 7fa3b3fff640 10 client.17481 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3b3fff640 1 -- 172.21.15.19:0/2206546263 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1c6a86ce0 con 0x55b1c6b00580 2024-04-23T18:44:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3b3fff640 10 client.17481 handle_command_reply: tid=0 2024-04-23T18:44:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3dddcd740 1 client.17481 shutdown 2024-04-23T18:44:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2206546263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 msgr2=0x55b1c6a8ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a8ffe0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc003cb0 tx=0x7fa3cc0042b0 comp rx=0 tx=0).stop 2024-04-23T18:44:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2206546263 shutdown_connections 2024-04-23T18:44:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b1c6b00580 0x55b1c6b02970 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1c6a89800 0x55b1c6a8ffe0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1c6a7ef40 0x55b1c6a8fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3b3fff640 0 client.17481 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.479+0000 7fa3dddcd740 1 --2- 172.21.15.19:0/2206546263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1c6a7e620 0x55b1c6a8f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2206546263 >> 172.21.15.19:0/2206546263 conn(0x55b1c6a7ab30 msgr2=0x55b1c6a942e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2206546263 shutdown_connections 2024-04-23T18:44:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3dddcd740 1 -- 172.21.15.19:0/2206546263 wait complete. 2024-04-23T18:44:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3dddcd740 20 client.17481 trim_cache size 0 max 16384 2024-04-23T18:44:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:44:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:44:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:44:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:44:08.660 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:57 smithi155 ceph-mon[33468]: pgmap v561: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:08.711 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:43:59 smithi155 ceph-mon[33468]: pgmap v562: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:08.762 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:01 smithi155 ceph-mon[33468]: pgmap v563: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:44:08.811 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:03 smithi155 ceph-mon[33468]: pgmap v564: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:08.945 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:05 smithi155 ceph-mon[33468]: pgmap v565: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-23T18:44:08.988 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:07 smithi155 ceph-mon[33468]: pgmap v566: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-23T18:44:09.145 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:57 smithi044 ceph-mon[32458]: pgmap v561: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:09.203 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:43:59 smithi044 ceph-mon[32458]: pgmap v562: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:09.262 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:01 smithi044 ceph-mon[32458]: pgmap v563: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:44:09.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:03 smithi044 ceph-mon[32458]: pgmap v564: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:09.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:05 smithi044 ceph-mon[32458]: pgmap v565: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-23T18:44:09.412 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:07 smithi044 ceph-mon[32458]: pgmap v566: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-23T18:44:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 -- 172.21.15.19:0/3380352892 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa3c00a3df0 msgr2=0x7fa3c00a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa3c00a3df0 0x7fa3c00a62b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc04b000 tx=0x7fa3cc04c000 comp rx=0 tx=0).stop 2024-04-23T18:44:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 -- 172.21.15.19:0/3380352892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3d4103200 msgr2=0x7fa3d4100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3d4103200 0x7fa3d4100e50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00e9f0 tx=0x7fa3bc00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:44:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 -- 172.21.15.19:0/3380352892 shutdown_connections 2024-04-23T18:44:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa3c00a3df0 0x7fa3c00a62b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4103b20 0x7fa3d4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d410c5d0 0x7fa3d41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 --2- 172.21.15.19:0/3380352892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3d4103200 0x7fa3d4100e50 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 -- 172.21.15.19:0/3380352892 >> 172.21.15.19:0/3380352892 conn(0x7fa3d40771d0 msgr2=0x7fa3d40fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 -- 172.21.15.19:0/3380352892 shutdown_connections 2024-04-23T18:44:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:41:57.480+0000 7fa3db0b9640 1 -- 172.21.15.19:0/3380352892 wait complete. 2024-04-23T18:44:10.189 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:57 smithi019 ceph-mon[29215]: pgmap v496: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:10.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:41:59 smithi019 ceph-mon[29215]: pgmap v497: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:10.287 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:01 smithi019 ceph-mon[29215]: pgmap v498: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-23T18:44:10.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:03 smithi019 ceph-mon[29215]: pgmap v499: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:10.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:05 smithi019 ceph-mon[29215]: pgmap v500: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-23T18:44:10.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:06 smithi019 ceph-mon[29215]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T18:44:10.604 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:07 smithi019 ceph-mon[29215]: pgmap v502: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:44:10.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:09 smithi019 ceph-mon[29215]: pgmap v503: 97 pgs: 97 active+clean; 4.2 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:44:10.707 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:09 smithi155 ceph-mon[33468]: pgmap v567: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-23T18:44:10.754 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:09 smithi044 ceph-mon[32458]: pgmap v567: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-23T18:44:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.579+0000 7f1b07a0b640 1 -- 172.21.15.19:0/3071474581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0006a360 msgr2=0x7f1b00106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.579+0000 7f1b07a0b640 1 --2- 172.21.15.19:0/3071474581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0006a360 0x7f1b00106ba0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009900 tx=0x7f1af002a410 comp rx=0 tx=0).stop 2024-04-23T18:44:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.579+0000 7f1b07a0b640 1 -- 172.21.15.19:0/3071474581 shutdown_connections 2024-04-23T18:44:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.579+0000 7f1b07a0b640 1 --2- 172.21.15.19:0/3071474581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b001005c0 0x7f1b00100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.579+0000 7f1b07a0b640 1 --2- 172.21.15.19:0/3071474581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0006a360 0x7f1b00106ba0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.579+0000 7f1b07a0b640 1 --2- 172.21.15.19:0/3071474581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b00069ef0 0x7f1b00106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.579+0000 7f1b07a0b640 1 -- 172.21.15.19:0/3071474581 >> 172.21.15.19:0/3071474581 conn(0x7f1b00077390 msgr2=0x7f1b000fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.580+0000 7f1b07a0b640 1 -- 172.21.15.19:0/3071474581 shutdown_connections 2024-04-23T18:44:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.580+0000 7f1b07a0b640 1 -- 172.21.15.19:0/3071474581 wait complete. 2024-04-23T18:44:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.580+0000 7f1b07a0b640 1 Processor -- start 2024-04-23T18:44:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.580+0000 7f1b07a0b640 1 -- start start 2024-04-23T18:44:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.580+0000 7f1b07a0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b00069ef0 0x7f1b001a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b07a0b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0006a360 0x7f1b001a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b07a0b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b001005c0 0x7f1b0019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b07a0b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001110c0 con 0x7f1b0006a360 2024-04-23T18:44:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b07a0b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00110f40 con 0x7f1b00069ef0 2024-04-23T18:44:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b07a0b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00111240 con 0x7f1b001005c0 2024-04-23T18:44:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b04fa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0006a360 0x7f1b001a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b04fa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0006a360 0x7f1b001a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34358/0 (socket says 172.21.15.19:34358) 2024-04-23T18:44:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b04fa6640 1 -- 172.21.15.19:0/265308117 learned_addr learned my addr 172.21.15.19:0/265308117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b057a7640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b00069ef0 0x7f1b001a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b05fa8640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b001005c0 0x7f1b0019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b04fa6640 1 -- 172.21.15.19:0/265308117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b001005c0 msgr2=0x7f1b0019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b001005c0 0x7f1b0019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b04fa6640 1 -- 172.21.15.19:0/265308117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b00069ef0 msgr2=0x7f1b001a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b00069ef0 0x7f1b001a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b04fa6640 1 -- 172.21.15.19:0/265308117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0019ada0 con 0x7f1b0006a360 2024-04-23T18:44:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b05fa8640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b001005c0 0x7f1b0019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0006a360 0x7f1b001a09b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1af0006010 tx=0x7f1af0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.581+0000 7f1af67fc640 1 -- 172.21.15.19:0/265308117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af003f030 con 0x7f1b0006a360 2024-04-23T18:44:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.582+0000 7f1af67fc640 1 -- 172.21.15.19:0/265308117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af0003aa0 con 0x7f1b0006a360 2024-04-23T18:44:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.582+0000 7f1b07a0b640 1 -- 172.21.15.19:0/265308117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0019b030 con 0x7f1b0006a360 2024-04-23T18:44:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.582+0000 7f1af67fc640 1 -- 172.21.15.19:0/265308117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0035070 con 0x7f1b0006a360 2024-04-23T18:44:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.582+0000 7f1b07a0b640 1 -- 172.21.15.19:0/265308117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b001a7330 con 0x7f1b0006a360 2024-04-23T18:44:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1af67fc640 1 -- 172.21.15.19:0/265308117 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1af0042030 con 0x7f1b0006a360 2024-04-23T18:44:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b0a7f5740 1 Processor -- start 2024-04-23T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b0a7f5740 1 -- start start 2024-04-23T18:44:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 0x55be5261ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261bf40 0x55be526262c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be52626800 0x55be52628bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55be52580970 con 0x55be5261bf40 2024-04-23T18:44:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55be5251d4a0 con 0x55be52626800 2024-04-23T18:44:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55be5257c930 con 0x55be5261b620 2024-04-23T18:44:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b057a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261bf40 0x55be526262c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b057a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261bf40 0x55be526262c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34362/0 (socket says 172.21.15.19:34362) 2024-04-23T18:44:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b057a7640 1 -- 172.21.15.19:0/2758381075 learned_addr learned my addr 172.21.15.19:0/2758381075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b05fa8640 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 0x55be5261ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:13.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be52626800 0x55be52628bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b057a7640 1 -- 172.21.15.19:0/2758381075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 msgr2=0x55be5261ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b057a7640 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 0x55be5261ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b057a7640 1 -- 172.21.15.19:0/2758381075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be52626800 msgr2=0x55be52628bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1af67fc640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1aec0a3dc0 0x7f1aec0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b057a7640 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be52626800 0x55be52628bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b057a7640 1 -- 172.21.15.19:0/2758381075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be52293210 con 0x55be5261bf40 2024-04-23T18:44:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be52626800 0x55be52628bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:44:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.585+0000 7f1b05fa8640 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 0x55be5261ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:44:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.586+0000 7f1b05fa8640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1aec0a3dc0 0x7f1aec0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.586+0000 7f1af67fc640 1 -- 172.21.15.19:0/265308117 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8557+0+0 (secure 0 0 0) 0x7f1af00f0c70 con 0x7f1b0006a360 2024-04-23T18:44:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.586+0000 7f1b057a7640 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261bf40 0x55be526262c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00ac70 tx=0x7f1afc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935af48a236fb095 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.586+0000 7f1aeb7fe640 1 -- 172.21.15.19:0/2758381075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc005220 con 0x55be5261bf40 2024-04-23T18:44:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.586+0000 7f1aeb7fe640 1 -- 172.21.15.19:0/2758381075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1afc0047a0 con 0x55be5261bf40 2024-04-23T18:44:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.586+0000 7f1aeb7fe640 1 -- 172.21.15.19:0/2758381075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc018780 con 0x55be5261bf40 2024-04-23T18:44:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2758381075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261bf40 msgr2=0x55be526262c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261bf40 0x55be526262c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00ac70 tx=0x7f1afc00f790 comp rx=0 tx=0).stop 2024-04-23T18:44:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b05fa8640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1aec0a3dc0 0x7f1aec0a6280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1af8005fd0 tx=0x7f1af8009500 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2758381075 shutdown_connections 2024-04-23T18:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be52626800 0x55be52628bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261bf40 0x55be526262c0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/2758381075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 0x55be5261ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2758381075 >> 172.21.15.19:0/2758381075 conn(0x55be52617b30 msgr2=0x55be52617f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2758381075 shutdown_connections 2024-04-23T18:44:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2758381075 wait complete. 2024-04-23T18:44:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 -- start start 2024-04-23T18:44:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.587+0000 7f1b0a7f5740 1 Processor -- start 2024-04-23T18:44:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 0x55be52626090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be52623380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be52626800 0x55be526238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55be52629430 con 0x55be52626800 2024-04-23T18:44:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55be5251b0b0 con 0x55be5261bf40 2024-04-23T18:44:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55be526295b0 con 0x55be5261b620 2024-04-23T18:44:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b0a7f5740 10 client.0 _ll_register_callbacks cb 0x55be52508d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b05fa8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be52626800 0x55be526238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b05fa8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be52626800 0x55be526238c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34378/0 (socket says 172.21.15.19:34378) 2024-04-23T18:44:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b05fa8640 1 -- 172.21.15.19:0/2863846841 learned_addr learned my addr 172.21.15.19:0/2863846841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be52623380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b057a7640 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 0x55be52626090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b05fa8640 1 -- 172.21.15.19:0/2863846841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 msgr2=0x55be52626090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b05fa8640 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 0x55be52626090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b05fa8640 1 -- 172.21.15.19:0/2863846841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 msgr2=0x55be52623380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b05fa8640 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be52623380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b05fa8640 1 -- 172.21.15.19:0/2863846841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5235f620 con 0x55be52626800 2024-04-23T18:44:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1b05fa8640 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be52626800 0x55be526238c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f1af8012930 tx=0x7f1af8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f9011d538dc4af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.588+0000 7f1ae8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:44:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/2863846841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af800fd60 con 0x55be52626800 2024-04-23T18:44:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2863846841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1afc0047a0 con 0x55be52626800 2024-04-23T18:44:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/2863846841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8557+0+0 (secure 0 0 0) 0x7f1af801f420 con 0x55be52626800 2024-04-23T18:44:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/2863846841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1af801f620 con 0x55be52626800 2024-04-23T18:44:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1b0a7f5740 10 client.17493 fetch_fsmap learned FSMap version 29 2024-04-23T18:44:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2863846841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be52635310 con 0x55be52626800 2024-04-23T18:44:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/2863846841 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1afc0047a0 con 0x55be52626800 2024-04-23T18:44:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/2863846841 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1af8027840 con 0x55be52626800 2024-04-23T18:44:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1b0a7f5740 10 client.17493 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:44:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1b0a7f5740 10 client.17493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1b0a7f5740 20 client.17493 populate_metadata read hostname 'smithi019' 2024-04-23T18:44:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55be526367d0 0x55be52638be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1b0a7f5740 4 client.17493 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:44:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.589+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2863846841 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be5258e390 con 0x55be526367d0 2024-04-23T18:44:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.590+0000 7f1b057a7640 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55be526367d0 0x55be52638be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.590+0000 7f1b057a7640 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55be526367d0 0x55be52638be0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e6885b826da7d2 server_cookie=bf82f1db5da00d95 in_seq=0 out_seq=0 2024-04-23T18:44:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.590+0000 7f1ae8ff9640 10 client.17493 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/2863846841 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be5258e390 con 0x55be526367d0 2024-04-23T18:44:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1ae8ff9640 10 client.17493 handle_command_reply: tid=0 2024-04-23T18:44:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 client.17493 shutdown 2024-04-23T18:44:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2863846841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be52626800 msgr2=0x55be526238c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be52626800 0x55be526238c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f1af8012930 tx=0x7f1af8004a40 comp rx=0 tx=0).stop 2024-04-23T18:44:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2863846841 shutdown_connections 2024-04-23T18:44:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55be526367d0 0x55be52638be0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be52623380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1ae8ff9640 0 client.17493 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be52626800 0x55be526238c0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/2863846841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5261b620 0x55be52626090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2863846841 >> 172.21.15.19:0/2863846841 conn(0x55be52617b30 msgr2=0x55be5261d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2863846841 shutdown_connections 2024-04-23T18:44:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/2863846841 wait complete. 2024-04-23T18:44:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.591+0000 7f1b0a7f5740 20 client.17493 trim_cache size 0 max 16384 2024-04-23T18:44:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.606+0000 7f1b0a7f5740 1 Processor -- start 2024-04-23T18:44:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.606+0000 7f1b0a7f5740 1 -- start start 2024-04-23T18:44:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be526322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be52616b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be52626800 0x55be5263e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001afda0 con 0x55be5261b620 2024-04-23T18:44:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001b18e0 con 0x55be5261bf40 2024-04-23T18:44:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001b0470 con 0x55be52626800 2024-04-23T18:44:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b057a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be526322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b057a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be526322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34388/0 (socket says 172.21.15.19:34388) 2024-04-23T18:44:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b057a7640 1 -- 172.21.15.19:0/1501895411 learned_addr learned my addr 172.21.15.19:0/1501895411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b057a7640 1 -- 172.21.15.19:0/1501895411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be52626800 msgr2=0x55be5263e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/1501895411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be52616b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b057a7640 1 --2- 172.21.15.19:0/1501895411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be52626800 0x55be5263e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b057a7640 1 -- 172.21.15.19:0/1501895411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 msgr2=0x55be52616b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b057a7640 1 --2- 172.21.15.19:0/1501895411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be52616b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b057a7640 1 -- 172.21.15.19:0/1501895411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be52623ec0 con 0x55be5261b620 2024-04-23T18:44:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/1501895411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be52616b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1b057a7640 1 --2- 172.21.15.19:0/1501895411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be526322f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f1afc004ab0 tx=0x7f1afc00af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531d8ceaa5a81723 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1aeaffd640 1 -- 172.21.15.19:0/1501895411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc02bb50 con 0x55be5261b620 2024-04-23T18:44:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1aeaffd640 1 -- 172.21.15.19:0/1501895411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1afc02bcf0 con 0x55be5261b620 2024-04-23T18:44:18.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.607+0000 7f1aeaffd640 1 -- 172.21.15.19:0/1501895411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc004200 con 0x55be5261b620 2024-04-23T18:44:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1501895411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 msgr2=0x55be526322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/1501895411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be526322f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f1afc004ab0 tx=0x7f1afc00af70 comp rx=0 tx=0).stop 2024-04-23T18:44:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1501895411 shutdown_connections 2024-04-23T18:44:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/1501895411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be52616b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/1501895411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be52626800 0x55be5263e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/1501895411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be526322f0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1501895411 >> 172.21.15.19:0/1501895411 conn(0x55be52617b30 msgr2=0x55be52616420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1501895411 shutdown_connections 2024-04-23T18:44:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1501895411 wait complete. 2024-04-23T18:44:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 -- start start 2024-04-23T18:44:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.608+0000 7f1b0a7f5740 1 Processor -- start 2024-04-23T18:44:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be5262a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be5262a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:19.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b0a7f5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be52626800 0x55be5262ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001b1210 con 0x55be5261b620 2024-04-23T18:44:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001b0b40 con 0x55be5261bf40 2024-04-23T18:44:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b0a7f5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001adda0 con 0x55be52626800 2024-04-23T18:44:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b0a7f5740 10 client.0 _ll_register_callbacks cb 0x7f1af8012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b057a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be5262a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b057a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be5262a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34392/0 (socket says 172.21.15.19:34392) 2024-04-23T18:44:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b057a7640 1 -- 172.21.15.19:0/1428250329 learned_addr learned my addr 172.21.15.19:0/1428250329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b05fa8640 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be52626800 0x55be5262ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be5262a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b057a7640 1 -- 172.21.15.19:0/1428250329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be52626800 msgr2=0x55be5262ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b057a7640 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be52626800 0x55be5262ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b057a7640 1 -- 172.21.15.19:0/1428250329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 msgr2=0x55be5262a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b057a7640 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be5262a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.609+0000 7f1b057a7640 1 -- 172.21.15.19:0/1428250329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5269bec0 con 0x55be5261b620 2024-04-23T18:44:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1b057a7640 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be5262a360 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1afc000c00 tx=0x7f1afc02b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d56809cded89f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1ae8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:44:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/1428250329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc039060 con 0x55be5261b620 2024-04-23T18:44:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/1428250329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8557+0+0 (secure 0 0 0) 0x7f1afc02b6d0 con 0x55be5261b620 2024-04-23T18:44:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1428250329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1afc02bcf0 con 0x55be5261b620 2024-04-23T18:44:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/1428250329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1afc02b8d0 con 0x55be5261b620 2024-04-23T18:44:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1b0a7f5740 10 client.17499 fetch_fsmap learned FSMap version 29 2024-04-23T18:44:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1428250329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be5269d120 con 0x55be5261b620 2024-04-23T18:44:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/1428250329 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1afc02bcf0 con 0x55be5261b620 2024-04-23T18:44:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/1428250329 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1afc0039d0 con 0x55be5261b620 2024-04-23T18:44:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1b0a7f5740 10 client.17499 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:44:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1b0a7f5740 10 client.17499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1b0a7f5740 20 client.17499 populate_metadata read hostname 'smithi019' 2024-04-23T18:44:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55be5269d640 0x55be5269fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1b0a7f5740 4 client.17499 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:44:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.610+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1428250329 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be52588200 con 0x55be5269d640 2024-04-23T18:44:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.611+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55be5269d640 0x55be5269fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.611+0000 7f1ae8ff9640 10 client.17499 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.611+0000 7f1b04fa6640 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55be5269d640 0x55be5269fa30 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48e07616ed513900 server_cookie=2b350b2da20e3345 in_seq=0 out_seq=0 2024-04-23T18:44:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.618+0000 7f1ae8ff9640 1 -- 172.21.15.19:0/1428250329 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55be52588200 con 0x55be5269d640 2024-04-23T18:44:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.618+0000 7f1ae8ff9640 10 client.17499 handle_command_reply: tid=0 2024-04-23T18:44:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.618+0000 7f1b0a7f5740 1 client.17499 shutdown 2024-04-23T18:44:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1428250329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 msgr2=0x55be5262a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be5262a360 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1afc000c00 tx=0x7f1afc02b2d0 comp rx=0 tx=0).stop 2024-04-23T18:44:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1428250329 shutdown_connections 2024-04-23T18:44:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55be5269d640 0x55be5269fa30 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be5261bf40 0x55be5262a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be52626800 0x55be5262ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1ae8ff9640 0 client.17499 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 1 --2- 172.21.15.19:0/1428250329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be5261b620 0x55be5262a360 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1428250329 >> 172.21.15.19:0/1428250329 conn(0x55be52617b30 msgr2=0x55be52616cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1428250329 shutdown_connections 2024-04-23T18:44:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 1 -- 172.21.15.19:0/1428250329 wait complete. 2024-04-23T18:44:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b0a7f5740 20 client.17499 trim_cache size 0 max 16384 2024-04-23T18:44:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:44:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:44:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:44:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:44:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b07a0b640 1 -- 172.21.15.19:0/265308117 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1aec0a3dc0 msgr2=0x7f1aec0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b07a0b640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1aec0a3dc0 0x7f1aec0a6280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1af8005fd0 tx=0x7f1af8009500 comp rx=0 tx=0).stop 2024-04-23T18:44:21.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b07a0b640 1 -- 172.21.15.19:0/265308117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0006a360 msgr2=0x7f1b001a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b07a0b640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0006a360 0x7f1b001a09b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1af0006010 tx=0x7f1af0004060 comp rx=0 tx=0).stop 2024-04-23T18:44:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b07a0b640 1 -- 172.21.15.19:0/265308117 shutdown_connections 2024-04-23T18:44:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b07a0b640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f1aec0a3dc0 0x7f1aec0a6280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b07a0b640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b001005c0 0x7f1b0019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b07a0b640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0006a360 0x7f1b001a09b0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b07a0b640 1 --2- 172.21.15.19:0/265308117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b00069ef0 0x7f1b001a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.619+0000 7f1b07a0b640 1 -- 172.21.15.19:0/265308117 >> 172.21.15.19:0/265308117 conn(0x7f1b00077390 msgr2=0x7f1b000fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.620+0000 7f1b07a0b640 1 -- 172.21.15.19:0/265308117 shutdown_connections 2024-04-23T18:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:10.620+0000 7f1b07a0b640 1 -- 172.21.15.19:0/265308117 wait complete. 2024-04-23T18:44:22.463 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:11 smithi019 ceph-mon[29215]: pgmap v504: 97 pgs: 97 active+clean; 4.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:44:22.512 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:13 smithi019 ceph-mon[29215]: pgmap v505: 97 pgs: 97 active+clean; 4.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:44:22.562 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:13 smithi019 ceph-mon[29215]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T18:44:22.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:15 smithi019 ceph-mon[29215]: pgmap v507: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T18:44:22.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:17 smithi019 ceph-mon[29215]: pgmap v508: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-23T18:44:22.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:19 smithi019 ceph-mon[29215]: pgmap v509: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-23T18:44:22.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:21 smithi019 ceph-mon[29215]: pgmap v510: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-23T18:44:22.815 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:11 smithi155 ceph-mon[33468]: pgmap v568: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:44:22.862 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:11 smithi155 ceph-mon[33468]: osdmap e114: 12 total, 12 up, 12 in 2024-04-23T18:44:22.946 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:13 smithi155 ceph-mon[33468]: pgmap v570: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:44:22.997 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:15 smithi155 ceph-mon[33468]: pgmap v571: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:44:23.096 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:17 smithi155 ceph-mon[33468]: pgmap v572: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:44:23.146 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:17 smithi155 ceph-mon[33468]: osdmap e115: 12 total, 12 up, 12 in 2024-04-23T18:44:23.196 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:18 smithi155 ceph-mon[33468]: pgmap v574: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T18:44:23.246 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:21 smithi155 ceph-mon[33468]: pgmap v575: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-23T18:44:23.296 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:11 smithi044 ceph-mon[32458]: pgmap v568: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:44:23.346 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:11 smithi044 ceph-mon[32458]: osdmap e114: 12 total, 12 up, 12 in 2024-04-23T18:44:23.396 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:13 smithi044 ceph-mon[32458]: pgmap v570: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:44:23.446 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:15 smithi044 ceph-mon[32458]: pgmap v571: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:44:23.538 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:17 smithi044 ceph-mon[32458]: pgmap v572: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:44:23.610 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:17 smithi044 ceph-mon[32458]: osdmap e115: 12 total, 12 up, 12 in 2024-04-23T18:44:23.696 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:18 smithi044 ceph-mon[32458]: pgmap v574: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T18:44:23.789 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:21 smithi044 ceph-mon[32458]: pgmap v575: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-23T18:44:23.823 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:23 smithi155 ceph-mon[33468]: pgmap v576: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:44:23.871 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:23 smithi044 ceph-mon[32458]: pgmap v576: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:44:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.719+0000 7fbf668cf640 1 -- 172.21.15.19:0/2065340507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6010c520 msgr2=0x7fbf6010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.719+0000 7fbf668cf640 1 --2- 172.21.15.19:0/2065340507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6010c520 0x7fbf6010e9c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009ca0 tx=0x7fbf5802a9e0 comp rx=0 tx=0).stop 2024-04-23T18:44:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.719+0000 7fbf668cf640 1 -- 172.21.15.19:0/2065340507 shutdown_connections 2024-04-23T18:44:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.719+0000 7fbf668cf640 1 --2- 172.21.15.19:0/2065340507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6010c520 0x7fbf6010e9c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.719+0000 7fbf668cf640 1 --2- 172.21.15.19:0/2065340507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf60103b00 0x7fbf6010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.719+0000 7fbf668cf640 1 --2- 172.21.15.19:0/2065340507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf601031e0 0x7fbf601035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.719+0000 7fbf668cf640 1 -- 172.21.15.19:0/2065340507 >> 172.21.15.19:0/2065340507 conn(0x7fbf60077fd0 msgr2=0x7fbf600783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.719+0000 7fbf668cf640 1 -- 172.21.15.19:0/2065340507 shutdown_connections 2024-04-23T18:44:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.719+0000 7fbf668cf640 1 -- 172.21.15.19:0/2065340507 wait complete. 2024-04-23T18:44:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.720+0000 7fbf668cf640 1 Processor -- start 2024-04-23T18:44:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.720+0000 7fbf668cf640 1 -- start start 2024-04-23T18:44:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf668cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf601031e0 0x7fbf60198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf668cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf60103b00 0x7fbf60198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf668cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6010c520 0x7fbf6019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf668cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf601121b0 con 0x7fbf6010c520 2024-04-23T18:44:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf668cf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60112030 con 0x7fbf601031e0 2024-04-23T18:44:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf668cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60112330 con 0x7fbf60103b00 2024-04-23T18:44:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf64e6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6010c520 0x7fbf6019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf64e6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6010c520 0x7fbf6019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45912/0 (socket says 172.21.15.19:45912) 2024-04-23T18:44:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf64e6c640 1 -- 172.21.15.19:0/1504433304 learned_addr learned my addr 172.21.15.19:0/1504433304 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf60103b00 0x7fbf60198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf601031e0 0x7fbf60198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf64e6c640 1 -- 172.21.15.19:0/1504433304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf60103b00 msgr2=0x7fbf60198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf64e6c640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf60103b00 0x7fbf60198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf64e6c640 1 -- 172.21.15.19:0/1504433304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf601031e0 msgr2=0x7fbf60198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf64e6c640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf601031e0 0x7fbf60198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf64e6c640 1 -- 172.21.15.19:0/1504433304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6019d5d0 con 0x7fbf6010c520 2024-04-23T18:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.721+0000 7fbf64e6c640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6010c520 0x7fbf6019cea0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fbf58006010 tx=0x7fbf5802fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.722+0000 7fbf5d7fa640 1 -- 172.21.15.19:0/1504433304 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58033040 con 0x7fbf6010c520 2024-04-23T18:44:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.722+0000 7fbf5d7fa640 1 -- 172.21.15.19:0/1504433304 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5802e7a0 con 0x7fbf6010c520 2024-04-23T18:44:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.722+0000 7fbf5d7fa640 1 -- 172.21.15.19:0/1504433304 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58043720 con 0x7fbf6010c520 2024-04-23T18:44:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.722+0000 7fbf668cf640 1 -- 172.21.15.19:0/1504433304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6019d860 con 0x7fbf6010c520 2024-04-23T18:44:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.722+0000 7fbf668cf640 1 -- 172.21.15.19:0/1504433304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf601a5150 con 0x7fbf6010c520 2024-04-23T18:44:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.724+0000 7fbf5d7fa640 1 -- 172.21.15.19:0/1504433304 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbf58036070 con 0x7fbf6010c520 2024-04-23T18:44:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.724+0000 7fbf695e5740 1 Processor -- start 2024-04-23T18:44:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.724+0000 7fbf695e5740 1 -- start start 2024-04-23T18:44:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.724+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295a520 0x55eb3295a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295ae40 0x55eb329696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb32969c10 0x55eb3296c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5d7fa640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbf4c0a3dc0 0x7fbf4c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf695e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eb328bf970 con 0x55eb3295ae40 2024-04-23T18:44:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf695e5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55eb3285c4a0 con 0x55eb32969c10 2024-04-23T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf695e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eb328bb930 con 0x55eb3295a520 2024-04-23T18:44:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295ae40 0x55eb329696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295ae40 0x55eb329696d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45924/0 (socket says 172.21.15.19:45924) 2024-04-23T18:44:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5ffff640 1 -- 172.21.15.19:0/2795550751 learned_addr learned my addr 172.21.15.19:0/2795550751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf64e6c640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbf4c0a3dc0 0x7fbf4c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/2795550751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295a520 0x55eb3295a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5d7fa640 1 -- 172.21.15.19:0/1504433304 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7fbf580f14f0 con 0x7fbf6010c520 2024-04-23T18:44:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/2795550751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb32969c10 0x55eb3296c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5ffff640 1 -- 172.21.15.19:0/2795550751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295a520 msgr2=0x55eb3295a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/2795550751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295a520 0x55eb3295a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5ffff640 1 -- 172.21.15.19:0/2795550751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb32969c10 msgr2=0x55eb3296c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/2795550751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb32969c10 0x55eb3296c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5ffff640 1 -- 172.21.15.19:0/2795550751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb325d2210 con 0x55eb3295ae40 2024-04-23T18:44:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/2795550751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295a520 0x55eb3295a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:44:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf64e6c640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbf4c0a3dc0 0x7fbf4c0a6280 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbf580bd190 tx=0x7fbf580b2920 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.725+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/2795550751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295ae40 0x55eb329696d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800ad00 tx=0x7fbf4800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c6fc3816963622 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf567fc640 1 -- 172.21.15.19:0/2795550751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4801d070 con 0x55eb3295ae40 2024-04-23T18:44:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf567fc640 1 -- 172.21.15.19:0/2795550751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf48018af0 con 0x55eb3295ae40 2024-04-23T18:44:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf567fc640 1 -- 172.21.15.19:0/2795550751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48010b40 con 0x55eb3295ae40 2024-04-23T18:44:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf695e5740 1 -- 172.21.15.19:0/2795550751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295ae40 msgr2=0x55eb329696d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf695e5740 1 --2- 172.21.15.19:0/2795550751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295ae40 0x55eb329696d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800ad00 tx=0x7fbf4800f790 comp rx=0 tx=0).stop 2024-04-23T18:44:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf695e5740 1 -- 172.21.15.19:0/2795550751 shutdown_connections 2024-04-23T18:44:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf695e5740 1 --2- 172.21.15.19:0/2795550751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb32969c10 0x55eb3296c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf695e5740 1 --2- 172.21.15.19:0/2795550751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295ae40 0x55eb329696d0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf695e5740 1 --2- 172.21.15.19:0/2795550751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295a520 0x55eb3295a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf695e5740 1 -- 172.21.15.19:0/2795550751 >> 172.21.15.19:0/2795550751 conn(0x55eb32956b30 msgr2=0x55eb32956f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf695e5740 1 -- 172.21.15.19:0/2795550751 shutdown_connections 2024-04-23T18:44:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.726+0000 7fbf695e5740 1 -- 172.21.15.19:0/2795550751 wait complete. 2024-04-23T18:44:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf695e5740 1 -- start start 2024-04-23T18:44:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf695e5740 1 Processor -- start 2024-04-23T18:44:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32969530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295ae40 0x55eb32966820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb32969c10 0x55eb32966d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf695e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eb3296c840 con 0x55eb3295a520 2024-04-23T18:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf695e5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55eb3285a0b0 con 0x55eb32969c10 2024-04-23T18:44:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf695e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eb3296c9c0 con 0x55eb3295ae40 2024-04-23T18:44:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf695e5740 10 client.0 _ll_register_callbacks cb 0x55eb32847d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32969530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32969530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45928/0 (socket says 172.21.15.19:45928) 2024-04-23T18:44:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/1138786612 learned_addr learned my addr 172.21.15.19:0/1138786612 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.727+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295ae40 0x55eb32966820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/1138786612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295ae40 msgr2=0x55eb32966820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295ae40 0x55eb32966820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/1138786612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb32969c10 msgr2=0x55eb32966d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb32969c10 0x55eb32966d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/1138786612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3269e620 con 0x55eb3295a520 2024-04-23T18:44:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32969530 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000bee0 tx=0x7fbf5000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3617be1dc9270975 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:44:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/1138786612 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50005220 con 0x55eb3295a520 2024-04-23T18:44:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/1138786612 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7fbf500053c0 con 0x55eb3295a520 2024-04-23T18:44:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf695e5740 1 -- 172.21.15.19:0/1138786612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf48018af0 con 0x55eb3295a520 2024-04-23T18:44:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/1138786612 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbf500055c0 con 0x55eb3295a520 2024-04-23T18:44:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf695e5740 10 client.17511 fetch_fsmap learned FSMap version 29 2024-04-23T18:44:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf695e5740 1 -- 172.21.15.19:0/1138786612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb32978800 con 0x55eb3295a520 2024-04-23T18:44:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.728+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/1138786612 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf48018af0 con 0x55eb3295a520 2024-04-23T18:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.729+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/1138786612 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbf50012950 con 0x55eb3295a520 2024-04-23T18:44:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.729+0000 7fbf695e5740 10 client.17511 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:44:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.729+0000 7fbf695e5740 10 client.17511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.729+0000 7fbf695e5740 20 client.17511 populate_metadata read hostname 'smithi019' 2024-04-23T18:44:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.729+0000 7fbf695e5740 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eb32979cc0 0x55eb3297c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.729+0000 7fbf695e5740 4 client.17511 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:44:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.729+0000 7fbf695e5740 1 -- 172.21.15.19:0/1138786612 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb328cd390 con 0x55eb32979cc0 2024-04-23T18:44:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.729+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eb32979cc0 0x55eb3297c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.730+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eb32979cc0 0x55eb3297c0b0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f8ecbfbd1cbfbe server_cookie=70d9614a5ba0fb09 in_seq=0 out_seq=0 2024-04-23T18:44:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.730+0000 7fbf3f7fe640 10 client.17511 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.730+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/1138786612 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb328cd390 con 0x55eb32979cc0 2024-04-23T18:44:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.730+0000 7fbf3f7fe640 10 client.17511 handle_command_reply: tid=0 2024-04-23T18:44:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.730+0000 7fbf695e5740 1 client.17511 shutdown 2024-04-23T18:44:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 1 -- 172.21.15.19:0/1138786612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 msgr2=0x55eb32969530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32969530 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000bee0 tx=0x7fbf5000bfd0 comp rx=0 tx=0).stop 2024-04-23T18:44:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 1 -- 172.21.15.19:0/1138786612 shutdown_connections 2024-04-23T18:44:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eb32979cc0 0x55eb3297c0b0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb32969c10 0x55eb32966d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295ae40 0x55eb32966820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf3f7fe640 0 client.17511 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 1 --2- 172.21.15.19:0/1138786612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32969530 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 1 -- 172.21.15.19:0/1138786612 >> 172.21.15.19:0/1138786612 conn(0x55eb32956b30 msgr2=0x55eb3295c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 1 -- 172.21.15.19:0/1138786612 shutdown_connections 2024-04-23T18:44:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 1 -- 172.21.15.19:0/1138786612 wait complete. 2024-04-23T18:44:29.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.731+0000 7fbf695e5740 20 client.17511 trim_cache size 0 max 16384 2024-04-23T18:44:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.746+0000 7fbf695e5740 1 Processor -- start 2024-04-23T18:44:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.746+0000 7fbf695e5740 1 -- start start 2024-04-23T18:44:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.746+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32975740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.746+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb3295ae40 0x55eb32961d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.746+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb32969c10 0x55eb32969060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.746+0000 7fbf695e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf601ab8a0 con 0x55eb3295a520 2024-04-23T18:44:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.746+0000 7fbf695e5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf601ad3e0 con 0x55eb3295ae40 2024-04-23T18:44:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.746+0000 7fbf695e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf601abf70 con 0x55eb32969c10 2024-04-23T18:44:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.746+0000 7fbf5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32975740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32975740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45932/0 (socket says 172.21.15.19:45932) 2024-04-23T18:44:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/249204676 learned_addr learned my addr 172.21.15.19:0/249204676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/249204676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb3295ae40 0x55eb32961d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/249204676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb32969c10 msgr2=0x55eb32969060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/249204676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb32969c10 0x55eb32969060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/249204676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb3295ae40 msgr2=0x55eb32961d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/249204676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb3295ae40 0x55eb32961d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/249204676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb32956960 con 0x55eb3295a520 2024-04-23T18:44:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/249204676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32975740 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000b620 tx=0x7fbf5000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df05055989340f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf55ffb640 1 -- 172.21.15.19:0/249204676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5002a070 con 0x55eb3295a520 2024-04-23T18:44:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf55ffb640 1 -- 172.21.15.19:0/249204676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5001d4b0 con 0x55eb3295a520 2024-04-23T18:44:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.747+0000 7fbf55ffb640 1 -- 172.21.15.19:0/249204676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf500106c0 con 0x55eb3295a520 2024-04-23T18:44:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 -- 172.21.15.19:0/249204676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 msgr2=0x55eb32975740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 --2- 172.21.15.19:0/249204676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32975740 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000b620 tx=0x7fbf5000b790 comp rx=0 tx=0).stop 2024-04-23T18:44:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 -- 172.21.15.19:0/249204676 shutdown_connections 2024-04-23T18:44:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 --2- 172.21.15.19:0/249204676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb3295ae40 0x55eb32961d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 --2- 172.21.15.19:0/249204676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb32969c10 0x55eb32969060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 --2- 172.21.15.19:0/249204676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb3295a520 0x55eb32975740 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 -- 172.21.15.19:0/249204676 >> 172.21.15.19:0/249204676 conn(0x55eb32956b30 msgr2=0x55eb32961670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 -- 172.21.15.19:0/249204676 shutdown_connections 2024-04-23T18:44:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 -- 172.21.15.19:0/249204676 wait complete. 2024-04-23T18:44:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 -- start start 2024-04-23T18:44:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.748+0000 7fbf695e5740 1 Processor -- start 2024-04-23T18:44:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb3295a520 0x55eb32971cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295ae40 0x55eb3296d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf695e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb32969c10 0x55eb3296d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf695e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf601acd10 con 0x55eb32969c10 2024-04-23T18:44:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf695e5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf601ac640 con 0x55eb3295a520 2024-04-23T18:44:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf695e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf601a98c0 con 0x55eb3295ae40 2024-04-23T18:44:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf695e5740 10 client.0 _ll_register_callbacks cb 0x7fbf5000bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb32969c10 0x55eb3296d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb32969c10 0x55eb3296d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45944/0 (socket says 172.21.15.19:45944) 2024-04-23T18:44:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/3749256370 learned_addr learned my addr 172.21.15.19:0/3749256370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb3295a520 0x55eb32971cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295ae40 0x55eb3296d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/3749256370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295ae40 msgr2=0x55eb3296d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295ae40 0x55eb3296d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/3749256370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb3295a520 msgr2=0x55eb32971cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb3295a520 0x55eb32971cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5f7fe640 1 -- 172.21.15.19:0/3749256370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3296dcb0 con 0x55eb32969c10 2024-04-23T18:44:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf5f7fe640 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb32969c10 0x55eb3296d770 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fbf50005fe0 tx=0x7fbf50005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778e22cecae7bfb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.749+0000 7fbf3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:44:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/3749256370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5002a070 con 0x55eb32969c10 2024-04-23T18:44:32.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/3749256370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7fbf50010aa0 con 0x55eb32969c10 2024-04-23T18:44:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf695e5740 1 -- 172.21.15.19:0/3749256370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf5001d4b0 con 0x55eb32969c10 2024-04-23T18:44:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/3749256370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbf5001dc00 con 0x55eb32969c10 2024-04-23T18:44:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf695e5740 10 client.17517 fetch_fsmap learned FSMap version 29 2024-04-23T18:44:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf695e5740 1 -- 172.21.15.19:0/3749256370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb329dfe90 con 0x55eb32969c10 2024-04-23T18:44:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/3749256370 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5001d4b0 con 0x55eb32969c10 2024-04-23T18:44:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/3749256370 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbf50026870 con 0x55eb32969c10 2024-04-23T18:44:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf695e5740 10 client.17517 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:44:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf695e5740 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf695e5740 20 client.17517 populate_metadata read hostname 'smithi019' 2024-04-23T18:44:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf695e5740 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eb329e0490 0x55eb329e2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf695e5740 4 client.17517 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:44:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.750+0000 7fbf695e5740 1 -- 172.21.15.19:0/3749256370 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb328c7200 con 0x55eb329e0490 2024-04-23T18:44:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.751+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eb329e0490 0x55eb329e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.751+0000 7fbf5ffff640 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eb329e0490 0x55eb329e2880 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49567da7f54e9c63 server_cookie=f8f32e64670ca80c in_seq=0 out_seq=0 2024-04-23T18:44:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.751+0000 7fbf3f7fe640 10 client.17517 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.756+0000 7fbf3f7fe640 1 -- 172.21.15.19:0/3749256370 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb328c7200 con 0x55eb329e0490 2024-04-23T18:44:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.756+0000 7fbf3f7fe640 10 client.17517 handle_command_reply: tid=0 2024-04-23T18:44:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.756+0000 7fbf695e5740 1 client.17517 shutdown 2024-04-23T18:44:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 1 -- 172.21.15.19:0/3749256370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb32969c10 msgr2=0x55eb3296d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb32969c10 0x55eb3296d770 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fbf50005fe0 tx=0x7fbf50005f30 comp rx=0 tx=0).stop 2024-04-23T18:44:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 1 -- 172.21.15.19:0/3749256370 shutdown_connections 2024-04-23T18:44:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55eb329e0490 0x55eb329e2880 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb32969c10 0x55eb3296d770 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb3295ae40 0x55eb3296d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf3f7fe640 0 client.17517 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 1 --2- 172.21.15.19:0/3749256370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb3295a520 0x55eb32971cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 1 -- 172.21.15.19:0/3749256370 >> 172.21.15.19:0/3749256370 conn(0x55eb32956b30 msgr2=0x55eb32955420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 1 -- 172.21.15.19:0/3749256370 shutdown_connections 2024-04-23T18:44:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 1 -- 172.21.15.19:0/3749256370 wait complete. 2024-04-23T18:44:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf695e5740 20 client.17517 trim_cache size 0 max 16384 2024-04-23T18:44:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:44:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:44:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:44:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:44:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf668cf640 1 -- 172.21.15.19:0/1504433304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbf4c0a3dc0 msgr2=0x7fbf4c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf668cf640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbf4c0a3dc0 0x7fbf4c0a6280 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbf580bd190 tx=0x7fbf580b2920 comp rx=0 tx=0).stop 2024-04-23T18:44:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf668cf640 1 -- 172.21.15.19:0/1504433304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6010c520 msgr2=0x7fbf6019cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf668cf640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6010c520 0x7fbf6019cea0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fbf58006010 tx=0x7fbf5802fc00 comp rx=0 tx=0).stop 2024-04-23T18:44:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf668cf640 1 -- 172.21.15.19:0/1504433304 shutdown_connections 2024-04-23T18:44:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf668cf640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbf4c0a3dc0 0x7fbf4c0a6280 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf668cf640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6010c520 0x7fbf6019cea0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf668cf640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf60103b00 0x7fbf60198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf668cf640 1 --2- 172.21.15.19:0/1504433304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf601031e0 0x7fbf60198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.757+0000 7fbf668cf640 1 -- 172.21.15.19:0/1504433304 >> 172.21.15.19:0/1504433304 conn(0x7fbf60077fd0 msgr2=0x7fbf600fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.758+0000 7fbf668cf640 1 -- 172.21.15.19:0/1504433304 shutdown_connections 2024-04-23T18:44:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:23.758+0000 7fbf668cf640 1 -- 172.21.15.19:0/1504433304 wait complete. 2024-04-23T18:44:35.397 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:25 smithi155 ceph-mon[33468]: pgmap v577: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-23T18:44:35.444 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:27 smithi155 ceph-mon[33468]: pgmap v578: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-23T18:44:35.485 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:29 smithi155 ceph-mon[33468]: pgmap v579: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-23T18:44:35.518 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:30 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:44:35.551 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:31 smithi155 ceph-mon[33468]: pgmap v580: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-04-23T18:44:35.585 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:31 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:44:35.618 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:31 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:44:35.680 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:31 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:44:35.779 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:33 smithi155 ceph-mon[33468]: pgmap v581: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-23T18:44:35.818 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:25 smithi044 ceph-mon[32458]: pgmap v577: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-23T18:44:35.852 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:27 smithi044 ceph-mon[32458]: pgmap v578: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-23T18:44:35.963 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:29 smithi044 ceph-mon[32458]: pgmap v579: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-23T18:44:36.002 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:30 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:44:36.035 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:31 smithi044 ceph-mon[32458]: pgmap v580: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-04-23T18:44:36.068 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:31 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:44:36.110 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:31 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:44:36.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:31 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:44:36.177 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:33 smithi044 ceph-mon[32458]: pgmap v581: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-23T18:44:36.212 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:35 smithi155 ceph-mon[33468]: pgmap v582: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-04-23T18:44:36.294 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:35 smithi044 ceph-mon[32458]: pgmap v582: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-04-23T18:44:36.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:23 smithi019 ceph-mon[29215]: pgmap v511: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-23T18:44:36.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:25 smithi019 ceph-mon[29215]: pgmap v512: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-23T18:44:36.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:27 smithi019 ceph-mon[29215]: pgmap v513: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:36.486 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:28 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:44:36.519 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:29 smithi019 ceph-mon[29215]: pgmap v514: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:36.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:29 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:44:36.594 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:29 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:44:36.627 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:29 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:44:36.661 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:31 smithi019 ceph-mon[29215]: pgmap v515: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:36.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:33 smithi019 ceph-mon[29215]: pgmap v516: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:36.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:35 smithi019 ceph-mon[29215]: pgmap v517: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.857+0000 7f6106194640 1 -- 172.21.15.19:0/822923035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f610010c520 msgr2=0x7f610010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.857+0000 7f6106194640 1 --2- 172.21.15.19:0/822923035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f610010c520 0x7f610010e9c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009ca0 tx=0x7f60f802a9e0 comp rx=0 tx=0).stop 2024-04-23T18:44:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.857+0000 7f6106194640 1 -- 172.21.15.19:0/822923035 shutdown_connections 2024-04-23T18:44:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.857+0000 7f6106194640 1 --2- 172.21.15.19:0/822923035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f610010c520 0x7f610010e9c0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.857+0000 7f6106194640 1 --2- 172.21.15.19:0/822923035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6100103b00 0x7f610010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.857+0000 7f6106194640 1 --2- 172.21.15.19:0/822923035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61001031e0 0x7f61001035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.857+0000 7f6106194640 1 -- 172.21.15.19:0/822923035 >> 172.21.15.19:0/822923035 conn(0x7f6100077fd0 msgr2=0x7f61000783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.857+0000 7f6106194640 1 -- 172.21.15.19:0/822923035 shutdown_connections 2024-04-23T18:44:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.857+0000 7f6106194640 1 -- 172.21.15.19:0/822923035 wait complete. 2024-04-23T18:44:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f6106194640 1 Processor -- start 2024-04-23T18:44:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f6106194640 1 -- start start 2024-04-23T18:44:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f6106194640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61001031e0 0x7f6100198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f6106194640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6100103b00 0x7f6100198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f6106194640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f610010c520 0x7f610019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f6106194640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61001121b0 con 0x7f6100103b00 2024-04-23T18:44:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f6106194640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6100112030 con 0x7f610010c520 2024-04-23T18:44:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f6106194640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6100112330 con 0x7f61001031e0 2024-04-23T18:44:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f60f6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6100103b00 0x7f6100198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f60f6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6100103b00 0x7f6100198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48082/0 (socket says 172.21.15.19:48082) 2024-04-23T18:44:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f60f6ffd640 1 -- 172.21.15.19:0/3058908132 learned_addr learned my addr 172.21.15.19:0/3058908132 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61001031e0 0x7f6100198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.858+0000 7f60fffff640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f610010c520 0x7f610019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f60f6ffd640 1 -- 172.21.15.19:0/3058908132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61001031e0 msgr2=0x7f6100198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61001031e0 0x7f6100198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f60f6ffd640 1 -- 172.21.15.19:0/3058908132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f610010c520 msgr2=0x7f610019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f610010c520 0x7f610019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f60f6ffd640 1 -- 172.21.15.19:0/3058908132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610019d640 con 0x7f6100103b00 2024-04-23T18:44:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6100103b00 0x7f6100198b90 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d770 tx=0x7f60e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f60fd7fa640 1 -- 172.21.15.19:0/3058908132 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8017070 con 0x7f6100103b00 2024-04-23T18:44:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f60fd7fa640 1 -- 172.21.15.19:0/3058908132 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60e80054d0 con 0x7f6100103b00 2024-04-23T18:44:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f6106194640 1 -- 172.21.15.19:0/3058908132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610019d930 con 0x7f6100103b00 2024-04-23T18:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f60fd7fa640 1 -- 172.21.15.19:0/3058908132 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8004c30 con 0x7f6100103b00 2024-04-23T18:44:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.859+0000 7f6106194640 1 -- 172.21.15.19:0/3058908132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61001a51c0 con 0x7f6100103b00 2024-04-23T18:44:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f60fd7fa640 1 -- 172.21.15.19:0/3058908132 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f60e8004e10 con 0x7f6100103b00 2024-04-23T18:44:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f6108f88740 1 Processor -- start 2024-04-23T18:44:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f6108f88740 1 -- start start 2024-04-23T18:44:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f6108f88740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ec8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f6108f88740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f6108f88740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050ed5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f6108f88740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b050e2d970 con 0x55b050ec8620 2024-04-23T18:44:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f6108f88740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b050dca4a0 con 0x55b050ec8f40 2024-04-23T18:44:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f6108f88740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b050e29930 con 0x55b050ed3800 2024-04-23T18:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f60fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ec8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f60fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ec8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48090/0 (socket says 172.21.15.19:48090) 2024-04-23T18:44:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f60fffff640 1 -- 172.21.15.19:0/3595739203 learned_addr learned my addr 172.21.15.19:0/3595739203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.862+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/3595739203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050ed5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/3595739203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60fd7fa640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60ec0a3dc0 0x7f60ec0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60fffff640 1 -- 172.21.15.19:0/3595739203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 msgr2=0x55b050ed5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60fffff640 1 --2- 172.21.15.19:0/3595739203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050ed5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60fffff640 1 -- 172.21.15.19:0/3595739203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 msgr2=0x55b050ed32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60fffff640 1 --2- 172.21.15.19:0/3595739203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60fffff640 1 -- 172.21.15.19:0/3595739203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b050b40210 con 0x55b050ec8620 2024-04-23T18:44:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/3595739203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:44:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60fffff640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60ec0a3dc0 0x7f60ec0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60fd7fa640 1 -- 172.21.15.19:0/3058908132 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7f60e80d0110 con 0x7f6100103b00 2024-04-23T18:44:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60fffff640 1 --2- 172.21.15.19:0/3595739203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ec8a00 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f60f802ebc0 tx=0x7f60f802ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03127465551d3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60f5ffb640 1 -- 172.21.15.19:0/3595739203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8033040 con 0x55b050ec8620 2024-04-23T18:44:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60f5ffb640 1 -- 172.21.15.19:0/3595739203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f804a040 con 0x55b050ec8620 2024-04-23T18:44:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60fffff640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60ec0a3dc0 0x7f60ec0a6280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60f804c000 tx=0x7f60f8049000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.863+0000 7f60f5ffb640 1 -- 172.21.15.19:0/3595739203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8038450 con 0x55b050ec8620 2024-04-23T18:44:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.864+0000 7f6108f88740 1 -- 172.21.15.19:0/3595739203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 msgr2=0x55b050ec8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.864+0000 7f6108f88740 1 --2- 172.21.15.19:0/3595739203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ec8a00 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f60f802ebc0 tx=0x7f60f802ebf0 comp rx=0 tx=0).stop 2024-04-23T18:44:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.864+0000 7f6108f88740 1 -- 172.21.15.19:0/3595739203 shutdown_connections 2024-04-23T18:44:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.864+0000 7f6108f88740 1 --2- 172.21.15.19:0/3595739203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.864+0000 7f6108f88740 1 --2- 172.21.15.19:0/3595739203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050ed5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.864+0000 7f6108f88740 1 --2- 172.21.15.19:0/3595739203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ec8a00 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.864+0000 7f6108f88740 1 -- 172.21.15.19:0/3595739203 >> 172.21.15.19:0/3595739203 conn(0x55b050ec4b30 msgr2=0x55b050ec4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.864+0000 7f6108f88740 1 -- 172.21.15.19:0/3595739203 shutdown_connections 2024-04-23T18:44:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.864+0000 7f6108f88740 1 -- 172.21.15.19:0/3595739203 wait complete. 2024-04-23T18:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.864+0000 7f6108f88740 1 -- start start 2024-04-23T18:44:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f6108f88740 1 Processor -- start 2024-04-23T18:44:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f6108f88740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ed30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f6108f88740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f6108f88740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050ed08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f6108f88740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b050ed6430 con 0x55b050ec8620 2024-04-23T18:44:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f6108f88740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b050dc80b0 con 0x55b050ec8f40 2024-04-23T18:44:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f6108f88740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b050ed65b0 con 0x55b050ed3800 2024-04-23T18:44:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f6108f88740 10 client.0 _ll_register_callbacks cb 0x55b050db5d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60ff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ed30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60ff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ed30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48096/0 (socket says 172.21.15.19:48096) 2024-04-23T18:44:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60ff7fe640 1 -- 172.21.15.19:0/3603578828 learned_addr learned my addr 172.21.15.19:0/3603578828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60fffff640 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050ed08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60ff7fe640 1 -- 172.21.15.19:0/3603578828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 msgr2=0x55b050ed08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050ed08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60ff7fe640 1 -- 172.21.15.19:0/3603578828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 msgr2=0x55b050ed03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60ff7fe640 1 -- 172.21.15.19:0/3603578828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b050c0c620 con 0x55b050ec8620 2024-04-23T18:44:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60fffff640 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050ed08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:44:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.865+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ed30b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b840 tx=0x7f60f000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b4a84920590d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f60deffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:44:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f60deffd640 1 -- 172.21.15.19:0/3603578828 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f001b440 con 0x55b050ec8620 2024-04-23T18:44:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f60deffd640 1 -- 172.21.15.19:0/3603578828 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7f60f001b5e0 con 0x55b050ec8620 2024-04-23T18:44:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f6108f88740 1 -- 172.21.15.19:0/3603578828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f804a040 con 0x55b050ec8620 2024-04-23T18:44:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f60deffd640 1 -- 172.21.15.19:0/3603578828 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60f001b7e0 con 0x55b050ec8620 2024-04-23T18:44:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f6108f88740 10 client.17529 fetch_fsmap learned FSMap version 29 2024-04-23T18:44:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f6108f88740 1 -- 172.21.15.19:0/3603578828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b050ee2350 con 0x55b050ec8620 2024-04-23T18:44:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f60deffd640 1 -- 172.21.15.19:0/3603578828 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f804a040 con 0x55b050ec8620 2024-04-23T18:44:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f60deffd640 1 -- 172.21.15.19:0/3603578828 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f60f0019020 con 0x55b050ec8620 2024-04-23T18:44:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f6108f88740 10 client.17529 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:44:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f6108f88740 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f6108f88740 20 client.17529 populate_metadata read hostname 'smithi019' 2024-04-23T18:44:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.866+0000 7f6108f88740 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b050ee37c0 0x55b050ee5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.867+0000 7f6108f88740 4 client.17529 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:44:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.867+0000 7f6108f88740 1 -- 172.21.15.19:0/3603578828 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b050e3b390 con 0x55b050ee37c0 2024-04-23T18:44:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.867+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b050ee37c0 0x55b050ee5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.867+0000 7f60deffd640 10 client.17529 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.867+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b050ee37c0 0x55b050ee5bd0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c32d066876256 server_cookie=13332e833d9911dc in_seq=0 out_seq=0 2024-04-23T18:44:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.868+0000 7f60deffd640 1 -- 172.21.15.19:0/3603578828 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b050e3b390 con 0x55b050ee37c0 2024-04-23T18:44:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.868+0000 7f60deffd640 10 client.17529 handle_command_reply: tid=0 2024-04-23T18:44:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.868+0000 7f6108f88740 1 client.17529 shutdown 2024-04-23T18:44:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.868+0000 7f6108f88740 1 -- 172.21.15.19:0/3603578828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 msgr2=0x55b050ed30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.868+0000 7f6108f88740 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ed30b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b840 tx=0x7f60f000bd00 comp rx=0 tx=0).stop 2024-04-23T18:44:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.869+0000 7f6108f88740 1 -- 172.21.15.19:0/3603578828 shutdown_connections 2024-04-23T18:44:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.869+0000 7f6108f88740 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b050ee37c0 0x55b050ee5bd0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.869+0000 7f6108f88740 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.869+0000 7f6108f88740 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050ed08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.869+0000 7f6108f88740 1 --2- 172.21.15.19:0/3603578828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050ed30b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.869+0000 7f60deffd640 0 client.17529 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.869+0000 7f6108f88740 1 -- 172.21.15.19:0/3603578828 >> 172.21.15.19:0/3603578828 conn(0x55b050ec4b30 msgr2=0x55b050eca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.869+0000 7f6108f88740 1 -- 172.21.15.19:0/3603578828 shutdown_connections 2024-04-23T18:44:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.869+0000 7f6108f88740 1 -- 172.21.15.19:0/3603578828 wait complete. 2024-04-23T18:44:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.869+0000 7f6108f88740 20 client.17529 trim_cache size 0 max 16384 2024-04-23T18:44:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.883+0000 7f6108f88740 1 Processor -- start 2024-04-23T18:44:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f6108f88740 1 -- start start 2024-04-23T18:44:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f6108f88740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f6108f88740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f6108f88740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050eeb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f6108f88740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61001ada60 con 0x55b050ec8620 2024-04-23T18:44:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f6108f88740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f61001af5a0 con 0x55b050ec8f40 2024-04-23T18:44:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f6108f88740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f61001ae130 con 0x55b050ed3800 2024-04-23T18:44:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60ff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60ff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48104/0 (socket says 172.21.15.19:48104) 2024-04-23T18:44:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60ff7fe640 1 -- 172.21.15.19:0/1830747620 learned_addr learned my addr 172.21.15.19:0/1830747620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/1830747620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60ff7fe640 1 -- 172.21.15.19:0/1830747620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 msgr2=0x55b050eeb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/1830747620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050eeb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60ff7fe640 1 -- 172.21.15.19:0/1830747620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 msgr2=0x55b050ed2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/1830747620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60ff7fe640 1 -- 172.21.15.19:0/1830747620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b050ed0c40 con 0x55b050ec8620 2024-04-23T18:44:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/1830747620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/1830747620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edf2f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f60f0024bd0 tx=0x7f60f0003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80eff23a8e524436 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.884+0000 7f60f57fa640 1 -- 172.21.15.19:0/1830747620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f001b440 con 0x55b050ec8620 2024-04-23T18:44:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f60f57fa640 1 -- 172.21.15.19:0/1830747620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f001be30 con 0x55b050ec8620 2024-04-23T18:44:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f60f57fa640 1 -- 172.21.15.19:0/1830747620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00052a0 con 0x55b050ec8620 2024-04-23T18:44:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f6108f88740 1 -- 172.21.15.19:0/1830747620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 msgr2=0x55b050edf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f6108f88740 1 --2- 172.21.15.19:0/1830747620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edf2f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f60f0024bd0 tx=0x7f60f0003b00 comp rx=0 tx=0).stop 2024-04-23T18:44:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f6108f88740 1 -- 172.21.15.19:0/1830747620 shutdown_connections 2024-04-23T18:44:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f6108f88740 1 --2- 172.21.15.19:0/1830747620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ec8f40 0x55b050ed2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f6108f88740 1 --2- 172.21.15.19:0/1830747620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ed3800 0x55b050eeb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f6108f88740 1 --2- 172.21.15.19:0/1830747620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edf2f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f6108f88740 1 -- 172.21.15.19:0/1830747620 >> 172.21.15.19:0/1830747620 conn(0x55b050ec4b30 msgr2=0x55b050ecb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f6108f88740 1 -- 172.21.15.19:0/1830747620 shutdown_connections 2024-04-23T18:44:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f6108f88740 1 -- 172.21.15.19:0/1830747620 wait complete. 2024-04-23T18:44:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.885+0000 7f6108f88740 1 -- start start 2024-04-23T18:44:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f6108f88740 1 Processor -- start 2024-04-23T18:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f6108f88740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f6108f88740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ec8f40 0x55b050ed70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f6108f88740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ed3800 0x55b050ed77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f6108f88740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61001aeed0 con 0x55b050ec8620 2024-04-23T18:44:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f6108f88740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f61001ae800 con 0x55b050ed3800 2024-04-23T18:44:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f6108f88740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f61001aba60 con 0x55b050ec8f40 2024-04-23T18:44:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f6108f88740 10 client.0 _ll_register_callbacks cb 0x7f60f00041b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f60ff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f60ff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48116/0 (socket says 172.21.15.19:48116) 2024-04-23T18:44:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f60ff7fe640 1 -- 172.21.15.19:0/1576807748 learned_addr learned my addr 172.21.15.19:0/1576807748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f60ff7fe640 1 -- 172.21.15.19:0/1576807748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ec8f40 msgr2=0x55b050ed70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ec8f40 0x55b050ed70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ec8f40 0x55b050ed70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f60ff7fe640 1 -- 172.21.15.19:0/1576807748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ed3800 msgr2=0x55b050ed77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ed3800 0x55b050ed77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f60ff7fe640 1 -- 172.21.15.19:0/1576807748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b050f48d50 con 0x55b050ec8620 2024-04-23T18:44:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.886+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ec8f40 0x55b050ed70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f60ff7fe640 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edbd60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b840 tx=0x7f60f001b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16d710a2a7d7daa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:45.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f60deffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:44:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f60deffd640 1 -- 172.21.15.19:0/1576807748 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f003c060 con 0x55b050ec8620 2024-04-23T18:44:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f60deffd640 1 -- 172.21.15.19:0/1576807748 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7f60f001b980 con 0x55b050ec8620 2024-04-23T18:44:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f6108f88740 1 -- 172.21.15.19:0/1576807748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f001be30 con 0x55b050ec8620 2024-04-23T18:44:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f60deffd640 1 -- 172.21.15.19:0/1576807748 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60f001bb80 con 0x55b050ec8620 2024-04-23T18:44:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f6108f88740 10 client.17535 fetch_fsmap learned FSMap version 29 2024-04-23T18:44:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f6108f88740 1 -- 172.21.15.19:0/1576807748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b050f49e70 con 0x55b050ec8620 2024-04-23T18:44:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f60deffd640 1 -- 172.21.15.19:0/1576807748 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f001be30 con 0x55b050ec8620 2024-04-23T18:44:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f60deffd640 1 -- 172.21.15.19:0/1576807748 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f60f0019040 con 0x55b050ec8620 2024-04-23T18:44:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f6108f88740 10 client.17535 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:44:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f6108f88740 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f6108f88740 20 client.17535 populate_metadata read hostname 'smithi019' 2024-04-23T18:44:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f6108f88740 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b050f4a480 0x55b050f4c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:46.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f6108f88740 4 client.17535 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:44:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.887+0000 7f6108f88740 1 -- 172.21.15.19:0/1576807748 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b050e35200 con 0x55b050f4a480 2024-04-23T18:44:46.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.888+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b050f4a480 0x55b050f4c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.888+0000 7f60deffd640 10 client.17535 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.888+0000 7f60f6ffd640 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b050f4a480 0x55b050f4c890 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c709805765772f9 server_cookie=78810ddd5f681e61 in_seq=0 out_seq=0 2024-04-23T18:44:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f60deffd640 1 -- 172.21.15.19:0/1576807748 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b050e35200 con 0x55b050f4a480 2024-04-23T18:44:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f60deffd640 10 client.17535 handle_command_reply: tid=0 2024-04-23T18:44:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 client.17535 shutdown 2024-04-23T18:44:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 -- 172.21.15.19:0/1576807748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 msgr2=0x55b050edbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edbd60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b840 tx=0x7f60f001b6f0 comp rx=0 tx=0).stop 2024-04-23T18:44:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 -- 172.21.15.19:0/1576807748 shutdown_connections 2024-04-23T18:44:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55b050f4a480 0x55b050f4c890 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050ed3800 0x55b050ed77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f60deffd640 0 client.17535 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b050ec8f40 0x55b050ed70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 --2- 172.21.15.19:0/1576807748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b050ec8620 0x55b050edbd60 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 -- 172.21.15.19:0/1576807748 >> 172.21.15.19:0/1576807748 conn(0x55b050ec4b30 msgr2=0x55b050ecfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 -- 172.21.15.19:0/1576807748 shutdown_connections 2024-04-23T18:44:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 1 -- 172.21.15.19:0/1576807748 wait complete. 2024-04-23T18:44:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6108f88740 20 client.17535 trim_cache size 0 max 16384 2024-04-23T18:44:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:44:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:44:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:44:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:44:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6106194640 1 -- 172.21.15.19:0/3058908132 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60ec0a3dc0 msgr2=0x7f60ec0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6106194640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60ec0a3dc0 0x7f60ec0a6280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60f804c000 tx=0x7f60f8049000 comp rx=0 tx=0).stop 2024-04-23T18:44:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6106194640 1 -- 172.21.15.19:0/3058908132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6100103b00 msgr2=0x7f6100198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.893+0000 7f6106194640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6100103b00 0x7f6100198b90 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d770 tx=0x7f60e800dc30 comp rx=0 tx=0).stop 2024-04-23T18:44:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.894+0000 7f6106194640 1 -- 172.21.15.19:0/3058908132 shutdown_connections 2024-04-23T18:44:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.894+0000 7f6106194640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f60ec0a3dc0 0x7f60ec0a6280 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.894+0000 7f6106194640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f610010c520 0x7f610019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.894+0000 7f6106194640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6100103b00 0x7f6100198b90 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.894+0000 7f6106194640 1 --2- 172.21.15.19:0/3058908132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61001031e0 0x7f6100198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.894+0000 7f6106194640 1 -- 172.21.15.19:0/3058908132 >> 172.21.15.19:0/3058908132 conn(0x7f6100077fd0 msgr2=0x7f61000fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.894+0000 7f6106194640 1 -- 172.21.15.19:0/3058908132 shutdown_connections 2024-04-23T18:44:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:36.894+0000 7f6106194640 1 -- 172.21.15.19:0/3058908132 wait complete. 2024-04-23T18:44:48.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:37 smithi019 ceph-mon[29215]: pgmap v518: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:44:48.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:39 smithi019 ceph-mon[29215]: pgmap v519: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:44:48.811 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:41 smithi019 ceph-mon[29215]: pgmap v520: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:48.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:43 smithi019 ceph-mon[29215]: pgmap v521: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:48.911 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:45 smithi019 ceph-mon[29215]: pgmap v522: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:48.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:47 smithi019 ceph-mon[29215]: pgmap v523: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:49.015 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:37 smithi155 ceph-mon[33468]: pgmap v583: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:44:49.145 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:39 smithi155 ceph-mon[33468]: pgmap v584: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:44:49.195 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:41 smithi155 ceph-mon[33468]: pgmap v585: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:44:49.245 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:43 smithi155 ceph-mon[33468]: pgmap v586: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:49.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:45 smithi155 ceph-mon[33468]: pgmap v587: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:49.345 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:47 smithi155 ceph-mon[33468]: pgmap v588: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:49.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:37 smithi044 ceph-mon[32458]: pgmap v583: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:44:49.415 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:39 smithi044 ceph-mon[32458]: pgmap v584: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:44:49.479 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:41 smithi044 ceph-mon[32458]: pgmap v585: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:44:49.515 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:43 smithi044 ceph-mon[32458]: pgmap v586: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:49.551 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:45 smithi044 ceph-mon[32458]: pgmap v587: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:49.582 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:47 smithi044 ceph-mon[32458]: pgmap v588: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.993+0000 7f7688ecb640 1 -- 172.21.15.19:0/3059306328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7684101a90 msgr2=0x7f7684105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.993+0000 7f7688ecb640 1 --2- 172.21.15.19:0/3059306328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7684101a90 0x7f7684105ad0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7678009900 tx=0x7f767802a410 comp rx=0 tx=0).stop 2024-04-23T18:44:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.993+0000 7f7688ecb640 1 -- 172.21.15.19:0/3059306328 shutdown_connections 2024-04-23T18:44:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.993+0000 7f7688ecb640 1 --2- 172.21.15.19:0/3059306328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76841061b0 0x7f768410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.993+0000 7f7688ecb640 1 --2- 172.21.15.19:0/3059306328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7684101a90 0x7f7684105ad0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.993+0000 7f7688ecb640 1 --2- 172.21.15.19:0/3059306328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76841010e0 0x7f76841014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.993+0000 7f7688ecb640 1 -- 172.21.15.19:0/3059306328 >> 172.21.15.19:0/3059306328 conn(0x7f76840771d0 msgr2=0x7f76840775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.993+0000 7f7688ecb640 1 -- 172.21.15.19:0/3059306328 shutdown_connections 2024-04-23T18:44:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.993+0000 7f7688ecb640 1 -- 172.21.15.19:0/3059306328 wait complete. 2024-04-23T18:44:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.994+0000 7f7688ecb640 1 Processor -- start 2024-04-23T18:44:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.994+0000 7f7688ecb640 1 -- start start 2024-04-23T18:44:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.994+0000 7f7688ecb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76841010e0 0x7f7684198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.994+0000 7f7688ecb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76841061b0 0x7f7684198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.994+0000 7f7688ecb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768419cf00 0x7f768419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.994+0000 7f7688ecb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76841110d0 con 0x7f76841061b0 2024-04-23T18:44:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.994+0000 7f7688ecb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7684110f50 con 0x7f76841010e0 2024-04-23T18:44:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.994+0000 7f7688ecb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7684111250 con 0x7f768419cf00 2024-04-23T18:44:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7681d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76841061b0 0x7f7684198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7681d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76841061b0 0x7f7684198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35410/0 (socket says 172.21.15.19:35410) 2024-04-23T18:44:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7681d9b640 1 -- 172.21.15.19:0/799952160 learned_addr learned my addr 172.21.15.19:0/799952160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f768259c640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76841010e0 0x7f7684198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7682d9d640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768419cf00 0x7f768419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7681d9b640 1 -- 172.21.15.19:0/799952160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768419cf00 msgr2=0x7f768419d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7681d9b640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768419cf00 0x7f768419d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7681d9b640 1 -- 172.21.15.19:0/799952160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76841010e0 msgr2=0x7f7684198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7681d9b640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76841010e0 0x7f7684198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7681d9b640 1 -- 172.21.15.19:0/799952160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768419d9f0 con 0x7f76841061b0 2024-04-23T18:44:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7681d9b640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76841061b0 0x7f7684198b80 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7678031a90 tx=0x7f7678004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f76737fe640 1 -- 172.21.15.19:0/799952160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767803f030 con 0x7f76841061b0 2024-04-23T18:44:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f76737fe640 1 -- 172.21.15.19:0/799952160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7678004490 con 0x7f76841061b0 2024-04-23T18:44:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7688ecb640 1 -- 172.21.15.19:0/799952160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768419dc80 con 0x7f76841061b0 2024-04-23T18:44:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f76737fe640 1 -- 172.21.15.19:0/799952160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678035070 con 0x7f76841061b0 2024-04-23T18:44:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.995+0000 7f7688ecb640 1 -- 172.21.15.19:0/799952160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76841a5440 con 0x7f76841061b0 2024-04-23T18:44:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.998+0000 7f76737fe640 1 -- 172.21.15.19:0/799952160 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7678042030 con 0x7f76841061b0 2024-04-23T18:44:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.998+0000 7f768bbdb740 1 Processor -- start 2024-04-23T18:44:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.998+0000 7f768bbdb740 1 -- start start 2024-04-23T18:44:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560caf9ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafabbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768bbdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5560caf03970 con 0x5560cafa9800 2024-04-23T18:44:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768bbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5560caea04a0 con 0x5560caf9ef40 2024-04-23T18:44:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768bbdb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5560caeff930 con 0x5560caf9e620 2024-04-23T18:44:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f7681d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafabbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f7681d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafabbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35420/0 (socket says 172.21.15.19:35420) 2024-04-23T18:44:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f7681d9b640 1 -- 172.21.15.19:0/2141039418 learned_addr learned my addr 172.21.15.19:0/2141039418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f7682d9d640 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560caf9ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f76737fe640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f766c0a3dc0 0x7f766c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768259c640 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f7682d9d640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f766c0a3dc0 0x7f766c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f76737fe640 1 -- 172.21.15.19:0/799952160 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7f76780f0dd0 con 0x7f76841061b0 2024-04-23T18:44:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768259c640 1 -- 172.21.15.19:0/2141039418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 msgr2=0x5560caf9ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768259c640 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560caf9ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768259c640 1 -- 172.21.15.19:0/2141039418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 msgr2=0x5560cafabbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768259c640 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafabbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f768259c640 1 -- 172.21.15.19:0/2141039418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560cac16210 con 0x5560caf9ef40 2024-04-23T18:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f7682d9d640 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560caf9ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:44:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:49.999+0000 7f7681d9b640 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafabbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:44:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.000+0000 7f768259c640 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa92c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f767400abf0 tx=0x7f767400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed07143da36b178b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.000+0000 7f7682d9d640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f766c0a3dc0 0x7f766c0a6280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f767c0128c0 tx=0x7f767c004ab0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.000+0000 7f7670ff9640 1 -- 172.21.15.19:0/2141039418 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674005220 con 0x5560caf9ef40 2024-04-23T18:44:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.000+0000 7f7670ff9640 1 -- 172.21.15.19:0/2141039418 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76740047a0 con 0x5560caf9ef40 2024-04-23T18:44:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.000+0000 7f7670ff9640 1 -- 172.21.15.19:0/2141039418 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76740186e0 con 0x5560caf9ef40 2024-04-23T18:44:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 -- 172.21.15.19:0/2141039418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 msgr2=0x5560cafa92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa92c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f767400abf0 tx=0x7f767400f790 comp rx=0 tx=0).stop 2024-04-23T18:44:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 -- 172.21.15.19:0/2141039418 shutdown_connections 2024-04-23T18:44:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa92c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafabbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 --2- 172.21.15.19:0/2141039418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560caf9ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 -- 172.21.15.19:0/2141039418 >> 172.21.15.19:0/2141039418 conn(0x5560caf9ab30 msgr2=0x5560caf9af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 -- 172.21.15.19:0/2141039418 shutdown_connections 2024-04-23T18:44:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 -- 172.21.15.19:0/2141039418 wait complete. 2024-04-23T18:44:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 -- start start 2024-04-23T18:44:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 Processor -- start 2024-04-23T18:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560cafa9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafa6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5560cafac430 con 0x5560cafa9800 2024-04-23T18:44:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5560cae9e0b0 con 0x5560caf9ef40 2024-04-23T18:44:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5560cafac5b0 con 0x5560caf9e620 2024-04-23T18:44:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.001+0000 7f768bbdb740 10 client.0 _ll_register_callbacks cb 0x5560cae8bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7682d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafa6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7681d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7682d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafa6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35432/0 (socket says 172.21.15.19:35432) 2024-04-23T18:44:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7682d9d640 1 -- 172.21.15.19:0/2553461048 learned_addr learned my addr 172.21.15.19:0/2553461048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f768259c640 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560cafa9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7681d9b640 1 -- 172.21.15.19:0/2553461048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 msgr2=0x5560cafa9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7681d9b640 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560cafa9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7681d9b640 1 -- 172.21.15.19:0/2553461048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 msgr2=0x5560cafa6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7681d9b640 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafa6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7681d9b640 1 -- 172.21.15.19:0/2553461048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560cace2620 con 0x5560caf9ef40 2024-04-23T18:44:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7682d9d640 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafa6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:44:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7681d9b640 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa6350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f76780bcca0 tx=0x7f767803b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba54100f60875f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-23T18:44:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7669ffb640 1 -- 172.21.15.19:0/2553461048 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678049070 con 0x5560caf9ef40 2024-04-23T18:44:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7669ffb640 1 -- 172.21.15.19:0/2553461048 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7f76780b8a60 con 0x5560caf9ef40 2024-04-23T18:44:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f768bbdb740 1 -- 172.21.15.19:0/2553461048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76740047a0 con 0x5560caf9ef40 2024-04-23T18:44:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f7669ffb640 1 -- 172.21.15.19:0/2553461048 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f76780bd410 con 0x5560caf9ef40 2024-04-23T18:44:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.002+0000 7f768bbdb740 10 client.26687 fetch_fsmap learned FSMap version 29 2024-04-23T18:44:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.003+0000 7f768bbdb740 1 -- 172.21.15.19:0/2553461048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560cafb8290 con 0x5560caf9ef40 2024-04-23T18:44:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.003+0000 7f7669ffb640 1 -- 172.21.15.19:0/2553461048 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76740047a0 con 0x5560caf9ef40 2024-04-23T18:44:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.003+0000 7f7669ffb640 1 -- 172.21.15.19:0/2553461048 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f76780ee740 con 0x5560caf9ef40 2024-04-23T18:44:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.003+0000 7f768bbdb740 10 client.26687 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:44:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.003+0000 7f768bbdb740 10 client.26687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.003+0000 7f768bbdb740 20 client.26687 populate_metadata read hostname 'smithi019' 2024-04-23T18:44:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.003+0000 7f768bbdb740 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5560cafb9810 0x5560cafbbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.003+0000 7f768bbdb740 4 client.26687 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:44:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.003+0000 7f768bbdb740 1 -- 172.21.15.19:0/2553461048 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560caf11390 con 0x5560cafb9810 2024-04-23T18:44:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.003+0000 7f768259c640 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5560cafb9810 0x5560cafbbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f768259c640 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5560cafb9810 0x5560cafbbc30 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=982f024c8bc63742 server_cookie=60e4f8812d5b83ec in_seq=0 out_seq=0 2024-04-23T18:44:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f7669ffb640 10 client.26687 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f7669ffb640 1 -- 172.21.15.19:0/2553461048 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560caf11390 con 0x5560cafb9810 2024-04-23T18:44:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f7669ffb640 10 client.26687 handle_command_reply: tid=0 2024-04-23T18:44:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f768bbdb740 1 client.26687 shutdown 2024-04-23T18:44:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f768bbdb740 1 -- 172.21.15.19:0/2553461048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 msgr2=0x5560cafa6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f768bbdb740 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa6350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f76780bcca0 tx=0x7f767803b6b0 comp rx=0 tx=0).stop 2024-04-23T18:44:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f768bbdb740 1 -- 172.21.15.19:0/2553461048 shutdown_connections 2024-04-23T18:44:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f768bbdb740 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5560cafb9810 0x5560cafbbc30 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f768bbdb740 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa6350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f768bbdb740 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafa6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f768bbdb740 1 --2- 172.21.15.19:0/2553461048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560cafa9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f7669ffb640 0 client.26687 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.004+0000 7f768bbdb740 1 -- 172.21.15.19:0/2553461048 >> 172.21.15.19:0/2553461048 conn(0x5560caf9ab30 msgr2=0x5560cafa0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.005+0000 7f768bbdb740 1 -- 172.21.15.19:0/2553461048 shutdown_connections 2024-04-23T18:44:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.005+0000 7f768bbdb740 1 -- 172.21.15.19:0/2553461048 wait complete. 2024-04-23T18:44:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.005+0000 7f768bbdb740 20 client.26687 trim_cache size 0 max 16384 2024-04-23T18:44:54.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768bbdb740 1 Processor -- start 2024-04-23T18:44:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768bbdb740 1 -- start start 2024-04-23T18:44:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560caf9e620 0x5560cafb52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cafa9800 0x5560cafc1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768bbdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76841ada00 con 0x5560caf9e620 2024-04-23T18:44:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768bbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76841af540 con 0x5560caf9ef40 2024-04-23T18:44:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768bbdb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76841ae0d0 con 0x5560cafa9800 2024-04-23T18:44:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560caf9e620 0x5560cafb52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560caf9e620 0x5560cafb52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35442/0 (socket says 172.21.15.19:35442) 2024-04-23T18:44:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768259c640 1 -- 172.21.15.19:0/1186328754 learned_addr learned my addr 172.21.15.19:0/1186328754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f7681d9b640 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f7682d9d640 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cafa9800 0x5560cafc1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768259c640 1 -- 172.21.15.19:0/1186328754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cafa9800 msgr2=0x5560cafc1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768259c640 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cafa9800 0x5560cafc1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768259c640 1 -- 172.21.15.19:0/1186328754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 msgr2=0x5560cafa8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768259c640 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768259c640 1 -- 172.21.15.19:0/1186328754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560caf9ed70 con 0x5560caf9e620 2024-04-23T18:44:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f7681d9b640 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f7682d9d640 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cafa9800 0x5560cafc1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.020+0000 7f768259c640 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560caf9e620 0x5560cafb52f0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f767400abf0 tx=0x7f767400ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be45d4eebc4d149 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f766bfff640 1 -- 172.21.15.19:0/1186328754 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767400a7a0 con 0x5560caf9e620 2024-04-23T18:44:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f766bfff640 1 -- 172.21.15.19:0/1186328754 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7674004660 con 0x5560caf9e620 2024-04-23T18:44:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f766bfff640 1 -- 172.21.15.19:0/1186328754 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674005160 con 0x5560caf9e620 2024-04-23T18:44:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f768bbdb740 1 -- 172.21.15.19:0/1186328754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560caf9e620 msgr2=0x5560cafb52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f768bbdb740 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560caf9e620 0x5560cafb52f0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f767400abf0 tx=0x7f767400ffc0 comp rx=0 tx=0).stop 2024-04-23T18:44:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f768bbdb740 1 -- 172.21.15.19:0/1186328754 shutdown_connections 2024-04-23T18:44:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f768bbdb740 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafa8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f768bbdb740 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cafa9800 0x5560cafc1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f768bbdb740 1 --2- 172.21.15.19:0/1186328754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560caf9e620 0x5560cafb52f0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f768bbdb740 1 -- 172.21.15.19:0/1186328754 >> 172.21.15.19:0/1186328754 conn(0x5560caf9ab30 msgr2=0x5560cafa6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f768bbdb740 1 -- 172.21.15.19:0/1186328754 shutdown_connections 2024-04-23T18:44:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f768bbdb740 1 -- 172.21.15.19:0/1186328754 wait complete. 2024-04-23T18:44:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.021+0000 7f768bbdb740 1 -- start start 2024-04-23T18:44:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f768bbdb740 1 Processor -- start 2024-04-23T18:44:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560cafb1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafacf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f768bbdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:56.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f768bbdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76841aee70 con 0x5560cafa9800 2024-04-23T18:44:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f768bbdb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76841ae7a0 con 0x5560caf9ef40 2024-04-23T18:44:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f768bbdb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76841aba00 con 0x5560caf9e620 2024-04-23T18:44:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f768bbdb740 10 client.0 _ll_register_callbacks cb 0x7f76780ee460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f7682d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f7682d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafad440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35456/0 (socket says 172.21.15.19:35456) 2024-04-23T18:44:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f7682d9d640 1 -- 172.21.15.19:0/227987728 learned_addr learned my addr 172.21.15.19:0/227987728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f768259c640 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560cafb1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f7681d9b640 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafacf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f7682d9d640 1 -- 172.21.15.19:0/227987728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 msgr2=0x5560cafb1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f7682d9d640 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560cafb1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f7682d9d640 1 -- 172.21.15.19:0/227987728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 msgr2=0x5560cafacf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f7682d9d640 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafacf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f7682d9d640 1 -- 172.21.15.19:0/227987728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560cafadc50 con 0x5560cafa9800 2024-04-23T18:44:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.022+0000 7f7681d9b640 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafacf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:44:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f7682d9d640 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafad440 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f767c013010 tx=0x7f767c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189ca05f4357951 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f7669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:44:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f7669ffb640 1 -- 172.21.15.19:0/227987728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c018070 con 0x5560cafa9800 2024-04-23T18:44:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f7669ffb640 1 -- 172.21.15.19:0/227987728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7f767c021c20 con 0x5560cafa9800 2024-04-23T18:44:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f768bbdb740 1 -- 172.21.15.19:0/227987728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7674004660 con 0x5560cafa9800 2024-04-23T18:44:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f7669ffb640 1 -- 172.21.15.19:0/227987728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f767c021e20 con 0x5560cafa9800 2024-04-23T18:44:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f768bbdb740 10 client.17553 fetch_fsmap learned FSMap version 29 2024-04-23T18:44:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f768bbdb740 1 -- 172.21.15.19:0/227987728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560cb01fd60 con 0x5560cafa9800 2024-04-23T18:44:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f7669ffb640 1 -- 172.21.15.19:0/227987728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7674004660 con 0x5560cafa9800 2024-04-23T18:44:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f7669ffb640 1 -- 172.21.15.19:0/227987728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f767c01b030 con 0x5560cafa9800 2024-04-23T18:44:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f768bbdb740 10 client.17553 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:44:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f768bbdb740 10 client.17553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f768bbdb740 20 client.17553 populate_metadata read hostname 'smithi019' 2024-04-23T18:44:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f768bbdb740 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5560cb0204e0 0x5560cb022900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f768bbdb740 4 client.17553 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:44:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.023+0000 7f768bbdb740 1 -- 172.21.15.19:0/227987728 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560caf0b200 con 0x5560cb0204e0 2024-04-23T18:44:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.024+0000 7f768259c640 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5560cb0204e0 0x5560cb022900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.024+0000 7f7669ffb640 10 client.17553 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.024+0000 7f768259c640 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5560cb0204e0 0x5560cb022900 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9356c61bfeafa08 server_cookie=b84646083a29622b in_seq=0 out_seq=0 2024-04-23T18:44:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f7669ffb640 1 -- 172.21.15.19:0/227987728 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560caf0b200 con 0x5560cb0204e0 2024-04-23T18:44:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f7669ffb640 10 client.17553 handle_command_reply: tid=0 2024-04-23T18:44:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 client.17553 shutdown 2024-04-23T18:44:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 -- 172.21.15.19:0/227987728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 msgr2=0x5560cafad440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafad440 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f767c013010 tx=0x7f767c004a40 comp rx=0 tx=0).stop 2024-04-23T18:44:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 -- 172.21.15.19:0/227987728 shutdown_connections 2024-04-23T18:44:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5560cb0204e0 0x5560cb022900 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560caf9ef40 0x5560cafacf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f7669ffb640 0 client.17553 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:44:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560cafa9800 0x5560cafad440 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 --2- 172.21.15.19:0/227987728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560caf9e620 0x5560cafb1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 -- 172.21.15.19:0/227987728 >> 172.21.15.19:0/227987728 conn(0x5560caf9ab30 msgr2=0x5560caf9ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 -- 172.21.15.19:0/227987728 shutdown_connections 2024-04-23T18:44:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 1 -- 172.21.15.19:0/227987728 wait complete. 2024-04-23T18:44:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f768bbdb740 20 client.17553 trim_cache size 0 max 16384 2024-04-23T18:44:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:44:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:44:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f7688ecb640 1 -- 172.21.15.19:0/799952160 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f766c0a3dc0 msgr2=0x7f766c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.029+0000 7f7688ecb640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f766c0a3dc0 0x7f766c0a6280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f767c0128c0 tx=0x7f767c004ab0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.030+0000 7f7688ecb640 1 -- 172.21.15.19:0/799952160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76841061b0 msgr2=0x7f7684198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.030+0000 7f7688ecb640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76841061b0 0x7f7684198b80 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7678031a90 tx=0x7f7678004270 comp rx=0 tx=0).stop 2024-04-23T18:44:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.030+0000 7f7688ecb640 1 -- 172.21.15.19:0/799952160 shutdown_connections 2024-04-23T18:44:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.030+0000 7f7688ecb640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f766c0a3dc0 0x7f766c0a6280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.030+0000 7f7688ecb640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768419cf00 0x7f768419d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.030+0000 7f7688ecb640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76841061b0 0x7f7684198b80 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.030+0000 7f7688ecb640 1 --2- 172.21.15.19:0/799952160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76841010e0 0x7f7684198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.030+0000 7f7688ecb640 1 -- 172.21.15.19:0/799952160 >> 172.21.15.19:0/799952160 conn(0x7f76840771d0 msgr2=0x7f768406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.030+0000 7f7688ecb640 1 -- 172.21.15.19:0/799952160 shutdown_connections 2024-04-23T18:44:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:42:50.030+0000 7f7688ecb640 1 -- 172.21.15.19:0/799952160 wait complete. 2024-04-23T18:44:58.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:49 smithi019 ceph-mon[29215]: pgmap v524: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:58.402 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:51 smithi019 ceph-mon[29215]: pgmap v525: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:58.410 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:53 smithi019 ceph-mon[29215]: pgmap v526: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:44:58.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:55 smithi019 ceph-mon[29215]: pgmap v527: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:58.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:57 smithi019 ceph-mon[29215]: pgmap v528: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:58.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:42:59 smithi019 ceph-mon[29215]: pgmap v529: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:58.497 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:01 smithi019 ceph-mon[29215]: pgmap v530: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:58.512 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:49 smithi155 ceph-mon[33468]: pgmap v589: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:58.522 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:51 smithi155 ceph-mon[33468]: pgmap v590: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:58.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:53 smithi155 ceph-mon[33468]: pgmap v591: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:44:58.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:55 smithi155 ceph-mon[33468]: pgmap v592: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:58.549 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:57 smithi155 ceph-mon[33468]: pgmap v593: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:58.556 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:49 smithi044 ceph-mon[32458]: pgmap v589: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:58.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:51 smithi044 ceph-mon[32458]: pgmap v590: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:58.570 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:53 smithi044 ceph-mon[32458]: pgmap v591: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:44:58.617 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:55 smithi044 ceph-mon[32458]: pgmap v592: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:44:58.684 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:57 smithi044 ceph-mon[32458]: pgmap v593: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:44:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.128+0000 7f991eefa640 1 -- 172.21.15.19:0/1238462546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9918104d50 msgr2=0x7f9918108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.128+0000 7f991eefa640 1 --2- 172.21.15.19:0/1238462546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9918104d50 0x7f9918108d90 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9908009900 tx=0x7f990802a3f0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.128+0000 7f991eefa640 1 -- 172.21.15.19:0/1238462546 shutdown_connections 2024-04-23T18:44:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 --2- 172.21.15.19:0/1238462546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918109470 0x7f9918111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 --2- 172.21.15.19:0/1238462546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9918104d50 0x7f9918108d90 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 --2- 172.21.15.19:0/1238462546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99181043a0 0x7f9918104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 -- 172.21.15.19:0/1238462546 >> 172.21.15.19:0/1238462546 conn(0x7f99180ffc40 msgr2=0x7f9918102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 -- 172.21.15.19:0/1238462546 shutdown_connections 2024-04-23T18:44:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 -- 172.21.15.19:0/1238462546 wait complete. 2024-04-23T18:44:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 Processor -- start 2024-04-23T18:44:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 -- start start 2024-04-23T18:44:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99181043a0 0x7f99181a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9918109470 0x7f99181a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991819a690 0x7f991819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99181143b0 con 0x7f9918109470 2024-04-23T18:44:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9918114230 con 0x7f99181043a0 2024-04-23T18:44:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.129+0000 7f991eefa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9918114530 con 0x7f991819a690 2024-04-23T18:44:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9918109470 0x7f99181a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9918109470 0x7f99181a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41286/0 (socket says 172.21.15.19:41286) 2024-04-23T18:44:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990ffff640 1 -- 172.21.15.19:0/3308752478 learned_addr learned my addr 172.21.15.19:0/3308752478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f991d497640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991819a690 0x7f991819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f991cc96640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99181043a0 0x7f99181a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990ffff640 1 -- 172.21.15.19:0/3308752478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991819a690 msgr2=0x7f991819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990ffff640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991819a690 0x7f991819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990ffff640 1 -- 172.21.15.19:0/3308752478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99181043a0 msgr2=0x7f99181a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990ffff640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99181043a0 0x7f99181a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990ffff640 1 -- 172.21.15.19:0/3308752478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991819b290 con 0x7f9918109470 2024-04-23T18:44:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f991d497640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991819a690 0x7f991819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:44:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990ffff640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9918109470 0x7f99181a0b10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9908032fd0 tx=0x7f9908004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990dffb640 1 -- 172.21.15.19:0/3308752478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990803f030 con 0x7f9918109470 2024-04-23T18:44:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f991eefa640 1 -- 172.21.15.19:0/3308752478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991819b520 con 0x7f9918109470 2024-04-23T18:44:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990dffb640 1 -- 172.21.15.19:0/3308752478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9908004490 con 0x7f9918109470 2024-04-23T18:44:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.130+0000 7f990dffb640 1 -- 172.21.15.19:0/3308752478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908035070 con 0x7f9918109470 2024-04-23T18:44:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.131+0000 7f991eefa640 1 -- 172.21.15.19:0/3308752478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99181a7660 con 0x7f9918109470 2024-04-23T18:44:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.133+0000 7f990dffb640 1 -- 172.21.15.19:0/3308752478 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9908042030 con 0x7f9918109470 2024-04-23T18:44:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.133+0000 7f9921c14740 1 Processor -- start 2024-04-23T18:44:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.133+0000 7f9921c14740 1 -- start start 2024-04-23T18:44:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f9921c14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37f620 0x557d6b37fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f9921c14740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b37ff40 0x557d6b38a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f9921c14740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b38a800 0x557d6b38cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f9921c14740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d6b2e4970 con 0x557d6b38a800 2024-04-23T18:44:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f9921c14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557d6b2814a0 con 0x557d6b37f620 2024-04-23T18:44:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f9921c14740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557d6b2e0930 con 0x557d6b37ff40 2024-04-23T18:44:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990dffb640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f99000a3dc0 0x7f99000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b38a800 0x557d6b38cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f991d497640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f99000a3dc0 0x7f99000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f991cc96640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b37ff40 0x557d6b38a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b38a800 0x557d6b38cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41288/0 (socket says 172.21.15.19:41288) 2024-04-23T18:44:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f991d497640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37f620 0x557d6b37fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990ffff640 1 -- 172.21.15.19:0/1605621133 learned_addr learned my addr 172.21.15.19:0/1605621133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:44:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990dffb640 1 -- 172.21.15.19:0/3308752478 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7f99080f0dc0 con 0x7f9918109470 2024-04-23T18:44:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990ffff640 1 -- 172.21.15.19:0/1605621133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b37ff40 msgr2=0x557d6b38a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990ffff640 1 --2- 172.21.15.19:0/1605621133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b37ff40 0x557d6b38a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990ffff640 1 -- 172.21.15.19:0/1605621133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37f620 msgr2=0x557d6b37fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990ffff640 1 --2- 172.21.15.19:0/1605621133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37f620 0x557d6b37fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990ffff640 1 -- 172.21.15.19:0/1605621133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6aff7210 con 0x557d6b38a800 2024-04-23T18:44:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f991d497640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f99000a3dc0 0x7f99000a6280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9910005cc0 tx=0x7f9910005c50 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.134+0000 7f990ffff640 1 --2- 172.21.15.19:0/1605621133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b38a800 0x557d6b38cbf0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f99080b96d0 tx=0x7f990808f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e1406aed150fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f98feffd640 1 -- 172.21.15.19:0/1605621133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908049070 con 0x557d6b38a800 2024-04-23T18:44:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f98feffd640 1 -- 172.21.15.19:0/1605621133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9908093e50 con 0x557d6b38a800 2024-04-23T18:44:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f98feffd640 1 -- 172.21.15.19:0/1605621133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99080c10b0 con 0x557d6b38a800 2024-04-23T18:44:59.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f9921c14740 1 -- 172.21.15.19:0/1605621133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b38a800 msgr2=0x557d6b38cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f9921c14740 1 --2- 172.21.15.19:0/1605621133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b38a800 0x557d6b38cbf0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f99080b96d0 tx=0x7f990808f7f0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f9921c14740 1 -- 172.21.15.19:0/1605621133 shutdown_connections 2024-04-23T18:44:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f9921c14740 1 --2- 172.21.15.19:0/1605621133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b38a800 0x557d6b38cbf0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f9921c14740 1 --2- 172.21.15.19:0/1605621133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b37ff40 0x557d6b38a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f9921c14740 1 --2- 172.21.15.19:0/1605621133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37f620 0x557d6b37fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f9921c14740 1 -- 172.21.15.19:0/1605621133 >> 172.21.15.19:0/1605621133 conn(0x557d6b37bb30 msgr2=0x557d6b37bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f9921c14740 1 -- 172.21.15.19:0/1605621133 shutdown_connections 2024-04-23T18:44:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.135+0000 7f9921c14740 1 -- 172.21.15.19:0/1605621133 wait complete. 2024-04-23T18:44:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f9921c14740 1 -- start start 2024-04-23T18:44:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f9921c14740 1 Processor -- start 2024-04-23T18:44:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f9921c14740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b38a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f9921c14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b3874f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f9921c14740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b387b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f9921c14740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d6b2e4970 con 0x557d6b37f620 2024-04-23T18:44:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f9921c14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557d6b27f0b0 con 0x557d6b37ff40 2024-04-23T18:44:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f9921c14740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557d6b38d1f0 con 0x557d6b38a800 2024-04-23T18:44:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f9921c14740 10 client.0 _ll_register_callbacks cb 0x557d6b26cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f991cc96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b38a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.136+0000 7f991cc96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b38a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41294/0 (socket says 172.21.15.19:41294) 2024-04-23T18:44:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f991cc96640 1 -- 172.21.15.19:0/2174131036 learned_addr learned my addr 172.21.15.19:0/2174131036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f991d497640 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b387b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f990ffff640 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b3874f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f991cc96640 1 -- 172.21.15.19:0/2174131036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 msgr2=0x557d6b387b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f991cc96640 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b387b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f991cc96640 1 -- 172.21.15.19:0/2174131036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 msgr2=0x557d6b3874f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f991cc96640 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b3874f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f991cc96640 1 -- 172.21.15.19:0/2174131036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6b0c3620 con 0x557d6b37f620 2024-04-23T18:45:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f990ffff640 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b3874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f991cc96640 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b38a190 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f991400bf00 tx=0x7f9914012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcad72b94be775a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f98f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f98f7fff640 1 -- 172.21.15.19:0/2174131036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991401f060 con 0x557d6b37f620 2024-04-23T18:45:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f98f7fff640 1 -- 172.21.15.19:0/2174131036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7f99140108a0 con 0x557d6b37f620 2024-04-23T18:45:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f9921c14740 1 -- 172.21.15.19:0/2174131036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9908093e50 con 0x557d6b37f620 2024-04-23T18:45:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f98f7fff640 1 -- 172.21.15.19:0/2174131036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9914010aa0 con 0x557d6b37f620 2024-04-23T18:45:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f9921c14740 10 client.17565 fetch_fsmap learned FSMap version 29 2024-04-23T18:45:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.137+0000 7f9921c14740 1 -- 172.21.15.19:0/2174131036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557d6b399080 con 0x557d6b37f620 2024-04-23T18:45:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.138+0000 7f98f7fff640 1 -- 172.21.15.19:0/2174131036 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9908093e50 con 0x557d6b37f620 2024-04-23T18:45:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.138+0000 7f98f7fff640 1 -- 172.21.15.19:0/2174131036 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9914019030 con 0x557d6b37f620 2024-04-23T18:45:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.138+0000 7f9921c14740 10 client.17565 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:45:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.138+0000 7f9921c14740 10 client.17565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.138+0000 7f9921c14740 20 client.17565 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.138+0000 7f9921c14740 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557d6b39a5e0 0x557d6b39ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.138+0000 7f9921c14740 4 client.17565 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.138+0000 7f9921c14740 1 -- 172.21.15.19:0/2174131036 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d6b2f2390 con 0x557d6b39a5e0 2024-04-23T18:45:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.138+0000 7f990ffff640 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557d6b39a5e0 0x557d6b39ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.139+0000 7f990ffff640 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557d6b39a5e0 0x557d6b39ca80 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b96167ba6e64ea server_cookie=78ce686e8b1c7f9a in_seq=0 out_seq=0 2024-04-23T18:45:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.139+0000 7f98f7fff640 10 client.17565 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.139+0000 7f98f7fff640 1 -- 172.21.15.19:0/2174131036 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d6b2f2390 con 0x557d6b39a5e0 2024-04-23T18:45:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.139+0000 7f98f7fff640 10 client.17565 handle_command_reply: tid=0 2024-04-23T18:45:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 client.17565 shutdown 2024-04-23T18:45:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 -- 172.21.15.19:0/2174131036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 msgr2=0x557d6b38a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b38a190 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f991400bf00 tx=0x7f9914012820 comp rx=0 tx=0).stop 2024-04-23T18:45:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 -- 172.21.15.19:0/2174131036 shutdown_connections 2024-04-23T18:45:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557d6b39a5e0 0x557d6b39ca80 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b3874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b387b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 --2- 172.21.15.19:0/2174131036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b38a190 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 -- 172.21.15.19:0/2174131036 >> 172.21.15.19:0/2174131036 conn(0x557d6b37bb30 msgr2=0x557d6b381860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f98f7fff640 0 client.17565 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 -- 172.21.15.19:0/2174131036 shutdown_connections 2024-04-23T18:45:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 1 -- 172.21.15.19:0/2174131036 wait complete. 2024-04-23T18:45:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.140+0000 7f9921c14740 20 client.17565 trim_cache size 0 max 16384 2024-04-23T18:45:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f9921c14740 1 Processor -- start 2024-04-23T18:45:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f9921c14740 1 -- start start 2024-04-23T18:45:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f9921c14740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b389d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f9921c14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b386dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f9921c14740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b387310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f9921c14740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99181afcc0 con 0x557d6b37f620 2024-04-23T18:45:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f9921c14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b1800 con 0x557d6b37ff40 2024-04-23T18:45:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f9921c14740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b0390 con 0x557d6b38a800 2024-04-23T18:45:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f991cc96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b389d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f991cc96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b389d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41302/0 (socket says 172.21.15.19:41302) 2024-04-23T18:45:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f991cc96640 1 -- 172.21.15.19:0/2995628382 learned_addr learned my addr 172.21.15.19:0/2995628382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f991d497640 1 --2- 172.21.15.19:0/2995628382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b387310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f990ffff640 1 --2- 172.21.15.19:0/2995628382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b386dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f991cc96640 1 -- 172.21.15.19:0/2995628382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 msgr2=0x557d6b387310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f991cc96640 1 --2- 172.21.15.19:0/2995628382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b387310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f991cc96640 1 -- 172.21.15.19:0/2995628382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 msgr2=0x557d6b386dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f991cc96640 1 --2- 172.21.15.19:0/2995628382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b386dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f991cc96640 1 -- 172.21.15.19:0/2995628382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6b387e70 con 0x557d6b37f620 2024-04-23T18:45:01.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.155+0000 7f990ffff640 1 --2- 172.21.15.19:0/2995628382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b386dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f991cc96640 1 --2- 172.21.15.19:0/2995628382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b389d30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f991400ba50 tx=0x7f99140056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3169e7437a2e49a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f98fe7fc640 1 -- 172.21.15.19:0/2995628382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914031070 con 0x557d6b37f620 2024-04-23T18:45:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f98fe7fc640 1 -- 172.21.15.19:0/2995628382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f991400f610 con 0x557d6b37f620 2024-04-23T18:45:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f98fe7fc640 1 -- 172.21.15.19:0/2995628382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99140223b0 con 0x557d6b37f620 2024-04-23T18:45:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f9921c14740 1 -- 172.21.15.19:0/2995628382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 msgr2=0x557d6b389d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f9921c14740 1 --2- 172.21.15.19:0/2995628382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b389d30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f991400ba50 tx=0x7f99140056d0 comp rx=0 tx=0).stop 2024-04-23T18:45:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f9921c14740 1 -- 172.21.15.19:0/2995628382 shutdown_connections 2024-04-23T18:45:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f9921c14740 1 --2- 172.21.15.19:0/2995628382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b386dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f9921c14740 1 --2- 172.21.15.19:0/2995628382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b387310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f9921c14740 1 --2- 172.21.15.19:0/2995628382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b389d30 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f9921c14740 1 -- 172.21.15.19:0/2995628382 >> 172.21.15.19:0/2995628382 conn(0x557d6b37bb30 msgr2=0x557d6b37a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f9921c14740 1 -- 172.21.15.19:0/2995628382 shutdown_connections 2024-04-23T18:45:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.156+0000 7f9921c14740 1 -- 172.21.15.19:0/2995628382 wait complete. 2024-04-23T18:45:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f9921c14740 1 -- start start 2024-04-23T18:45:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f9921c14740 1 Processor -- start 2024-04-23T18:45:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f9921c14740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b3f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f9921c14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b3f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f9921c14740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b38f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f9921c14740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b1130 con 0x557d6b37f620 2024-04-23T18:45:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f9921c14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b0a60 con 0x557d6b37ff40 2024-04-23T18:45:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f9921c14740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f99181adcc0 con 0x557d6b38a800 2024-04-23T18:45:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f9921c14740 10 client.0 _ll_register_callbacks cb 0x7f991400bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f991cc96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b3f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f991cc96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b3f8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41318/0 (socket says 172.21.15.19:41318) 2024-04-23T18:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.157+0000 7f991cc96640 1 -- 172.21.15.19:0/3057192610 learned_addr learned my addr 172.21.15.19:0/3057192610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f990ffff640 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b3f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f991d497640 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b38f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f991cc96640 1 -- 172.21.15.19:0/3057192610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 msgr2=0x557d6b38f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f991cc96640 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b38f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f991cc96640 1 -- 172.21.15.19:0/3057192610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 msgr2=0x557d6b3f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f991cc96640 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b3f94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f991cc96640 1 -- 172.21.15.19:0/3057192610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6b38fe30 con 0x557d6b37f620 2024-04-23T18:45:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f991d497640 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b38f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f991cc96640 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b3f8fb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9914010700 tx=0x7f991400ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3161c8c9518758 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f98f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f98f7fff640 1 -- 172.21.15.19:0/3057192610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914031040 con 0x557d6b37f620 2024-04-23T18:45:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f98f7fff640 1 -- 172.21.15.19:0/3057192610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8513+0+0 (secure 0 0 0) 0x7f9914010e40 con 0x557d6b37f620 2024-04-23T18:45:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f9921c14740 1 -- 172.21.15.19:0/3057192610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f991400f610 con 0x557d6b37f620 2024-04-23T18:45:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f98f7fff640 1 -- 172.21.15.19:0/3057192610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f991401f040 con 0x557d6b37f620 2024-04-23T18:45:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f9921c14740 10 client.17571 fetch_fsmap learned FSMap version 29 2024-04-23T18:45:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.158+0000 7f9921c14740 1 -- 172.21.15.19:0/3057192610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557d6b4006f0 con 0x557d6b37f620 2024-04-23T18:45:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f98f7fff640 1 -- 172.21.15.19:0/3057192610 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f991400f610 con 0x557d6b37f620 2024-04-23T18:45:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f98f7fff640 1 -- 172.21.15.19:0/3057192610 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9914019040 con 0x557d6b37f620 2024-04-23T18:45:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f9921c14740 10 client.17571 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:45:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f9921c14740 10 client.17571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f9921c14740 20 client.17571 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f9921c14740 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557d6b400ca0 0x557d6b403090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f9921c14740 4 client.17571 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f9921c14740 1 -- 172.21.15.19:0/3057192610 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d6b2ec200 con 0x557d6b400ca0 2024-04-23T18:45:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f990ffff640 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557d6b400ca0 0x557d6b403090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f990ffff640 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557d6b400ca0 0x557d6b403090 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73f774af267ac6e server_cookie=f9319c526b593918 in_seq=0 out_seq=0 2024-04-23T18:45:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:03.159+0000 7f98f7fff640 10 client.17571 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:01.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:03 smithi019 ceph-mon[29215]: pgmap v531: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:01.961 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:44:59 smithi155 ceph-mon[33468]: pgmap v594: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:01.969 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:01 smithi155 ceph-mon[33468]: pgmap v595: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:01.976 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:44:59 smithi044 ceph-mon[32458]: pgmap v594: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:01.983 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:01 smithi044 ceph-mon[32458]: pgmap v595: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f98f7fff640 1 -- 172.21.15.19:0/3057192610 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d6b2ec200 con 0x557d6b400ca0 2024-04-23T18:45:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f98f7fff640 10 client.17571 handle_command_reply: tid=0 2024-04-23T18:45:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 client.17571 shutdown 2024-04-23T18:45:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 -- 172.21.15.19:0/3057192610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 msgr2=0x557d6b3f8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b3f8fb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9914010700 tx=0x7f991400ba80 comp rx=0 tx=0).stop 2024-04-23T18:45:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 -- 172.21.15.19:0/3057192610 shutdown_connections 2024-04-23T18:45:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557d6b400ca0 0x557d6b403090 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d6b37ff40 0x557d6b3f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d6b38a800 0x557d6b38f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 --2- 172.21.15.19:0/3057192610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6b37f620 0x557d6b3f8fb0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f98f7fff640 0 client.17571 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 -- 172.21.15.19:0/3057192610 >> 172.21.15.19:0/3057192610 conn(0x557d6b37bb30 msgr2=0x557d6b37acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 -- 172.21.15.19:0/3057192610 shutdown_connections 2024-04-23T18:45:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 1 -- 172.21.15.19:0/3057192610 wait complete. 2024-04-23T18:45:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.316+0000 7f9921c14740 20 client.17571 trim_cache size 0 max 16384 2024-04-23T18:45:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 -- 172.21.15.19:0/3308752478 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f99000a3dc0 msgr2=0x7f99000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f99000a3dc0 0x7f99000a6280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9910005cc0 tx=0x7f9910005c50 comp rx=0 tx=0).stop 2024-04-23T18:45:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 -- 172.21.15.19:0/3308752478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9918109470 msgr2=0x7f99181a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9918109470 0x7f99181a0b10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9908032fd0 tx=0x7f9908004270 comp rx=0 tx=0).stop 2024-04-23T18:45:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 -- 172.21.15.19:0/3308752478 shutdown_connections 2024-04-23T18:45:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f99000a3dc0 0x7f99000a6280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f991819a690 0x7f991819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9918109470 0x7f99181a0b10 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 --2- 172.21.15.19:0/3308752478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99181043a0 0x7f99181a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 -- 172.21.15.19:0/3308752478 >> 172.21.15.19:0/3308752478 conn(0x7f99180ffc40 msgr2=0x7f9918101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 -- 172.21.15.19:0/3308752478 shutdown_connections 2024-04-23T18:45:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:05.317+0000 7f991eefa640 1 -- 172.21.15.19:0/3308752478 wait complete. 2024-04-23T18:45:02.495 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:05 smithi019 ceph-mon[29215]: pgmap v532: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:45:02.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:06 smithi019 ceph-mon[29215]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T18:45:02.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:07 smithi019 ceph-mon[29215]: pgmap v534: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:45:02.519 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:09 smithi019 ceph-mon[29215]: pgmap v535: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:45:02.526 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:10 smithi019 ceph-mon[29215]: pgmap v536: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:45:02.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:13 smithi019 ceph-mon[29215]: pgmap v537: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:45:02.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:13 smithi019 ceph-mon[29215]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T18:45:02.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:15 smithi019 ceph-mon[29215]: pgmap v539: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 606 B/s wr, 10 op/s 2024-04-23T18:45:02.606 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:17 smithi019 ceph-mon[29215]: pgmap v540: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:45:02.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:17 smithi019 ceph-mon[29215]: daemon mds.e restarted 2024-04-23T18:45:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.419+0000 7f82848a0640 1 -- 172.21.15.19:0/2605669356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828010c5d0 msgr2=0x7f828010ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.419+0000 7f82848a0640 1 --2- 172.21.15.19:0/2605669356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828010c5d0 0x7f828010ea00 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8278009ca0 tx=0x7f827802a950 comp rx=0 tx=0).stop 2024-04-23T18:45:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.420+0000 7f82848a0640 1 -- 172.21.15.19:0/2605669356 shutdown_connections 2024-04-23T18:45:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.420+0000 7f82848a0640 1 --2- 172.21.15.19:0/2605669356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8280103b20 0x7f828010c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.420+0000 7f82848a0640 1 --2- 172.21.15.19:0/2605669356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828010c5d0 0x7f828010ea00 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.420+0000 7f82848a0640 1 --2- 172.21.15.19:0/2605669356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8280103200 0x7f82801035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.420+0000 7f82848a0640 1 -- 172.21.15.19:0/2605669356 >> 172.21.15.19:0/2605669356 conn(0x7f82800771d0 msgr2=0x7f82800775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.420+0000 7f82848a0640 1 -- 172.21.15.19:0/2605669356 shutdown_connections 2024-04-23T18:45:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.420+0000 7f82848a0640 1 -- 172.21.15.19:0/2605669356 wait complete. 2024-04-23T18:45:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.420+0000 7f82848a0640 1 Processor -- start 2024-04-23T18:45:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f82848a0640 1 -- start start 2024-04-23T18:45:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f82848a0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8280103200 0x7f8280198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f82848a0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8280103b20 0x7f8280198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f82848a0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828010c5d0 0x7f828019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f82848a0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82801121f0 con 0x7f828010c5d0 2024-04-23T18:45:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f82848a0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8280112070 con 0x7f8280103200 2024-04-23T18:45:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f82848a0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8280112370 con 0x7f8280103b20 2024-04-23T18:45:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f827ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828010c5d0 0x7f828019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f827ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828010c5d0 0x7f828019cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55394/0 (socket says 172.21.15.19:55394) 2024-04-23T18:45:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f827ed9d640 1 -- 172.21.15.19:0/4262470470 learned_addr learned my addr 172.21.15.19:0/4262470470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f827e59c640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8280103200 0x7f8280198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f8275d9b640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8280103b20 0x7f8280198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f827ed9d640 1 -- 172.21.15.19:0/4262470470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8280103b20 msgr2=0x7f8280198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f827ed9d640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8280103b20 0x7f8280198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f827ed9d640 1 -- 172.21.15.19:0/4262470470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8280103200 msgr2=0x7f8280198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f827ed9d640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8280103200 0x7f8280198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.421+0000 7f827ed9d640 1 -- 172.21.15.19:0/4262470470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828019d630 con 0x7f828010c5d0 2024-04-23T18:45:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.422+0000 7f827ed9d640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828010c5d0 0x7f828019cf00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f827802ae70 tx=0x7f827802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.422+0000 7f8277fff640 1 -- 172.21.15.19:0/4262470470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278034040 con 0x7f828010c5d0 2024-04-23T18:45:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.422+0000 7f82848a0640 1 -- 172.21.15.19:0/4262470470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828019d8c0 con 0x7f828010c5d0 2024-04-23T18:45:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.422+0000 7f8277fff640 1 -- 172.21.15.19:0/4262470470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82780308f0 con 0x7f828010c5d0 2024-04-23T18:45:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.422+0000 7f8277fff640 1 -- 172.21.15.19:0/4262470470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82780047c0 con 0x7f828010c5d0 2024-04-23T18:45:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.422+0000 7f82848a0640 1 -- 172.21.15.19:0/4262470470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82801a51b0 con 0x7f828010c5d0 2024-04-23T18:45:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.424+0000 7f8277fff640 1 -- 172.21.15.19:0/4262470470 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8278037020 con 0x7f828010c5d0 2024-04-23T18:45:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f82875bc740 1 Processor -- start 2024-04-23T18:45:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f82875bc740 1 -- start start 2024-04-23T18:45:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f8277fff640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f826c0a3dc0 0x7f826c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f82875bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 0x56414e434a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f82875bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e43f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f82875bc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e441bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f82875bc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56414e399970 con 0x56414e43f800 2024-04-23T18:45:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f82875bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56414e3364a0 con 0x56414e434620 2024-04-23T18:45:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f82875bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56414e395930 con 0x56414e434f40 2024-04-23T18:45:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f8275d9b640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f826c0a3dc0 0x7f826c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f827ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e441bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f8277fff640 1 -- 172.21.15.19:0/4262470470 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7f82780f1760 con 0x7f828010c5d0 2024-04-23T18:45:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f827ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e441bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55402/0 (socket says 172.21.15.19:55402) 2024-04-23T18:45:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f827e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 0x56414e434a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.425+0000 7f827ed9d640 1 -- 172.21.15.19:0/1342531583 learned_addr learned my addr 172.21.15.19:0/1342531583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f8275d9b640 1 --2- 172.21.15.19:0/1342531583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e43f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f827ed9d640 1 -- 172.21.15.19:0/1342531583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 msgr2=0x56414e43f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f827ed9d640 1 --2- 172.21.15.19:0/1342531583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e43f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f827ed9d640 1 -- 172.21.15.19:0/1342531583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 msgr2=0x56414e434a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f827ed9d640 1 --2- 172.21.15.19:0/1342531583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 0x56414e434a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f827ed9d640 1 -- 172.21.15.19:0/1342531583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414e0ac210 con 0x56414e43f800 2024-04-23T18:45:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f827e59c640 1 --2- 172.21.15.19:0/1342531583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 0x56414e434a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:45:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f8275d9b640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f826c0a3dc0 0x7f826c0a6280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f828010e270 tx=0x7f82680137d0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f827ed9d640 1 --2- 172.21.15.19:0/1342531583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e441bf0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f82780bd770 tx=0x7f827802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34813e6f504fd3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f8274d99640 1 -- 172.21.15.19:0/1342531583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82780bd320 con 0x56414e43f800 2024-04-23T18:45:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f8274d99640 1 -- 172.21.15.19:0/1342531583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82780bdee0 con 0x56414e43f800 2024-04-23T18:45:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.426+0000 7f8274d99640 1 -- 172.21.15.19:0/1342531583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82780955c0 con 0x56414e43f800 2024-04-23T18:45:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 -- 172.21.15.19:0/1342531583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 msgr2=0x56414e441bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 --2- 172.21.15.19:0/1342531583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e441bf0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f82780bd770 tx=0x7f827802a950 comp rx=0 tx=0).stop 2024-04-23T18:45:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 -- 172.21.15.19:0/1342531583 shutdown_connections 2024-04-23T18:45:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 --2- 172.21.15.19:0/1342531583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e441bf0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 --2- 172.21.15.19:0/1342531583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e43f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 --2- 172.21.15.19:0/1342531583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 0x56414e434a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 -- 172.21.15.19:0/1342531583 >> 172.21.15.19:0/1342531583 conn(0x56414e430b30 msgr2=0x56414e430f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 -- 172.21.15.19:0/1342531583 shutdown_connections 2024-04-23T18:45:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 -- 172.21.15.19:0/1342531583 wait complete. 2024-04-23T18:45:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 -- start start 2024-04-23T18:45:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.427+0000 7f82875bc740 1 Processor -- start 2024-04-23T18:45:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f82875bc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e43f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f82875bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e43c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f82875bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e43f800 0x56414e43ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f82875bc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56414e4421f0 con 0x56414e434620 2024-04-23T18:45:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f82875bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56414e3340b0 con 0x56414e43f800 2024-04-23T18:45:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f82875bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56414e442370 con 0x56414e434f40 2024-04-23T18:45:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f82875bc740 10 client.0 _ll_register_callbacks cb 0x56414e321d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e43f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e43f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55410/0 (socket says 172.21.15.19:55410) 2024-04-23T18:45:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827e59c640 1 -- 172.21.15.19:0/1757741890 learned_addr learned my addr 172.21.15.19:0/1757741890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f8275d9b640 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e43c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827ed9d640 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e43f800 0x56414e43ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827e59c640 1 -- 172.21.15.19:0/1757741890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 msgr2=0x56414e43c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827e59c640 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e43c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827e59c640 1 -- 172.21.15.19:0/1757741890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e43f800 msgr2=0x56414e43ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827e59c640 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e43f800 0x56414e43ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827e59c640 1 -- 172.21.15.19:0/1757741890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414e178620 con 0x56414e434620 2024-04-23T18:45:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827ed9d640 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e43f800 0x56414e43ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f827e59c640 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e43f190 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f827000bf80 tx=0x7f8270012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dec9f2f5bdc04a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f8265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.428+0000 7f8265ffb640 1 -- 172.21.15.19:0/1757741890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270005220 con 0x56414e434620 2024-04-23T18:45:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f82875bc740 1 -- 172.21.15.19:0/1757741890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82780bdee0 con 0x56414e434620 2024-04-23T18:45:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f8265ffb640 1 -- 172.21.15.19:0/1757741890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7f82700053c0 con 0x56414e434620 2024-04-23T18:45:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f8265ffb640 1 -- 172.21.15.19:0/1757741890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82700055c0 con 0x56414e434620 2024-04-23T18:45:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f82875bc740 10 client.17586 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f82875bc740 1 -- 172.21.15.19:0/1757741890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56414e44e1a0 con 0x56414e434620 2024-04-23T18:45:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f8265ffb640 1 -- 172.21.15.19:0/1757741890 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82780bdee0 con 0x56414e434620 2024-04-23T18:45:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f8265ffb640 1 -- 172.21.15.19:0/1757741890 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8270019070 con 0x56414e434620 2024-04-23T18:45:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f82875bc740 10 client.17586 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f82875bc740 10 client.17586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f82875bc740 20 client.17586 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f82875bc740 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56414e44f660 0x56414e451a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f82875bc740 4 client.17586 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f82875bc740 1 -- 172.21.15.19:0/1757741890 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56414e3a7390 con 0x56414e44f660 2024-04-23T18:45:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.429+0000 7f8275d9b640 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56414e44f660 0x56414e451a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.430+0000 7f8275d9b640 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56414e44f660 0x56414e451a90 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3e9a75e9246c4d server_cookie=cf4b714386c7ad22 in_seq=0 out_seq=0 2024-04-23T18:45:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.430+0000 7f8265ffb640 10 client.17586 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.430+0000 7f8265ffb640 1 -- 172.21.15.19:0/1757741890 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56414e3a7390 con 0x56414e44f660 2024-04-23T18:45:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.430+0000 7f8265ffb640 10 client.17586 handle_command_reply: tid=0 2024-04-23T18:45:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.430+0000 7f82875bc740 1 client.17586 shutdown 2024-04-23T18:45:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 1 -- 172.21.15.19:0/1757741890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 msgr2=0x56414e43f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e43f190 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f827000bf80 tx=0x7f8270012820 comp rx=0 tx=0).stop 2024-04-23T18:45:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 1 -- 172.21.15.19:0/1757741890 shutdown_connections 2024-04-23T18:45:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56414e44f660 0x56414e451a90 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e43f800 0x56414e43ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e43c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f8265ffb640 0 client.17586 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 1 --2- 172.21.15.19:0/1757741890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e43f190 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 1 -- 172.21.15.19:0/1757741890 >> 172.21.15.19:0/1757741890 conn(0x56414e430b30 msgr2=0x56414e4368b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 1 -- 172.21.15.19:0/1757741890 shutdown_connections 2024-04-23T18:45:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 1 -- 172.21.15.19:0/1757741890 wait complete. 2024-04-23T18:45:04.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.431+0000 7f82875bc740 20 client.17586 trim_cache size 0 max 16384 2024-04-23T18:45:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.446+0000 7f82875bc740 1 Processor -- start 2024-04-23T18:45:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.446+0000 7f82875bc740 1 -- start start 2024-04-23T18:45:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.446+0000 7f82875bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 0x56414e44afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.446+0000 7f82875bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e42fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.446+0000 7f82875bc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e4573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.446+0000 7f82875bc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82801ada30 con 0x56414e43f800 2024-04-23T18:45:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f82875bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f82801af570 con 0x56414e434620 2024-04-23T18:45:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f82875bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82801ae100 con 0x56414e434f40 2024-04-23T18:45:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e4573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e4573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55422/0 (socket says 172.21.15.19:55422) 2024-04-23T18:45:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827ed9d640 1 -- 172.21.15.19:0/3483952081 learned_addr learned my addr 172.21.15.19:0/3483952081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827e59c640 1 --2- 172.21.15.19:0/3483952081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 0x56414e44afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f8275d9b640 1 --2- 172.21.15.19:0/3483952081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e42fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827ed9d640 1 -- 172.21.15.19:0/3483952081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 msgr2=0x56414e42fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827ed9d640 1 --2- 172.21.15.19:0/3483952081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e42fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827ed9d640 1 -- 172.21.15.19:0/3483952081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 msgr2=0x56414e44afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827ed9d640 1 --2- 172.21.15.19:0/3483952081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 0x56414e44afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827ed9d640 1 -- 172.21.15.19:0/3483952081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414e43b430 con 0x56414e43f800 2024-04-23T18:45:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827e59c640 1 --2- 172.21.15.19:0/3483952081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 0x56414e44afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:45:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f827ed9d640 1 --2- 172.21.15.19:0/3483952081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e4573a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f82780c5f20 tx=0x7f8278003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb53160a4a765bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f8267fff640 1 -- 172.21.15.19:0/3483952081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82780bd140 con 0x56414e43f800 2024-04-23T18:45:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f8267fff640 1 -- 172.21.15.19:0/3483952081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82780bdee0 con 0x56414e43f800 2024-04-23T18:45:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.447+0000 7f8267fff640 1 -- 172.21.15.19:0/3483952081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82780a80b0 con 0x56414e43f800 2024-04-23T18:45:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 -- 172.21.15.19:0/3483952081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 msgr2=0x56414e4573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 --2- 172.21.15.19:0/3483952081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e4573a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f82780c5f20 tx=0x7f8278003ea0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 -- 172.21.15.19:0/3483952081 shutdown_connections 2024-04-23T18:45:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 --2- 172.21.15.19:0/3483952081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e43f800 0x56414e4573a0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 --2- 172.21.15.19:0/3483952081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e434f40 0x56414e42fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 --2- 172.21.15.19:0/3483952081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434620 0x56414e44afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 -- 172.21.15.19:0/3483952081 >> 172.21.15.19:0/3483952081 conn(0x56414e430b30 msgr2=0x56414e42f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 -- 172.21.15.19:0/3483952081 shutdown_connections 2024-04-23T18:45:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 -- 172.21.15.19:0/3483952081 wait complete. 2024-04-23T18:45:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 -- start start 2024-04-23T18:45:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.448+0000 7f82875bc740 1 Processor -- start 2024-04-23T18:45:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f82875bc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e447730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:05.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f82875bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434f40 0x56414e444b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f82875bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e43f800 0x56414e445050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f82875bc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82801aeea0 con 0x56414e434620 2024-04-23T18:45:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f82875bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f82801ae7d0 con 0x56414e434f40 2024-04-23T18:45:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f82875bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82801aba10 con 0x56414e43f800 2024-04-23T18:45:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f82875bc740 10 client.0 _ll_register_callbacks cb 0x7f827000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e447730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e447730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55434/0 (socket says 172.21.15.19:55434) 2024-04-23T18:45:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827e59c640 1 -- 172.21.15.19:0/3026331745 learned_addr learned my addr 172.21.15.19:0/3026331745 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f8275d9b640 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434f40 0x56414e444b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827ed9d640 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e43f800 0x56414e445050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827e59c640 1 -- 172.21.15.19:0/3026331745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e43f800 msgr2=0x56414e445050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827e59c640 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e43f800 0x56414e445050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827e59c640 1 -- 172.21.15.19:0/3026331745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434f40 msgr2=0x56414e444b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827e59c640 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434f40 0x56414e444b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827e59c640 1 -- 172.21.15.19:0/3026331745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414e434d70 con 0x56414e434620 2024-04-23T18:45:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827ed9d640 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e43f800 0x56414e445050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f827e59c640 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e447730 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f8270010cd0 tx=0x7f827000bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a777801e0f1d1ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f8265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f8265ffb640 1 -- 172.21.15.19:0/3026331745 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270027070 con 0x56414e434620 2024-04-23T18:45:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f8265ffb640 1 -- 172.21.15.19:0/3026331745 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7f82700106a0 con 0x56414e434620 2024-04-23T18:45:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.449+0000 7f82875bc740 1 -- 172.21.15.19:0/3026331745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82780bdee0 con 0x56414e434620 2024-04-23T18:45:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f8265ffb640 1 -- 172.21.15.19:0/3026331745 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f827003a050 con 0x56414e434620 2024-04-23T18:45:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f82875bc740 10 client.17592 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f82875bc740 1 -- 172.21.15.19:0/3026331745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56414e4b5910 con 0x56414e434620 2024-04-23T18:45:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f8265ffb640 1 -- 172.21.15.19:0/3026331745 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82780bdee0 con 0x56414e434620 2024-04-23T18:45:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f8265ffb640 1 -- 172.21.15.19:0/3026331745 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8270019050 con 0x56414e434620 2024-04-23T18:45:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f82875bc740 10 client.17592 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f82875bc740 10 client.17592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f82875bc740 20 client.17592 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f82875bc740 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56414e4b6110 0x56414e4b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f82875bc740 4 client.17592 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f82875bc740 1 -- 172.21.15.19:0/3026331745 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56414e3a1200 con 0x56414e4b6110 2024-04-23T18:45:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f8275d9b640 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56414e4b6110 0x56414e4b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f8265ffb640 10 client.17592 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.450+0000 7f8275d9b640 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56414e4b6110 0x56414e4b8530 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6831a39b854bfb4 server_cookie=6e854731a5f6a26b in_seq=0 out_seq=0 2024-04-23T18:45:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.456+0000 7f8265ffb640 1 -- 172.21.15.19:0/3026331745 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56414e3a1200 con 0x56414e4b6110 2024-04-23T18:45:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.456+0000 7f8265ffb640 10 client.17592 handle_command_reply: tid=0 2024-04-23T18:45:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.456+0000 7f82875bc740 1 client.17592 shutdown 2024-04-23T18:45:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.456+0000 7f82875bc740 1 -- 172.21.15.19:0/3026331745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 msgr2=0x56414e447730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.456+0000 7f82875bc740 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e447730 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f8270010cd0 tx=0x7f827000bb00 comp rx=0 tx=0).stop 2024-04-23T18:45:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82875bc740 1 -- 172.21.15.19:0/3026331745 shutdown_connections 2024-04-23T18:45:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82875bc740 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56414e4b6110 0x56414e4b8530 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82875bc740 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56414e434f40 0x56414e444b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82875bc740 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e43f800 0x56414e445050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f8265ffb640 0 client.17592 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82875bc740 1 --2- 172.21.15.19:0/3026331745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56414e434620 0x56414e447730 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82875bc740 1 -- 172.21.15.19:0/3026331745 >> 172.21.15.19:0/3026331745 conn(0x56414e430b30 msgr2=0x56414e42fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82875bc740 1 -- 172.21.15.19:0/3026331745 shutdown_connections 2024-04-23T18:45:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82875bc740 1 -- 172.21.15.19:0/3026331745 wait complete. 2024-04-23T18:45:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82875bc740 20 client.17592 trim_cache size 0 max 16384 2024-04-23T18:45:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82848a0640 1 -- 172.21.15.19:0/4262470470 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f826c0a3dc0 msgr2=0x7f826c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82848a0640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f826c0a3dc0 0x7f826c0a6280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f828010e270 tx=0x7f82680137d0 comp rx=0 tx=0).stop 2024-04-23T18:45:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82848a0640 1 -- 172.21.15.19:0/4262470470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828010c5d0 msgr2=0x7f828019cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82848a0640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828010c5d0 0x7f828019cf00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f827802ae70 tx=0x7f827802f5d0 comp rx=0 tx=0).stop 2024-04-23T18:45:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82848a0640 1 -- 172.21.15.19:0/4262470470 shutdown_connections 2024-04-23T18:45:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82848a0640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f826c0a3dc0 0x7f826c0a6280 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82848a0640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828010c5d0 0x7f828019cf00 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82848a0640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8280103b20 0x7f8280198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82848a0640 1 --2- 172.21.15.19:0/4262470470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8280103200 0x7f8280198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.457+0000 7f82848a0640 1 -- 172.21.15.19:0/4262470470 >> 172.21.15.19:0/4262470470 conn(0x7f82800771d0 msgr2=0x7f82800fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.458+0000 7f82848a0640 1 -- 172.21.15.19:0/4262470470 shutdown_connections 2024-04-23T18:45:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:18.458+0000 7f82848a0640 1 -- 172.21.15.19:0/4262470470 wait complete. 2024-04-23T18:45:06.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:18 smithi019 ceph-mon[29215]: osdmap e113: 12 total, 12 up, 12 in 2024-04-23T18:45:06.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:18 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6834/3131205149,v1:172.21.15.44:6835/3131205149] up:boot 2024-04-23T18:45:06.175 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:18 smithi019 ceph-mon[29215]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:45:06.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:18 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:45:06.188 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:19 smithi019 ceph-mon[29215]: pgmap v542: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-23T18:45:06.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:21 smithi019 ceph-mon[29215]: pgmap v543: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 639 B/s wr, 12 op/s 2024-04-23T18:45:06.200 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:23 smithi019 ceph-mon[29215]: pgmap v544: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 537 B/s wr, 10 op/s 2024-04-23T18:45:06.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:25 smithi019 ceph-mon[29215]: pgmap v545: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:45:06.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:27 smithi019 ceph-mon[29215]: pgmap v546: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:45:06.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:29 smithi019 ceph-mon[29215]: pgmap v547: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 388 B/s wr, 10 op/s 2024-04-23T18:45:06.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:29 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:06.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:30 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:06.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:30 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:06.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:30 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:45:06.355 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:03 smithi155 ceph-mon[33468]: pgmap v596: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:06.421 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:05 smithi155 ceph-mon[33468]: pgmap v597: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:45:06.430 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:03 smithi044 ceph-mon[32458]: pgmap v596: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:06.439 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:05 smithi044 ceph-mon[32458]: pgmap v597: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:45:06.448 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:45:06.457 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: in thread 7f657ee11700 thread_name:md_log_replay 2024-04-23T18:45:06.466 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:45:06.474 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f65901d2d20] 2024-04-23T18:45:06.483 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55580a54f38f] 2024-04-23T18:45:06.493 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55580a552b23] 2024-04-23T18:45:06.521 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 4: (MDLog::_replay_thread()+0x617) [0x55580a5ad647] 2024-04-23T18:45:06.530 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55580a5b6d31] 2024-04-23T18:45:06.588 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f65901c81ca] 2024-04-23T18:45:06.595 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 7: clone() 2024-04-23T18:45:06.602 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2024-04-23T18:45:06.073+0000 7f657ee11700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:45:06.609 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: in thread 7f657ee11700 thread_name:md_log_replay 2024-04-23T18:45:06.617 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2024-04-23T18:45:06.625 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:45:06.694 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f65901d2d20] 2024-04-23T18:45:06.738 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55580a54f38f] 2024-04-23T18:45:06.750 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55580a552b23] 2024-04-23T18:45:06.759 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 4: (MDLog::_replay_thread()+0x617) [0x55580a5ad647] 2024-04-23T18:45:06.767 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55580a5b6d31] 2024-04-23T18:45:06.776 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f65901c81ca] 2024-04-23T18:45:06.784 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 7: clone() 2024-04-23T18:45:06.794 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:45:06.803 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2024-04-23T18:45:06.857 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 0> 2024-04-23T18:45:06.073+0000 7f657ee11700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:45:06.887 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: in thread 7f657ee11700 thread_name:md_log_replay 2024-04-23T18:45:06.896 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2024-04-23T18:45:06.904 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:45:06.913 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f65901d2d20] 2024-04-23T18:45:06.921 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55580a54f38f] 2024-04-23T18:45:06.929 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55580a552b23] 2024-04-23T18:45:06.937 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 4: (MDLog::_replay_thread()+0x617) [0x55580a5ad647] 2024-04-23T18:45:07.020 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55580a5b6d31] 2024-04-23T18:45:07.026 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f65901c81ca] 2024-04-23T18:45:07.032 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 7: clone() 2024-04-23T18:45:07.038 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:45:07.045 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2024-04-23T18:45:07.059 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 0> 2024-04-23T18:45:06.073+0000 7f657ee11700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:45:07.066 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: in thread 7f657ee11700 thread_name:md_log_replay 2024-04-23T18:45:07.074 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2024-04-23T18:45:07.081 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:45:07.125 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f65901d2d20] 2024-04-23T18:45:07.185 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55580a54f38f] 2024-04-23T18:45:07.192 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55580a552b23] 2024-04-23T18:45:07.199 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 4: (MDLog::_replay_thread()+0x617) [0x55580a5ad647] 2024-04-23T18:45:07.206 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55580a5b6d31] 2024-04-23T18:45:07.214 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f65901c81ca] 2024-04-23T18:45:07.220 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 7: clone() 2024-04-23T18:45:07.228 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:45:07.293 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69126]: 2024-04-23T18:45:07.301 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 podman[69449]: 2024-04-23 18:45:06.129963451 +0000 UTC m=+0.022546625 container died 8cecf12f4df369b9fc1a5ec8789b3e7b969a724587c458ca082f2d4eb9537c00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-23T18:45:07.312 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:06 smithi155 ceph-mon[33468]: osdmap e116: 12 total, 12 up, 12 in 2024-04-23T18:45:07.319 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:06 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/746115411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:45:07.326 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-mon[32458]: osdmap e116: 12 total, 12 up, 12 in 2024-04-23T18:45:07.351 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:06 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/746115411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:45:07.357 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 podman[69449]: 2024-04-23 18:45:06.799494312 +0000 UTC m=+0.692077486 container remove 8cecf12f4df369b9fc1a5ec8789b3e7b969a724587c458ca082f2d4eb9537c00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:45:07.363 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:06 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:45:07.368 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:07 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-23T18:45:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.557+0000 7fc291b08640 1 -- 172.21.15.19:0/368935487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc28c106fe0 msgr2=0x7fc28c1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.557+0000 7fc291b08640 1 --2- 172.21.15.19:0/368935487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc28c106fe0 0x7fc28c1073c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009930 tx=0x7fc27c02a420 comp rx=0 tx=0).stop 2024-04-23T18:45:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.557+0000 7fc291b08640 1 -- 172.21.15.19:0/368935487 shutdown_connections 2024-04-23T18:45:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.557+0000 7fc291b08640 1 --2- 172.21.15.19:0/368935487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc28c1011a0 0x7fc28c101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.557+0000 7fc291b08640 1 --2- 172.21.15.19:0/368935487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c101b40 0x7fc28c10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.557+0000 7fc291b08640 1 --2- 172.21.15.19:0/368935487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc28c106fe0 0x7fc28c1073c0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.557+0000 7fc291b08640 1 -- 172.21.15.19:0/368935487 >> 172.21.15.19:0/368935487 conn(0x7fc28c076b80 msgr2=0x7fc28c076f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.557+0000 7fc291b08640 1 -- 172.21.15.19:0/368935487 shutdown_connections 2024-04-23T18:45:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.557+0000 7fc291b08640 1 -- 172.21.15.19:0/368935487 wait complete. 2024-04-23T18:45:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc291b08640 1 Processor -- start 2024-04-23T18:45:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc291b08640 1 -- start start 2024-04-23T18:45:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc291b08640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c1011a0 0x7fc28c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc291b08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc28c101b40 0x7fc28c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc291b08640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc28c106fe0 0x7fc28c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc291b08640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c112210 con 0x7fc28c106fe0 2024-04-23T18:45:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc291b08640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c112090 con 0x7fc28c101b40 2024-04-23T18:45:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc291b08640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c112390 con 0x7fc28c1011a0 2024-04-23T18:45:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc28bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc28c106fe0 0x7fc28c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc28bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc28c106fe0 0x7fc28c19cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59414/0 (socket says 172.21.15.19:59414) 2024-04-23T18:45:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.558+0000 7fc28bfff640 1 -- 172.21.15.19:0/4273888811 learned_addr learned my addr 172.21.15.19:0/4273888811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc282ffd640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc28c101b40 0x7fc28c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c1011a0 0x7fc28c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc28bfff640 1 -- 172.21.15.19:0/4273888811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c1011a0 msgr2=0x7fc28c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc28bfff640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c1011a0 0x7fc28c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc28bfff640 1 -- 172.21.15.19:0/4273888811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc28c101b40 msgr2=0x7fc28c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc28bfff640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc28c101b40 0x7fc28c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc28bfff640 1 -- 172.21.15.19:0/4273888811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c19d690 con 0x7fc28c106fe0 2024-04-23T18:45:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc28bfff640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc28c106fe0 0x7fc28c19cf60 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc28400cad0 tx=0x7fc28400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc2897fa640 1 -- 172.21.15.19:0/4273888811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28400ae20 con 0x7fc28c106fe0 2024-04-23T18:45:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc2897fa640 1 -- 172.21.15.19:0/4273888811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc284005020 con 0x7fc28c106fe0 2024-04-23T18:45:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc291b08640 1 -- 172.21.15.19:0/4273888811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c19d980 con 0x7fc28c106fe0 2024-04-23T18:45:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc2897fa640 1 -- 172.21.15.19:0/4273888811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28400f6b0 con 0x7fc28c106fe0 2024-04-23T18:45:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.559+0000 7fc291b08640 1 -- 172.21.15.19:0/4273888811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c1a5210 con 0x7fc28c106fe0 2024-04-23T18:45:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc2897fa640 1 -- 172.21.15.19:0/4273888811 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc2840051d0 con 0x7fc28c106fe0 2024-04-23T18:45:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc29481d740 1 Processor -- start 2024-04-23T18:45:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc29481d740 1 -- start start 2024-04-23T18:45:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc29481d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a32858900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc29481d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32858e40 0x557a328676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc29481d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32867c10 0x557a3286a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc29481d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a327bd970 con 0x557a32858520 2024-04-23T18:45:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc29481d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557a3275a4a0 con 0x557a32858e40 2024-04-23T18:45:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc29481d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557a327b9930 con 0x557a32867c10 2024-04-23T18:45:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc282ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a32858900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc282ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a32858900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59428/0 (socket says 172.21.15.19:59428) 2024-04-23T18:45:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc282ffd640 1 -- 172.21.15.19:0/1787167059 learned_addr learned my addr 172.21.15.19:0/1787167059 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc2897fa640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc2780a3dc0 0x7fc2780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.562+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/1787167059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32858e40 0x557a328676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc282ffd640 1 --2- 172.21.15.19:0/1787167059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32867c10 0x557a3286a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc28bfff640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc2780a3dc0 0x7fc2780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc282ffd640 1 -- 172.21.15.19:0/1787167059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32867c10 msgr2=0x557a3286a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc282ffd640 1 --2- 172.21.15.19:0/1787167059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32867c10 0x557a3286a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc282ffd640 1 -- 172.21.15.19:0/1787167059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32858e40 msgr2=0x557a328676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc282ffd640 1 --2- 172.21.15.19:0/1787167059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32858e40 0x557a328676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc282ffd640 1 -- 172.21.15.19:0/1787167059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a324d0210 con 0x557a32858520 2024-04-23T18:45:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc2897fa640 1 -- 172.21.15.19:0/4273888811 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7fc2840d0760 con 0x7fc28c106fe0 2024-04-23T18:45:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc282ffd640 1 --2- 172.21.15.19:0/1787167059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a32858900 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc2740125b0 tx=0x7fc27400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184a2e677b30bafc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc281ffb640 1 -- 172.21.15.19:0/1787167059 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274005020 con 0x557a32858520 2024-04-23T18:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc28bfff640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc2780a3dc0 0x7fc2780a6280 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc2840cdf90 tx=0x7fc284005890 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc281ffb640 1 -- 172.21.15.19:0/1787167059 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc274005670 con 0x557a32858520 2024-04-23T18:45:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.563+0000 7fc281ffb640 1 -- 172.21.15.19:0/1787167059 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27400e730 con 0x557a32858520 2024-04-23T18:45:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.564+0000 7fc29481d740 1 -- 172.21.15.19:0/1787167059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 msgr2=0x557a32858900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.564+0000 7fc29481d740 1 --2- 172.21.15.19:0/1787167059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a32858900 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc2740125b0 tx=0x7fc27400b8c0 comp rx=0 tx=0).stop 2024-04-23T18:45:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.564+0000 7fc29481d740 1 -- 172.21.15.19:0/1787167059 shutdown_connections 2024-04-23T18:45:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.564+0000 7fc29481d740 1 --2- 172.21.15.19:0/1787167059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32858e40 0x557a328676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.564+0000 7fc29481d740 1 --2- 172.21.15.19:0/1787167059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32867c10 0x557a3286a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.564+0000 7fc29481d740 1 --2- 172.21.15.19:0/1787167059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a32858900 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.564+0000 7fc29481d740 1 -- 172.21.15.19:0/1787167059 >> 172.21.15.19:0/1787167059 conn(0x557a32854b30 msgr2=0x557a32854f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.564+0000 7fc29481d740 1 -- 172.21.15.19:0/1787167059 shutdown_connections 2024-04-23T18:45:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.564+0000 7fc29481d740 1 -- 172.21.15.19:0/1787167059 wait complete. 2024-04-23T18:45:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.564+0000 7fc29481d740 1 -- start start 2024-04-23T18:45:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc29481d740 1 Processor -- start 2024-04-23T18:45:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc29481d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a328674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc29481d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858e40 0x557a328647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc29481d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32867c10 0x557a32864d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc29481d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a3286a840 con 0x557a32858520 2024-04-23T18:45:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc29481d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557a327580b0 con 0x557a32867c10 2024-04-23T18:45:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc29481d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557a3286a9c0 con 0x557a32858e40 2024-04-23T18:45:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc29481d740 10 client.0 _ll_register_callbacks cb 0x557a32745d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc282ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a328674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc282ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a328674f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59430/0 (socket says 172.21.15.19:59430) 2024-04-23T18:45:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc282ffd640 1 -- 172.21.15.19:0/4202671948 learned_addr learned my addr 172.21.15.19:0/4202671948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc282ffd640 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32867c10 0x557a32864d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858e40 0x557a328647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc282ffd640 1 -- 172.21.15.19:0/4202671948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858e40 msgr2=0x557a328647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc282ffd640 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858e40 0x557a328647e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc282ffd640 1 -- 172.21.15.19:0/4202671948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32867c10 msgr2=0x557a32864d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc282ffd640 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32867c10 0x557a32864d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc282ffd640 1 -- 172.21.15.19:0/4202671948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3259c620 con 0x557a32858520 2024-04-23T18:45:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.565+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858e40 0x557a328647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc282ffd640 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a328674f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc274005d00 tx=0x7fc274004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcbc8b7389eb048 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc26affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc26affd640 1 -- 172.21.15.19:0/4202671948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274021020 con 0x557a32858520 2024-04-23T18:45:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc26affd640 1 -- 172.21.15.19:0/4202671948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7fc274004d80 con 0x557a32858520 2024-04-23T18:45:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc29481d740 1 -- 172.21.15.19:0/4202671948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc274005670 con 0x557a32858520 2024-04-23T18:45:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc26affd640 1 -- 172.21.15.19:0/4202671948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc274004020 con 0x557a32858520 2024-04-23T18:45:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc29481d740 10 client.17604 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc29481d740 1 -- 172.21.15.19:0/4202671948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a32876760 con 0x557a32858520 2024-04-23T18:45:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc26affd640 1 -- 172.21.15.19:0/4202671948 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc274005670 con 0x557a32858520 2024-04-23T18:45:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc26affd640 1 -- 172.21.15.19:0/4202671948 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc2740035f0 con 0x557a32858520 2024-04-23T18:45:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc29481d740 10 client.17604 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc29481d740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc29481d740 20 client.17604 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc29481d740 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a32877b70 0x557a32879f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc29481d740 4 client.17604 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.566+0000 7fc29481d740 1 -- 172.21.15.19:0/4202671948 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a327cb390 con 0x557a32877b70 2024-04-23T18:45:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.567+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a32877b70 0x557a32879f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.567+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a32877b70 0x557a32879f60 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1238c6925206b231 server_cookie=747ed84295564a76 in_seq=0 out_seq=0 2024-04-23T18:45:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.567+0000 7fc26affd640 10 client.17604 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc26affd640 1 -- 172.21.15.19:0/4202671948 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a327cb390 con 0x557a32877b70 2024-04-23T18:45:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc26affd640 10 client.17604 handle_command_reply: tid=0 2024-04-23T18:45:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 client.17604 shutdown 2024-04-23T18:45:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 -- 172.21.15.19:0/4202671948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 msgr2=0x557a328674f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a328674f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc274005d00 tx=0x7fc274004ca0 comp rx=0 tx=0).stop 2024-04-23T18:45:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 -- 172.21.15.19:0/4202671948 shutdown_connections 2024-04-23T18:45:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a32877b70 0x557a32879f60 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32867c10 0x557a32864d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858e40 0x557a328647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc26affd640 0 client.17604 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 --2- 172.21.15.19:0/4202671948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858520 0x557a328674f0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 -- 172.21.15.19:0/4202671948 >> 172.21.15.19:0/4202671948 conn(0x557a32854b30 msgr2=0x557a3285a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 -- 172.21.15.19:0/4202671948 shutdown_connections 2024-04-23T18:45:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 1 -- 172.21.15.19:0/4202671948 wait complete. 2024-04-23T18:45:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.568+0000 7fc29481d740 20 client.17604 trim_cache size 0 max 16384 2024-04-23T18:45:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.583+0000 7fc29481d740 1 Processor -- start 2024-04-23T18:45:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.583+0000 7fc29481d740 1 -- start start 2024-04-23T18:45:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc29481d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32858520 0x557a3285ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc29481d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a328640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc29481d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32867c10 0x557a32867020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc29481d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1ab810 con 0x557a32858e40 2024-04-23T18:45:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc29481d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1ad350 con 0x557a32858520 2024-04-23T18:45:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc29481d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1abee0 con 0x557a32867c10 2024-04-23T18:45:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc28b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a328640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc28b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a328640c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59446/0 (socket says 172.21.15.19:59446) 2024-04-23T18:45:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc28b7fe640 1 -- 172.21.15.19:0/79262310 learned_addr learned my addr 172.21.15.19:0/79262310 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc28b7fe640 1 -- 172.21.15.19:0/79262310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32867c10 msgr2=0x557a32867020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc282ffd640 1 --2- 172.21.15.19:0/79262310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32858520 0x557a3285ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/79262310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32867c10 0x557a32867020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc28b7fe640 1 -- 172.21.15.19:0/79262310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32858520 msgr2=0x557a3285ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/79262310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32858520 0x557a3285ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc28b7fe640 1 -- 172.21.15.19:0/79262310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a32854960 con 0x557a32858e40 2024-04-23T18:45:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/79262310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a328640c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009930 tx=0x7fc27c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32df13d2dbdadcc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.584+0000 7fc2817fa640 1 -- 172.21.15.19:0/79262310 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c035040 con 0x557a32858e40 2024-04-23T18:45:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc2817fa640 1 -- 172.21.15.19:0/79262310 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc27c0054d0 con 0x557a32858e40 2024-04-23T18:45:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc2817fa640 1 -- 172.21.15.19:0/79262310 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c0452c0 con 0x557a32858e40 2024-04-23T18:45:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc29481d740 1 -- 172.21.15.19:0/79262310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 msgr2=0x557a328640c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc29481d740 1 --2- 172.21.15.19:0/79262310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a328640c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009930 tx=0x7fc27c02a420 comp rx=0 tx=0).stop 2024-04-23T18:45:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc29481d740 1 -- 172.21.15.19:0/79262310 shutdown_connections 2024-04-23T18:45:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc29481d740 1 --2- 172.21.15.19:0/79262310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32867c10 0x557a32867020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc29481d740 1 --2- 172.21.15.19:0/79262310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a328640c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc29481d740 1 --2- 172.21.15.19:0/79262310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32858520 0x557a3285ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc29481d740 1 -- 172.21.15.19:0/79262310 >> 172.21.15.19:0/79262310 conn(0x557a32854b30 msgr2=0x557a3285a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc29481d740 1 -- 172.21.15.19:0/79262310 shutdown_connections 2024-04-23T18:45:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc29481d740 1 -- 172.21.15.19:0/79262310 wait complete. 2024-04-23T18:45:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.585+0000 7fc29481d740 1 -- start start 2024-04-23T18:45:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc29481d740 1 Processor -- start 2024-04-23T18:45:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc29481d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858520 0x557a32870150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc29481d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a32861f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc29481d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32867c10 0x557a32860a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc29481d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1acc80 con 0x557a32858e40 2024-04-23T18:45:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc29481d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1ac5b0 con 0x557a32867c10 2024-04-23T18:45:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc29481d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1a9810 con 0x557a32858520 2024-04-23T18:45:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc29481d740 10 client.0 _ll_register_callbacks cb 0x7fc2740046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc28b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a32861f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc28b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a32861f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59450/0 (socket says 172.21.15.19:59450) 2024-04-23T18:45:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc28b7fe640 1 -- 172.21.15.19:0/1566416205 learned_addr learned my addr 172.21.15.19:0/1566416205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc282ffd640 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858520 0x557a32870150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc28b7fe640 1 -- 172.21.15.19:0/1566416205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858520 msgr2=0x557a32870150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858520 0x557a32870150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc28b7fe640 1 -- 172.21.15.19:0/1566416205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32867c10 msgr2=0x557a32860a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32867c10 0x557a32860a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc28b7fe640 1 -- 172.21.15.19:0/1566416205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a32860f90 con 0x557a32858e40 2024-04-23T18:45:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.586+0000 7fc282ffd640 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858520 0x557a32870150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc28b7fe640 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a32861f10 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fc27c031440 tx=0x7fc27c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5db84bee664e3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc26affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc26affd640 1 -- 172.21.15.19:0/1566416205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c053060 con 0x557a32858e40 2024-04-23T18:45:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc26affd640 1 -- 172.21.15.19:0/1566416205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7fc27c005230 con 0x557a32858e40 2024-04-23T18:45:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc29481d740 1 -- 172.21.15.19:0/1566416205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc27c0054d0 con 0x557a32858e40 2024-04-23T18:45:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc26affd640 1 -- 172.21.15.19:0/1566416205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc27c035040 con 0x557a32858e40 2024-04-23T18:45:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc29481d740 10 client.17610 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc29481d740 1 -- 172.21.15.19:0/1566416205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a328dd510 con 0x557a32858e40 2024-04-23T18:45:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc26affd640 1 -- 172.21.15.19:0/1566416205 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc27c0054d0 con 0x557a32858e40 2024-04-23T18:45:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc26affd640 1 -- 172.21.15.19:0/1566416205 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc27c0035f0 con 0x557a32858e40 2024-04-23T18:45:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc29481d740 10 client.17610 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc29481d740 10 client.17610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc29481d740 20 client.17610 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc29481d740 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a328ddc60 0x557a328e0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc29481d740 4 client.17610 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.587+0000 7fc29481d740 1 -- 172.21.15.19:0/1566416205 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a327c5200 con 0x557a328ddc60 2024-04-23T18:45:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.588+0000 7fc282ffd640 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a328ddc60 0x557a328e0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:10.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.588+0000 7fc282ffd640 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a328ddc60 0x557a328e0080 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ab0fc6b48555d33 server_cookie=4a2cfb57f5a0021f in_seq=0 out_seq=0 2024-04-23T18:45:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.588+0000 7fc26affd640 10 client.17610 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.592+0000 7fc26affd640 1 -- 172.21.15.19:0/1566416205 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a327c5200 con 0x557a328ddc60 2024-04-23T18:45:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.592+0000 7fc26affd640 10 client.17610 handle_command_reply: tid=0 2024-04-23T18:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.592+0000 7fc29481d740 1 client.17610 shutdown 2024-04-23T18:45:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 1 -- 172.21.15.19:0/1566416205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 msgr2=0x557a32861f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a32861f10 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fc27c031440 tx=0x7fc27c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 1 -- 172.21.15.19:0/1566416205 shutdown_connections 2024-04-23T18:45:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x557a328ddc60 0x557a328e0080 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a32867c10 0x557a32860a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a32858e40 0x557a32861f10 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc26affd640 0 client.17610 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 1 --2- 172.21.15.19:0/1566416205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a32858520 0x557a32870150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 1 -- 172.21.15.19:0/1566416205 >> 172.21.15.19:0/1566416205 conn(0x557a32854b30 msgr2=0x557a3285fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 1 -- 172.21.15.19:0/1566416205 shutdown_connections 2024-04-23T18:45:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 1 -- 172.21.15.19:0/1566416205 wait complete. 2024-04-23T18:45:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc29481d740 20 client.17610 trim_cache size 0 max 16384 2024-04-23T18:45:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc291b08640 1 -- 172.21.15.19:0/4273888811 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc2780a3dc0 msgr2=0x7fc2780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc291b08640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc2780a3dc0 0x7fc2780a6280 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc2840cdf90 tx=0x7fc284005890 comp rx=0 tx=0).stop 2024-04-23T18:45:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc291b08640 1 -- 172.21.15.19:0/4273888811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc28c106fe0 msgr2=0x7fc28c19cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.593+0000 7fc291b08640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc28c106fe0 0x7fc28c19cf60 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc28400cad0 tx=0x7fc28400cf90 comp rx=0 tx=0).stop 2024-04-23T18:45:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.594+0000 7fc291b08640 1 -- 172.21.15.19:0/4273888811 shutdown_connections 2024-04-23T18:45:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.594+0000 7fc291b08640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc2780a3dc0 0x7fc2780a6280 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.594+0000 7fc291b08640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc28c101b40 0x7fc28c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.594+0000 7fc291b08640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc28c106fe0 0x7fc28c19cf60 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.594+0000 7fc291b08640 1 --2- 172.21.15.19:0/4273888811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c1011a0 0x7fc28c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.594+0000 7fc291b08640 1 -- 172.21.15.19:0/4273888811 >> 172.21.15.19:0/4273888811 conn(0x7fc28c076b80 msgr2=0x7fc28c0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.594+0000 7fc291b08640 1 -- 172.21.15.19:0/4273888811 shutdown_connections 2024-04-23T18:45:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:31.594+0000 7fc291b08640 1 -- 172.21.15.19:0/4273888811 wait complete. 2024-04-23T18:45:11.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:31 smithi019 ceph-mon[29215]: pgmap v548: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T18:45:11.244 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:33 smithi019 ceph-mon[29215]: pgmap v549: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:11.253 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:35 smithi019 ceph-mon[29215]: pgmap v550: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:11.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:37 smithi019 ceph-mon[29215]: pgmap v551: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:11.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:39 smithi019 ceph-mon[29215]: pgmap v552: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:11.279 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:41 smithi019 ceph-mon[29215]: pgmap v553: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:11.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:43 smithi019 ceph-mon[29215]: pgmap v554: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:11.352 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:07 smithi155 ceph-mon[33468]: pgmap v599: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:45:11.359 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:09 smithi155 ceph-mon[33468]: pgmap v600: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:45:11.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:07 smithi044 ceph-mon[32458]: pgmap v599: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:45:11.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:09 smithi044 ceph-mon[32458]: pgmap v600: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.692+0000 7fade2e2f640 1 -- 172.21.15.19:0/1190377147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faddc1016a0 msgr2=0x7faddc10e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.692+0000 7fade2e2f640 1 --2- 172.21.15.19:0/1190377147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faddc1016a0 0x7faddc10e3f0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009900 tx=0x7fadd002a410 comp rx=0 tx=0).stop 2024-04-23T18:45:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.692+0000 7fade2e2f640 1 -- 172.21.15.19:0/1190377147 shutdown_connections 2024-04-23T18:45:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.692+0000 7fade2e2f640 1 --2- 172.21.15.19:0/1190377147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc10e930 0x7faddc110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.692+0000 7fade2e2f640 1 --2- 172.21.15.19:0/1190377147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faddc1016a0 0x7faddc10e3f0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.692+0000 7fade2e2f640 1 --2- 172.21.15.19:0/1190377147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faddc100d80 0x7faddc101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.692+0000 7fade2e2f640 1 -- 172.21.15.19:0/1190377147 >> 172.21.15.19:0/1190377147 conn(0x7faddc077fd0 msgr2=0x7faddc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.692+0000 7fade2e2f640 1 -- 172.21.15.19:0/1190377147 shutdown_connections 2024-04-23T18:45:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.692+0000 7fade2e2f640 1 -- 172.21.15.19:0/1190377147 wait complete. 2024-04-23T18:45:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7fade2e2f640 1 Processor -- start 2024-04-23T18:45:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7fade2e2f640 1 -- start start 2024-04-23T18:45:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7fade2e2f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc100d80 0x7faddc1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7fade2e2f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faddc1016a0 0x7faddc1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7fade2e2f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faddc10e930 0x7faddc19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7fade2e2f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1143c0 con 0x7faddc1016a0 2024-04-23T18:45:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7fade2e2f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faddc114240 con 0x7faddc100d80 2024-04-23T18:45:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7fade2e2f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faddc114540 con 0x7faddc10e930 2024-04-23T18:45:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7faddbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faddc1016a0 0x7faddc1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7faddbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faddc1016a0 0x7faddc1a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59838/0 (socket says 172.21.15.19:59838) 2024-04-23T18:45:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7faddbfff640 1 -- 172.21.15.19:0/1867602626 learned_addr learned my addr 172.21.15.19:0/1867602626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7faddbfff640 1 -- 172.21.15.19:0/1867602626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faddc10e930 msgr2=0x7faddc19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7faddbfff640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faddc10e930 0x7faddc19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7fade0bcb640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc100d80 0x7faddc1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7faddbfff640 1 -- 172.21.15.19:0/1867602626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc100d80 msgr2=0x7faddc1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7faddbfff640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc100d80 0x7faddc1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.693+0000 7faddbfff640 1 -- 172.21.15.19:0/1867602626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc19afd0 con 0x7faddc1016a0 2024-04-23T18:45:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.694+0000 7faddbfff640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faddc1016a0 0x7faddc1a0bd0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fadd0005940 tx=0x7fadd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.694+0000 7fadd9ffb640 1 -- 172.21.15.19:0/1867602626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd003f030 con 0x7faddc1016a0 2024-04-23T18:45:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.694+0000 7fadd9ffb640 1 -- 172.21.15.19:0/1867602626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd0003cb0 con 0x7faddc1016a0 2024-04-23T18:45:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.694+0000 7fade2e2f640 1 -- 172.21.15.19:0/1867602626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc19b260 con 0x7faddc1016a0 2024-04-23T18:45:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.694+0000 7fadd9ffb640 1 -- 172.21.15.19:0/1867602626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0035070 con 0x7faddc1016a0 2024-04-23T18:45:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.694+0000 7fade2e2f640 1 -- 172.21.15.19:0/1867602626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1a74e0 con 0x7faddc1016a0 2024-04-23T18:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.696+0000 7fadd9ffb640 1 -- 172.21.15.19:0/1867602626 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fadd0042030 con 0x7faddc1016a0 2024-04-23T18:45:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade5c18740 1 Processor -- start 2024-04-23T18:45:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade5c18740 1 -- start start 2024-04-23T18:45:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade5c18740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x56175720fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade5c18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175720ff40 0x56175721a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade5c18740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175721a800 0x56175721cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade5c18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561757174970 con 0x56175720ff40 2024-04-23T18:45:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade5c18740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5617571114a0 con 0x56175721a800 2024-04-23T18:45:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade5c18740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561757170930 con 0x56175720f620 2024-04-23T18:45:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fadd9ffb640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fadc40a3dc0 0x7fadc40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade0bcb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175720ff40 0x56175721a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade0bcb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175720ff40 0x56175721a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59850/0 (socket says 172.21.15.19:59850) 2024-04-23T18:45:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade0bcb640 1 -- 172.21.15.19:0/2860814432 learned_addr learned my addr 172.21.15.19:0/2860814432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade13cc640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fadc40a3dc0 0x7fadc40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7faddbfff640 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175721a800 0x56175721cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade13cc640 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x56175720fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fadd9ffb640 1 -- 172.21.15.19:0/1867602626 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7fadd00f1100 con 0x7faddc1016a0 2024-04-23T18:45:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade0bcb640 1 -- 172.21.15.19:0/2860814432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 msgr2=0x56175720fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade0bcb640 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x56175720fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade0bcb640 1 -- 172.21.15.19:0/2860814432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175721a800 msgr2=0x56175721cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade0bcb640 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175721a800 0x56175721cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade0bcb640 1 -- 172.21.15.19:0/2860814432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561756e87210 con 0x56175720ff40 2024-04-23T18:45:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7faddbfff640 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175721a800 0x56175721cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.697+0000 7fade13cc640 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x56175720fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.698+0000 7fade0bcb640 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175720ff40 0x56175721a2c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009bc0 tx=0x7fadcc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b4170f81dcf98a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.698+0000 7fade13cc640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fadc40a3dc0 0x7fadc40a6280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fadd40092f0 tx=0x7fadd4009420 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.698+0000 7fadc2ffd640 1 -- 172.21.15.19:0/2860814432 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc016070 con 0x56175720ff40 2024-04-23T18:45:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.698+0000 7fadc2ffd640 1 -- 172.21.15.19:0/2860814432 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadcc0054d0 con 0x56175720ff40 2024-04-23T18:45:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.698+0000 7fadc2ffd640 1 -- 172.21.15.19:0/2860814432 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc011470 con 0x56175720ff40 2024-04-23T18:45:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.698+0000 7fade5c18740 1 -- 172.21.15.19:0/2860814432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175720ff40 msgr2=0x56175721a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.698+0000 7fade5c18740 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175720ff40 0x56175721a2c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009bc0 tx=0x7fadcc00e6c0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 -- 172.21.15.19:0/2860814432 shutdown_connections 2024-04-23T18:45:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175721a800 0x56175721cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175720ff40 0x56175721a2c0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 --2- 172.21.15.19:0/2860814432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x56175720fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 -- 172.21.15.19:0/2860814432 >> 172.21.15.19:0/2860814432 conn(0x56175720bb30 msgr2=0x56175720bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 -- 172.21.15.19:0/2860814432 shutdown_connections 2024-04-23T18:45:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 -- 172.21.15.19:0/2860814432 wait complete. 2024-04-23T18:45:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 -- start start 2024-04-23T18:45:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 Processor -- start 2024-04-23T18:45:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720f620 0x56175721a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720ff40 0x561757217440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.699+0000 7fade5c18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572179a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade5c18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56175721d430 con 0x56175721a800 2024-04-23T18:45:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade5c18740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56175710f0b0 con 0x56175720f620 2024-04-23T18:45:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade5c18740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56175721d5b0 con 0x56175720ff40 2024-04-23T18:45:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade5c18740 10 client.0 _ll_register_callbacks cb 0x5617570fcd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade13cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade13cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572179a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59866/0 (socket says 172.21.15.19:59866) 2024-04-23T18:45:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade13cc640 1 -- 172.21.15.19:0/735134595 learned_addr learned my addr 172.21.15.19:0/735134595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade0bcb640 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720f620 0x56175721a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7faddbfff640 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720ff40 0x561757217440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade13cc640 1 -- 172.21.15.19:0/735134595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720ff40 msgr2=0x561757217440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade13cc640 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720ff40 0x561757217440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade13cc640 1 -- 172.21.15.19:0/735134595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720f620 msgr2=0x56175721a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade13cc640 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720f620 0x56175721a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade13cc640 1 -- 172.21.15.19:0/735134595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561756f53620 con 0x56175721a800 2024-04-23T18:45:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7faddbfff640 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720ff40 0x561757217440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fade13cc640 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572179a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fadd4012170 tx=0x7fadd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffd55c6c223065c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fadbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.700+0000 7fadbbfff640 1 -- 172.21.15.19:0/735134595 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4011470 con 0x56175721a800 2024-04-23T18:45:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fadbbfff640 1 -- 172.21.15.19:0/735134595 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7fadd4011610 con 0x56175721a800 2024-04-23T18:45:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fade5c18740 1 -- 172.21.15.19:0/735134595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadcc0054d0 con 0x56175721a800 2024-04-23T18:45:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fadbbfff640 1 -- 172.21.15.19:0/735134595 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fadd4011810 con 0x56175721a800 2024-04-23T18:45:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fade5c18740 10 client.17622 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fade5c18740 1 -- 172.21.15.19:0/735134595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561757229330 con 0x56175721a800 2024-04-23T18:45:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fadbbfff640 1 -- 172.21.15.19:0/735134595 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadcc0054d0 con 0x56175721a800 2024-04-23T18:45:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fadbbfff640 1 -- 172.21.15.19:0/735134595 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fadd401a070 con 0x56175721a800 2024-04-23T18:45:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fade5c18740 10 client.17622 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fade5c18740 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fade5c18740 20 client.17622 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fade5c18740 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56175722a7f0 0x56175722cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fade5c18740 4 client.17622 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fade5c18740 1 -- 172.21.15.19:0/735134595 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561757182390 con 0x56175722a7f0 2024-04-23T18:45:13.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.701+0000 7fade0bcb640 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56175722a7f0 0x56175722cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.702+0000 7fadbbfff640 10 client.17622 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.702+0000 7fade0bcb640 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56175722a7f0 0x56175722cc40 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f4040f6a9dc743 server_cookie=3e63b4458e12b188 in_seq=0 out_seq=0 2024-04-23T18:45:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fadbbfff640 1 -- 172.21.15.19:0/735134595 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561757182390 con 0x56175722a7f0 2024-04-23T18:45:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fadbbfff640 10 client.17622 handle_command_reply: tid=0 2024-04-23T18:45:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 client.17622 shutdown 2024-04-23T18:45:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 -- 172.21.15.19:0/735134595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 msgr2=0x5617572179a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572179a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fadd4012170 tx=0x7fadd4004a40 comp rx=0 tx=0).stop 2024-04-23T18:45:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 -- 172.21.15.19:0/735134595 shutdown_connections 2024-04-23T18:45:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56175722a7f0 0x56175722cc40 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572179a0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720ff40 0x561757217440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 --2- 172.21.15.19:0/735134595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720f620 0x56175721a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fadbbfff640 0 client.17622 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 -- 172.21.15.19:0/735134595 >> 172.21.15.19:0/735134595 conn(0x56175720bb30 msgr2=0x561757211860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 -- 172.21.15.19:0/735134595 shutdown_connections 2024-04-23T18:45:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 1 -- 172.21.15.19:0/735134595 wait complete. 2024-04-23T18:45:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.703+0000 7fade5c18740 20 client.17622 trim_cache size 0 max 16384 2024-04-23T18:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.718+0000 7fade5c18740 1 Processor -- start 2024-04-23T18:45:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.718+0000 7fade5c18740 1 -- start start 2024-04-23T18:45:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.718+0000 7fade5c18740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x5617572262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.718+0000 7fade5c18740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 0x56175720ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.718+0000 7fade5c18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572325a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.718+0000 7fade5c18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1afd10 con 0x56175721a800 2024-04-23T18:45:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.718+0000 7fade5c18740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1b1850 con 0x56175720ff40 2024-04-23T18:45:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.718+0000 7fade5c18740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1b03e0 con 0x56175720f620 2024-04-23T18:45:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade13cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade13cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572325a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59876/0 (socket says 172.21.15.19:59876) 2024-04-23T18:45:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade13cc640 1 -- 172.21.15.19:0/1575530696 learned_addr learned my addr 172.21.15.19:0/1575530696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade13cc640 1 -- 172.21.15.19:0/1575530696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 msgr2=0x5617572262f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade0bcb640 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x5617572262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade13cc640 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x5617572262f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade13cc640 1 -- 172.21.15.19:0/1575530696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 msgr2=0x56175720ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7faddbfff640 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 0x56175720ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade13cc640 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 0x56175720ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade13cc640 1 -- 172.21.15.19:0/1575530696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561757215990 con 0x56175721a800 2024-04-23T18:45:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade0bcb640 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x5617572262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7faddbfff640 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 0x56175720ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fade13cc640 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572325a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fadd4026b10 tx=0x7fadd4005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394d4d839f03b23c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fadc27fc640 1 -- 172.21.15.19:0/1575530696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4033070 con 0x56175721a800 2024-04-23T18:45:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fadc27fc640 1 -- 172.21.15.19:0/1575530696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd4012260 con 0x56175721a800 2024-04-23T18:45:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.719+0000 7fadc27fc640 1 -- 172.21.15.19:0/1575530696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd401c560 con 0x56175721a800 2024-04-23T18:45:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 -- 172.21.15.19:0/1575530696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 msgr2=0x5617572325a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572325a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fadd4026b10 tx=0x7fadd4005d10 comp rx=0 tx=0).stop 2024-04-23T18:45:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 -- 172.21.15.19:0/1575530696 shutdown_connections 2024-04-23T18:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 0x56175720ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x5617572325a0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 --2- 172.21.15.19:0/1575530696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x5617572262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 -- 172.21.15.19:0/1575530696 >> 172.21.15.19:0/1575530696 conn(0x56175720bb30 msgr2=0x56175720a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 -- 172.21.15.19:0/1575530696 shutdown_connections 2024-04-23T18:45:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 -- 172.21.15.19:0/1575530696 wait complete. 2024-04-23T18:45:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 -- start start 2024-04-23T18:45:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.720+0000 7fade5c18740 1 Processor -- start 2024-04-23T18:45:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade5c18740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x561757222c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade5c18740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 0x56175721ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade5c18740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x561757220510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade5c18740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1b1180 con 0x56175721a800 2024-04-23T18:45:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade5c18740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1b0ab0 con 0x56175720ff40 2024-04-23T18:45:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade5c18740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1adcf0 con 0x56175720f620 2024-04-23T18:45:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade5c18740 10 client.0 _ll_register_callbacks cb 0x7fadd40099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade13cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x561757220510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade13cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x561757220510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59884/0 (socket says 172.21.15.19:59884) 2024-04-23T18:45:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade13cc640 1 -- 172.21.15.19:0/1434592754 learned_addr learned my addr 172.21.15.19:0/1434592754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade0bcb640 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x561757222c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7faddbfff640 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 0x56175721ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade13cc640 1 -- 172.21.15.19:0/1434592754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 msgr2=0x561757222c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade13cc640 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x561757222c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade13cc640 1 -- 172.21.15.19:0/1434592754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 msgr2=0x56175721ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade13cc640 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 0x56175721ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade13cc640 1 -- 172.21.15.19:0/1434592754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561757220d90 con 0x56175721a800 2024-04-23T18:45:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7faddbfff640 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 0x56175721ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fade13cc640 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x561757220510 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009fd0 tx=0x7fadd40264b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8211cc69b4c163f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.721+0000 7fadbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fadbbfff640 1 -- 172.21.15.19:0/1434592754 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4033070 con 0x56175721a800 2024-04-23T18:45:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fadbbfff640 1 -- 172.21.15.19:0/1434592754 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7fadd4011bb0 con 0x56175721a800 2024-04-23T18:45:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fade5c18740 1 -- 172.21.15.19:0/1434592754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd4012260 con 0x56175721a800 2024-04-23T18:45:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fadbbfff640 1 -- 172.21.15.19:0/1434592754 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fadd4047050 con 0x56175721a800 2024-04-23T18:45:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fade5c18740 10 client.17628 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fade5c18740 1 -- 172.21.15.19:0/1434592754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561757290e10 con 0x56175721a800 2024-04-23T18:45:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fadbbfff640 1 -- 172.21.15.19:0/1434592754 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd4012260 con 0x56175721a800 2024-04-23T18:45:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fadbbfff640 1 -- 172.21.15.19:0/1434592754 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fadd401a070 con 0x56175721a800 2024-04-23T18:45:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fade5c18740 10 client.17628 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fade5c18740 10 client.17628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:14.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fade5c18740 20 client.17628 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fade5c18740 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561757291410 0x561757293830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fade5c18740 4 client.17628 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.722+0000 7fade5c18740 1 -- 172.21.15.19:0/1434592754 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56175717c200 con 0x561757291410 2024-04-23T18:45:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.723+0000 7fade0bcb640 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561757291410 0x561757293830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.723+0000 7fade0bcb640 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561757291410 0x561757293830 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1228d478248affc server_cookie=7354806d9426e6b2 in_seq=0 out_seq=0 2024-04-23T18:45:14.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.723+0000 7fadbbfff640 10 client.17628 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fadbbfff640 1 -- 172.21.15.19:0/1434592754 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56175717c200 con 0x561757291410 2024-04-23T18:45:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fadbbfff640 10 client.17628 handle_command_reply: tid=0 2024-04-23T18:45:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 client.17628 shutdown 2024-04-23T18:45:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 -- 172.21.15.19:0/1434592754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 msgr2=0x561757220510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x561757220510 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009fd0 tx=0x7fadd40264b0 comp rx=0 tx=0).stop 2024-04-23T18:45:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 -- 172.21.15.19:0/1434592754 shutdown_connections 2024-04-23T18:45:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561757291410 0x561757293830 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56175720ff40 0x56175721ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56175721a800 0x561757220510 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 --2- 172.21.15.19:0/1434592754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175720f620 0x561757222c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fadbbfff640 0 client.17628 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 -- 172.21.15.19:0/1434592754 >> 172.21.15.19:0/1434592754 conn(0x56175720bb30 msgr2=0x56175720acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 -- 172.21.15.19:0/1434592754 shutdown_connections 2024-04-23T18:45:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 1 -- 172.21.15.19:0/1434592754 wait complete. 2024-04-23T18:45:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade5c18740 20 client.17628 trim_cache size 0 max 16384 2024-04-23T18:45:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 -- 172.21.15.19:0/1867602626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fadc40a3dc0 msgr2=0x7fadc40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fadc40a3dc0 0x7fadc40a6280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fadd40092f0 tx=0x7fadd4009420 comp rx=0 tx=0).stop 2024-04-23T18:45:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 -- 172.21.15.19:0/1867602626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faddc1016a0 msgr2=0x7faddc1a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faddc1016a0 0x7faddc1a0bd0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fadd0005940 tx=0x7fadd00042f0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 -- 172.21.15.19:0/1867602626 shutdown_connections 2024-04-23T18:45:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fadc40a3dc0 0x7fadc40a6280 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faddc10e930 0x7faddc19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faddc1016a0 0x7faddc1a0bd0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 --2- 172.21.15.19:0/1867602626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc100d80 0x7faddc1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 -- 172.21.15.19:0/1867602626 >> 172.21.15.19:0/1867602626 conn(0x7faddc077fd0 msgr2=0x7faddc10ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 -- 172.21.15.19:0/1867602626 shutdown_connections 2024-04-23T18:45:15.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:44.730+0000 7fade2e2f640 1 -- 172.21.15.19:0/1867602626 wait complete. 2024-04-23T18:45:15.215 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:45 smithi019 ceph-mon[29215]: pgmap v555: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:15.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:47 smithi019 ceph-mon[29215]: pgmap v556: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:15.232 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:49 smithi019 ceph-mon[29215]: pgmap v557: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:15.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:51 smithi019 ceph-mon[29215]: pgmap v558: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:15.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:53 smithi019 ceph-mon[29215]: pgmap v559: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:15.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:55 smithi019 ceph-mon[29215]: pgmap v560: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:15.266 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:11 smithi155 ceph-mon[33468]: pgmap v601: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:45:15.384 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:13 smithi155 ceph-mon[33468]: pgmap v602: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:45:15.396 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:13 smithi155 ceph-mon[33468]: osdmap e117: 12 total, 12 up, 12 in 2024-04-23T18:45:15.404 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:11 smithi044 ceph-mon[32458]: pgmap v601: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:45:15.410 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:13 smithi044 ceph-mon[32458]: pgmap v602: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:45:15.415 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:13 smithi044 ceph-mon[32458]: osdmap e117: 12 total, 12 up, 12 in 2024-04-23T18:45:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.831+0000 7ff8f0929640 1 -- 172.21.15.19:0/287248042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8ec10c520 msgr2=0x7ff8ec10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.831+0000 7ff8f0929640 1 --2- 172.21.15.19:0/287248042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8ec10c520 0x7ff8ec10e9c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009ca0 tx=0x7ff8e402a9e0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.831+0000 7ff8f0929640 1 -- 172.21.15.19:0/287248042 shutdown_connections 2024-04-23T18:45:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.831+0000 7ff8f0929640 1 --2- 172.21.15.19:0/287248042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ec103b00 0x7ff8ec10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.831+0000 7ff8f0929640 1 --2- 172.21.15.19:0/287248042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8ec10c520 0x7ff8ec10e9c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.831+0000 7ff8f0929640 1 --2- 172.21.15.19:0/287248042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8ec1031e0 0x7ff8ec1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.831+0000 7ff8f0929640 1 -- 172.21.15.19:0/287248042 >> 172.21.15.19:0/287248042 conn(0x7ff8ec077fd0 msgr2=0x7ff8ec0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.831+0000 7ff8f0929640 1 -- 172.21.15.19:0/287248042 shutdown_connections 2024-04-23T18:45:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.831+0000 7ff8f0929640 1 -- 172.21.15.19:0/287248042 wait complete. 2024-04-23T18:45:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8f0929640 1 Processor -- start 2024-04-23T18:45:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8f0929640 1 -- start start 2024-04-23T18:45:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8f0929640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ec1031e0 0x7ff8ec1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8f0929640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8ec103b00 0x7ff8ec198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8f0929640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8ec10c520 0x7ff8ec19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:15.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8f0929640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec112200 con 0x7ff8ec103b00 2024-04-23T18:45:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8f0929640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec112080 con 0x7ff8ec1031e0 2024-04-23T18:45:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8f0929640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec112380 con 0x7ff8ec10c520 2024-04-23T18:45:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8e9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8ec103b00 0x7ff8ec198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8e9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8ec103b00 0x7ff8ec198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59726/0 (socket says 172.21.15.19:59726) 2024-04-23T18:45:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8e9d9b640 1 -- 172.21.15.19:0/1127116541 learned_addr learned my addr 172.21.15.19:0/1127116541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8ec10c520 0x7ff8ec19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ec1031e0 0x7ff8ec1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8e9d9b640 1 -- 172.21.15.19:0/1127116541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8ec10c520 msgr2=0x7ff8ec19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.832+0000 7ff8e9d9b640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8ec10c520 0x7ff8ec19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.833+0000 7ff8e9d9b640 1 -- 172.21.15.19:0/1127116541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ec1031e0 msgr2=0x7ff8ec1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.833+0000 7ff8e9d9b640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ec1031e0 0x7ff8ec1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.833+0000 7ff8e9d9b640 1 -- 172.21.15.19:0/1127116541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec19d640 con 0x7ff8ec103b00 2024-04-23T18:45:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.833+0000 7ff8e9d9b640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8ec103b00 0x7ff8ec198be0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000d950 tx=0x7ff8e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.833+0000 7ff8db7fe640 1 -- 172.21.15.19:0/1127116541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0017030 con 0x7ff8ec103b00 2024-04-23T18:45:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.833+0000 7ff8db7fe640 1 -- 172.21.15.19:0/1127116541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e0004020 con 0x7ff8ec103b00 2024-04-23T18:45:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.833+0000 7ff8f0929640 1 -- 172.21.15.19:0/1127116541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec19d930 con 0x7ff8ec103b00 2024-04-23T18:45:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.833+0000 7ff8db7fe640 1 -- 172.21.15.19:0/1127116541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00047a0 con 0x7ff8ec103b00 2024-04-23T18:45:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.833+0000 7ff8f0929640 1 -- 172.21.15.19:0/1127116541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec1a51c0 con 0x7ff8ec103b00 2024-04-23T18:45:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.835+0000 7ff8db7fe640 1 -- 172.21.15.19:0/1127116541 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff8e0009a30 con 0x7ff8ec103b00 2024-04-23T18:45:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8f3640740 1 Processor -- start 2024-04-23T18:45:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8f3640740 1 -- start start 2024-04-23T18:45:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158620 0x55e73a158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a1632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a163800 0x55e73a165bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8f3640740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e73a0bd970 con 0x55e73a158f40 2024-04-23T18:45:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8f3640740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e73a05a4a0 con 0x55e73a163800 2024-04-23T18:45:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8f3640740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e73a0b9930 con 0x55e73a158620 2024-04-23T18:45:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8db7fe640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8d40a3dc0 0x7ff8d40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8ea59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a1632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8d40a3dc0 0x7ff8d40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8ea59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a1632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59738/0 (socket says 172.21.15.19:59738) 2024-04-23T18:45:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8ea59c640 1 -- 172.21.15.19:0/2551052693 learned_addr learned my addr 172.21.15.19:0/2551052693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.836+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/2551052693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158620 0x55e73a158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8db7fe640 1 -- 172.21.15.19:0/1127116541 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7ff8e00d00e0 con 0x7ff8ec103b00 2024-04-23T18:45:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8e9d9b640 1 --2- 172.21.15.19:0/2551052693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a163800 0x55e73a165bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8ea59c640 1 -- 172.21.15.19:0/2551052693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158620 msgr2=0x55e73a158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/2551052693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158620 0x55e73a158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8ea59c640 1 -- 172.21.15.19:0/2551052693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a163800 msgr2=0x55e73a165bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/2551052693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a163800 0x55e73a165bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8ea59c640 1 -- 172.21.15.19:0/2551052693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e739dd0210 con 0x55e73a158f40 2024-04-23T18:45:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8d40a3dc0 0x7ff8d40a6280 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005fe0 tx=0x7ff8e4009360 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/2551052693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a1632c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00bcf0 tx=0x7ff8dc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81bd8fe381a633b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8d8ff9640 1 -- 172.21.15.19:0/2551052693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc015070 con 0x55e73a158f40 2024-04-23T18:45:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8d8ff9640 1 -- 172.21.15.19:0/2551052693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc01a910 con 0x55e73a158f40 2024-04-23T18:45:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.837+0000 7ff8d8ff9640 1 -- 172.21.15.19:0/2551052693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc011860 con 0x55e73a158f40 2024-04-23T18:45:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.838+0000 7ff8f3640740 1 -- 172.21.15.19:0/2551052693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 msgr2=0x55e73a1632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.838+0000 7ff8f3640740 1 --2- 172.21.15.19:0/2551052693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a1632c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00bcf0 tx=0x7ff8dc00f730 comp rx=0 tx=0).stop 2024-04-23T18:45:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.838+0000 7ff8f3640740 1 -- 172.21.15.19:0/2551052693 shutdown_connections 2024-04-23T18:45:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.838+0000 7ff8f3640740 1 --2- 172.21.15.19:0/2551052693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a163800 0x55e73a165bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.838+0000 7ff8f3640740 1 --2- 172.21.15.19:0/2551052693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a1632c0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.838+0000 7ff8f3640740 1 --2- 172.21.15.19:0/2551052693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158620 0x55e73a158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.838+0000 7ff8f3640740 1 -- 172.21.15.19:0/2551052693 >> 172.21.15.19:0/2551052693 conn(0x55e73a154b30 msgr2=0x55e73a154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.838+0000 7ff8f3640740 1 -- 172.21.15.19:0/2551052693 shutdown_connections 2024-04-23T18:45:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.838+0000 7ff8f3640740 1 -- 172.21.15.19:0/2551052693 wait complete. 2024-04-23T18:45:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.838+0000 7ff8f3640740 1 -- start start 2024-04-23T18:45:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8f3640740 1 Processor -- start 2024-04-23T18:45:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a163160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a160450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a163800 0x55e73a1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8f3640740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e73a166430 con 0x55e73a158f40 2024-04-23T18:45:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8f3640740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e73a0580b0 con 0x55e73a158620 2024-04-23T18:45:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8f3640740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e73a1665b0 con 0x55e73a163800 2024-04-23T18:45:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8f3640740 10 client.0 _ll_register_callbacks cb 0x55e73a045d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8e9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8e9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a160450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59746/0 (socket says 172.21.15.19:59746) 2024-04-23T18:45:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8e9d9b640 1 -- 172.21.15.19:0/1019451226 learned_addr learned my addr 172.21.15.19:0/1019451226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a163160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a163800 0x55e73a1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8e9d9b640 1 -- 172.21.15.19:0/1019451226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a163800 msgr2=0x55e73a1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8e9d9b640 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a163800 0x55e73a1609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8e9d9b640 1 -- 172.21.15.19:0/1019451226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 msgr2=0x55e73a163160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8e9d9b640 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a163160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8e9d9b640 1 -- 172.21.15.19:0/1019451226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e739e9c620 con 0x55e73a158f40 2024-04-23T18:45:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8e9d9b640 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a160450 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff8e009c0e0 tx=0x7ff8e0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c73b736105abfd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.839+0000 7ff8d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1019451226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00b4020 con 0x55e73a158f40 2024-04-23T18:45:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1019451226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7ff8e00cdb80 con 0x55e73a158f40 2024-04-23T18:45:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8f3640740 1 -- 172.21.15.19:0/1019451226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8dc01a910 con 0x55e73a158f40 2024-04-23T18:45:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1019451226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8e00a0500 con 0x55e73a158f40 2024-04-23T18:45:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8f3640740 10 client.17640 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8f3640740 1 -- 172.21.15.19:0/1019451226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e73a172330 con 0x55e73a158f40 2024-04-23T18:45:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1019451226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc01a910 con 0x55e73a158f40 2024-04-23T18:45:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1019451226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff8e0097ea0 con 0x55e73a158f40 2024-04-23T18:45:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8f3640740 10 client.17640 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8f3640740 10 client.17640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8f3640740 20 client.17640 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e73a1737a0 0x55e73a175b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8f3640740 4 client.17640 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8f3640740 1 -- 172.21.15.19:0/1019451226 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e73a0cb390 con 0x55e73a1737a0 2024-04-23T18:45:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.840+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e73a1737a0 0x55e73a175b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.841+0000 7ff8d1ffb640 10 client.17640 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.841+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e73a1737a0 0x55e73a175b90 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=861959753c1d0fea server_cookie=b020fdcbcb6b82d3 in_seq=0 out_seq=0 2024-04-23T18:45:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1019451226 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e73a0cb390 con 0x55e73a1737a0 2024-04-23T18:45:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8d1ffb640 10 client.17640 handle_command_reply: tid=0 2024-04-23T18:45:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 client.17640 shutdown 2024-04-23T18:45:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 -- 172.21.15.19:0/1019451226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 msgr2=0x55e73a160450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a160450 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff8e009c0e0 tx=0x7ff8e0005760 comp rx=0 tx=0).stop 2024-04-23T18:45:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 -- 172.21.15.19:0/1019451226 shutdown_connections 2024-04-23T18:45:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e73a1737a0 0x55e73a175b90 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a163800 0x55e73a1609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a158f40 0x55e73a160450 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1019451226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a163160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8d1ffb640 0 client.17640 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 -- 172.21.15.19:0/1019451226 >> 172.21.15.19:0/1019451226 conn(0x55e73a154b30 msgr2=0x55e73a15a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 -- 172.21.15.19:0/1019451226 shutdown_connections 2024-04-23T18:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 1 -- 172.21.15.19:0/1019451226 wait complete. 2024-04-23T18:45:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.842+0000 7ff8f3640740 20 client.17640 trim_cache size 0 max 16384 2024-04-23T18:45:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.857+0000 7ff8f3640740 1 Processor -- start 2024-04-23T18:45:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.857+0000 7ff8f3640740 1 -- start start 2024-04-23T18:45:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.857+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a16f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.857+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158f40 0x55e73a15b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.857+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:17.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.857+0000 7ff8f3640740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ab890 con 0x55e73a163800 2024-04-23T18:45:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.857+0000 7ff8f3640740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ad3d0 con 0x55e73a158620 2024-04-23T18:45:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.857+0000 7ff8f3640740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1abf60 con 0x55e73a158f40 2024-04-23T18:45:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ead9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ead9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a162c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59754/0 (socket says 172.21.15.19:59754) 2024-04-23T18:45:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ead9d640 1 -- 172.21.15.19:0/3757606338 learned_addr learned my addr 172.21.15.19:0/3757606338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/3757606338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a16f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ead9d640 1 -- 172.21.15.19:0/3757606338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158f40 msgr2=0x55e73a15b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/3757606338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158f40 0x55e73a15b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ead9d640 1 -- 172.21.15.19:0/3757606338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 msgr2=0x55e73a16f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/3757606338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a16f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ead9d640 1 -- 172.21.15.19:0/3757606338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e73a158d30 con 0x55e73a163800 2024-04-23T18:45:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/3757606338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a16f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/3757606338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a162c90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009ca0 tx=0x7ff8e4047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0264dd9c4ec7504 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8d3fff640 1 -- 172.21.15.19:0/3757606338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4048040 con 0x55e73a163800 2024-04-23T18:45:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8d3fff640 1 -- 172.21.15.19:0/3757606338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e402fd90 con 0x55e73a163800 2024-04-23T18:45:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.858+0000 7ff8d3fff640 1 -- 172.21.15.19:0/3757606338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4033040 con 0x55e73a163800 2024-04-23T18:45:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 -- 172.21.15.19:0/3757606338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 msgr2=0x55e73a162c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 --2- 172.21.15.19:0/3757606338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a162c90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009ca0 tx=0x7ff8e4047000 comp rx=0 tx=0).stop 2024-04-23T18:45:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 -- 172.21.15.19:0/3757606338 shutdown_connections 2024-04-23T18:45:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 --2- 172.21.15.19:0/3757606338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a162c90 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 --2- 172.21.15.19:0/3757606338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158f40 0x55e73a15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 --2- 172.21.15.19:0/3757606338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a16f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 -- 172.21.15.19:0/3757606338 >> 172.21.15.19:0/3757606338 conn(0x55e73a154b30 msgr2=0x55e73a15b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 -- 172.21.15.19:0/3757606338 shutdown_connections 2024-04-23T18:45:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 -- 172.21.15.19:0/3757606338 wait complete. 2024-04-23T18:45:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 -- start start 2024-04-23T18:45:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.859+0000 7ff8f3640740 1 Processor -- start 2024-04-23T18:45:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a16bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158f40 0x55e73a167030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8f3640740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a1675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8f3640740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1acd00 con 0x55e73a163800 2024-04-23T18:45:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8f3640740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ac630 con 0x55e73a158620 2024-04-23T18:45:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8f3640740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1a9890 con 0x55e73a158f40 2024-04-23T18:45:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8f3640740 10 client.0 _ll_register_callbacks cb 0x7ff8e009bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ead9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a1675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ead9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a1675d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59768/0 (socket says 172.21.15.19:59768) 2024-04-23T18:45:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ead9d640 1 -- 172.21.15.19:0/1421089470 learned_addr learned my addr 172.21.15.19:0/1421089470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a16bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8e9d9b640 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158f40 0x55e73a167030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ead9d640 1 -- 172.21.15.19:0/1421089470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158f40 msgr2=0x55e73a167030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158f40 0x55e73a167030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ead9d640 1 -- 172.21.15.19:0/1421089470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 msgr2=0x55e73a16bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a16bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ead9d640 1 -- 172.21.15.19:0/1421089470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e73a167e00 con 0x55e73a163800 2024-04-23T18:45:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a16bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:45:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8ead9d640 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a1675d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004af0 tx=0x7ff8e405ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10a42cc4d296bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1421089470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4062060 con 0x55e73a163800 2024-04-23T18:45:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1421089470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7ff8e4004600 con 0x55e73a163800 2024-04-23T18:45:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.860+0000 7ff8f3640740 1 -- 172.21.15.19:0/1421089470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e402fd90 con 0x55e73a163800 2024-04-23T18:45:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1421089470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8e4048040 con 0x55e73a163800 2024-04-23T18:45:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8f3640740 10 client.17646 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8f3640740 1 -- 172.21.15.19:0/1421089470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e73a1d9df0 con 0x55e73a163800 2024-04-23T18:45:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1421089470 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e402fd90 con 0x55e73a163800 2024-04-23T18:45:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1421089470 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff8e403e450 con 0x55e73a163800 2024-04-23T18:45:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8f3640740 10 client.17646 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8f3640740 10 client.17646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8f3640740 20 client.17646 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e73a1da4e0 0x55e73a1dc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8f3640740 4 client.17646 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8f3640740 1 -- 172.21.15.19:0/1421089470 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e73a0c5200 con 0x55e73a1da4e0 2024-04-23T18:45:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.861+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e73a1da4e0 0x55e73a1dc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.862+0000 7ff8ea59c640 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e73a1da4e0 0x55e73a1dc900 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be2e3d15a2a2baa1 server_cookie=2b4460ad7f3d6a55 in_seq=0 out_seq=0 2024-04-23T18:45:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.862+0000 7ff8d1ffb640 10 client.17646 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.866+0000 7ff8d1ffb640 1 -- 172.21.15.19:0/1421089470 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e73a0c5200 con 0x55e73a1da4e0 2024-04-23T18:45:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.866+0000 7ff8d1ffb640 10 client.17646 handle_command_reply: tid=0 2024-04-23T18:45:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.866+0000 7ff8f3640740 1 client.17646 shutdown 2024-04-23T18:45:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 1 -- 172.21.15.19:0/1421089470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 msgr2=0x55e73a1675d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a1675d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004af0 tx=0x7ff8e405ae70 comp rx=0 tx=0).stop 2024-04-23T18:45:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 1 -- 172.21.15.19:0/1421089470 shutdown_connections 2024-04-23T18:45:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55e73a1da4e0 0x55e73a1dc900 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e73a163800 0x55e73a1675d0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8d1ffb640 0 client.17646 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73a158f40 0x55e73a167030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 1 --2- 172.21.15.19:0/1421089470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e73a158620 0x55e73a16bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 1 -- 172.21.15.19:0/1421089470 >> 172.21.15.19:0/1421089470 conn(0x55e73a154b30 msgr2=0x55e73a153cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 1 -- 172.21.15.19:0/1421089470 shutdown_connections 2024-04-23T18:45:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 1 -- 172.21.15.19:0/1421089470 wait complete. 2024-04-23T18:45:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f3640740 20 client.17646 trim_cache size 0 max 16384 2024-04-23T18:45:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:19.012 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:15 smithi044 ceph-mon[32458]: pgmap v604: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 460 B/s wr, 10 op/s 2024-04-23T18:45:19.020 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:15 smithi155 ceph-mon[33468]: pgmap v604: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 460 B/s wr, 10 op/s 2024-04-23T18:45:19.027 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:17 smithi155 ceph-mon[33468]: pgmap v605: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-04-23T18:45:19.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:17 smithi155 ceph-mon[33468]: daemon mds.e restarted 2024-04-23T18:45:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 -- 172.21.15.19:0/1127116541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8d40a3dc0 msgr2=0x7ff8d40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8d40a3dc0 0x7ff8d40a6280 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005fe0 tx=0x7ff8e4009360 comp rx=0 tx=0).stop 2024-04-23T18:45:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 -- 172.21.15.19:0/1127116541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8ec103b00 msgr2=0x7ff8ec198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8ec103b00 0x7ff8ec198be0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000d950 tx=0x7ff8e000de10 comp rx=0 tx=0).stop 2024-04-23T18:45:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 -- 172.21.15.19:0/1127116541 shutdown_connections 2024-04-23T18:45:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7ff8d40a3dc0 0x7ff8d40a6280 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8ec10c520 0x7ff8ec19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8ec103b00 0x7ff8ec198be0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 --2- 172.21.15.19:0/1127116541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ec1031e0 0x7ff8ec1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 -- 172.21.15.19:0/1127116541 >> 172.21.15.19:0/1127116541 conn(0x7ff8ec077fd0 msgr2=0x7ff8ec0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.867+0000 7ff8f0929640 1 -- 172.21.15.19:0/1127116541 shutdown_connections 2024-04-23T18:45:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:43:57.868+0000 7ff8f0929640 1 -- 172.21.15.19:0/1127116541 wait complete. 2024-04-23T18:45:19.271 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 2. 2024-04-23T18:45:19.280 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 systemd[1]: Stopped Ceph mds.e for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:45:19.289 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 systemd[1]: Starting Ceph mds.e for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:45:19.298 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 podman[69544]: 2024-04-23 18:45:17.21434175 +0000 UTC m=+0.019000832 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:45:19.308 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 podman[69544]: 2024-04-23 18:45:17.333878397 +0000 UTC m=+0.138537450 container create 77a9f69890bbf83c572470daa41d9cd61c522ef6f34a283dcdaa206d22a215ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:45:19.317 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 podman[69544]: 2024-04-23 18:45:17.586608333 +0000 UTC m=+0.391267387 container init 77a9f69890bbf83c572470daa41d9cd61c522ef6f34a283dcdaa206d22a215ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-23T18:45:19.384 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 podman[69544]: 2024-04-23 18:45:17.589615558 +0000 UTC m=+0.394274620 container start 77a9f69890bbf83c572470daa41d9cd61c522ef6f34a283dcdaa206d22a215ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:45:19.392 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 ceph-mds[69562]: -- 172.21.15.44:0/3690462836 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b130049a40 con 0x55b13001d000 2024-04-23T18:45:19.401 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69558]: starting mds.e at 2024-04-23T18:45:19.410 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 bash[69544]: 77a9f69890bbf83c572470daa41d9cd61c522ef6f34a283dcdaa206d22a215ef 2024-04-23T18:45:19.420 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:45:17 smithi044 systemd[1]: Started Ceph mds.e for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:45:19.429 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:17 smithi044 ceph-mon[32458]: pgmap v605: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-04-23T18:45:19.438 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:17 smithi044 ceph-mon[32458]: daemon mds.e restarted 2024-04-23T18:45:19.447 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:18 smithi044 ceph-mon[32458]: osdmap e118: 12 total, 12 up, 12 in 2024-04-23T18:45:19.455 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:18 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6834/332625709,v1:172.21.15.44:6835/332625709] up:boot 2024-04-23T18:45:19.492 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:18 smithi044 ceph-mon[32458]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:45:19.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:18 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:45:19.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:57 smithi019 ceph-mon[29215]: pgmap v561: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:19.581 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:43:59 smithi019 ceph-mon[29215]: pgmap v562: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:19.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:01 smithi019 ceph-mon[29215]: pgmap v563: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:45:19.601 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:03 smithi019 ceph-mon[29215]: pgmap v564: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:19.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:05 smithi019 ceph-mon[29215]: pgmap v565: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-23T18:45:19.618 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:07 smithi019 ceph-mon[29215]: pgmap v566: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-23T18:45:19.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:09 smithi019 ceph-mon[29215]: pgmap v567: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-23T18:45:19.725 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:18 smithi155 ceph-mon[33468]: osdmap e118: 12 total, 12 up, 12 in 2024-04-23T18:45:19.733 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:18 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6834/332625709,v1:172.21.15.44:6835/332625709] up:boot 2024-04-23T18:45:19.742 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:18 smithi155 ceph-mon[33468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:45:19.750 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:18 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:45:19.760 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-23T18:45:19.768 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-23T18:45:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.967+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/2945796784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3d4103b00 msgr2=0x7fe3d410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.967+0000 7fe3d8ee8640 1 --2- 172.21.15.19:0/2945796784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3d4103b00 0x7fe3d410bfe0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009900 tx=0x7fe3c802a510 comp rx=0 tx=0).stop 2024-04-23T18:45:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.967+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/2945796784 shutdown_connections 2024-04-23T18:45:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.967+0000 7fe3d8ee8640 1 --2- 172.21.15.19:0/2945796784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d410c520 0x7fe3d410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.967+0000 7fe3d8ee8640 1 --2- 172.21.15.19:0/2945796784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3d4103b00 0x7fe3d410bfe0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.967+0000 7fe3d8ee8640 1 --2- 172.21.15.19:0/2945796784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3d41031e0 0x7fe3d41035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.967+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/2945796784 >> 172.21.15.19:0/2945796784 conn(0x7fe3d4077fd0 msgr2=0x7fe3d40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.967+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/2945796784 shutdown_connections 2024-04-23T18:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.967+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/2945796784 wait complete. 2024-04-23T18:45:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d8ee8640 1 Processor -- start 2024-04-23T18:45:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d8ee8640 1 -- start start 2024-04-23T18:45:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d8ee8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3d41031e0 0x7fe3d4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d8ee8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3d4103b00 0x7fe3d4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d8ee8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d410c520 0x7fe3d419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d8ee8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41121b0 con 0x7fe3d41031e0 2024-04-23T18:45:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d8ee8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d4112030 con 0x7fe3d4103b00 2024-04-23T18:45:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d8ee8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d4112330 con 0x7fe3d410c520 2024-04-23T18:45:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3d41031e0 0x7fe3d4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3d41031e0 0x7fe3d4198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44382/0 (socket says 172.21.15.19:44382) 2024-04-23T18:45:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d259c640 1 -- 172.21.15.19:0/1683099247 learned_addr learned my addr 172.21.15.19:0/1683099247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.968+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d410c520 0x7fe3d419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3d1d9b640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3d4103b00 0x7fe3d4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3d259c640 1 -- 172.21.15.19:0/1683099247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d410c520 msgr2=0x7fe3d419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d410c520 0x7fe3d419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3d259c640 1 -- 172.21.15.19:0/1683099247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3d4103b00 msgr2=0x7fe3d4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3d4103b00 0x7fe3d4198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3d259c640 1 -- 172.21.15.19:0/1683099247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d419d6b0 con 0x7fe3d41031e0 2024-04-23T18:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3d1d9b640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3d4103b00 0x7fe3d4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3d41031e0 0x7fe3d4198710 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400e9f0 tx=0x7fe3c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3c37fe640 1 -- 172.21.15.19:0/1683099247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c400cd80 con 0x7fe3d41031e0 2024-04-23T18:45:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3c37fe640 1 -- 172.21.15.19:0/1683099247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c40108e0 con 0x7fe3d41031e0 2024-04-23T18:45:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/1683099247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d419d9a0 con 0x7fe3d41031e0 2024-04-23T18:45:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3c37fe640 1 -- 172.21.15.19:0/1683099247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4018980 con 0x7fe3d41031e0 2024-04-23T18:45:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.969+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/1683099247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d41a5230 con 0x7fe3d41031e0 2024-04-23T18:45:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.971+0000 7fe3c37fe640 1 -- 172.21.15.19:0/1683099247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe3c4010430 con 0x7fe3d41031e0 2024-04-23T18:45:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3dbbfe740 1 Processor -- start 2024-04-23T18:45:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3dbbfe740 1 -- start start 2024-04-23T18:45:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9daa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9e5800 0x5568de9e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568de93f970 con 0x5568de9da620 2024-04-23T18:45:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5568de8dc4a0 con 0x5568de9e5800 2024-04-23T18:45:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5568de93b930 con 0x5568de9daf40 2024-04-23T18:45:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3d2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9daa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3d2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9daa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44396/0 (socket says 172.21.15.19:44396) 2024-04-23T18:45:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3d2d9d640 1 -- 172.21.15.19:0/1951799445 learned_addr learned my addr 172.21.15.19:0/1951799445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3c37fe640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3bc0a3dc0 0x7fe3bc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3d1d9b640 1 --2- 172.21.15.19:0/1951799445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9e5800 0x5568de9e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.972+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1951799445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3d2d9d640 1 -- 172.21.15.19:0/1951799445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 msgr2=0x5568de9e52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/1951799445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9e52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3d2d9d640 1 -- 172.21.15.19:0/1951799445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9e5800 msgr2=0x5568de9e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/1951799445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9e5800 0x5568de9e7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3d2d9d640 1 -- 172.21.15.19:0/1951799445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568de652210 con 0x5568de9da620 2024-04-23T18:45:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1951799445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9e52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3bc0a3dc0 0x7fe3bc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3c37fe640 1 -- 172.21.15.19:0/1683099247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7fe3c40d0bc0 con 0x7fe3d41031e0 2024-04-23T18:45:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/1951799445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9daa00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc011590 tx=0x7fe3cc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74c928c8b22b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3c0ff9640 1 -- 172.21.15.19:0/1951799445 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc017050 con 0x5568de9da620 2024-04-23T18:45:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3c0ff9640 1 -- 172.21.15.19:0/1951799445 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3cc01bb90 con 0x5568de9da620 2024-04-23T18:45:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.973+0000 7fe3c0ff9640 1 -- 172.21.15.19:0/1951799445 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc025dc0 con 0x5568de9da620 2024-04-23T18:45:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1951799445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 msgr2=0x5568de9daa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3bc0a3dc0 0x7fe3bc0a6280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc022000 tx=0x7fe3cc023000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1951799445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9daa00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc011590 tx=0x7fe3cc011a50 comp rx=0 tx=0).stop 2024-04-23T18:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1951799445 shutdown_connections 2024-04-23T18:45:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1951799445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9e5800 0x5568de9e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1951799445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9e52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1951799445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9daa00 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1951799445 >> 172.21.15.19:0/1951799445 conn(0x5568de9d6b30 msgr2=0x5568de9d6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1951799445 shutdown_connections 2024-04-23T18:45:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1951799445 wait complete. 2024-04-23T18:45:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.974+0000 7fe3dbbfe740 1 -- start start 2024-04-23T18:45:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3dbbfe740 1 Processor -- start 2024-04-23T18:45:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 0x5568de9e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9e23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9e5800 0x5568de9e2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568de9e8430 con 0x5568de9e5800 2024-04-23T18:45:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5568de8da0b0 con 0x5568de9da620 2024-04-23T18:45:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5568de9e85b0 con 0x5568de9daf40 2024-04-23T18:45:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3dbbfe740 10 client.0 _ll_register_callbacks cb 0x5568de8c7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3d2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9e5800 0x5568de9e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3d2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9e5800 0x5568de9e2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44400/0 (socket says 172.21.15.19:44400) 2024-04-23T18:45:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3d2d9d640 1 -- 172.21.15.19:0/4269415228 learned_addr learned my addr 172.21.15.19:0/4269415228 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3d259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 0x5568de9e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3d1d9b640 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9e23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3d2d9d640 1 -- 172.21.15.19:0/4269415228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 msgr2=0x5568de9e23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9e23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.975+0000 7fe3d2d9d640 1 -- 172.21.15.19:0/4269415228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 msgr2=0x5568de9e50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.976+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 0x5568de9e50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.976+0000 7fe3d2d9d640 1 -- 172.21.15.19:0/4269415228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568de71e620 con 0x5568de9e5800 2024-04-23T18:45:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.976+0000 7fe3d259c640 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 0x5568de9e50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:45:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.976+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9e5800 0x5568de9e2900 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00f600 tx=0x7fe3cc025f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4f1e4da9643282 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.976+0000 7fe3b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.978+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/4269415228 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc01b990 con 0x5568de9e5800 2024-04-23T18:45:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.978+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/4269415228 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8469+0+0 (secure 0 0 0) 0x7fe3cc004bd0 con 0x5568de9e5800 2024-04-23T18:45:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.978+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/4269415228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3cc01bb90 con 0x5568de9e5800 2024-04-23T18:45:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.978+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/4269415228 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3cc03b050 con 0x5568de9e5800 2024-04-23T18:45:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.978+0000 7fe3dbbfe740 10 client.17658 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.978+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/4269415228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568de9f42d0 con 0x5568de9e5800 2024-04-23T18:45:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.979+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/4269415228 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3cc01bb90 con 0x5568de9e5800 2024-04-23T18:45:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.979+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/4269415228 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe3cc02d860 con 0x5568de9e5800 2024-04-23T18:45:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.979+0000 7fe3dbbfe740 10 client.17658 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.979+0000 7fe3dbbfe740 10 client.17658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.979+0000 7fe3dbbfe740 20 client.17658 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.979+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568de9f5730 0x5568de9f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.979+0000 7fe3dbbfe740 4 client.17658 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:21.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.979+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/4269415228 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568de94d390 con 0x5568de9f5730 2024-04-23T18:45:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.979+0000 7fe3d259c640 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568de9f5730 0x5568de9f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3b9ffb640 10 client.17658 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3d259c640 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568de9f5730 0x5568de9f7b20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880d5b7629af7eb server_cookie=c4977ae257af08d in_seq=0 out_seq=0 2024-04-23T18:45:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/4269415228 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568de94d390 con 0x5568de9f5730 2024-04-23T18:45:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3b9ffb640 10 client.17658 handle_command_reply: tid=0 2024-04-23T18:45:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3dbbfe740 1 client.17658 shutdown 2024-04-23T18:45:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/4269415228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9e5800 msgr2=0x5568de9e2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9e5800 0x5568de9e2900 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00f600 tx=0x7fe3cc025f70 comp rx=0 tx=0).stop 2024-04-23T18:45:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/4269415228 shutdown_connections 2024-04-23T18:45:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568de9f5730 0x5568de9f7b20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9e5800 0x5568de9e2900 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9e23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/4269415228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 0x5568de9e50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3b9ffb640 0 client.17658 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.980+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/4269415228 >> 172.21.15.19:0/4269415228 conn(0x5568de9d6b30 msgr2=0x5568de9dc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.981+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/4269415228 shutdown_connections 2024-04-23T18:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.981+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/4269415228 wait complete. 2024-04-23T18:45:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.981+0000 7fe3dbbfe740 20 client.17658 trim_cache size 0 max 16384 2024-04-23T18:45:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.999+0000 7fe3dbbfe740 1 Processor -- start 2024-04-23T18:45:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:10.999+0000 7fe3dbbfe740 1 -- start start 2024-04-23T18:45:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9d6b30 0x5568de9f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 0x5568de9fd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41adab0 con 0x5568de9d6b30 2024-04-23T18:45:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41af5f0 con 0x5568de9da620 2024-04-23T18:45:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41ae180 con 0x5568de9daf40 2024-04-23T18:45:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9d6b30 0x5568de9f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9d6b30 0x5568de9f12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44406/0 (socket says 172.21.15.19:44406) 2024-04-23T18:45:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d259c640 1 -- 172.21.15.19:0/1964732707 learned_addr learned my addr 172.21.15.19:0/1964732707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/1964732707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d1d9b640 1 --2- 172.21.15.19:0/1964732707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 0x5568de9fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d259c640 1 -- 172.21.15.19:0/1964732707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 msgr2=0x5568de9fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1964732707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9fd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d259c640 1 -- 172.21.15.19:0/1964732707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 msgr2=0x5568de9fd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1964732707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 0x5568de9fd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d259c640 1 -- 172.21.15.19:0/1964732707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568de9d9e20 con 0x5568de9d6b30 2024-04-23T18:45:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1964732707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9d6b30 0x5568de9f12f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400c6a0 tx=0x7fe3c4098dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab54b4a3df1afbe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.000+0000 7fe3bbfff640 1 -- 172.21.15.19:0/1964732707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c409de60 con 0x5568de9d6b30 2024-04-23T18:45:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3bbfff640 1 -- 172.21.15.19:0/1964732707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c40a0ee0 con 0x5568de9d6b30 2024-04-23T18:45:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3bbfff640 1 -- 172.21.15.19:0/1964732707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4010a90 con 0x5568de9d6b30 2024-04-23T18:45:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1964732707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9d6b30 msgr2=0x5568de9f12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1964732707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9d6b30 0x5568de9f12f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400c6a0 tx=0x7fe3c4098dd0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1964732707 shutdown_connections 2024-04-23T18:45:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1964732707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9da620 0x5568de9fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1964732707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1964732707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9d6b30 0x5568de9f12f0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1964732707 >> 172.21.15.19:0/1964732707 conn(0x5568de9e2a60 msgr2=0x5568de9d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1964732707 shutdown_connections 2024-04-23T18:45:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.001+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1964732707 wait complete. 2024-04-23T18:45:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3dbbfe740 1 -- start start 2024-04-23T18:45:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3dbbfe740 1 Processor -- start 2024-04-23T18:45:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9d6b30 0x5568de9edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9eb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3dbbfe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9eb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41aef20 con 0x5568de9da620 2024-04-23T18:45:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41ae850 con 0x5568de9d6b30 2024-04-23T18:45:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3dbbfe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41aba90 con 0x5568de9daf40 2024-04-23T18:45:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3dbbfe740 10 client.0 _ll_register_callbacks cb 0x7fe3cc009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3d1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3d1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9eb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44416/0 (socket says 172.21.15.19:44416) 2024-04-23T18:45:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3d1d9b640 1 -- 172.21.15.19:0/1167805042 learned_addr learned my addr 172.21.15.19:0/1167805042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3d2d9d640 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9eb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9d6b30 0x5568de9edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3d1d9b640 1 -- 172.21.15.19:0/1167805042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 msgr2=0x5568de9eb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3d1d9b640 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9eb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3d1d9b640 1 -- 172.21.15.19:0/1167805042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9d6b30 msgr2=0x5568de9edce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3d1d9b640 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9d6b30 0x5568de9edce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.002+0000 7fe3d1d9b640 1 -- 172.21.15.19:0/1167805042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568de9ebeb0 con 0x5568de9da620 2024-04-23T18:45:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3d1d9b640 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9eb070 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009900 tx=0x7fe3c8031800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29479a9b92d2bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/1167805042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8035040 con 0x5568de9da620 2024-04-23T18:45:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/1167805042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8557+0+0 (secure 0 0 0) 0x7fe3c8004250 con 0x5568de9da620 2024-04-23T18:45:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1167805042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c40a0ee0 con 0x5568de9da620 2024-04-23T18:45:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/1167805042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3c8004450 con 0x5568de9da620 2024-04-23T18:45:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3dbbfe740 10 client.17664 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1167805042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568dea5bb40 con 0x5568de9da620 2024-04-23T18:45:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/1167805042 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c40a0ee0 con 0x5568de9da620 2024-04-23T18:45:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/1167805042 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe3c8004b00 con 0x5568de9da620 2024-04-23T18:45:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3dbbfe740 10 client.17664 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3dbbfe740 10 client.17664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3dbbfe740 20 client.17664 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568de9e5800 0x5568dea5e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3dbbfe740 4 client.17664 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.003+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1167805042 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568de9dbaa0 con 0x5568de9e5800 2024-04-23T18:45:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.004+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568de9e5800 0x5568dea5e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.004+0000 7fe3b9ffb640 10 client.17664 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.004+0000 7fe3d259c640 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568de9e5800 0x5568dea5e3b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=427c566b582b06b9 server_cookie=cdb5ec478e8246aa in_seq=0 out_seq=0 2024-04-23T18:45:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3b9ffb640 1 -- 172.21.15.19:0/1167805042 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568de9dbaa0 con 0x5568de9e5800 2024-04-23T18:45:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3b9ffb640 10 client.17664 handle_command_reply: tid=0 2024-04-23T18:45:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 client.17664 shutdown 2024-04-23T18:45:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1167805042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 msgr2=0x5568de9eb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9eb070 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009900 tx=0x7fe3c8031800 comp rx=0 tx=0).stop 2024-04-23T18:45:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1167805042 shutdown_connections 2024-04-23T18:45:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568de9e5800 0x5568dea5e3b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568de9daf40 0x5568de9eb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568de9da620 0x5568de9eb070 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3b9ffb640 0 client.17664 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 --2- 172.21.15.19:0/1167805042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568de9d6b30 0x5568de9edce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1167805042 >> 172.21.15.19:0/1167805042 conn(0x5568de9e2a60 msgr2=0x5568de9d5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1167805042 shutdown_connections 2024-04-23T18:45:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 1 -- 172.21.15.19:0/1167805042 wait complete. 2024-04-23T18:45:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.025+0000 7fe3dbbfe740 20 client.17664 trim_cache size 0 max 16384 2024-04-23T18:45:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:23.373 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:19 smithi155 ceph-mon[33468]: pgmap v607: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-23T18:45:23.427 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:21 smithi155 ceph-mon[33468]: pgmap v608: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 639 B/s wr, 12 op/s 2024-04-23T18:45:23.433 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:19 smithi044 ceph-mon[32458]: pgmap v607: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-23T18:45:23.439 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:21 smithi044 ceph-mon[32458]: pgmap v608: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 639 B/s wr, 12 op/s 2024-04-23T18:45:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/1683099247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3bc0a3dc0 msgr2=0x7fe3bc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3bc0a3dc0 0x7fe3bc0a6280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc022000 tx=0x7fe3cc023000 comp rx=0 tx=0).stop 2024-04-23T18:45:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/1683099247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3d41031e0 msgr2=0x7fe3d4198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3d41031e0 0x7fe3d4198710 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400e9f0 tx=0x7fe3c400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/1683099247 shutdown_connections 2024-04-23T18:45:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe3bc0a3dc0 0x7fe3bc0a6280 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3d4103b00 0x7fe3d4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d410c520 0x7fe3d419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 --2- 172.21.15.19:0/1683099247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3d41031e0 0x7fe3d4198710 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/1683099247 >> 172.21.15.19:0/1683099247 conn(0x7fe3d4077fd0 msgr2=0x7fe3d40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/1683099247 shutdown_connections 2024-04-23T18:45:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:11.026+0000 7fe3d8ee8640 1 -- 172.21.15.19:0/1683099247 wait complete. 2024-04-23T18:45:23.633 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:11 smithi019 ceph-mon[29215]: pgmap v568: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:45:23.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:11 smithi019 ceph-mon[29215]: osdmap e114: 12 total, 12 up, 12 in 2024-04-23T18:45:23.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:13 smithi019 ceph-mon[29215]: pgmap v570: 97 pgs: 97 active+clean; 4.2 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:45:23.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:15 smithi019 ceph-mon[29215]: pgmap v571: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:45:23.725 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:17 smithi019 ceph-mon[29215]: pgmap v572: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:45:23.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:17 smithi019 ceph-mon[29215]: osdmap e115: 12 total, 12 up, 12 in 2024-04-23T18:45:23.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:18 smithi019 ceph-mon[29215]: pgmap v574: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T18:45:23.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:21 smithi019 ceph-mon[29215]: pgmap v575: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-23T18:45:23.756 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:23 smithi019 ceph-mon[29215]: pgmap v576: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:45:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.125+0000 7f8701537640 1 -- 172.21.15.19:0/2165882010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86fc1021d0 msgr2=0x7f86fc1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.125+0000 7f8701537640 1 --2- 172.21.15.19:0/2165882010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86fc1021d0 0x7f86fc1025b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009900 tx=0x7f86f402a410 comp rx=0 tx=0).stop 2024-04-23T18:45:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.125+0000 7f8701537640 1 -- 172.21.15.19:0/2165882010 shutdown_connections 2024-04-23T18:45:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.125+0000 7f8701537640 1 --2- 172.21.15.19:0/2165882010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc102b80 0x7f86fc106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.125+0000 7f8701537640 1 --2- 172.21.15.19:0/2165882010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86fc1072a0 0x7f86fc10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.125+0000 7f8701537640 1 --2- 172.21.15.19:0/2165882010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86fc1021d0 0x7f86fc1025b0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.125+0000 7f8701537640 1 -- 172.21.15.19:0/2165882010 >> 172.21.15.19:0/2165882010 conn(0x7f86fc0793e0 msgr2=0x7f86fc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.125+0000 7f8701537640 1 -- 172.21.15.19:0/2165882010 shutdown_connections 2024-04-23T18:45:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.125+0000 7f8701537640 1 -- 172.21.15.19:0/2165882010 wait complete. 2024-04-23T18:45:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.125+0000 7f8701537640 1 Processor -- start 2024-04-23T18:45:24.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f8701537640 1 -- start start 2024-04-23T18:45:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f8701537640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86fc1021d0 0x7f86fc198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f8701537640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc1072a0 0x7f86fc198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f8701537640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86fc19cf10 0x7f86fc19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f8701537640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc112230 con 0x7f86fc1021d0 2024-04-23T18:45:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f8701537640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1120b0 con 0x7f86fc1072a0 2024-04-23T18:45:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f8701537640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1123b0 con 0x7f86fc19cf10 2024-04-23T18:45:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f86faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86fc1021d0 0x7f86fc198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f86faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86fc1021d0 0x7f86fc198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58630/0 (socket says 172.21.15.19:58630) 2024-04-23T18:45:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f86faffd640 1 -- 172.21.15.19:0/1579818280 learned_addr learned my addr 172.21.15.19:0/1579818280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc1072a0 0x7f86fc198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86fc19cf10 0x7f86fc19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f86faffd640 1 -- 172.21.15.19:0/1579818280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86fc19cf10 msgr2=0x7f86fc19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f86faffd640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86fc19cf10 0x7f86fc19d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f86faffd640 1 -- 172.21.15.19:0/1579818280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc1072a0 msgr2=0x7f86fc198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f86faffd640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc1072a0 0x7f86fc198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.126+0000 7f86faffd640 1 -- 172.21.15.19:0/1579818280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc19da00 con 0x7f86fc1021d0 2024-04-23T18:45:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.127+0000 7f86faffd640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86fc1021d0 0x7f86fc198650 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f86f4005ab0 tx=0x7f86f4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.127+0000 7f86f3fff640 1 -- 172.21.15.19:0/1579818280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f403f070 con 0x7f86fc1021d0 2024-04-23T18:45:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.127+0000 7f86f3fff640 1 -- 172.21.15.19:0/1579818280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f4031d60 con 0x7f86fc1021d0 2024-04-23T18:45:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.127+0000 7f8701537640 1 -- 172.21.15.19:0/1579818280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc19dc90 con 0x7f86fc1021d0 2024-04-23T18:45:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.127+0000 7f86f3fff640 1 -- 172.21.15.19:0/1579818280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4035030 con 0x7f86fc1021d0 2024-04-23T18:45:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.127+0000 7f8701537640 1 -- 172.21.15.19:0/1579818280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc1a5450 con 0x7f86fc1021d0 2024-04-23T18:45:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.129+0000 7f86f3fff640 1 -- 172.21.15.19:0/1579818280 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f86f4042070 con 0x7f86fc1021d0 2024-04-23T18:45:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f870424d740 1 Processor -- start 2024-04-23T18:45:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f870424d740 1 -- start start 2024-04-23T18:45:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f870424d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f870424d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63f1f40 0x5604f63fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f870424d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63fc800 0x5604f63febf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f870424d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5604f6356970 con 0x5604f63f1620 2024-04-23T18:45:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f870424d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5604f62f34a0 con 0x5604f63f1f40 2024-04-23T18:45:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f870424d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5604f6352930 con 0x5604f63fc800 2024-04-23T18:45:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f86f3fff640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f86e80a3dc0 0x7f86e80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f86fb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f86fb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63f1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58646/0 (socket says 172.21.15.19:58646) 2024-04-23T18:45:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f86fb7fe640 1 -- 172.21.15.19:0/493265144 learned_addr learned my addr 172.21.15.19:0/493265144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.130+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f86e80a3dc0 0x7f86e80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/493265144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63fc800 0x5604f63febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86f3fff640 1 -- 172.21.15.19:0/1579818280 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7f86f40f1060 con 0x7f86fc1021d0 2024-04-23T18:45:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86faffd640 1 --2- 172.21.15.19:0/493265144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63f1f40 0x5604f63fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86fb7fe640 1 -- 172.21.15.19:0/493265144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63fc800 msgr2=0x5604f63febf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/493265144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63fc800 0x5604f63febf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86fb7fe640 1 -- 172.21.15.19:0/493265144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63f1f40 msgr2=0x5604f63fc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/493265144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63f1f40 0x5604f63fc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86fb7fe640 1 -- 172.21.15.19:0/493265144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f6069210 con 0x5604f63f1620 2024-04-23T18:45:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/493265144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63f1a00 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f86ec012c40 tx=0x7f86ec010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f067cc9c9d67d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f86e80a3dc0 0x7f86e80a6280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f86ec010c40 tx=0x7f86ec017000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86f17fa640 1 -- 172.21.15.19:0/493265144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec023020 con 0x5604f63f1620 2024-04-23T18:45:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86f17fa640 1 -- 172.21.15.19:0/493265144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec009ca0 con 0x5604f63f1620 2024-04-23T18:45:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.131+0000 7f86f17fa640 1 -- 172.21.15.19:0/493265144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec01f930 con 0x5604f63f1620 2024-04-23T18:45:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.132+0000 7f870424d740 1 -- 172.21.15.19:0/493265144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 msgr2=0x5604f63f1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.132+0000 7f870424d740 1 --2- 172.21.15.19:0/493265144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63f1a00 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f86ec012c40 tx=0x7f86ec010740 comp rx=0 tx=0).stop 2024-04-23T18:45:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.132+0000 7f870424d740 1 -- 172.21.15.19:0/493265144 shutdown_connections 2024-04-23T18:45:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.132+0000 7f870424d740 1 --2- 172.21.15.19:0/493265144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63f1f40 0x5604f63fc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.132+0000 7f870424d740 1 --2- 172.21.15.19:0/493265144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63fc800 0x5604f63febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.132+0000 7f870424d740 1 --2- 172.21.15.19:0/493265144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63f1a00 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.132+0000 7f870424d740 1 -- 172.21.15.19:0/493265144 >> 172.21.15.19:0/493265144 conn(0x5604f63edb30 msgr2=0x5604f63edf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.132+0000 7f870424d740 1 -- 172.21.15.19:0/493265144 shutdown_connections 2024-04-23T18:45:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.132+0000 7f870424d740 1 -- 172.21.15.19:0/493265144 wait complete. 2024-04-23T18:45:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f870424d740 1 -- start start 2024-04-23T18:45:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f870424d740 1 Processor -- start 2024-04-23T18:45:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f870424d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63fc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f870424d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 0x5604f63f9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f870424d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63fc800 0x5604f63f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f870424d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5604f63ff3d0 con 0x5604f63f1620 2024-04-23T18:45:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f870424d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5604f62f10b0 con 0x5604f63fc800 2024-04-23T18:45:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f870424d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5604f63ff550 con 0x5604f63f1f40 2024-04-23T18:45:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f870424d740 10 client.0 _ll_register_callbacks cb 0x5604f62ded30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63fc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58658/0 (socket says 172.21.15.19:58658) 2024-04-23T18:45:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86faffd640 1 -- 172.21.15.19:0/2800520844 learned_addr learned my addr 172.21.15.19:0/2800520844 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63fc800 0x5604f63f99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86faffd640 1 -- 172.21.15.19:0/2800520844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 msgr2=0x5604f63f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 0x5604f63f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86faffd640 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 0x5604f63f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86faffd640 1 -- 172.21.15.19:0/2800520844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63fc800 msgr2=0x5604f63f99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86faffd640 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63fc800 0x5604f63f99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86faffd640 1 -- 172.21.15.19:0/2800520844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f6135620 con 0x5604f63f1620 2024-04-23T18:45:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 0x5604f63f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.133+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63fc800 0x5604f63f99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f86faffd640 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63fc180 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f86f4003fd0 tx=0x7f86f4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e1e0a0533d796e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f86e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f86e27fc640 1 -- 172.21.15.19:0/2800520844 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40d5070 con 0x5604f63f1620 2024-04-23T18:45:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f86e27fc640 1 -- 172.21.15.19:0/2800520844 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7f86f40ee9d0 con 0x5604f63f1620 2024-04-23T18:45:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f870424d740 1 -- 172.21.15.19:0/2800520844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5604f6364390 con 0x5604f63f1620 2024-04-23T18:45:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f86e27fc640 1 -- 172.21.15.19:0/2800520844 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86f4049040 con 0x5604f63f1620 2024-04-23T18:45:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f870424d740 10 client.17676 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f870424d740 1 -- 172.21.15.19:0/2800520844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604f640b320 con 0x5604f63f1620 2024-04-23T18:45:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f86e27fc640 1 -- 172.21.15.19:0/2800520844 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5604f6364390 con 0x5604f63f1620 2024-04-23T18:45:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f86e27fc640 1 -- 172.21.15.19:0/2800520844 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f86f40bd200 con 0x5604f63f1620 2024-04-23T18:45:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f870424d740 10 client.17676 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f870424d740 10 client.17676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f870424d740 20 client.17676 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f870424d740 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5604f640c7d0 0x5604f640ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f870424d740 4 client.17676 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.134+0000 7f870424d740 1 -- 172.21.15.19:0/2800520844 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604f635e200 con 0x5604f640c7d0 2024-04-23T18:45:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.135+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5604f640c7d0 0x5604f640ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.135+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5604f640c7d0 0x5604f640ec40 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c2b63d438b943b server_cookie=1017d3f35859310c in_seq=0 out_seq=0 2024-04-23T18:45:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.135+0000 7f86e27fc640 10 client.17676 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f86e27fc640 1 -- 172.21.15.19:0/2800520844 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604f635e200 con 0x5604f640c7d0 2024-04-23T18:45:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f86e27fc640 10 client.17676 handle_command_reply: tid=0 2024-04-23T18:45:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 client.17676 shutdown 2024-04-23T18:45:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 -- 172.21.15.19:0/2800520844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 msgr2=0x5604f63fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63fc180 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f86f4003fd0 tx=0x7f86f4009260 comp rx=0 tx=0).stop 2024-04-23T18:45:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 -- 172.21.15.19:0/2800520844 shutdown_connections 2024-04-23T18:45:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5604f640c7d0 0x5604f640ec40 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63fc800 0x5604f63f99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 0x5604f63f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f86e27fc640 0 client.17676 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 --2- 172.21.15.19:0/2800520844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1620 0x5604f63fc180 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 -- 172.21.15.19:0/2800520844 >> 172.21.15.19:0/2800520844 conn(0x5604f63edb30 msgr2=0x5604f63f3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 -- 172.21.15.19:0/2800520844 shutdown_connections 2024-04-23T18:45:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 1 -- 172.21.15.19:0/2800520844 wait complete. 2024-04-23T18:45:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.136+0000 7f870424d740 20 client.17676 trim_cache size 0 max 16384 2024-04-23T18:45:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.151+0000 7f870424d740 1 Processor -- start 2024-04-23T18:45:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.151+0000 7f870424d740 1 -- start start 2024-04-23T18:45:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f870424d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1620 0x5604f6408290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f870424d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1f40 0x5604f63ecb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f870424d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63fc800 0x5604f63fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f870424d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1adb00 con 0x5604f63f1f40 2024-04-23T18:45:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f870424d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1af640 con 0x5604f63fc800 2024-04-23T18:45:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f870424d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1ae1d0 con 0x5604f63f1620 2024-04-23T18:45:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86fa7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1f40 0x5604f63ecb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86fa7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1f40 0x5604f63ecb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58670/0 (socket says 172.21.15.19:58670) 2024-04-23T18:45:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86fa7fc640 1 -- 172.21.15.19:0/762977608 learned_addr learned my addr 172.21.15.19:0/762977608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86fa7fc640 1 -- 172.21.15.19:0/762977608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1620 msgr2=0x5604f6408290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/762977608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1620 0x5604f6408290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86fa7fc640 1 -- 172.21.15.19:0/762977608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63fc800 msgr2=0x5604f63fbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/762977608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63fc800 0x5604f63fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86fa7fc640 1 -- 172.21.15.19:0/762977608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f63f1d70 con 0x5604f63f1f40 2024-04-23T18:45:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/762977608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1f40 0x5604f63ecb30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x5604f63eecc0 tx=0x7f86e4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dd71d0d9423710 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86f0ff9640 1 -- 172.21.15.19:0/762977608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4005020 con 0x5604f63f1f40 2024-04-23T18:45:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86f0ff9640 1 -- 172.21.15.19:0/762977608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e40047a0 con 0x5604f63f1f40 2024-04-23T18:45:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.152+0000 7f86f0ff9640 1 -- 172.21.15.19:0/762977608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e401f350 con 0x5604f63f1f40 2024-04-23T18:45:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.153+0000 7f870424d740 1 -- 172.21.15.19:0/762977608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1f40 msgr2=0x5604f63ecb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.153+0000 7f870424d740 1 --2- 172.21.15.19:0/762977608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1f40 0x5604f63ecb30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x5604f63eecc0 tx=0x7f86e4012820 comp rx=0 tx=0).stop 2024-04-23T18:45:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.153+0000 7f870424d740 1 -- 172.21.15.19:0/762977608 shutdown_connections 2024-04-23T18:45:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.153+0000 7f870424d740 1 --2- 172.21.15.19:0/762977608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63fc800 0x5604f63fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.153+0000 7f870424d740 1 --2- 172.21.15.19:0/762977608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63f1f40 0x5604f63ecb30 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.153+0000 7f870424d740 1 --2- 172.21.15.19:0/762977608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1620 0x5604f6408290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.153+0000 7f870424d740 1 -- 172.21.15.19:0/762977608 >> 172.21.15.19:0/762977608 conn(0x5604f63edb30 msgr2=0x5604f63ec420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.153+0000 7f870424d740 1 -- 172.21.15.19:0/762977608 shutdown_connections 2024-04-23T18:45:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.153+0000 7f870424d740 1 -- 172.21.15.19:0/762977608 wait complete. 2024-04-23T18:45:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.153+0000 7f870424d740 1 -- start start 2024-04-23T18:45:26.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f870424d740 1 Processor -- start 2024-04-23T18:45:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f870424d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63f1620 0x5604f6404990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f870424d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 0x5604f63f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f870424d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63fc800 0x5604f63f6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f870424d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1aef70 con 0x5604f63fc800 2024-04-23T18:45:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f870424d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1ae8a0 con 0x5604f63f1620 2024-04-23T18:45:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f870424d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1abb00 con 0x5604f63f1f40 2024-04-23T18:45:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f870424d740 10 client.0 _ll_register_callbacks cb 0x7f86f40bcae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86fb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63fc800 0x5604f63f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86fb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63fc800 0x5604f63f6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58674/0 (socket says 172.21.15.19:58674) 2024-04-23T18:45:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86fb7fe640 1 -- 172.21.15.19:0/3788982662 learned_addr learned my addr 172.21.15.19:0/3788982662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86faffd640 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63f1620 0x5604f6404990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 0x5604f63f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86fb7fe640 1 -- 172.21.15.19:0/3788982662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 msgr2=0x5604f63f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 0x5604f63f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86fb7fe640 1 -- 172.21.15.19:0/3788982662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63f1620 msgr2=0x5604f6404990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63f1620 0x5604f6404990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86fb7fe640 1 -- 172.21.15.19:0/3788982662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f63f67e0 con 0x5604f63fc800 2024-04-23T18:45:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.154+0000 7f86fa7fc640 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 0x5604f63f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f86fb7fe640 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63fc800 0x5604f63f6220 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00cfd0 tx=0x7f86ec0124d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e4474562767bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f86e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f86e27fc640 1 -- 172.21.15.19:0/3788982662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec02c070 con 0x5604f63fc800 2024-04-23T18:45:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f86e27fc640 1 -- 172.21.15.19:0/3788982662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7f86ec00cd50 con 0x5604f63fc800 2024-04-23T18:45:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f870424d740 1 -- 172.21.15.19:0/3788982662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e40047a0 con 0x5604f63fc800 2024-04-23T18:45:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f86e27fc640 1 -- 172.21.15.19:0/3788982662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86ec023050 con 0x5604f63fc800 2024-04-23T18:45:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f870424d740 10 client.17682 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f870424d740 1 -- 172.21.15.19:0/3788982662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604f6472ac0 con 0x5604f63fc800 2024-04-23T18:45:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f86e27fc640 1 -- 172.21.15.19:0/3788982662 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e40047a0 con 0x5604f63fc800 2024-04-23T18:45:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f86e27fc640 1 -- 172.21.15.19:0/3788982662 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f86ec0054e0 con 0x5604f63fc800 2024-04-23T18:45:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f870424d740 10 client.17682 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f870424d740 10 client.17682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f870424d740 20 client.17682 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f870424d740 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5604f6472f80 0x5604f64753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f870424d740 4 client.17682 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.155+0000 7f870424d740 1 -- 172.21.15.19:0/3788982662 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604f62da5f0 con 0x5604f6472f80 2024-04-23T18:45:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.156+0000 7f86faffd640 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5604f6472f80 0x5604f64753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.156+0000 7f86faffd640 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5604f6472f80 0x5604f64753a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620a15f40027e157 server_cookie=dee3f6ac1523087c in_seq=0 out_seq=0 2024-04-23T18:45:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.156+0000 7f86e27fc640 10 client.17682 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f86e27fc640 1 -- 172.21.15.19:0/3788982662 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604f62da5f0 con 0x5604f6472f80 2024-04-23T18:45:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f86e27fc640 10 client.17682 handle_command_reply: tid=0 2024-04-23T18:45:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 client.17682 shutdown 2024-04-23T18:45:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 -- 172.21.15.19:0/3788982662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63fc800 msgr2=0x5604f63f6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63fc800 0x5604f63f6220 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00cfd0 tx=0x7f86ec0124d0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 -- 172.21.15.19:0/3788982662 shutdown_connections 2024-04-23T18:45:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5604f6472f80 0x5604f64753a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5604f63fc800 0x5604f63f6220 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f86e27fc640 0 client.17682 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f63f1f40 0x5604f63f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 --2- 172.21.15.19:0/3788982662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604f63f1620 0x5604f6404990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 -- 172.21.15.19:0/3788982662 >> 172.21.15.19:0/3788982662 conn(0x5604f63edb30 msgr2=0x5604f63f49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 -- 172.21.15.19:0/3788982662 shutdown_connections 2024-04-23T18:45:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 1 -- 172.21.15.19:0/3788982662 wait complete. 2024-04-23T18:45:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f870424d740 20 client.17682 trim_cache size 0 max 16384 2024-04-23T18:45:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f8701537640 1 -- 172.21.15.19:0/1579818280 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f86e80a3dc0 msgr2=0x7f86e80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.166+0000 7f8701537640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f86e80a3dc0 0x7f86e80a6280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f86ec010c40 tx=0x7f86ec017000 comp rx=0 tx=0).stop 2024-04-23T18:45:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.167+0000 7f8701537640 1 -- 172.21.15.19:0/1579818280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86fc1021d0 msgr2=0x7f86fc198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.167+0000 7f8701537640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86fc1021d0 0x7f86fc198650 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f86f4005ab0 tx=0x7f86f4003c80 comp rx=0 tx=0).stop 2024-04-23T18:45:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.167+0000 7f8701537640 1 -- 172.21.15.19:0/1579818280 shutdown_connections 2024-04-23T18:45:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.167+0000 7f8701537640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f86e80a3dc0 0x7f86e80a6280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.167+0000 7f8701537640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc1072a0 0x7f86fc198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.167+0000 7f8701537640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86fc19cf10 0x7f86fc19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.167+0000 7f8701537640 1 --2- 172.21.15.19:0/1579818280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86fc1021d0 0x7f86fc198650 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.167+0000 7f8701537640 1 -- 172.21.15.19:0/1579818280 >> 172.21.15.19:0/1579818280 conn(0x7f86fc0793e0 msgr2=0x7f86fc0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.167+0000 7f8701537640 1 -- 172.21.15.19:0/1579818280 shutdown_connections 2024-04-23T18:45:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:24.167+0000 7f8701537640 1 -- 172.21.15.19:0/1579818280 wait complete. 2024-04-23T18:45:27.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:23 smithi155 ceph-mon[33468]: pgmap v609: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 536 B/s wr, 10 op/s 2024-04-23T18:45:27.576 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:25 smithi155 ceph-mon[33468]: pgmap v610: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:45:27.585 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:23 smithi044 ceph-mon[32458]: pgmap v609: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 536 B/s wr, 10 op/s 2024-04-23T18:45:27.593 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:25 smithi044 ceph-mon[32458]: pgmap v610: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:45:27.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:25 smithi019 ceph-mon[29215]: pgmap v577: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-23T18:45:27.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:27 smithi019 ceph-mon[29215]: pgmap v578: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-23T18:45:27.617 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:29 smithi019 ceph-mon[29215]: pgmap v579: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-23T18:45:27.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:30 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:27.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:31 smithi019 ceph-mon[29215]: pgmap v580: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-04-23T18:45:27.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:31 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:27.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:31 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:27.727 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:31 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:45:27.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:33 smithi019 ceph-mon[29215]: pgmap v581: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-23T18:45:27.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:35 smithi019 ceph-mon[29215]: pgmap v582: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-04-23T18:45:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.266+0000 7fd0d779b640 1 -- 172.21.15.19:0/1679472675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0d0103b20 msgr2=0x7fd0d010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.266+0000 7fd0d779b640 1 --2- 172.21.15.19:0/1679472675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0d0103b20 0x7fd0d010c000 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009900 tx=0x7fd0c002a480 comp rx=0 tx=0).stop 2024-04-23T18:45:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.266+0000 7fd0d779b640 1 -- 172.21.15.19:0/1679472675 shutdown_connections 2024-04-23T18:45:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.266+0000 7fd0d779b640 1 --2- 172.21.15.19:0/1679472675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0d010c5d0 0x7fd0d010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.266+0000 7fd0d779b640 1 --2- 172.21.15.19:0/1679472675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0d0103b20 0x7fd0d010c000 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.266+0000 7fd0d779b640 1 --2- 172.21.15.19:0/1679472675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0d0103200 0x7fd0d01035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.266+0000 7fd0d779b640 1 -- 172.21.15.19:0/1679472675 >> 172.21.15.19:0/1679472675 conn(0x7fd0d00771d0 msgr2=0x7fd0d00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.266+0000 7fd0d779b640 1 -- 172.21.15.19:0/1679472675 shutdown_connections 2024-04-23T18:45:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.266+0000 7fd0d779b640 1 -- 172.21.15.19:0/1679472675 wait complete. 2024-04-23T18:45:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d779b640 1 Processor -- start 2024-04-23T18:45:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d779b640 1 -- start start 2024-04-23T18:45:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d779b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0d0103200 0x7fd0d0100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d779b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0d0103b20 0x7fd0d0101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d779b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0d010c5d0 0x7fd0d01018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d779b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d0112210 con 0x7fd0d0103b20 2024-04-23T18:45:28.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d779b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d0112090 con 0x7fd0d010c5d0 2024-04-23T18:45:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d779b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d0112390 con 0x7fd0d0103200 2024-04-23T18:45:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d4d36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0d0103b20 0x7fd0d0101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d4d36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0d0103b20 0x7fd0d0101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51330/0 (socket says 172.21.15.19:51330) 2024-04-23T18:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d4d36640 1 -- 172.21.15.19:0/954155688 learned_addr learned my addr 172.21.15.19:0/954155688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.267+0000 7fd0d5537640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0d0103200 0x7fd0d0100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0d010c5d0 0x7fd0d01018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0d4d36640 1 -- 172.21.15.19:0/954155688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0d0103200 msgr2=0x7fd0d0100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0d0103200 0x7fd0d0100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0d4d36640 1 -- 172.21.15.19:0/954155688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0d010c5d0 msgr2=0x7fd0d01018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0d010c5d0 0x7fd0d01018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0d4d36640 1 -- 172.21.15.19:0/954155688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d01a6c70 con 0x7fd0d0103b20 2024-04-23T18:45:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0d010c5d0 0x7fd0d01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0d0103b20 0x7fd0d0101390 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0005940 tx=0x7fd0c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0c67fc640 1 -- 172.21.15.19:0/954155688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c003f030 con 0x7fd0d0103b20 2024-04-23T18:45:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0c67fc640 1 -- 172.21.15.19:0/954155688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c0003cb0 con 0x7fd0d0103b20 2024-04-23T18:45:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0d779b640 1 -- 172.21.15.19:0/954155688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d01a6f00 con 0x7fd0d0103b20 2024-04-23T18:45:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0c67fc640 1 -- 172.21.15.19:0/954155688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0035070 con 0x7fd0d0103b20 2024-04-23T18:45:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.268+0000 7fd0d779b640 1 -- 172.21.15.19:0/954155688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d01a7360 con 0x7fd0d0103b20 2024-04-23T18:45:28.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.271+0000 7fd0c67fc640 1 -- 172.21.15.19:0/954155688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd0c0042030 con 0x7fd0d0103b20 2024-04-23T18:45:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.271+0000 7fd0da592740 1 Processor -- start 2024-04-23T18:45:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.271+0000 7fd0da592740 1 -- start start 2024-04-23T18:45:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0da592740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c19a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0da592740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c19f40 0x561dc3c242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0da592740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c24800 0x561dc3c26bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0da592740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561dc3b7e970 con 0x561dc3c19620 2024-04-23T18:45:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0da592740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561dc3b1b4a0 con 0x561dc3c24800 2024-04-23T18:45:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0da592740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561dc3b7a930 con 0x561dc3c19f40 2024-04-23T18:45:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0c67fc640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0bc0a3df0 0x7fd0bc0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5d38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c19a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5d38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c19a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51342/0 (socket says 172.21.15.19:51342) 2024-04-23T18:45:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5d38640 1 -- 172.21.15.19:0/1429019235 learned_addr learned my addr 172.21.15.19:0/1429019235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0bc0a3df0 0x7fd0bc0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0c67fc640 1 -- 172.21.15.19:0/954155688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7fd0c00f0ca0 con 0x7fd0d0103b20 2024-04-23T18:45:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5537640 1 --2- 172.21.15.19:0/1429019235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c19f40 0x561dc3c242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/1429019235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c24800 0x561dc3c26bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5d38640 1 -- 172.21.15.19:0/1429019235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c19f40 msgr2=0x561dc3c242c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/1429019235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c19f40 0x561dc3c242c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5d38640 1 -- 172.21.15.19:0/1429019235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c24800 msgr2=0x561dc3c26bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/1429019235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c24800 0x561dc3c26bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5d38640 1 -- 172.21.15.19:0/1429019235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc3891210 con 0x561dc3c19620 2024-04-23T18:45:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.272+0000 7fd0d5537640 1 --2- 172.21.15.19:0/1429019235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c19f40 0x561dc3c242c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:45:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/1429019235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c19a00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8014f10 tx=0x7fd0c80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87715afe288b2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0bc0a3df0 0x7fd0bc0a62b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8011cb0 tx=0x7fd0c8016000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0bb7fe640 1 -- 172.21.15.19:0/1429019235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8024070 con 0x561dc3c19620 2024-04-23T18:45:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0bb7fe640 1 -- 172.21.15.19:0/1429019235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c800bc20 con 0x561dc3c19620 2024-04-23T18:45:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0bb7fe640 1 -- 172.21.15.19:0/1429019235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8004020 con 0x561dc3c19620 2024-04-23T18:45:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0da592740 1 -- 172.21.15.19:0/1429019235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 msgr2=0x561dc3c19a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0da592740 1 --2- 172.21.15.19:0/1429019235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c19a00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8014f10 tx=0x7fd0c80117b0 comp rx=0 tx=0).stop 2024-04-23T18:45:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0da592740 1 -- 172.21.15.19:0/1429019235 shutdown_connections 2024-04-23T18:45:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0da592740 1 --2- 172.21.15.19:0/1429019235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c24800 0x561dc3c26bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0da592740 1 --2- 172.21.15.19:0/1429019235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c19f40 0x561dc3c242c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0da592740 1 --2- 172.21.15.19:0/1429019235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c19a00 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.273+0000 7fd0da592740 1 -- 172.21.15.19:0/1429019235 >> 172.21.15.19:0/1429019235 conn(0x561dc3c15b30 msgr2=0x561dc3c15f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 1 -- 172.21.15.19:0/1429019235 shutdown_connections 2024-04-23T18:45:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 1 -- 172.21.15.19:0/1429019235 wait complete. 2024-04-23T18:45:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 1 -- start start 2024-04-23T18:45:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 1 Processor -- start 2024-04-23T18:45:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19620 0x561dc3c24200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c214f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 0x561dc3c21b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561dc3c273d0 con 0x561dc3c19f40 2024-04-23T18:45:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561dc3b190b0 con 0x561dc3c19620 2024-04-23T18:45:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561dc3c27550 con 0x561dc3c24800 2024-04-23T18:45:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.274+0000 7fd0da592740 10 client.0 _ll_register_callbacks cb 0x561dc3b06d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d4d36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c214f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d4d36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c214f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51350/0 (socket says 172.21.15.19:51350) 2024-04-23T18:45:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d4d36640 1 -- 172.21.15.19:0/1946909532 learned_addr learned my addr 172.21.15.19:0/1946909532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d5537640 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19620 0x561dc3c24200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 0x561dc3c21b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d4d36640 1 -- 172.21.15.19:0/1946909532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 msgr2=0x561dc3c21b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 0x561dc3c21b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d4d36640 1 -- 172.21.15.19:0/1946909532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19620 msgr2=0x561dc3c24200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19620 0x561dc3c24200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d4d36640 1 -- 172.21.15.19:0/1946909532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc395d620 con 0x561dc3c19f40 2024-04-23T18:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 0x561dc3c21b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c214f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00bc940 tx=0x7fd0c002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97493dc836002c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/1946909532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0049050 con 0x561dc3c19f40 2024-04-23T18:45:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/1946909532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7fd0c00ee670 con 0x561dc3c19f40 2024-04-23T18:45:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0da592740 1 -- 172.21.15.19:0/1946909532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0c800bc20 con 0x561dc3c19f40 2024-04-23T18:45:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/1946909532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd0c00bcc40 con 0x561dc3c19f40 2024-04-23T18:45:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0da592740 10 client.17694 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.275+0000 7fd0da592740 1 -- 172.21.15.19:0/1946909532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561dc3c33380 con 0x561dc3c19f40 2024-04-23T18:45:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/1946909532 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c800bc20 con 0x561dc3c19f40 2024-04-23T18:45:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/1946909532 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd0c00b92b0 con 0x561dc3c19f40 2024-04-23T18:45:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0da592740 10 client.17694 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0da592740 10 client.17694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0da592740 20 client.17694 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0da592740 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561dc3c34880 0x561dc3c36c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0da592740 4 client.17694 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0da592740 1 -- 172.21.15.19:0/1946909532 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dc3b8c390 con 0x561dc3c34880 2024-04-23T18:45:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0d5537640 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561dc3c34880 0x561dc3c36c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0d5537640 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561dc3c34880 0x561dc3c36c70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348fc016c0da04dd server_cookie=ea48ae5b6a3efb67 in_seq=0 out_seq=0 2024-04-23T18:45:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.276+0000 7fd0b8ff9640 10 client.17694 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/1946909532 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561dc3b8c390 con 0x561dc3c34880 2024-04-23T18:45:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0b8ff9640 10 client.17694 handle_command_reply: tid=0 2024-04-23T18:45:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0da592740 1 client.17694 shutdown 2024-04-23T18:45:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0da592740 1 -- 172.21.15.19:0/1946909532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 msgr2=0x561dc3c214f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0da592740 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c214f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00bc940 tx=0x7fd0c002a480 comp rx=0 tx=0).stop 2024-04-23T18:45:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0da592740 1 -- 172.21.15.19:0/1946909532 shutdown_connections 2024-04-23T18:45:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0da592740 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561dc3c34880 0x561dc3c36c70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0da592740 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 0x561dc3c21b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0da592740 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c214f0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0b8ff9640 0 client.17694 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0da592740 1 --2- 172.21.15.19:0/1946909532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19620 0x561dc3c24200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.277+0000 7fd0da592740 1 -- 172.21.15.19:0/1946909532 >> 172.21.15.19:0/1946909532 conn(0x561dc3c15b30 msgr2=0x561dc3c1b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.278+0000 7fd0da592740 1 -- 172.21.15.19:0/1946909532 shutdown_connections 2024-04-23T18:45:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.278+0000 7fd0da592740 1 -- 172.21.15.19:0/1946909532 wait complete. 2024-04-23T18:45:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.278+0000 7fd0da592740 20 client.17694 trim_cache size 0 max 16384 2024-04-23T18:45:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0da592740 1 Processor -- start 2024-04-23T18:45:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0da592740 1 -- start start 2024-04-23T18:45:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0da592740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c19620 0x561dc3c30290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0da592740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c14b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0da592740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c24800 0x561dc3c23d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0da592740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01afda0 con 0x561dc3c19f40 2024-04-23T18:45:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0da592740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01b18e0 con 0x561dc3c24800 2024-04-23T18:45:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0da592740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01b0470 con 0x561dc3c19620 2024-04-23T18:45:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0d4d36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c14b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0d4d36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c14b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51362/0 (socket says 172.21.15.19:51362) 2024-04-23T18:45:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0d4d36640 1 -- 172.21.15.19:0/831494054 learned_addr learned my addr 172.21.15.19:0/831494054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/831494054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c24800 0x561dc3c23d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:30.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0d4d36640 1 -- 172.21.15.19:0/831494054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c19620 msgr2=0x561dc3c30290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/831494054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c19620 0x561dc3c30290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0d4d36640 1 -- 172.21.15.19:0/831494054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c24800 msgr2=0x561dc3c23d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/831494054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c24800 0x561dc3c23d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0d4d36640 1 -- 172.21.15.19:0/831494054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc3c36fe0 con 0x561dc3c19f40 2024-04-23T18:45:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.293+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/831494054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c14b30 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00ee4a0 tx=0x7fd0c00bb150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b834961e2807ca75 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0baffd640 1 -- 172.21.15.19:0/831494054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0049050 con 0x561dc3c19f40 2024-04-23T18:45:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0baffd640 1 -- 172.21.15.19:0/831494054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c009f020 con 0x561dc3c19f40 2024-04-23T18:45:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0baffd640 1 -- 172.21.15.19:0/831494054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00ee670 con 0x561dc3c19f40 2024-04-23T18:45:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0da592740 1 -- 172.21.15.19:0/831494054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 msgr2=0x561dc3c14b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0da592740 1 --2- 172.21.15.19:0/831494054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c14b30 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00ee4a0 tx=0x7fd0c00bb150 comp rx=0 tx=0).stop 2024-04-23T18:45:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0da592740 1 -- 172.21.15.19:0/831494054 shutdown_connections 2024-04-23T18:45:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0da592740 1 --2- 172.21.15.19:0/831494054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c24800 0x561dc3c23d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0da592740 1 --2- 172.21.15.19:0/831494054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19f40 0x561dc3c14b30 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0da592740 1 --2- 172.21.15.19:0/831494054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c19620 0x561dc3c30290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0da592740 1 -- 172.21.15.19:0/831494054 >> 172.21.15.19:0/831494054 conn(0x561dc3c15b30 msgr2=0x561dc3c14420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0da592740 1 -- 172.21.15.19:0/831494054 shutdown_connections 2024-04-23T18:45:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.294+0000 7fd0da592740 1 -- 172.21.15.19:0/831494054 wait complete. 2024-04-23T18:45:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0da592740 1 -- start start 2024-04-23T18:45:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0da592740 1 Processor -- start 2024-04-23T18:45:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0da592740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c2cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0da592740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19f40 0x561dc3c1df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0da592740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 0x561dc3c1e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0da592740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01b1210 con 0x561dc3c19620 2024-04-23T18:45:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0da592740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01b0b40 con 0x561dc3c19f40 2024-04-23T18:45:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0da592740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01adda0 con 0x561dc3c24800 2024-04-23T18:45:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0da592740 10 client.0 _ll_register_callbacks cb 0x7fd0c00bc7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0d5537640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c2cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0d5537640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c2cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51372/0 (socket says 172.21.15.19:51372) 2024-04-23T18:45:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0d5537640 1 -- 172.21.15.19:0/2166146538 learned_addr learned my addr 172.21.15.19:0/2166146538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.295+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19f40 0x561dc3c1df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0d5d38640 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 0x561dc3c1e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0d5537640 1 -- 172.21.15.19:0/2166146538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 msgr2=0x561dc3c1e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0d5537640 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 0x561dc3c1e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0d5537640 1 -- 172.21.15.19:0/2166146538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19f40 msgr2=0x561dc3c1df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0d5537640 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19f40 0x561dc3c1df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0d5537640 1 -- 172.21.15.19:0/2166146538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc3c1ed00 con 0x561dc3c19620 2024-04-23T18:45:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19f40 0x561dc3c1df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0d5537640 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c2cbf0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x561dc3c37520 tx=0x7fd0cc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd881220ea9ff6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/2166146538 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0201b0 con 0x561dc3c19620 2024-04-23T18:45:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/2166146538 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7fd0cc020350 con 0x561dc3c19620 2024-04-23T18:45:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0da592740 1 -- 172.21.15.19:0/2166146538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0c009f020 con 0x561dc3c19620 2024-04-23T18:45:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/2166146538 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd0cc02f050 con 0x561dc3c19620 2024-04-23T18:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0da592740 10 client.17700 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0da592740 1 -- 172.21.15.19:0/2166146538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561dc3c9aef0 con 0x561dc3c19620 2024-04-23T18:45:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.296+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/2166146538 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c009f020 con 0x561dc3c19620 2024-04-23T18:45:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.297+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/2166146538 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd0cc011b70 con 0x561dc3c19620 2024-04-23T18:45:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.297+0000 7fd0da592740 10 client.17700 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.297+0000 7fd0da592740 10 client.17700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.297+0000 7fd0da592740 20 client.17700 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.297+0000 7fd0da592740 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561dc3c9b660 0x561dc3c9da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.297+0000 7fd0da592740 4 client.17700 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.297+0000 7fd0da592740 1 -- 172.21.15.19:0/2166146538 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561dc3b86200 con 0x561dc3c9b660 2024-04-23T18:45:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.297+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561dc3c9b660 0x561dc3c9da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.297+0000 7fd0b8ff9640 10 client.17700 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.297+0000 7fd0d4d36640 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561dc3c9b660 0x561dc3c9da80 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474bb5c4d9e76807 server_cookie=ef2809a69e88d25 in_seq=0 out_seq=0 2024-04-23T18:45:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.302+0000 7fd0b8ff9640 1 -- 172.21.15.19:0/2166146538 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561dc3b86200 con 0x561dc3c9b660 2024-04-23T18:45:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.302+0000 7fd0b8ff9640 10 client.17700 handle_command_reply: tid=0 2024-04-23T18:45:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.302+0000 7fd0da592740 1 client.17700 shutdown 2024-04-23T18:45:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.302+0000 7fd0da592740 1 -- 172.21.15.19:0/2166146538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 msgr2=0x561dc3c2cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.302+0000 7fd0da592740 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c2cbf0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x561dc3c37520 tx=0x7fd0cc00be70 comp rx=0 tx=0).stop 2024-04-23T18:45:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0da592740 1 -- 172.21.15.19:0/2166146538 shutdown_connections 2024-04-23T18:45:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0da592740 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x561dc3c9b660 0x561dc3c9da80 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0da592740 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dc3c19f40 0x561dc3c1df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0da592740 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc3c24800 0x561dc3c1e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0da592740 1 --2- 172.21.15.19:0/2166146538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561dc3c19620 0x561dc3c2cbf0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0b8ff9640 0 client.17700 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0da592740 1 -- 172.21.15.19:0/2166146538 >> 172.21.15.19:0/2166146538 conn(0x561dc3c15b30 msgr2=0x561dc3c14420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0da592740 1 -- 172.21.15.19:0/2166146538 shutdown_connections 2024-04-23T18:45:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0da592740 1 -- 172.21.15.19:0/2166146538 wait complete. 2024-04-23T18:45:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0da592740 20 client.17700 trim_cache size 0 max 16384 2024-04-23T18:45:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0d779b640 1 -- 172.21.15.19:0/954155688 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0bc0a3df0 msgr2=0x7fd0bc0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0d779b640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0bc0a3df0 0x7fd0bc0a62b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8011cb0 tx=0x7fd0c8016000 comp rx=0 tx=0).stop 2024-04-23T18:45:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0d779b640 1 -- 172.21.15.19:0/954155688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0d0103b20 msgr2=0x7fd0d0101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0d779b640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0d0103b20 0x7fd0d0101390 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0005940 tx=0x7fd0c00042f0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0d779b640 1 -- 172.21.15.19:0/954155688 shutdown_connections 2024-04-23T18:45:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0d779b640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fd0bc0a3df0 0x7fd0bc0a62b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0d779b640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0d010c5d0 0x7fd0d01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0d779b640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0d0103b20 0x7fd0d0101390 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0d779b640 1 --2- 172.21.15.19:0/954155688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0d0103200 0x7fd0d0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.303+0000 7fd0d779b640 1 -- 172.21.15.19:0/954155688 >> 172.21.15.19:0/954155688 conn(0x7fd0d00771d0 msgr2=0x7fd0d00ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.304+0000 7fd0d779b640 1 -- 172.21.15.19:0/954155688 shutdown_connections 2024-04-23T18:45:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:37.304+0000 7fd0d779b640 1 -- 172.21.15.19:0/954155688 wait complete. 2024-04-23T18:45:31.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:37 smithi019 ceph-mon[29215]: pgmap v583: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:45:31.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:39 smithi019 ceph-mon[29215]: pgmap v584: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:45:31.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:41 smithi019 ceph-mon[29215]: pgmap v585: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:45:31.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:43 smithi019 ceph-mon[29215]: pgmap v586: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:31.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:45 smithi019 ceph-mon[29215]: pgmap v587: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:31.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:47 smithi019 ceph-mon[29215]: pgmap v588: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:31.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:49 smithi019 ceph-mon[29215]: pgmap v589: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:31.782 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:27 smithi155 ceph-mon[33468]: pgmap v611: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:45:31.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:29 smithi155 ceph-mon[33468]: pgmap v612: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 388 B/s wr, 10 op/s 2024-04-23T18:45:31.799 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:27 smithi044 ceph-mon[32458]: pgmap v611: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:45:31.809 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:29 smithi044 ceph-mon[32458]: pgmap v612: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 388 B/s wr, 10 op/s 2024-04-23T18:45:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.402+0000 7fab623d8640 1 -- 172.21.15.19:0/3750234215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c103aa0 msgr2=0x7fab5c10bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.402+0000 7fab623d8640 1 --2- 172.21.15.19:0/3750234215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c103aa0 0x7fab5c10bf80 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fab44009900 tx=0x7fab4402a480 comp rx=0 tx=0).stop 2024-04-23T18:45:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.403+0000 7fab623d8640 1 -- 172.21.15.19:0/3750234215 shutdown_connections 2024-04-23T18:45:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.403+0000 7fab623d8640 1 --2- 172.21.15.19:0/3750234215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c10c550 0x7fab5c10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.403+0000 7fab623d8640 1 --2- 172.21.15.19:0/3750234215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c103aa0 0x7fab5c10bf80 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.403+0000 7fab623d8640 1 --2- 172.21.15.19:0/3750234215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5c103180 0x7fab5c103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.403+0000 7fab623d8640 1 -- 172.21.15.19:0/3750234215 >> 172.21.15.19:0/3750234215 conn(0x7fab5c077fd0 msgr2=0x7fab5c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.403+0000 7fab623d8640 1 -- 172.21.15.19:0/3750234215 shutdown_connections 2024-04-23T18:45:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.403+0000 7fab623d8640 1 -- 172.21.15.19:0/3750234215 wait complete. 2024-04-23T18:45:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.403+0000 7fab623d8640 1 Processor -- start 2024-04-23T18:45:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.403+0000 7fab623d8640 1 -- start start 2024-04-23T18:45:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab623d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5c103180 0x7fab5c100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab623d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c103aa0 0x7fab5c101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab623d8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c10c550 0x7fab5c101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab623d8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c1121b0 con 0x7fab5c103aa0 2024-04-23T18:45:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab623d8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c112030 con 0x7fab5c103180 2024-04-23T18:45:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab623d8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c112330 con 0x7fab5c10c550 2024-04-23T18:45:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c103aa0 0x7fab5c101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c103aa0 0x7fab5c101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36244/0 (socket says 172.21.15.19:36244) 2024-04-23T18:45:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2715812288 learned_addr learned my addr 172.21.15.19:0/2715812288 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab5bfff640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5c103180 0x7fab5c100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab60975640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c10c550 0x7fab5c101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2715812288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c10c550 msgr2=0x7fab5c101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c10c550 0x7fab5c101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2715812288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5c103180 msgr2=0x7fab5c100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5c103180 0x7fab5c100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2715812288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c1a69d0 con 0x7fab5c103aa0 2024-04-23T18:45:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.404+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c103aa0 0x7fab5c101310 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fab44032fd0 tx=0x7fab44030630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.405+0000 7fab597fa640 1 -- 172.21.15.19:0/2715812288 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4403f070 con 0x7fab5c103aa0 2024-04-23T18:45:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.405+0000 7fab597fa640 1 -- 172.21.15.19:0/2715812288 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab44030d00 con 0x7fab5c103aa0 2024-04-23T18:45:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.405+0000 7fab597fa640 1 -- 172.21.15.19:0/2715812288 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44035030 con 0x7fab5c103aa0 2024-04-23T18:45:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.405+0000 7fab623d8640 1 -- 172.21.15.19:0/2715812288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c1a6c60 con 0x7fab5c103aa0 2024-04-23T18:45:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.405+0000 7fab623d8640 1 -- 172.21.15.19:0/2715812288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c1a7100 con 0x7fab5c103aa0 2024-04-23T18:45:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.407+0000 7fab597fa640 1 -- 172.21.15.19:0/2715812288 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab44042070 con 0x7fab5c103aa0 2024-04-23T18:45:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.407+0000 7fab651c9740 1 Processor -- start 2024-04-23T18:45:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.407+0000 7fab651c9740 1 -- start start 2024-04-23T18:45:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab651c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 0x55697cdcca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab651c9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdccf40 0x55697cdd72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab651c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cdd9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab651c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55697cd31970 con 0x55697cdd7800 2024-04-23T18:45:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab651c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55697ccce4a0 con 0x55697cdcc620 2024-04-23T18:45:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab651c9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55697cd2d930 con 0x55697cdccf40 2024-04-23T18:45:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cdd9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cdd9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36256/0 (socket says 172.21.15.19:36256) 2024-04-23T18:45:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab5b7fe640 1 -- 172.21.15.19:0/3289799647 learned_addr learned my addr 172.21.15.19:0/3289799647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab597fa640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fab480a3dc0 0x7fab480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab60975640 1 --2- 172.21.15.19:0/3289799647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 0x55697cdcca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab5bfff640 1 --2- 172.21.15.19:0/3289799647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdccf40 0x55697cdd72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab60975640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fab480a3dc0 0x7fab480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab597fa640 1 -- 172.21.15.19:0/2715812288 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7fab440f1480 con 0x7fab5c103aa0 2024-04-23T18:45:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab5b7fe640 1 -- 172.21.15.19:0/3289799647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdccf40 msgr2=0x55697cdd72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/3289799647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdccf40 0x55697cdd72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab5b7fe640 1 -- 172.21.15.19:0/3289799647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 msgr2=0x55697cdcca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/3289799647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 0x55697cdcca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab5b7fe640 1 -- 172.21.15.19:0/3289799647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55697ca44210 con 0x55697cdd7800 2024-04-23T18:45:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab60975640 1 --2- 172.21.15.19:0/3289799647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 0x55697cdcca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/3289799647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cdd9bf0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fab440eeb70 tx=0x7fab44005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66296318b5e7d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.408+0000 7fab60975640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fab480a3dc0 0x7fab480a6280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab54005c40 tx=0x7fab54005bd0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab527fc640 1 -- 172.21.15.19:0/3289799647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44049070 con 0x55697cdd7800 2024-04-23T18:45:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab527fc640 1 -- 172.21.15.19:0/3289799647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab440bdb20 con 0x55697cdd7800 2024-04-23T18:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab527fc640 1 -- 172.21.15.19:0/3289799647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440b9a90 con 0x55697cdd7800 2024-04-23T18:45:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab651c9740 1 -- 172.21.15.19:0/3289799647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 msgr2=0x55697cdd9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab651c9740 1 --2- 172.21.15.19:0/3289799647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cdd9bf0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fab440eeb70 tx=0x7fab44005fd0 comp rx=0 tx=0).stop 2024-04-23T18:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab651c9740 1 -- 172.21.15.19:0/3289799647 shutdown_connections 2024-04-23T18:45:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab651c9740 1 --2- 172.21.15.19:0/3289799647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cdd9bf0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab651c9740 1 --2- 172.21.15.19:0/3289799647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdccf40 0x55697cdd72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab651c9740 1 --2- 172.21.15.19:0/3289799647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 0x55697cdcca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab651c9740 1 -- 172.21.15.19:0/3289799647 >> 172.21.15.19:0/3289799647 conn(0x55697cdc8b30 msgr2=0x55697cdc8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab651c9740 1 -- 172.21.15.19:0/3289799647 shutdown_connections 2024-04-23T18:45:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.409+0000 7fab651c9740 1 -- 172.21.15.19:0/3289799647 wait complete. 2024-04-23T18:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab651c9740 1 -- start start 2024-04-23T18:45:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab651c9740 1 Processor -- start 2024-04-23T18:45:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab651c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 0x55697cdd7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab651c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdd4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab651c9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdd7800 0x55697cdd4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab651c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55697cdda1f0 con 0x55697cdccf40 2024-04-23T18:45:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab651c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55697cccc0b0 con 0x55697cdcc620 2024-04-23T18:45:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab651c9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55697cdda370 con 0x55697cdd7800 2024-04-23T18:45:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab651c9740 10 client.0 _ll_register_callbacks cb 0x55697ccb9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdd4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdd4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36268/0 (socket says 172.21.15.19:36268) 2024-04-23T18:45:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2541974249 learned_addr learned my addr 172.21.15.19:0/2541974249 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab5bfff640 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 0x55697cdd7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.410+0000 7fab60975640 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdd7800 0x55697cdd4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2541974249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdd7800 msgr2=0x55697cdd4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdd7800 0x55697cdd4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2541974249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 msgr2=0x55697cdd7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 0x55697cdd7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2541974249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55697cb10620 con 0x55697cdccf40 2024-04-23T18:45:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdd4420 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fab440bd3b0 tx=0x7fab4402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cac1837fa255fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab3b7fe640 1 -- 172.21.15.19:0/2541974249 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44049040 con 0x55697cdccf40 2024-04-23T18:45:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab3b7fe640 1 -- 172.21.15.19:0/2541974249 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7fab440b9170 con 0x55697cdccf40 2024-04-23T18:45:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab651c9740 1 -- 172.21.15.19:0/2541974249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab440bdb20 con 0x55697cdccf40 2024-04-23T18:45:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab3b7fe640 1 -- 172.21.15.19:0/2541974249 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fab440b96c0 con 0x55697cdccf40 2024-04-23T18:45:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab651c9740 10 client.17712 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab651c9740 1 -- 172.21.15.19:0/2541974249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55697cdda1f0 con 0x55697cdccf40 2024-04-23T18:45:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.411+0000 7fab3b7fe640 1 -- 172.21.15.19:0/2541974249 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab440bdb20 con 0x55697cdccf40 2024-04-23T18:45:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.412+0000 7fab3b7fe640 1 -- 172.21.15.19:0/2541974249 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fab440035f0 con 0x55697cdccf40 2024-04-23T18:45:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.412+0000 7fab651c9740 10 client.17712 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.412+0000 7fab651c9740 10 client.17712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.412+0000 7fab651c9740 20 client.17712 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.412+0000 7fab651c9740 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55697cde72a0 0x55697cde9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.412+0000 7fab651c9740 4 client.17712 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.412+0000 7fab651c9740 1 -- 172.21.15.19:0/2541974249 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55697cd3f390 con 0x55697cde72a0 2024-04-23T18:45:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.412+0000 7fab5bfff640 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55697cde72a0 0x55697cde9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:33.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.412+0000 7fab5bfff640 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55697cde72a0 0x55697cde9690 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f648c39eed26b7b9 server_cookie=82b5be4748e9a724 in_seq=0 out_seq=0 2024-04-23T18:45:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.412+0000 7fab3b7fe640 10 client.17712 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab3b7fe640 1 -- 172.21.15.19:0/2541974249 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55697cd3f390 con 0x55697cde72a0 2024-04-23T18:45:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab3b7fe640 10 client.17712 handle_command_reply: tid=0 2024-04-23T18:45:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 client.17712 shutdown 2024-04-23T18:45:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 -- 172.21.15.19:0/2541974249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 msgr2=0x55697cdd4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdd4420 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fab440bd3b0 tx=0x7fab4402a480 comp rx=0 tx=0).stop 2024-04-23T18:45:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 -- 172.21.15.19:0/2541974249 shutdown_connections 2024-04-23T18:45:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55697cde72a0 0x55697cde9690 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdd7800 0x55697cdd4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdd4420 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 --2- 172.21.15.19:0/2541974249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdcc620 0x55697cdd7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab3b7fe640 0 client.17712 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 -- 172.21.15.19:0/2541974249 >> 172.21.15.19:0/2541974249 conn(0x55697cdc8b30 msgr2=0x55697cdce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 -- 172.21.15.19:0/2541974249 shutdown_connections 2024-04-23T18:45:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 1 -- 172.21.15.19:0/2541974249 wait complete. 2024-04-23T18:45:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.413+0000 7fab651c9740 20 client.17712 trim_cache size 0 max 16384 2024-04-23T18:45:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.428+0000 7fab651c9740 1 Processor -- start 2024-04-23T18:45:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.428+0000 7fab651c9740 1 -- start start 2024-04-23T18:45:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab651c9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdcc620 0x55697cde3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab651c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdc7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab651c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdd7800 0x55697cdeef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab651c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c1afaf0 con 0x55697cdccf40 2024-04-23T18:45:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab651c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c1b1630 con 0x55697cdd7800 2024-04-23T18:45:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab651c9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c1b01c0 con 0x55697cdcc620 2024-04-23T18:45:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdc7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdc7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36280/0 (socket says 172.21.15.19:36280) 2024-04-23T18:45:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2443590221 learned_addr learned my addr 172.21.15.19:0/2443590221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab5bfff640 1 --2- 172.21.15.19:0/2443590221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdcc620 0x55697cde3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab60975640 1 --2- 172.21.15.19:0/2443590221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdd7800 0x55697cdeef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2443590221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdcc620 msgr2=0x55697cde3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2443590221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdcc620 0x55697cde3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2443590221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdd7800 msgr2=0x55697cdeef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2443590221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdd7800 0x55697cdeef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2443590221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55697cdd2970 con 0x55697cdccf40 2024-04-23T18:45:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab60975640 1 --2- 172.21.15.19:0/2443590221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdd7800 0x55697cdeef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2443590221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdc7b30 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fab440bb6d0 tx=0x7fab44003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823f35cc7ecf83c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab51ffb640 1 -- 172.21.15.19:0/2443590221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44049070 con 0x55697cdccf40 2024-04-23T18:45:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab51ffb640 1 -- 172.21.15.19:0/2443590221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab440bdb20 con 0x55697cdccf40 2024-04-23T18:45:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.429+0000 7fab51ffb640 1 -- 172.21.15.19:0/2443590221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440bebe0 con 0x55697cdccf40 2024-04-23T18:45:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.430+0000 7fab651c9740 1 -- 172.21.15.19:0/2443590221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 msgr2=0x55697cdc7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.430+0000 7fab651c9740 1 --2- 172.21.15.19:0/2443590221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdc7b30 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fab440bb6d0 tx=0x7fab44003990 comp rx=0 tx=0).stop 2024-04-23T18:45:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.430+0000 7fab651c9740 1 -- 172.21.15.19:0/2443590221 shutdown_connections 2024-04-23T18:45:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.430+0000 7fab651c9740 1 --2- 172.21.15.19:0/2443590221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdd7800 0x55697cdeef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.430+0000 7fab651c9740 1 --2- 172.21.15.19:0/2443590221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdccf40 0x55697cdc7b30 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.430+0000 7fab651c9740 1 --2- 172.21.15.19:0/2443590221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdcc620 0x55697cde3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.430+0000 7fab651c9740 1 -- 172.21.15.19:0/2443590221 >> 172.21.15.19:0/2443590221 conn(0x55697cdc8b30 msgr2=0x55697cdc7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.430+0000 7fab651c9740 1 -- 172.21.15.19:0/2443590221 shutdown_connections 2024-04-23T18:45:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.430+0000 7fab651c9740 1 -- 172.21.15.19:0/2443590221 wait complete. 2024-04-23T18:45:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.430+0000 7fab651c9740 1 -- start start 2024-04-23T18:45:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab651c9740 1 Processor -- start 2024-04-23T18:45:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab651c9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdcc620 0x55697cddf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab651c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdccf40 0x55697cddc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab651c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cddcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab651c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c1b0f60 con 0x55697cdd7800 2024-04-23T18:45:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab651c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c1b0890 con 0x55697cdccf40 2024-04-23T18:45:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab651c9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c1adaf0 con 0x55697cdcc620 2024-04-23T18:45:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab651c9740 10 client.0 _ll_register_callbacks cb 0x7fab44004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab60975640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cddcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab60975640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cddcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36292/0 (socket says 172.21.15.19:36292) 2024-04-23T18:45:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab60975640 1 -- 172.21.15.19:0/3640595165 learned_addr learned my addr 172.21.15.19:0/3640595165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab5bfff640 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdcc620 0x55697cddf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdccf40 0x55697cddc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab60975640 1 -- 172.21.15.19:0/3640595165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdcc620 msgr2=0x55697cddf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab60975640 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdcc620 0x55697cddf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab60975640 1 -- 172.21.15.19:0/3640595165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdccf40 msgr2=0x55697cddc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab60975640 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdccf40 0x55697cddc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab60975640 1 -- 172.21.15.19:0/3640595165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55697cddd3e0 con 0x55697cdd7800 2024-04-23T18:45:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdccf40 0x55697cddc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab60975640 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cddcea0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fab5400fd00 tx=0x7fab54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a344d6b46bc6e0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.431+0000 7fab3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab3b7fe640 1 -- 172.21.15.19:0/3640595165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54004b20 con 0x55697cdd7800 2024-04-23T18:45:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab3b7fe640 1 -- 172.21.15.19:0/3640595165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7fab54004cc0 con 0x55697cdd7800 2024-04-23T18:45:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab651c9740 1 -- 172.21.15.19:0/3640595165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab440bdb20 con 0x55697cdd7800 2024-04-23T18:45:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab3b7fe640 1 -- 172.21.15.19:0/3640595165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fab54011eb0 con 0x55697cdd7800 2024-04-23T18:45:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab651c9740 10 client.17718 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab651c9740 1 -- 172.21.15.19:0/3640595165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55697ce4d4f0 con 0x55697cdd7800 2024-04-23T18:45:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab3b7fe640 1 -- 172.21.15.19:0/3640595165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab440bdb20 con 0x55697cdd7800 2024-04-23T18:45:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab3b7fe640 1 -- 172.21.15.19:0/3640595165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fab5401e070 con 0x55697cdd7800 2024-04-23T18:45:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab651c9740 10 client.17718 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab651c9740 10 client.17718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab651c9740 20 client.17718 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab651c9740 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55697ce4dc50 0x55697ce50070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab651c9740 4 client.17718 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.432+0000 7fab651c9740 1 -- 172.21.15.19:0/3640595165 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55697cdd4cf0 con 0x55697ce4dc50 2024-04-23T18:45:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.433+0000 7fab5bfff640 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55697ce4dc50 0x55697ce50070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.433+0000 7fab5bfff640 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55697ce4dc50 0x55697ce50070 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976afa8e03257de8 server_cookie=1edba36b501fae2 in_seq=0 out_seq=0 2024-04-23T18:45:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.433+0000 7fab3b7fe640 10 client.17718 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.438+0000 7fab3b7fe640 1 -- 172.21.15.19:0/3640595165 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55697cdd4cf0 con 0x55697ce4dc50 2024-04-23T18:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.438+0000 7fab3b7fe640 10 client.17718 handle_command_reply: tid=0 2024-04-23T18:45:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 client.17718 shutdown 2024-04-23T18:45:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 -- 172.21.15.19:0/3640595165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 msgr2=0x55697cddcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cddcea0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fab5400fd00 tx=0x7fab54004a40 comp rx=0 tx=0).stop 2024-04-23T18:45:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 -- 172.21.15.19:0/3640595165 shutdown_connections 2024-04-23T18:45:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55697ce4dc50 0x55697ce50070 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55697cdccf40 0x55697cddc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55697cdd7800 0x55697cddcea0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab3b7fe640 0 client.17718 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 --2- 172.21.15.19:0/3640595165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55697cdcc620 0x55697cddf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 -- 172.21.15.19:0/3640595165 >> 172.21.15.19:0/3640595165 conn(0x55697cdc8b30 msgr2=0x55697cdc7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 -- 172.21.15.19:0/3640595165 shutdown_connections 2024-04-23T18:45:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 1 -- 172.21.15.19:0/3640595165 wait complete. 2024-04-23T18:45:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab651c9740 20 client.17718 trim_cache size 0 max 16384 2024-04-23T18:45:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab623d8640 1 -- 172.21.15.19:0/2715812288 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fab480a3dc0 msgr2=0x7fab480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.439+0000 7fab623d8640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fab480a3dc0 0x7fab480a6280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab54005c40 tx=0x7fab54005bd0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.440+0000 7fab623d8640 1 -- 172.21.15.19:0/2715812288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c103aa0 msgr2=0x7fab5c101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.440+0000 7fab623d8640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c103aa0 0x7fab5c101310 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fab44032fd0 tx=0x7fab44030630 comp rx=0 tx=0).stop 2024-04-23T18:45:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.440+0000 7fab623d8640 1 -- 172.21.15.19:0/2715812288 shutdown_connections 2024-04-23T18:45:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.440+0000 7fab623d8640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fab480a3dc0 0x7fab480a6280 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.440+0000 7fab623d8640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c10c550 0x7fab5c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.440+0000 7fab623d8640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c103aa0 0x7fab5c101310 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.440+0000 7fab623d8640 1 --2- 172.21.15.19:0/2715812288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab5c103180 0x7fab5c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.440+0000 7fab623d8640 1 -- 172.21.15.19:0/2715812288 >> 172.21.15.19:0/2715812288 conn(0x7fab5c077fd0 msgr2=0x7fab5c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.440+0000 7fab623d8640 1 -- 172.21.15.19:0/2715812288 shutdown_connections 2024-04-23T18:45:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:44:50.440+0000 7fab623d8640 1 -- 172.21.15.19:0/2715812288 wait complete. 2024-04-23T18:45:35.556 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:31 smithi155 ceph-mon[33468]: pgmap v613: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T18:45:35.569 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:31 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:35.583 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:31 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:35.596 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:31 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:35.606 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:31 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:45:35.620 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:33 smithi155 ceph-mon[33468]: pgmap v614: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:35.627 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:31 smithi044 ceph-mon[32458]: pgmap v613: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T18:45:35.638 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:31 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:35.712 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:31 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:35.722 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:31 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:35.730 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:31 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:45:35.739 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:33 smithi044 ceph-mon[32458]: pgmap v614: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:35.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:51 smithi019 ceph-mon[29215]: pgmap v590: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:35.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:53 smithi019 ceph-mon[29215]: pgmap v591: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:35.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:55 smithi019 ceph-mon[29215]: pgmap v592: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:35.786 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:57 smithi019 ceph-mon[29215]: pgmap v593: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:35.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:44:59 smithi019 ceph-mon[29215]: pgmap v594: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:35.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:01 smithi019 ceph-mon[29215]: pgmap v595: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.539+0000 7f26bada2640 1 -- 172.21.15.19:0/2763822158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103200 msgr2=0x7f26b41035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.539+0000 7f26bada2640 1 --2- 172.21.15.19:0/2763822158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103200 0x7f26b41035e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f269c009900 tx=0x7f269c02a480 comp rx=0 tx=0).stop 2024-04-23T18:45:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.539+0000 7f26bada2640 1 -- 172.21.15.19:0/2763822158 shutdown_connections 2024-04-23T18:45:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.539+0000 7f26bada2640 1 --2- 172.21.15.19:0/2763822158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b4103b20 0x7f26b410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.539+0000 7f26bada2640 1 --2- 172.21.15.19:0/2763822158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b410c5d0 0x7f26b410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.539+0000 7f26bada2640 1 --2- 172.21.15.19:0/2763822158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103200 0x7f26b41035e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.539+0000 7f26bada2640 1 -- 172.21.15.19:0/2763822158 >> 172.21.15.19:0/2763822158 conn(0x7f26b40771d0 msgr2=0x7f26b40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.539+0000 7f26bada2640 1 -- 172.21.15.19:0/2763822158 shutdown_connections 2024-04-23T18:45:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.539+0000 7f26bada2640 1 -- 172.21.15.19:0/2763822158 wait complete. 2024-04-23T18:45:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26bada2640 1 Processor -- start 2024-04-23T18:45:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26bada2640 1 -- start start 2024-04-23T18:45:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26bada2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103200 0x7f26b4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26bada2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b4103b20 0x7f26b4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26bada2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b410c5d0 0x7f26b419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26bada2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4112220 con 0x7f26b4103200 2024-04-23T18:45:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26bada2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41120a0 con 0x7f26b410c5d0 2024-04-23T18:45:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26bada2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41123a0 con 0x7f26b4103b20 2024-04-23T18:45:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26b8b3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103200 0x7f26b4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26b8b3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103200 0x7f26b4198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42774/0 (socket says 172.21.15.19:42774) 2024-04-23T18:45:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26b8b3e640 1 -- 172.21.15.19:0/4071896960 learned_addr learned my addr 172.21.15.19:0/4071896960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26b3fff640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b4103b20 0x7f26b4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.540+0000 7f26b933f640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b410c5d0 0x7f26b419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26b8b3e640 1 -- 172.21.15.19:0/4071896960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b4103b20 msgr2=0x7f26b4198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b4103b20 0x7f26b4198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26b8b3e640 1 -- 172.21.15.19:0/4071896960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b410c5d0 msgr2=0x7f26b419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b410c5d0 0x7f26b419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26b8b3e640 1 -- 172.21.15.19:0/4071896960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b419d6a0 con 0x7f26b4103200 2024-04-23T18:45:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26b3fff640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b4103b20 0x7f26b4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103200 0x7f26b4198700 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f269c005940 tx=0x7f269c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26b1ffb640 1 -- 172.21.15.19:0/4071896960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c03f030 con 0x7f26b4103200 2024-04-23T18:45:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26b1ffb640 1 -- 172.21.15.19:0/4071896960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f269c003aa0 con 0x7f26b4103200 2024-04-23T18:45:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26bada2640 1 -- 172.21.15.19:0/4071896960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b419d930 con 0x7f26b4103200 2024-04-23T18:45:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26b1ffb640 1 -- 172.21.15.19:0/4071896960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c035070 con 0x7f26b4103200 2024-04-23T18:45:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.541+0000 7f26bada2640 1 -- 172.21.15.19:0/4071896960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b41a5220 con 0x7f26b4103200 2024-04-23T18:45:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26b1ffb640 1 -- 172.21.15.19:0/4071896960 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f269c042030 con 0x7f26b4103200 2024-04-23T18:45:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26bdb99740 1 Processor -- start 2024-04-23T18:45:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26bdb99740 1 -- start start 2024-04-23T18:45:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39620 0x5567f9d39a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 0x5567f9d442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d44800 0x5567f9d46bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26bdb99740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567f9c9e970 con 0x5567f9d44800 2024-04-23T18:45:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26bdb99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5567f9c3b4a0 con 0x5567f9d39620 2024-04-23T18:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26bdb99740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567f9c9a930 con 0x5567f9d39f40 2024-04-23T18:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26b1ffb640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26a00a3dc0 0x7f26a00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.544+0000 7f26b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d44800 0x5567f9d46bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d44800 0x5567f9d46bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42786/0 (socket says 172.21.15.19:42786) 2024-04-23T18:45:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b3fff640 1 -- 172.21.15.19:0/553919040 learned_addr learned my addr 172.21.15.19:0/553919040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b933f640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26a00a3dc0 0x7f26a00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 0x5567f9d442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b933f640 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39620 0x5567f9d39a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b1ffb640 1 -- 172.21.15.19:0/4071896960 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7f269c0f0fa0 con 0x7f26b4103200 2024-04-23T18:45:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b3fff640 1 -- 172.21.15.19:0/553919040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 msgr2=0x5567f9d442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b3fff640 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 0x5567f9d442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b3fff640 1 -- 172.21.15.19:0/553919040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39620 msgr2=0x5567f9d39a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b3fff640 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39620 0x5567f9d39a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b3fff640 1 -- 172.21.15.19:0/553919040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f99b1210 con 0x5567f9d44800 2024-04-23T18:45:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 0x5567f9d442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b933f640 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39620 0x5567f9d39a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b3fff640 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d44800 0x5567f9d46bf0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f26a400bcb0 tx=0x7f26a400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e647dc00565d5658 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26b933f640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26a00a3dc0 0x7f26a00a6280 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009270 tx=0x7f26ac009b20 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26aaffd640 1 -- 172.21.15.19:0/553919040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4011440 con 0x5567f9d44800 2024-04-23T18:45:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26aaffd640 1 -- 172.21.15.19:0/553919040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a4011a90 con 0x5567f9d44800 2024-04-23T18:45:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.545+0000 7f26aaffd640 1 -- 172.21.15.19:0/553919040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a401ba70 con 0x5567f9d44800 2024-04-23T18:45:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.546+0000 7f26bdb99740 1 -- 172.21.15.19:0/553919040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d44800 msgr2=0x5567f9d46bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.546+0000 7f26bdb99740 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d44800 0x5567f9d46bf0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f26a400bcb0 tx=0x7f26a400f730 comp rx=0 tx=0).stop 2024-04-23T18:45:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.546+0000 7f26bdb99740 1 -- 172.21.15.19:0/553919040 shutdown_connections 2024-04-23T18:45:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.546+0000 7f26bdb99740 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d44800 0x5567f9d46bf0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.546+0000 7f26bdb99740 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 0x5567f9d442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.546+0000 7f26bdb99740 1 --2- 172.21.15.19:0/553919040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39620 0x5567f9d39a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.546+0000 7f26bdb99740 1 -- 172.21.15.19:0/553919040 >> 172.21.15.19:0/553919040 conn(0x5567f9d35b30 msgr2=0x5567f9d35f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.546+0000 7f26bdb99740 1 -- 172.21.15.19:0/553919040 shutdown_connections 2024-04-23T18:45:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.546+0000 7f26bdb99740 1 -- 172.21.15.19:0/553919040 wait complete. 2024-04-23T18:45:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.546+0000 7f26bdb99740 1 -- start start 2024-04-23T18:45:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26bdb99740 1 Processor -- start 2024-04-23T18:45:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d44120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 0x5567f9d41410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d44800 0x5567f9d41950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26bdb99740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567f9d47430 con 0x5567f9d39620 2024-04-23T18:45:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26bdb99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5567f9c390b0 con 0x5567f9d44800 2024-04-23T18:45:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26bdb99740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567f9d475b0 con 0x5567f9d39f40 2024-04-23T18:45:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26bdb99740 10 client.0 _ll_register_callbacks cb 0x5567f9c26d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b8b3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d44120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b8b3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d44120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42796/0 (socket says 172.21.15.19:42796) 2024-04-23T18:45:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b8b3e640 1 -- 172.21.15.19:0/4152284571 learned_addr learned my addr 172.21.15.19:0/4152284571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b3fff640 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 0x5567f9d41410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b933f640 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d44800 0x5567f9d41950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b8b3e640 1 -- 172.21.15.19:0/4152284571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 msgr2=0x5567f9d41410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 0x5567f9d41410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b8b3e640 1 -- 172.21.15.19:0/4152284571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d44800 msgr2=0x5567f9d41950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d44800 0x5567f9d41950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b8b3e640 1 -- 172.21.15.19:0/4152284571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567f9a7d620 con 0x5567f9d39620 2024-04-23T18:45:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.547+0000 7f26b3fff640 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 0x5567f9d41410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d44120 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f269c0bced0 tx=0x7f269c041d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa61c793496c49c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f2693fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f2693fff640 1 -- 172.21.15.19:0/4152284571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c049040 con 0x5567f9d39620 2024-04-23T18:45:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f2693fff640 1 -- 172.21.15.19:0/4152284571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7f269c0ee970 con 0x5567f9d39620 2024-04-23T18:45:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f26bdb99740 1 -- 172.21.15.19:0/4152284571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a4011a90 con 0x5567f9d39620 2024-04-23T18:45:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f2693fff640 1 -- 172.21.15.19:0/4152284571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f269c0bc8a0 con 0x5567f9d39620 2024-04-23T18:45:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f26bdb99740 10 client.17730 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f26bdb99740 1 -- 172.21.15.19:0/4152284571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567f9d532c0 con 0x5567f9d39620 2024-04-23T18:45:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f2693fff640 1 -- 172.21.15.19:0/4152284571 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a4011a90 con 0x5567f9d39620 2024-04-23T18:45:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f2693fff640 1 -- 172.21.15.19:0/4152284571 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f269c0b95b0 con 0x5567f9d39620 2024-04-23T18:45:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f26bdb99740 10 client.17730 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f26bdb99740 10 client.17730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.548+0000 7f26bdb99740 20 client.17730 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.549+0000 7f26bdb99740 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567f9d54720 0x5567f9d56b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.549+0000 7f26bdb99740 4 client.17730 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.549+0000 7f26bdb99740 1 -- 172.21.15.19:0/4152284571 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567f9cac390 con 0x5567f9d54720 2024-04-23T18:45:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.549+0000 7f26b3fff640 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567f9d54720 0x5567f9d56b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.549+0000 7f26b3fff640 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567f9d54720 0x5567f9d56b90 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c82c7d0aa102180 server_cookie=1a65dcc24169b474 in_seq=0 out_seq=0 2024-04-23T18:45:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.549+0000 7f2693fff640 10 client.17730 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f2693fff640 1 -- 172.21.15.19:0/4152284571 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567f9cac390 con 0x5567f9d54720 2024-04-23T18:45:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f2693fff640 10 client.17730 handle_command_reply: tid=0 2024-04-23T18:45:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f26bdb99740 1 client.17730 shutdown 2024-04-23T18:45:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f26bdb99740 1 -- 172.21.15.19:0/4152284571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 msgr2=0x5567f9d44120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f26bdb99740 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d44120 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f269c0bced0 tx=0x7f269c041d10 comp rx=0 tx=0).stop 2024-04-23T18:45:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f26bdb99740 1 -- 172.21.15.19:0/4152284571 shutdown_connections 2024-04-23T18:45:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f26bdb99740 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567f9d54720 0x5567f9d56b90 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f26bdb99740 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d44800 0x5567f9d41950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f26bdb99740 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d39f40 0x5567f9d41410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f2693fff640 0 client.17730 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f26bdb99740 1 --2- 172.21.15.19:0/4152284571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d44120 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.550+0000 7f26bdb99740 1 -- 172.21.15.19:0/4152284571 >> 172.21.15.19:0/4152284571 conn(0x5567f9d35b30 msgr2=0x5567f9d3b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.551+0000 7f26bdb99740 1 -- 172.21.15.19:0/4152284571 shutdown_connections 2024-04-23T18:45:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.551+0000 7f26bdb99740 1 -- 172.21.15.19:0/4152284571 wait complete. 2024-04-23T18:45:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.551+0000 7f26bdb99740 20 client.17730 trim_cache size 0 max 16384 2024-04-23T18:45:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.568+0000 7f26bdb99740 1 Processor -- start 2024-04-23T18:45:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.568+0000 7f26bdb99740 1 -- start start 2024-04-23T18:45:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.568+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d43c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.568+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 0x5567f9d5c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.568+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 0x5567f9d5c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.568+0000 7f26bdb99740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41adac0 con 0x5567f9d39620 2024-04-23T18:45:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.568+0000 7f26bdb99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41af600 con 0x5567f9d39f40 2024-04-23T18:45:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.568+0000 7f26bdb99740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41ae190 con 0x5567f9d44800 2024-04-23T18:45:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b8b3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d43c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b8b3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d43c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42798/0 (socket says 172.21.15.19:42798) 2024-04-23T18:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b8b3e640 1 -- 172.21.15.19:0/1242762380 learned_addr learned my addr 172.21.15.19:0/1242762380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b3fff640 1 --2- 172.21.15.19:0/1242762380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 0x5567f9d5c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b933f640 1 --2- 172.21.15.19:0/1242762380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 0x5567f9d5c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b8b3e640 1 -- 172.21.15.19:0/1242762380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 msgr2=0x5567f9d5c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/1242762380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 0x5567f9d5c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b8b3e640 1 -- 172.21.15.19:0/1242762380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 msgr2=0x5567f9d5c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:37.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/1242762380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 0x5567f9d5c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b8b3e640 1 -- 172.21.15.19:0/1242762380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f9d403c0 con 0x5567f9d39620 2024-04-23T18:45:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b933f640 1 --2- 172.21.15.19:0/1242762380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 0x5567f9d5c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/1242762380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d43c50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f269c006010 tx=0x7f269c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb7e6d9dc4dad8c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26aa7fc640 1 -- 172.21.15.19:0/1242762380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c049070 con 0x5567f9d39620 2024-04-23T18:45:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26aa7fc640 1 -- 172.21.15.19:0/1242762380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f269c0bcf40 con 0x5567f9d39620 2024-04-23T18:45:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.569+0000 7f26aa7fc640 1 -- 172.21.15.19:0/1242762380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c0b95b0 con 0x5567f9d39620 2024-04-23T18:45:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 -- 172.21.15.19:0/1242762380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 msgr2=0x5567f9d43c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 --2- 172.21.15.19:0/1242762380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d43c50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f269c006010 tx=0x7f269c02a480 comp rx=0 tx=0).stop 2024-04-23T18:45:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 -- 172.21.15.19:0/1242762380 shutdown_connections 2024-04-23T18:45:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 --2- 172.21.15.19:0/1242762380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 0x5567f9d5c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 --2- 172.21.15.19:0/1242762380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 0x5567f9d5c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 --2- 172.21.15.19:0/1242762380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d43c50 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 -- 172.21.15.19:0/1242762380 >> 172.21.15.19:0/1242762380 conn(0x5567f9d35b30 msgr2=0x5567f9d34cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 -- 172.21.15.19:0/1242762380 shutdown_connections 2024-04-23T18:45:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 -- 172.21.15.19:0/1242762380 wait complete. 2024-04-23T18:45:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 -- start start 2024-04-23T18:45:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.570+0000 7f26bdb99740 1 Processor -- start 2024-04-23T18:45:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d4a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 0x5567f9d4a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26bdb99740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 0x5567f9d4ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26bdb99740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41aef30 con 0x5567f9d39620 2024-04-23T18:45:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26bdb99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41ae860 con 0x5567f9d39f40 2024-04-23T18:45:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26bdb99740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41abac0 con 0x5567f9d44800 2024-04-23T18:45:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26bdb99740 10 client.0 _ll_register_callbacks cb 0x7f269c0b9220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b8b3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d4a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b8b3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d4a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42800/0 (socket says 172.21.15.19:42800) 2024-04-23T18:45:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b8b3e640 1 -- 172.21.15.19:0/1725956258 learned_addr learned my addr 172.21.15.19:0/1725956258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b933f640 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 0x5567f9d4ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b3fff640 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 0x5567f9d4a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b8b3e640 1 -- 172.21.15.19:0/1725956258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 msgr2=0x5567f9d4ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 0x5567f9d4ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b8b3e640 1 -- 172.21.15.19:0/1725956258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 msgr2=0x5567f9d4a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 0x5567f9d4a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b8b3e640 1 -- 172.21.15.19:0/1725956258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567f9db9cf0 con 0x5567f9d39620 2024-04-23T18:45:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.571+0000 7f26b3fff640 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 0x5567f9d4a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f26b8b3e640 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d4a1b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f269c0bcc40 tx=0x7f269c0eeba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c390ab938feb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f2693fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f2693fff640 1 -- 172.21.15.19:0/1725956258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c086060 con 0x5567f9d39620 2024-04-23T18:45:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f2693fff640 1 -- 172.21.15.19:0/1725956258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7f269c0c5920 con 0x5567f9d39620 2024-04-23T18:45:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f26bdb99740 1 -- 172.21.15.19:0/1725956258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f269c0bcf40 con 0x5567f9d39620 2024-04-23T18:45:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f2693fff640 1 -- 172.21.15.19:0/1725956258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f269c049040 con 0x5567f9d39620 2024-04-23T18:45:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f26bdb99740 10 client.17736 fetch_fsmap learned FSMap version 30 2024-04-23T18:45:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f26bdb99740 1 -- 172.21.15.19:0/1725956258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567f9db9cf0 con 0x5567f9d39620 2024-04-23T18:45:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f2693fff640 1 -- 172.21.15.19:0/1725956258 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f269c0bcf40 con 0x5567f9d39620 2024-04-23T18:45:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f2693fff640 1 -- 172.21.15.19:0/1725956258 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f269c0bc420 con 0x5567f9d39620 2024-04-23T18:45:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f26bdb99740 10 client.17736 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:45:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f26bdb99740 10 client.17736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f26bdb99740 20 client.17736 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f26bdb99740 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567f9dbb210 0x5567f9dbd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:38.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f26bdb99740 4 client.17736 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.572+0000 7f26bdb99740 1 -- 172.21.15.19:0/1725956258 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567f9ca6200 con 0x5567f9dbb210 2024-04-23T18:45:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.573+0000 7f26b3fff640 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567f9dbb210 0x5567f9dbd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.573+0000 7f26b3fff640 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567f9dbb210 0x5567f9dbd600 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=311f29caa56569b8 server_cookie=d4026c1e060a04a1 in_seq=0 out_seq=0 2024-04-23T18:45:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:03.573+0000 7f2693fff640 10 client.17736 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:38.831 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:03 smithi019 ceph-mon[29215]: pgmap v596: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f2693fff640 1 -- 172.21.15.19:0/1725956258 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567f9ca6200 con 0x5567f9dbb210 2024-04-23T18:45:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f2693fff640 10 client.17736 handle_command_reply: tid=0 2024-04-23T18:45:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f26bdb99740 1 client.17736 shutdown 2024-04-23T18:45:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f26bdb99740 1 -- 172.21.15.19:0/1725956258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 msgr2=0x5567f9d4a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f26bdb99740 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d4a1b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f269c0bcc40 tx=0x7f269c0eeba0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f26bdb99740 1 -- 172.21.15.19:0/1725956258 shutdown_connections 2024-04-23T18:45:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f26bdb99740 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5567f9dbb210 0x5567f9dbd600 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f26bdb99740 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f9d39f40 0x5567f9d4a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f26bdb99740 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567f9d44800 0x5567f9d4ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f26bdb99740 1 --2- 172.21.15.19:0/1725956258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567f9d39620 0x5567f9d4a1b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f2693fff640 0 client.17736 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.319+0000 7f26bdb99740 1 -- 172.21.15.19:0/1725956258 >> 172.21.15.19:0/1725956258 conn(0x5567f9d35b30 msgr2=0x5567f9d40c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.320+0000 7f26bdb99740 1 -- 172.21.15.19:0/1725956258 shutdown_connections 2024-04-23T18:45:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.320+0000 7f26bdb99740 1 -- 172.21.15.19:0/1725956258 wait complete. 2024-04-23T18:45:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.320+0000 7f26bdb99740 20 client.17736 trim_cache size 0 max 16384 2024-04-23T18:45:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 -- 172.21.15.19:0/4071896960 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26a00a3dc0 msgr2=0x7f26a00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26a00a3dc0 0x7f26a00a6280 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009270 tx=0x7f26ac009b20 comp rx=0 tx=0).stop 2024-04-23T18:45:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 -- 172.21.15.19:0/4071896960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103200 msgr2=0x7f26b4198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103200 0x7f26b4198700 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f269c005940 tx=0x7f269c004270 comp rx=0 tx=0).stop 2024-04-23T18:45:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 -- 172.21.15.19:0/4071896960 shutdown_connections 2024-04-23T18:45:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26a00a3dc0 0x7f26a00a6280 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b410c5d0 0x7f26b419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b4103b20 0x7f26b4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 --2- 172.21.15.19:0/4071896960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103200 0x7f26b4198700 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 -- 172.21.15.19:0/4071896960 >> 172.21.15.19:0/4071896960 conn(0x7f26b40771d0 msgr2=0x7f26b40fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 -- 172.21.15.19:0/4071896960 shutdown_connections 2024-04-23T18:45:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:05.321+0000 7f26bada2640 1 -- 172.21.15.19:0/4071896960 wait complete. 2024-04-23T18:45:39.317 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:05 smithi019 ceph-mon[29215]: pgmap v597: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:45:39.326 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:35 smithi155 ceph-mon[33468]: pgmap v615: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:39.335 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:37 smithi155 ceph-mon[33468]: pgmap v616: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:39.345 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:35 smithi044 ceph-mon[32458]: pgmap v615: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:39.354 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:37 smithi044 ceph-mon[32458]: pgmap v616: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:39.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.283+0000 7fdc6b364640 1 -- 172.21.15.19:0/2145432147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc64076480 msgr2=0x7fdc64076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:39.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.283+0000 7fdc6b364640 1 --2- 172.21.15.19:0/2145432147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc64076480 0x7fdc64076860 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009ca0 tx=0x7fdc6002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.283+0000 7fdc6b364640 1 -- 172.21.15.19:0/2145432147 shutdown_connections 2024-04-23T18:45:39.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.283+0000 7fdc6b364640 1 --2- 172.21.15.19:0/2145432147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc64076da0 0x7fdc641128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.283+0000 7fdc6b364640 1 --2- 172.21.15.19:0/2145432147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc64112de0 0x7fdc641151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.283+0000 7fdc6b364640 1 --2- 172.21.15.19:0/2145432147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc64076480 0x7fdc64076860 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.283+0000 7fdc6b364640 1 -- 172.21.15.19:0/2145432147 >> 172.21.15.19:0/2145432147 conn(0x7fdc640ffc10 msgr2=0x7fdc64102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.283+0000 7fdc6b364640 1 -- 172.21.15.19:0/2145432147 shutdown_connections 2024-04-23T18:45:39.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.283+0000 7fdc6b364640 1 -- 172.21.15.19:0/2145432147 wait complete. 2024-04-23T18:45:39.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc6b364640 1 Processor -- start 2024-04-23T18:45:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc6b364640 1 -- start start 2024-04-23T18:45:39.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc6b364640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc64076480 0x7fdc6419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc6b364640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc64076da0 0x7fdc6419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc6b364640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc64112de0 0x7fdc641a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc6b364640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641187e0 con 0x7fdc64076da0 2024-04-23T18:45:39.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc6b364640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64118660 con 0x7fdc64076480 2024-04-23T18:45:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc6b364640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64118960 con 0x7fdc64112de0 2024-04-23T18:45:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc688ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc64076da0 0x7fdc6419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:39.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc688ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc64076da0 0x7fdc6419f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42824/0 (socket says 172.21.15.19:42824) 2024-04-23T18:45:39.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc688ff640 1 -- 172.21.15.19:0/746115411 learned_addr learned my addr 172.21.15.19:0/746115411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc69100640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc64076480 0x7fdc6419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:39.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc69901640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc64112de0 0x7fdc641a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc688ff640 1 -- 172.21.15.19:0/746115411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc64112de0 msgr2=0x7fdc641a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:39.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc688ff640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc64112de0 0x7fdc641a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc688ff640 1 -- 172.21.15.19:0/746115411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc64076480 msgr2=0x7fdc6419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:39.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc688ff640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc64076480 0x7fdc6419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.284+0000 7fdc688ff640 1 -- 172.21.15.19:0/746115411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc641a3d60 con 0x7fdc64076da0 2024-04-23T18:45:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.285+0000 7fdc688ff640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc64076da0 0x7fdc6419f300 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400c890 tx=0x7fdc5400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.285+0000 7fdc5a7fc640 1 -- 172.21.15.19:0/746115411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54004220 con 0x7fdc64076da0 2024-04-23T18:45:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.285+0000 7fdc5a7fc640 1 -- 172.21.15.19:0/746115411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc540043c0 con 0x7fdc64076da0 2024-04-23T18:45:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.285+0000 7fdc6b364640 1 -- 172.21.15.19:0/746115411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc641a4050 con 0x7fdc64076da0 2024-04-23T18:45:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.285+0000 7fdc5a7fc640 1 -- 172.21.15.19:0/746115411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54004a00 con 0x7fdc64076da0 2024-04-23T18:45:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.285+0000 7fdc6b364640 1 -- 172.21.15.19:0/746115411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc641ab8e0 con 0x7fdc64076da0 2024-04-23T18:45:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.286+0000 7fdc6b364640 1 -- 172.21.15.19:0/746115411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc64068d70 con 0x7fdc64076da0 2024-04-23T18:45:39.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.288+0000 7fdc5a7fc640 1 -- 172.21.15.19:0/746115411 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc540079b0 con 0x7fdc64076da0 2024-04-23T18:45:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.288+0000 7fdc5a7fc640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdc500a3d20 0x7fdc500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:39.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.289+0000 7fdc69100640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdc500a3d20 0x7fdc500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.289+0000 7fdc5a7fc640 1 -- 172.21.15.19:0/746115411 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8557+0+0 (secure 0 0 0) 0x7fdc540d0a80 con 0x7fdc64076da0 2024-04-23T18:45:39.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.289+0000 7fdc69100640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdc500a3d20 0x7fdc500a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdc6002f500 tx=0x7fdc6002f310 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:40.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.293+0000 7fdc5a7fc640 1 -- 172.21.15.19:0/746115411 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc54014030 con 0x7fdc64076da0 2024-04-23T18:45:40.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.518+0000 7fdc6b364640 1 -- 172.21.15.19:0/746115411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc641a00c0 con 0x7fdc64076da0 2024-04-23T18:45:40.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.519+0000 7fdc5a7fc640 1 -- 172.21.15.19:0/746115411 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11917 (secure 0 0 0) 0x7fdc641a00c0 con 0x7fdc64076da0 2024-04-23T18:45:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:45:40.059 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:43:17.885463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26705":{"gid":26705,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/3131205149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":3131205149},{"type":"v1","addr":"172.21.15.44:6835","nonce":3131205149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24508,24469,24521,24491]},"id":1}]} 2024-04-23T18:45:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-04-23T18:45:40.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 -- 172.21.15.19:0/746115411 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdc500a3d20 msgr2=0x7fdc500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:40.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdc500a3d20 0x7fdc500a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdc6002f500 tx=0x7fdc6002f310 comp rx=0 tx=0).stop 2024-04-23T18:45:40.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 -- 172.21.15.19:0/746115411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc64076da0 msgr2=0x7fdc6419f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:40.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc64076da0 0x7fdc6419f300 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400c890 tx=0x7fdc5400cd50 comp rx=0 tx=0).stop 2024-04-23T18:45:40.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 -- 172.21.15.19:0/746115411 shutdown_connections 2024-04-23T18:45:40.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fdc500a3d20 0x7fdc500a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc64112de0 0x7fdc641a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc64076da0 0x7fdc6419f300 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 --2- 172.21.15.19:0/746115411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc64076480 0x7fdc6419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 -- 172.21.15.19:0/746115411 >> 172.21.15.19:0/746115411 conn(0x7fdc640ffc10 msgr2=0x7fdc64102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:40.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 -- 172.21.15.19:0/746115411 shutdown_connections 2024-04-23T18:45:40.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:06.521+0000 7fdc6b364640 1 -- 172.21.15.19:0/746115411 wait complete. 2024-04-23T18:45:40.303 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --set-id 46eff5eb --release --await' 2024-04-23T18:45:40.313 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --set-id 46eff5eb --release --await 2024-04-23T18:45:40.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:39 smithi155 ceph-mon[33468]: pgmap v617: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:40.334 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:39 smithi044 ceph-mon[32458]: pgmap v617: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:40.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:06 smithi019 ceph-mon[29215]: osdmap e116: 12 total, 12 up, 12 in 2024-04-23T18:45:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:06 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/746115411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:45:40.426 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:07 smithi019 ceph-mon[29215]: pgmap v599: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:45:40.439 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:09 smithi019 ceph-mon[29215]: pgmap v600: 97 pgs: 97 active+clean; 4.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:45:40.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:11 smithi019 ceph-mon[29215]: pgmap v601: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:45:40.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:13 smithi019 ceph-mon[29215]: pgmap v602: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T18:45:40.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:13 smithi019 ceph-mon[29215]: osdmap e117: 12 total, 12 up, 12 in 2024-04-23T18:45:40.488 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:15 smithi019 ceph-mon[29215]: pgmap v604: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 460 B/s wr, 10 op/s 2024-04-23T18:45:40.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:17 smithi019 ceph-mon[29215]: pgmap v605: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-04-23T18:45:40.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:17 smithi019 ceph-mon[29215]: daemon mds.e restarted 2024-04-23T18:45:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.422+0000 7ff0169bb640 1 -- 172.21.15.19:0/799504337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0101010e0 msgr2=0x7ff0101014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.422+0000 7ff0169bb640 1 --2- 172.21.15.19:0/799504337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0101010e0 0x7ff0101014c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff000009900 tx=0x7ff00002a410 comp rx=0 tx=0).stop 2024-04-23T18:45:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.422+0000 7ff0169bb640 1 -- 172.21.15.19:0/799504337 shutdown_connections 2024-04-23T18:45:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.422+0000 7ff0169bb640 1 --2- 172.21.15.19:0/799504337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff010101a90 0x7ff010105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.422+0000 7ff0169bb640 1 --2- 172.21.15.19:0/799504337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0101061b0 0x7ff01010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.422+0000 7ff0169bb640 1 --2- 172.21.15.19:0/799504337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0101010e0 0x7ff0101014c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.422+0000 7ff0169bb640 1 -- 172.21.15.19:0/799504337 >> 172.21.15.19:0/799504337 conn(0x7ff0100771d0 msgr2=0x7ff0100775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.422+0000 7ff0169bb640 1 -- 172.21.15.19:0/799504337 shutdown_connections 2024-04-23T18:45:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.422+0000 7ff0169bb640 1 -- 172.21.15.19:0/799504337 wait complete. 2024-04-23T18:45:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff0169bb640 1 Processor -- start 2024-04-23T18:45:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff0169bb640 1 -- start start 2024-04-23T18:45:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff0169bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0101010e0 0x7ff0101986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff0169bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0101061b0 0x7ff010198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff0169bb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff01019cf20 0x7ff01019d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff0169bb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff010112230 con 0x7ff0101061b0 2024-04-23T18:45:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff0169bb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0101120b0 con 0x7ff0101010e0 2024-04-23T18:45:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff0169bb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0101123b0 con 0x7ff01019cf20 2024-04-23T18:45:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff00f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0101061b0 0x7ff010198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff00f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0101061b0 0x7ff010198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60906/0 (socket says 172.21.15.19:60906) 2024-04-23T18:45:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.423+0000 7ff00f7fe640 1 -- 172.21.15.19:0/1286682449 learned_addr learned my addr 172.21.15.19:0/1286682449 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:40.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff00ffff640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0101010e0 0x7ff0101986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff014f58640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff01019cf20 0x7ff01019d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff00f7fe640 1 -- 172.21.15.19:0/1286682449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff01019cf20 msgr2=0x7ff01019d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff00f7fe640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff01019cf20 0x7ff01019d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff00f7fe640 1 -- 172.21.15.19:0/1286682449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0101010e0 msgr2=0x7ff0101986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff00f7fe640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0101010e0 0x7ff0101986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff00f7fe640 1 -- 172.21.15.19:0/1286682449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01019da10 con 0x7ff0101061b0 2024-04-23T18:45:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff00f7fe640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0101061b0 0x7ff010198bf0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff00400e9c0 tx=0x7ff00400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:40.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff00d7fa640 1 -- 172.21.15.19:0/1286682449 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00400cd80 con 0x7ff0101061b0 2024-04-23T18:45:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff00d7fa640 1 -- 172.21.15.19:0/1286682449 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0040054d0 con 0x7ff0101061b0 2024-04-23T18:45:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff0169bb640 1 -- 172.21.15.19:0/1286682449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01019dd00 con 0x7ff0101061b0 2024-04-23T18:45:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff00d7fa640 1 -- 172.21.15.19:0/1286682449 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004010430 con 0x7ff0101061b0 2024-04-23T18:45:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.424+0000 7ff0169bb640 1 -- 172.21.15.19:0/1286682449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0101a54f0 con 0x7ff0101061b0 2024-04-23T18:45:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff00d7fa640 1 -- 172.21.15.19:0/1286682449 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff004004fd0 con 0x7ff0101061b0 2024-04-23T18:45:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff0197ae740 1 Processor -- start 2024-04-23T18:45:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff0197ae740 1 -- start start 2024-04-23T18:45:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 0x5568e15bea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15bef40 0x5568e15c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff0197ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568e1523970 con 0x5568e15c9800 2024-04-23T18:45:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff0197ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5568e14c04a0 con 0x5568e15bef40 2024-04-23T18:45:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff0197ae740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5568e151f930 con 0x5568e15be620 2024-04-23T18:45:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff00d7fa640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feff80a3dc0 0x7feff80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff00f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff014f58640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 0x5568e15bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff00f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60908/0 (socket says 172.21.15.19:60908) 2024-04-23T18:45:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff00ffff640 1 --2- 172.21.15.19:0/4027439108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15bef40 0x5568e15c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff00f7fe640 1 -- 172.21.15.19:0/4027439108 learned_addr learned my addr 172.21.15.19:0/4027439108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.427+0000 7ff014f58640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feff80a3dc0 0x7feff80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7ff00d7fa640 1 -- 172.21.15.19:0/1286682449 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7ff0040d0bd0 con 0x7ff0101061b0 2024-04-23T18:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7ff014f58640 1 -- 172.21.15.19:0/4027439108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15bef40 msgr2=0x5568e15c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7ff014f58640 1 --2- 172.21.15.19:0/4027439108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15bef40 0x5568e15c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7ff014f58640 1 -- 172.21.15.19:0/4027439108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 msgr2=0x5568e15cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7ff014f58640 1 --2- 172.21.15.19:0/4027439108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7ff014f58640 1 -- 172.21.15.19:0/4027439108 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e1236210 con 0x5568e15be620 2024-04-23T18:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7ff014f58640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feff80a3dc0 0x7feff80a6280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff00800fc90 tx=0x7ff008018000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7ff014f58640 1 --2- 172.21.15.19:0/4027439108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 0x5568e15bea00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff008011cc0 tx=0x7ff00800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a743c6770828fc29 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7feff67fc640 1 -- 172.21.15.19:0/4027439108 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008004b20 con 0x5568e15be620 2024-04-23T18:45:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7feff67fc640 1 -- 172.21.15.19:0/4027439108 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff008009ce0 con 0x5568e15be620 2024-04-23T18:45:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.428+0000 7feff67fc640 1 -- 172.21.15.19:0/4027439108 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008005400 con 0x5568e15be620 2024-04-23T18:45:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.429+0000 7ff0197ae740 1 -- 172.21.15.19:0/4027439108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 msgr2=0x5568e15bea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.429+0000 7ff0197ae740 1 --2- 172.21.15.19:0/4027439108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 0x5568e15bea00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff008011cc0 tx=0x7ff00800f790 comp rx=0 tx=0).stop 2024-04-23T18:45:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.429+0000 7ff0197ae740 1 -- 172.21.15.19:0/4027439108 shutdown_connections 2024-04-23T18:45:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.429+0000 7ff0197ae740 1 --2- 172.21.15.19:0/4027439108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15bef40 0x5568e15c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.429+0000 7ff0197ae740 1 --2- 172.21.15.19:0/4027439108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.429+0000 7ff0197ae740 1 --2- 172.21.15.19:0/4027439108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 0x5568e15bea00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.429+0000 7ff0197ae740 1 -- 172.21.15.19:0/4027439108 >> 172.21.15.19:0/4027439108 conn(0x5568e15bab30 msgr2=0x5568e15baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.429+0000 7ff0197ae740 1 -- 172.21.15.19:0/4027439108 shutdown_connections 2024-04-23T18:45:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.429+0000 7ff0197ae740 1 -- 172.21.15.19:0/4027439108 wait complete. 2024-04-23T18:45:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.429+0000 7ff0197ae740 1 -- start start 2024-04-23T18:45:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff0197ae740 1 Processor -- start 2024-04-23T18:45:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15be620 0x5568e15c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15bef40 0x5568e15c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff0197ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568e15cc3d0 con 0x5568e15c9800 2024-04-23T18:45:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff0197ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5568e14be0b0 con 0x5568e15be620 2024-04-23T18:45:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff0197ae740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5568e15cc550 con 0x5568e15bef40 2024-04-23T18:45:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff0197ae740 10 client.0 _ll_register_callbacks cb 0x5568e14abd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff014f58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff014f58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60924/0 (socket says 172.21.15.19:60924) 2024-04-23T18:45:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff014f58640 1 -- 172.21.15.19:0/1190567632 learned_addr learned my addr 172.21.15.19:0/1190567632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff00f7fe640 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15bef40 0x5568e15c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff00ffff640 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15be620 0x5568e15c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff014f58640 1 -- 172.21.15.19:0/1190567632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15bef40 msgr2=0x5568e15c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff014f58640 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15bef40 0x5568e15c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff014f58640 1 -- 172.21.15.19:0/1190567632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15be620 msgr2=0x5568e15c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.430+0000 7ff014f58640 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15be620 0x5568e15c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.431+0000 7ff014f58640 1 -- 172.21.15.19:0/1190567632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e1302620 con 0x5568e15c9800 2024-04-23T18:45:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.431+0000 7ff014f58640 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15c6ad0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff008009540 tx=0x7ff008004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a379968ff933ea80 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.431+0000 7fefef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.431+0000 7fefef7fe640 1 -- 172.21.15.19:0/1190567632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008011850 con 0x5568e15c9800 2024-04-23T18:45:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.431+0000 7fefef7fe640 1 -- 172.21.15.19:0/1190567632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7ff008010a10 con 0x5568e15c9800 2024-04-23T18:45:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.431+0000 7ff0197ae740 1 -- 172.21.15.19:0/1190567632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff008009ce0 con 0x5568e15c9800 2024-04-23T18:45:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.431+0000 7fefef7fe640 1 -- 172.21.15.19:0/1190567632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff008010c10 con 0x5568e15c9800 2024-04-23T18:45:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.431+0000 7ff0197ae740 10 client.17757 fetch_fsmap learned FSMap version 31 2024-04-23T18:45:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.431+0000 7ff0197ae740 1 -- 172.21.15.19:0/1190567632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5568e15d8360 con 0x5568e15c9800 2024-04-23T18:45:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.431+0000 7fefef7fe640 1 -- 172.21.15.19:0/1190567632 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff008009ce0 con 0x5568e15c9800 2024-04-23T18:45:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.432+0000 7fefef7fe640 1 -- 172.21.15.19:0/1190567632 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff00802e070 con 0x5568e15c9800 2024-04-23T18:45:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.432+0000 7ff0197ae740 10 client.17757 fetch_fsmap finished waiting for FSMap version 31 2024-04-23T18:45:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.432+0000 7ff0197ae740 10 client.17757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.432+0000 7ff0197ae740 20 client.17757 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.432+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568e15d97d0 0x5568e15dbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.432+0000 7ff0197ae740 4 client.17757 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.432+0000 7ff0197ae740 1 -- 172.21.15.19:0/1190567632 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568e1531390 con 0x5568e15d97d0 2024-04-23T18:45:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.432+0000 7ff00ffff640 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568e15d97d0 0x5568e15dbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.432+0000 7ff00ffff640 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568e15d97d0 0x5568e15dbbc0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c3736e7c910b0c9 server_cookie=63ec3884c335d28a in_seq=0 out_seq=0 2024-04-23T18:45:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.432+0000 7fefef7fe640 10 client.17757 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7fefef7fe640 1 -- 172.21.15.19:0/1190567632 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568e1531390 con 0x5568e15d97d0 2024-04-23T18:45:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7fefef7fe640 10 client.17757 handle_command_reply: tid=0 2024-04-23T18:45:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7ff0197ae740 1 client.17757 shutdown 2024-04-23T18:45:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7ff0197ae740 1 -- 172.21.15.19:0/1190567632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 msgr2=0x5568e15c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15c6ad0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff008009540 tx=0x7ff008004ec0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7ff0197ae740 1 -- 172.21.15.19:0/1190567632 shutdown_connections 2024-04-23T18:45:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568e15d97d0 0x5568e15dbbc0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15c9800 0x5568e15c6ad0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15bef40 0x5568e15c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1190567632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15be620 0x5568e15c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7ff0197ae740 1 -- 172.21.15.19:0/1190567632 >> 172.21.15.19:0/1190567632 conn(0x5568e15bab30 msgr2=0x5568e15c08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7fefef7fe640 0 client.17757 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.433+0000 7ff0197ae740 1 -- 172.21.15.19:0/1190567632 shutdown_connections 2024-04-23T18:45:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.434+0000 7ff0197ae740 1 -- 172.21.15.19:0/1190567632 wait complete. 2024-04-23T18:45:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.434+0000 7ff0197ae740 20 client.17757 trim_cache size 0 max 16384 2024-04-23T18:45:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff0197ae740 1 Processor -- start 2024-04-23T18:45:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff0197ae740 1 -- start start 2024-04-23T18:45:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15be620 0x5568e15c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15bef40 0x5568e15d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 0x5568e15c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff0197ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0101ada60 con 0x5568e15be620 2024-04-23T18:45:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff0197ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0101af5a0 con 0x5568e15c9800 2024-04-23T18:45:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff0197ae740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0101ae130 con 0x5568e15bef40 2024-04-23T18:45:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff00ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15be620 0x5568e15c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff00ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15be620 0x5568e15c1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60938/0 (socket says 172.21.15.19:60938) 2024-04-23T18:45:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff00ffff640 1 -- 172.21.15.19:0/3042409736 learned_addr learned my addr 172.21.15.19:0/3042409736 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff014f58640 1 --2- 172.21.15.19:0/3042409736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 0x5568e15c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff00f7fe640 1 --2- 172.21.15.19:0/3042409736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15bef40 0x5568e15d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff00ffff640 1 -- 172.21.15.19:0/3042409736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15bef40 msgr2=0x5568e15d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff00ffff640 1 --2- 172.21.15.19:0/3042409736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15bef40 0x5568e15d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff00ffff640 1 -- 172.21.15.19:0/3042409736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 msgr2=0x5568e15c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff00ffff640 1 --2- 172.21.15.19:0/3042409736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 0x5568e15c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff00ffff640 1 -- 172.21.15.19:0/3042409736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e15c70d0 con 0x5568e15be620 2024-04-23T18:45:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.449+0000 7ff00ffff640 1 --2- 172.21.15.19:0/3042409736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15be620 0x5568e15c1d30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff00002a930 tx=0x7ff0000092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1c9bd5e4064b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7feff5ffb640 1 -- 172.21.15.19:0/3042409736 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00004b070 con 0x5568e15be620 2024-04-23T18:45:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7feff5ffb640 1 -- 172.21.15.19:0/3042409736 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff000046dc0 con 0x5568e15be620 2024-04-23T18:45:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7feff5ffb640 1 -- 172.21.15.19:0/3042409736 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000035040 con 0x5568e15be620 2024-04-23T18:45:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7ff0197ae740 1 -- 172.21.15.19:0/3042409736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15be620 msgr2=0x5568e15c1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7ff0197ae740 1 --2- 172.21.15.19:0/3042409736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15be620 0x5568e15c1d30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff00002a930 tx=0x7ff0000092d0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7ff0197ae740 1 -- 172.21.15.19:0/3042409736 shutdown_connections 2024-04-23T18:45:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7ff0197ae740 1 --2- 172.21.15.19:0/3042409736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 0x5568e15c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7ff0197ae740 1 --2- 172.21.15.19:0/3042409736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15bef40 0x5568e15d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7ff0197ae740 1 --2- 172.21.15.19:0/3042409736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15be620 0x5568e15c1d30 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7ff0197ae740 1 -- 172.21.15.19:0/3042409736 >> 172.21.15.19:0/3042409736 conn(0x5568e15bab30 msgr2=0x5568e15b9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7ff0197ae740 1 -- 172.21.15.19:0/3042409736 shutdown_connections 2024-04-23T18:45:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.450+0000 7ff0197ae740 1 -- 172.21.15.19:0/3042409736 wait complete. 2024-04-23T18:45:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 1 -- start start 2024-04-23T18:45:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 1 Processor -- start 2024-04-23T18:45:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 0x5568e15d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15bef40 0x5568e15d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 0x5568e15cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0101aeed0 con 0x5568e15bef40 2024-04-23T18:45:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0101ae800 con 0x5568e15c9800 2024-04-23T18:45:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0101aba40 con 0x5568e15be620 2024-04-23T18:45:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 10 client.0 _ll_register_callbacks cb 0x7ff008011ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15bef40 0x5568e15d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15bef40 0x5568e15d1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60942/0 (socket says 172.21.15.19:60942) 2024-04-23T18:45:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00f7fe640 1 -- 172.21.15.19:0/1615599442 learned_addr learned my addr 172.21.15.19:0/1615599442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00ffff640 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 0x5568e15d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff014f58640 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 0x5568e15cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00f7fe640 1 -- 172.21.15.19:0/1615599442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 msgr2=0x5568e15d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00f7fe640 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 0x5568e15d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00f7fe640 1 -- 172.21.15.19:0/1615599442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 msgr2=0x5568e15cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00f7fe640 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 0x5568e15cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00f7fe640 1 -- 172.21.15.19:0/1615599442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e15cd000 con 0x5568e15bef40 2024-04-23T18:45:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff014f58640 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 0x5568e15cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00ffff640 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 0x5568e15d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:45:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff00f7fe640 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15bef40 0x5568e15d1940 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff0040ce950 tx=0x7ff0040049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9097090d15787e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7fefef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7fefef7fe640 1 -- 172.21.15.19:0/1615599442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0040046c0 con 0x5568e15bef40 2024-04-23T18:45:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7fefef7fe640 1 -- 172.21.15.19:0/1615599442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7ff004099210 con 0x5568e15bef40 2024-04-23T18:45:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 1 -- 172.21.15.19:0/1615599442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff000046dc0 con 0x5568e15bef40 2024-04-23T18:45:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7fefef7fe640 1 -- 172.21.15.19:0/1615599442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff00409d2a0 con 0x5568e15bef40 2024-04-23T18:45:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 10 client.17763 fetch_fsmap learned FSMap version 31 2024-04-23T18:45:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7ff0197ae740 1 -- 172.21.15.19:0/1615599442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5568e163f660 con 0x5568e15bef40 2024-04-23T18:45:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.452+0000 7fefef7fe640 1 -- 172.21.15.19:0/1615599442 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff000046dc0 con 0x5568e15bef40 2024-04-23T18:45:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.453+0000 7fefef7fe640 1 -- 172.21.15.19:0/1615599442 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff00409b6d0 con 0x5568e15bef40 2024-04-23T18:45:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.453+0000 7ff0197ae740 10 client.17763 fetch_fsmap finished waiting for FSMap version 31 2024-04-23T18:45:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.453+0000 7ff0197ae740 10 client.17763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.453+0000 7ff0197ae740 20 client.17763 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.453+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568e163fdb0 0x5568e16421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.453+0000 7ff0197ae740 4 client.17763 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.453+0000 7ff0197ae740 1 -- 172.21.15.19:0/1615599442 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568e152b200 con 0x5568e163fdb0 2024-04-23T18:45:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.453+0000 7ff00ffff640 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568e163fdb0 0x5568e16421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.453+0000 7ff00ffff640 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568e163fdb0 0x5568e16421d0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7dcdfe52c6ef939 server_cookie=7d382b78f07d44aa in_seq=0 out_seq=0 2024-04-23T18:45:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.453+0000 7fefef7fe640 10 client.17763 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7fefef7fe640 1 -- 172.21.15.19:0/1615599442 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568e152b200 con 0x5568e163fdb0 2024-04-23T18:45:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7fefef7fe640 10 client.17763 handle_command_reply: tid=0 2024-04-23T18:45:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 client.17763 shutdown 2024-04-23T18:45:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 -- 172.21.15.19:0/1615599442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15bef40 msgr2=0x5568e15d1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15bef40 0x5568e15d1940 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff0040ce950 tx=0x7ff0040049b0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 -- 172.21.15.19:0/1615599442 shutdown_connections 2024-04-23T18:45:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x5568e163fdb0 0x5568e16421d0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568e15c9800 0x5568e15cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e15bef40 0x5568e15d1940 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7fefef7fe640 0 client.17763 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 --2- 172.21.15.19:0/1615599442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e15be620 0x5568e15d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 -- 172.21.15.19:0/1615599442 >> 172.21.15.19:0/1615599442 conn(0x5568e15bab30 msgr2=0x5568e15c5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 -- 172.21.15.19:0/1615599442 shutdown_connections 2024-04-23T18:45:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 1 -- 172.21.15.19:0/1615599442 wait complete. 2024-04-23T18:45:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.462+0000 7ff0197ae740 20 client.17763 trim_cache size 0 max 16384 2024-04-23T18:45:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 -- 172.21.15.19:0/1286682449 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feff80a3dc0 msgr2=0x7feff80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feff80a3dc0 0x7feff80a6280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff00800fc90 tx=0x7ff008018000 comp rx=0 tx=0).stop 2024-04-23T18:45:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 -- 172.21.15.19:0/1286682449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0101061b0 msgr2=0x7ff010198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0101061b0 0x7ff010198bf0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff00400e9c0 tx=0x7ff00400ee80 comp rx=0 tx=0).stop 2024-04-23T18:45:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 -- 172.21.15.19:0/1286682449 shutdown_connections 2024-04-23T18:45:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feff80a3dc0 0x7feff80a6280 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff01019cf20 0x7ff01019d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0101061b0 0x7ff010198bf0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 --2- 172.21.15.19:0/1286682449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0101010e0 0x7ff0101986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 -- 172.21.15.19:0/1286682449 >> 172.21.15.19:0/1286682449 conn(0x7ff0100771d0 msgr2=0x7ff0100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 -- 172.21.15.19:0/1286682449 shutdown_connections 2024-04-23T18:45:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:18.463+0000 7ff0169bb640 1 -- 172.21.15.19:0/1286682449 wait complete. 2024-04-23T18:45:43.634 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:18 smithi019 ceph-mon[29215]: osdmap e118: 12 total, 12 up, 12 in 2024-04-23T18:45:43.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:18 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6834/332625709,v1:172.21.15.44:6835/332625709] up:boot 2024-04-23T18:45:43.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:18 smithi019 ceph-mon[29215]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:45:43.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:18 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:45:43.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:19 smithi019 ceph-mon[29215]: pgmap v607: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-23T18:45:43.747 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:21 smithi019 ceph-mon[29215]: pgmap v608: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 639 B/s wr, 12 op/s 2024-04-23T18:45:43.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:23 smithi019 ceph-mon[29215]: pgmap v609: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 536 B/s wr, 10 op/s 2024-04-23T18:45:43.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:25 smithi019 ceph-mon[29215]: pgmap v610: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:45:43.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:27 smithi019 ceph-mon[29215]: pgmap v611: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:45:43.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:29 smithi019 ceph-mon[29215]: pgmap v612: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 388 B/s wr, 10 op/s 2024-04-23T18:45:43.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:41 smithi155 ceph-mon[33468]: pgmap v618: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:43.797 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:41 smithi044 ceph-mon[32458]: pgmap v618: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.553+0000 7f844800b640 1 -- 172.21.15.19:0/2301434464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8440103b20 msgr2=0x7f844010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.553+0000 7f844800b640 1 --2- 172.21.15.19:0/2301434464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8440103b20 0x7f844010c000 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8430009900 tx=0x7f843002a480 comp rx=0 tx=0).stop 2024-04-23T18:45:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.553+0000 7f844800b640 1 -- 172.21.15.19:0/2301434464 shutdown_connections 2024-04-23T18:45:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.553+0000 7f844800b640 1 --2- 172.21.15.19:0/2301434464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f844010c540 0x7f844010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.553+0000 7f844800b640 1 --2- 172.21.15.19:0/2301434464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8440103b20 0x7f844010c000 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.553+0000 7f844800b640 1 --2- 172.21.15.19:0/2301434464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8440103200 0x7f84401035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.553+0000 7f844800b640 1 -- 172.21.15.19:0/2301434464 >> 172.21.15.19:0/2301434464 conn(0x7f8440077fd0 msgr2=0x7f84400783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.553+0000 7f844800b640 1 -- 172.21.15.19:0/2301434464 shutdown_connections 2024-04-23T18:45:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.553+0000 7f844800b640 1 -- 172.21.15.19:0/2301434464 wait complete. 2024-04-23T18:45:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f844800b640 1 Processor -- start 2024-04-23T18:45:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f844800b640 1 -- start start 2024-04-23T18:45:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f844800b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8440103200 0x7f8440198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f844800b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8440103b20 0x7f8440198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f844800b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f844010c540 0x7f844019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f844800b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84401121f0 con 0x7f844010c540 2024-04-23T18:45:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f844800b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8440112070 con 0x7f8440103b20 2024-04-23T18:45:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f844800b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8440112370 con 0x7f8440103200 2024-04-23T18:45:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f84465a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f844010c540 0x7f844019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f84465a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f844010c540 0x7f844019d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35040/0 (socket says 172.21.15.19:35040) 2024-04-23T18:45:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f84465a8640 1 -- 172.21.15.19:0/794171784 learned_addr learned my addr 172.21.15.19:0/794171784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f8445da7640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8440103200 0x7f8440198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f84455a6640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8440103b20 0x7f8440198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f84465a8640 1 -- 172.21.15.19:0/794171784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8440103200 msgr2=0x7f8440198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f84465a8640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8440103200 0x7f8440198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f84465a8640 1 -- 172.21.15.19:0/794171784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8440103b20 msgr2=0x7f8440198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f84465a8640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8440103b20 0x7f8440198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.554+0000 7f84465a8640 1 -- 172.21.15.19:0/794171784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844019d730 con 0x7f844010c540 2024-04-23T18:45:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.555+0000 7f84465a8640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f844010c540 0x7f844019d000 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f843800eb10 tx=0x7f843800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.555+0000 7f8436ffd640 1 -- 172.21.15.19:0/794171784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438004b20 con 0x7f844010c540 2024-04-23T18:45:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.555+0000 7f844800b640 1 -- 172.21.15.19:0/794171784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844019da20 con 0x7f844010c540 2024-04-23T18:45:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.555+0000 7f8436ffd640 1 -- 172.21.15.19:0/794171784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8438004cc0 con 0x7f844010c540 2024-04-23T18:45:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.555+0000 7f8436ffd640 1 -- 172.21.15.19:0/794171784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438010690 con 0x7f844010c540 2024-04-23T18:45:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.555+0000 7f844800b640 1 -- 172.21.15.19:0/794171784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84401a52b0 con 0x7f844010c540 2024-04-23T18:45:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.557+0000 7f8436ffd640 1 -- 172.21.15.19:0/794171784 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8438005020 con 0x7f844010c540 2024-04-23T18:45:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f844adf4740 1 Processor -- start 2024-04-23T18:45:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f844adf4740 1 -- start start 2024-04-23T18:45:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f844adf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc58a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f844adf4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f844adf4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc65bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f844adf4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560abcbbd970 con 0x560abcc63800 2024-04-23T18:45:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f844adf4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560abcb5a4a0 con 0x560abcc58f40 2024-04-23T18:45:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f844adf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560abcbb9930 con 0x560abcc58620 2024-04-23T18:45:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f8436ffd640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f842c0a3dc0 0x7f842c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f84455a6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc58a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f8445da7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f84455a6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc58a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.19:44854/0 (socket says 172.21.15.19:44854) 2024-04-23T18:45:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f8445da7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.19:33990/0 (socket says 172.21.15.19:33990) 2024-04-23T18:45:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f8445da7640 1 -- 172.21.15.19:0/1736550901 learned_addr learned my addr 172.21.15.19:0/1736550901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f84465a8640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f842c0a3dc0 0x7f842c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.558+0000 7f8436ffd640 1 -- 172.21.15.19:0/794171784 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7f84380d0d10 con 0x7f844010c540 2024-04-23T18:45:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f84455a6640 1 --2- 172.21.15.19:0/1736550901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc65bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f8445da7640 1 -- 172.21.15.19:0/1736550901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 msgr2=0x560abcc58a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f8445da7640 1 --2- 172.21.15.19:0/1736550901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc58a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f8445da7640 1 -- 172.21.15.19:0/1736550901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 msgr2=0x560abcc65bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f8445da7640 1 --2- 172.21.15.19:0/1736550901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc65bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f8445da7640 1 -- 172.21.15.19:0/1736550901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abc8d0210 con 0x560abcc58f40 2024-04-23T18:45:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f84455a6640 1 --2- 172.21.15.19:0/1736550901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc65bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f8445da7640 1 --2- 172.21.15.19:0/1736550901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc632c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f843c009cf0 tx=0x7f843c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2990254fc6c32569 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f84465a8640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f842c0a3dc0 0x7f842c0a6280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f843809c980 tx=0x7f843806e6c0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f842bfff640 1 -- 172.21.15.19:0/1736550901 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c004230 con 0x560abcc58f40 2024-04-23T18:45:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f842bfff640 1 -- 172.21.15.19:0/1736550901 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843c0043d0 con 0x560abcc58f40 2024-04-23T18:45:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.559+0000 7f842bfff640 1 -- 172.21.15.19:0/1736550901 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c019880 con 0x560abcc58f40 2024-04-23T18:45:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 -- 172.21.15.19:0/1736550901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 msgr2=0x560abcc632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 --2- 172.21.15.19:0/1736550901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc632c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f843c009cf0 tx=0x7f843c011810 comp rx=0 tx=0).stop 2024-04-23T18:45:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 -- 172.21.15.19:0/1736550901 shutdown_connections 2024-04-23T18:45:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 --2- 172.21.15.19:0/1736550901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc632c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 --2- 172.21.15.19:0/1736550901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc65bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 --2- 172.21.15.19:0/1736550901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc58a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 -- 172.21.15.19:0/1736550901 >> 172.21.15.19:0/1736550901 conn(0x560abcc54b30 msgr2=0x560abcc54f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 -- 172.21.15.19:0/1736550901 shutdown_connections 2024-04-23T18:45:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 -- 172.21.15.19:0/1736550901 wait complete. 2024-04-23T18:45:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 -- start start 2024-04-23T18:45:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.560+0000 7f844adf4740 1 Processor -- start 2024-04-23T18:45:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f844adf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f844adf4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f844adf4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f844adf4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560abcc66430 con 0x560abcc63800 2024-04-23T18:45:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f844adf4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560abcb580b0 con 0x560abcc58f40 2024-04-23T18:45:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f844adf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560abcc665b0 con 0x560abcc58620 2024-04-23T18:45:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f844adf4740 10 client.0 _ll_register_callbacks cb 0x560abcb45d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84455a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84455a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35062/0 (socket says 172.21.15.19:35062) 2024-04-23T18:45:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84455a6640 1 -- 172.21.15.19:0/1651292506 learned_addr learned my addr 172.21.15.19:0/1651292506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84455a6640 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f8445da7640 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84455a6640 1 -- 172.21.15.19:0/1651292506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 msgr2=0x560abcc630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84455a6640 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc630b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84455a6640 1 -- 172.21.15.19:0/1651292506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 msgr2=0x560abcc603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84455a6640 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84455a6640 1 -- 172.21.15.19:0/1651292506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abc99c620 con 0x560abcc63800 2024-04-23T18:45:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f8445da7640 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:45:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84455a6640 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc608e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8430000c00 tx=0x7f843002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17c1ead519669e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.561+0000 7f84297fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f84297fa640 1 -- 172.21.15.19:0/1651292506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430035070 con 0x560abcc63800 2024-04-23T18:45:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f84297fa640 1 -- 172.21.15.19:0/1651292506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7f8430005050 con 0x560abcc63800 2024-04-23T18:45:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f844adf4740 1 -- 172.21.15.19:0/1651292506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843c0043d0 con 0x560abcc63800 2024-04-23T18:45:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f84297fa640 1 -- 172.21.15.19:0/1651292506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8430052050 con 0x560abcc63800 2024-04-23T18:45:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f844adf4740 10 client.17775 fetch_fsmap learned FSMap version 31 2024-04-23T18:45:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f844adf4740 1 -- 172.21.15.19:0/1651292506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560abcc722a0 con 0x560abcc63800 2024-04-23T18:45:45.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f84297fa640 1 -- 172.21.15.19:0/1651292506 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843c0043d0 con 0x560abcc63800 2024-04-23T18:45:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f84297fa640 1 -- 172.21.15.19:0/1651292506 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8430004c20 con 0x560abcc63800 2024-04-23T18:45:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f844adf4740 10 client.17775 fetch_fsmap finished waiting for FSMap version 31 2024-04-23T18:45:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f844adf4740 10 client.17775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f844adf4740 20 client.17775 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f844adf4740 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560abcc73760 0x560abcc75b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f844adf4740 4 client.17775 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.562+0000 7f844adf4740 1 -- 172.21.15.19:0/1651292506 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560abcbcb390 con 0x560abcc73760 2024-04-23T18:45:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.563+0000 7f8445da7640 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560abcc73760 0x560abcc75b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.563+0000 7f84297fa640 10 client.17775 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.563+0000 7f8445da7640 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560abcc73760 0x560abcc75b50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6075d88e308dfa server_cookie=a0f55752129ce5ff in_seq=0 out_seq=0 2024-04-23T18:45:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f84297fa640 1 -- 172.21.15.19:0/1651292506 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560abcbcb390 con 0x560abcc73760 2024-04-23T18:45:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f84297fa640 10 client.17775 handle_command_reply: tid=0 2024-04-23T18:45:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 client.17775 shutdown 2024-04-23T18:45:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 -- 172.21.15.19:0/1651292506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 msgr2=0x560abcc608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc608e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8430000c00 tx=0x7f843002a480 comp rx=0 tx=0).stop 2024-04-23T18:45:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 -- 172.21.15.19:0/1651292506 shutdown_connections 2024-04-23T18:45:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560abcc73760 0x560abcc75b50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58f40 0x560abcc603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc608e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 --2- 172.21.15.19:0/1651292506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f84297fa640 0 client.17775 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 -- 172.21.15.19:0/1651292506 >> 172.21.15.19:0/1651292506 conn(0x560abcc54b30 msgr2=0x560abcc5a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 -- 172.21.15.19:0/1651292506 shutdown_connections 2024-04-23T18:45:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 1 -- 172.21.15.19:0/1651292506 wait complete. 2024-04-23T18:45:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.564+0000 7f844adf4740 20 client.17775 trim_cache size 0 max 16384 2024-04-23T18:45:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f844adf4740 1 Processor -- start 2024-04-23T18:45:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f844adf4740 1 -- start start 2024-04-23T18:45:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f844adf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc6f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f844adf4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc58f40 0x560abcc62be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f844adf4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc63800 0x560abcc7b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f844adf4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84401adac0 con 0x560abcc58f40 2024-04-23T18:45:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f844adf4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f84401af600 con 0x560abcc63800 2024-04-23T18:45:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f844adf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84401ae190 con 0x560abcc58620 2024-04-23T18:45:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f8445da7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc58f40 0x560abcc62be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f8445da7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc58f40 0x560abcc62be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35064/0 (socket says 172.21.15.19:35064) 2024-04-23T18:45:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f8445da7640 1 -- 172.21.15.19:0/4257893477 learned_addr learned my addr 172.21.15.19:0/4257893477 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f84455a6640 1 --2- 172.21.15.19:0/4257893477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc63800 0x560abcc7b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f84455a6640 1 --2- 172.21.15.19:0/4257893477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc6f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f8445da7640 1 -- 172.21.15.19:0/4257893477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 msgr2=0x560abcc6f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f8445da7640 1 --2- 172.21.15.19:0/4257893477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc6f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f8445da7640 1 -- 172.21.15.19:0/4257893477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc63800 msgr2=0x560abcc7b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f8445da7640 1 --2- 172.21.15.19:0/4257893477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc63800 0x560abcc7b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f8445da7640 1 -- 172.21.15.19:0/4257893477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abcc58d70 con 0x560abcc58f40 2024-04-23T18:45:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.579+0000 7f84455a6640 1 --2- 172.21.15.19:0/4257893477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc6f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.580+0000 7f8445da7640 1 --2- 172.21.15.19:0/4257893477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc58f40 0x560abcc62be0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f843c009cf0 tx=0x7f843c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2fec2bc9b8a09d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.580+0000 7f842b7fe640 1 -- 172.21.15.19:0/4257893477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c02f020 con 0x560abcc58f40 2024-04-23T18:45:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.580+0000 7f842b7fe640 1 -- 172.21.15.19:0/4257893477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843c02bd50 con 0x560abcc58f40 2024-04-23T18:45:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.580+0000 7f842b7fe640 1 -- 172.21.15.19:0/4257893477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c005420 con 0x560abcc58f40 2024-04-23T18:45:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 -- 172.21.15.19:0/4257893477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc58f40 msgr2=0x560abcc62be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 --2- 172.21.15.19:0/4257893477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc58f40 0x560abcc62be0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f843c009cf0 tx=0x7f843c011810 comp rx=0 tx=0).stop 2024-04-23T18:45:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 -- 172.21.15.19:0/4257893477 shutdown_connections 2024-04-23T18:45:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 --2- 172.21.15.19:0/4257893477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc63800 0x560abcc7b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 --2- 172.21.15.19:0/4257893477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc58f40 0x560abcc62be0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 --2- 172.21.15.19:0/4257893477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58620 0x560abcc6f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 -- 172.21.15.19:0/4257893477 >> 172.21.15.19:0/4257893477 conn(0x560abcc54b30 msgr2=0x560abcc5ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 -- 172.21.15.19:0/4257893477 shutdown_connections 2024-04-23T18:45:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 -- 172.21.15.19:0/4257893477 wait complete. 2024-04-23T18:45:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 -- start start 2024-04-23T18:45:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 Processor -- start 2024-04-23T18:45:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58620 0x560abcc6bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58f40 0x560abcc66f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc67480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84401aef30 con 0x560abcc63800 2024-04-23T18:45:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f84401ae860 con 0x560abcc58620 2024-04-23T18:45:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84401abaa0 con 0x560abcc58f40 2024-04-23T18:45:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.581+0000 7f844adf4740 10 client.0 _ll_register_callbacks cb 0x7f8430003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84455a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc67480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84455a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc67480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35074/0 (socket says 172.21.15.19:35074) 2024-04-23T18:45:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84455a6640 1 -- 172.21.15.19:0/1107828241 learned_addr learned my addr 172.21.15.19:0/1107828241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f8445da7640 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58f40 0x560abcc66f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84455a6640 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58620 0x560abcc6bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84455a6640 1 -- 172.21.15.19:0/1107828241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58f40 msgr2=0x560abcc66f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84455a6640 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58f40 0x560abcc66f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84455a6640 1 -- 172.21.15.19:0/1107828241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58620 msgr2=0x560abcc6bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84455a6640 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58620 0x560abcc6bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84455a6640 1 -- 172.21.15.19:0/1107828241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abcc67c60 con 0x560abcc63800 2024-04-23T18:45:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84455a6640 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc67480 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8430005e00 tx=0x7f8430031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd9c3c5462106d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84297fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84297fa640 1 -- 172.21.15.19:0/1107828241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430035030 con 0x560abcc63800 2024-04-23T18:45:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84297fa640 1 -- 172.21.15.19:0/1107828241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7f8430031c00 con 0x560abcc63800 2024-04-23T18:45:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f844adf4740 1 -- 172.21.15.19:0/1107828241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843c02bd50 con 0x560abcc63800 2024-04-23T18:45:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f84297fa640 1 -- 172.21.15.19:0/1107828241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8430031e00 con 0x560abcc63800 2024-04-23T18:45:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f844adf4740 10 client.17781 fetch_fsmap learned FSMap version 31 2024-04-23T18:45:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.582+0000 7f844adf4740 1 -- 172.21.15.19:0/1107828241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560abccd9cb0 con 0x560abcc63800 2024-04-23T18:45:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f84297fa640 1 -- 172.21.15.19:0/1107828241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843c02bd50 con 0x560abcc63800 2024-04-23T18:45:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f84297fa640 1 -- 172.21.15.19:0/1107828241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8430005260 con 0x560abcc63800 2024-04-23T18:45:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f844adf4740 10 client.17781 fetch_fsmap finished waiting for FSMap version 31 2024-04-23T18:45:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f844adf4740 10 client.17781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f844adf4740 20 client.17781 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f844adf4740 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560abccda370 0x560abccdc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f844adf4740 4 client.17781 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f844adf4740 1 -- 172.21.15.19:0/1107828241 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560abcbc5200 con 0x560abccda370 2024-04-23T18:45:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f8445da7640 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560abccda370 0x560abccdc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f84297fa640 10 client.17781 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.583+0000 7f8445da7640 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560abccda370 0x560abccdc790 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41d65c72b9469aba server_cookie=6ebdc981a0fb3dc5 in_seq=0 out_seq=0 2024-04-23T18:45:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f84297fa640 1 -- 172.21.15.19:0/1107828241 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560abcbc5200 con 0x560abccda370 2024-04-23T18:45:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f84297fa640 10 client.17781 handle_command_reply: tid=0 2024-04-23T18:45:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 client.17781 shutdown 2024-04-23T18:45:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 -- 172.21.15.19:0/1107828241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 msgr2=0x560abcc67480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc67480 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8430005e00 tx=0x7f8430031400 comp rx=0 tx=0).stop 2024-04-23T18:45:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 -- 172.21.15.19:0/1107828241 shutdown_connections 2024-04-23T18:45:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x560abccda370 0x560abccdc790 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560abcc63800 0x560abcc67480 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560abcc58f40 0x560abcc66f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f84297fa640 0 client.17781 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 --2- 172.21.15.19:0/1107828241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560abcc58620 0x560abcc6bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 -- 172.21.15.19:0/1107828241 >> 172.21.15.19:0/1107828241 conn(0x560abcc54b30 msgr2=0x560abcc59fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 -- 172.21.15.19:0/1107828241 shutdown_connections 2024-04-23T18:45:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 1 -- 172.21.15.19:0/1107828241 wait complete. 2024-04-23T18:45:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.587+0000 7f844adf4740 20 client.17781 trim_cache size 0 max 16384 2024-04-23T18:45:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 -- 172.21.15.19:0/794171784 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f842c0a3dc0 msgr2=0x7f842c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f842c0a3dc0 0x7f842c0a6280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f843809c980 tx=0x7f843806e6c0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 -- 172.21.15.19:0/794171784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f844010c540 msgr2=0x7f844019d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f844010c540 0x7f844019d000 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f843800eb10 tx=0x7f843800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 -- 172.21.15.19:0/794171784 shutdown_connections 2024-04-23T18:45:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f842c0a3dc0 0x7f842c0a6280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8440103b20 0x7f8440198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f844010c540 0x7f844019d000 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 --2- 172.21.15.19:0/794171784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8440103200 0x7f8440198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 -- 172.21.15.19:0/794171784 >> 172.21.15.19:0/794171784 conn(0x7f8440077fd0 msgr2=0x7f84400ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 -- 172.21.15.19:0/794171784 shutdown_connections 2024-04-23T18:45:46.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:31.588+0000 7f844800b640 1 -- 172.21.15.19:0/794171784 wait complete. 2024-04-23T18:45:46.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:31 smithi019 ceph-mon[29215]: pgmap v613: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T18:45:46.603 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:31 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:46.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:31 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:46.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:31 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:46.628 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:31 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:45:46.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:33 smithi019 ceph-mon[29215]: pgmap v614: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:46.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:35 smithi019 ceph-mon[29215]: pgmap v615: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:46.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:37 smithi019 ceph-mon[29215]: pgmap v616: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:46.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:39 smithi019 ceph-mon[29215]: pgmap v617: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:46.721 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:41 smithi019 ceph-mon[29215]: pgmap v618: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:46.731 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:43 smithi155 ceph-mon[33468]: pgmap v619: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:46.738 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:45 smithi155 ceph-mon[33468]: pgmap v620: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:46.747 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:43 smithi044 ceph-mon[32458]: pgmap v619: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:46.759 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:45 smithi044 ceph-mon[32458]: pgmap v620: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:43 smithi019 ceph-mon[29215]: pgmap v619: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:46.808 DEBUG:teuthology.parallel:result is None 2024-04-23T18:45:46.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-23T18:45:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.688+0000 7f153e19b640 1 -- 172.21.15.19:0/1927118863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1538103200 msgr2=0x7f15381035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.688+0000 7f153e19b640 1 --2- 172.21.15.19:0/1927118863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1538103200 0x7f15381035e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1528009900 tx=0x7f152802a480 comp rx=0 tx=0).stop 2024-04-23T18:45:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.688+0000 7f153e19b640 1 -- 172.21.15.19:0/1927118863 shutdown_connections 2024-04-23T18:45:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.688+0000 7f153e19b640 1 --2- 172.21.15.19:0/1927118863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1538103b20 0x7f153810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.688+0000 7f153e19b640 1 --2- 172.21.15.19:0/1927118863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f153810c5d0 0x7f153810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.688+0000 7f153e19b640 1 --2- 172.21.15.19:0/1927118863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1538103200 0x7f15381035e0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.688+0000 7f153e19b640 1 -- 172.21.15.19:0/1927118863 >> 172.21.15.19:0/1927118863 conn(0x7f15380771d0 msgr2=0x7f15380775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.688+0000 7f153e19b640 1 -- 172.21.15.19:0/1927118863 shutdown_connections 2024-04-23T18:45:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.688+0000 7f153e19b640 1 -- 172.21.15.19:0/1927118863 wait complete. 2024-04-23T18:45:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.689+0000 7f153e19b640 1 Processor -- start 2024-04-23T18:45:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.689+0000 7f153e19b640 1 -- start start 2024-04-23T18:45:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.689+0000 7f153e19b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1538103200 0x7f1538100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.689+0000 7f153e19b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1538103b20 0x7f1538101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.689+0000 7f153e19b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f153810c5d0 0x7f15381018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.689+0000 7f153e19b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1538112250 con 0x7f153810c5d0 2024-04-23T18:45:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.689+0000 7f153e19b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15381120d0 con 0x7f1538103200 2024-04-23T18:45:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.689+0000 7f153e19b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15381123d0 con 0x7f1538103b20 2024-04-23T18:45:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.689+0000 7f1537fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f153810c5d0 0x7f15381018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f1537fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f153810c5d0 0x7f15381018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60056/0 (socket says 172.21.15.19:60056) 2024-04-23T18:45:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f1537fff640 1 -- 172.21.15.19:0/3499616181 learned_addr learned my addr 172.21.15.19:0/3499616181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f152effd640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1538103b20 0x7f1538101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f15377fe640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1538103200 0x7f1538100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f1537fff640 1 -- 172.21.15.19:0/3499616181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1538103b20 msgr2=0x7f1538101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f1537fff640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1538103b20 0x7f1538101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f1537fff640 1 -- 172.21.15.19:0/3499616181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1538103200 msgr2=0x7f1538100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f1537fff640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1538103200 0x7f1538100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f1537fff640 1 -- 172.21.15.19:0/3499616181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15381a6b50 con 0x7f153810c5d0 2024-04-23T18:45:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f15377fe640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1538103200 0x7f1538100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f1537fff640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f153810c5d0 0x7f15381018d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f153000ccb0 tx=0x7f153000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f15357fa640 1 -- 172.21.15.19:0/3499616181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153000f450 con 0x7f153810c5d0 2024-04-23T18:45:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f15357fa640 1 -- 172.21.15.19:0/3499616181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f153000faa0 con 0x7f153810c5d0 2024-04-23T18:45:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f153e19b640 1 -- 172.21.15.19:0/3499616181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15381a6e40 con 0x7f153810c5d0 2024-04-23T18:45:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f15357fa640 1 -- 172.21.15.19:0/3499616181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530018b00 con 0x7f153810c5d0 2024-04-23T18:45:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.690+0000 7f153e19b640 1 -- 172.21.15.19:0/3499616181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15381a7360 con 0x7f153810c5d0 2024-04-23T18:45:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.693+0000 7f15357fa640 1 -- 172.21.15.19:0/3499616181 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f153000fc50 con 0x7f153810c5d0 2024-04-23T18:45:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.693+0000 7f1540f92740 1 Processor -- start 2024-04-23T18:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.693+0000 7f1540f92740 1 -- start start 2024-04-23T18:45:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.693+0000 7f1540f92740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bd620 0x55bf9f9bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.693+0000 7f1540f92740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.693+0000 7f1540f92740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.693+0000 7f1540f92740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bf9f922970 con 0x55bf9f9c8800 2024-04-23T18:45:47.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.693+0000 7f1540f92740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf9f8bf4a0 con 0x55bf9f9bd620 2024-04-23T18:45:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.693+0000 7f1540f92740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bf9f91e930 con 0x55bf9f9bdf40 2024-04-23T18:45:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f15357fa640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f15240a3e40 0x7f15240a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:47.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60072/0 (socket says 172.21.15.19:60072) 2024-04-23T18:45:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152effd640 1 -- 172.21.15.19:0/1464087780 learned_addr learned my addr 172.21.15.19:0/1464087780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f15377fe640 1 --2- 172.21.15.19:0/1464087780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152effd640 1 --2- 172.21.15.19:0/1464087780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bd620 0x55bf9f9bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f1537fff640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f15240a3e40 0x7f15240a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f15357fa640 1 -- 172.21.15.19:0/3499616181 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7f15300d0e90 con 0x7f153810c5d0 2024-04-23T18:45:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152effd640 1 -- 172.21.15.19:0/1464087780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bdf40 msgr2=0x55bf9f9c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152effd640 1 --2- 172.21.15.19:0/1464087780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152effd640 1 -- 172.21.15.19:0/1464087780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bd620 msgr2=0x55bf9f9bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152effd640 1 --2- 172.21.15.19:0/1464087780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bd620 0x55bf9f9bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152effd640 1 -- 172.21.15.19:0/1464087780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9f635210 con 0x55bf9f9c8800 2024-04-23T18:45:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152effd640 1 --2- 172.21.15.19:0/1464087780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9cabf0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f15200058c0 tx=0x7f1520005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733c01a1842fdba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f1537fff640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f15240a3e40 0x7f15240a6300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f153009cb00 tx=0x7f153000ff70 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152dffb640 1 -- 172.21.15.19:0/1464087780 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520004030 con 0x55bf9f9c8800 2024-04-23T18:45:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152dffb640 1 -- 172.21.15.19:0/1464087780 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f152000f7c0 con 0x55bf9f9c8800 2024-04-23T18:45:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.694+0000 7f152dffb640 1 -- 172.21.15.19:0/1464087780 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520003970 con 0x55bf9f9c8800 2024-04-23T18:45:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.695+0000 7f1540f92740 1 -- 172.21.15.19:0/1464087780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 msgr2=0x55bf9f9cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.695+0000 7f1540f92740 1 --2- 172.21.15.19:0/1464087780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9cabf0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f15200058c0 tx=0x7f1520005850 comp rx=0 tx=0).stop 2024-04-23T18:45:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.695+0000 7f1540f92740 1 -- 172.21.15.19:0/1464087780 shutdown_connections 2024-04-23T18:45:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.695+0000 7f1540f92740 1 --2- 172.21.15.19:0/1464087780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9cabf0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.695+0000 7f1540f92740 1 --2- 172.21.15.19:0/1464087780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.695+0000 7f1540f92740 1 --2- 172.21.15.19:0/1464087780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bd620 0x55bf9f9bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.695+0000 7f1540f92740 1 -- 172.21.15.19:0/1464087780 >> 172.21.15.19:0/1464087780 conn(0x55bf9f9b9b30 msgr2=0x55bf9f9b9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.695+0000 7f1540f92740 1 -- 172.21.15.19:0/1464087780 shutdown_connections 2024-04-23T18:45:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.695+0000 7f1540f92740 1 -- 172.21.15.19:0/1464087780 wait complete. 2024-04-23T18:45:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.695+0000 7f1540f92740 1 -- start start 2024-04-23T18:45:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f1540f92740 1 Processor -- start 2024-04-23T18:45:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f1540f92740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bd620 0x55bf9f9c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f1540f92740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9c5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f1540f92740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f1540f92740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bf9f9cb430 con 0x55bf9f9c8800 2024-04-23T18:45:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f1540f92740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf9f8bd0b0 con 0x55bf9f9bdf40 2024-04-23T18:45:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f1540f92740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bf9f9cb5b0 con 0x55bf9f9bd620 2024-04-23T18:45:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f1540f92740 10 client.0 _ll_register_callbacks cb 0x55bf9f8aad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f152effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f152effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60080/0 (socket says 172.21.15.19:60080) 2024-04-23T18:45:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f152effd640 1 -- 172.21.15.19:0/322434193 learned_addr learned my addr 172.21.15.19:0/322434193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f152effd640 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bd620 0x55bf9f9c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.696+0000 7f15377fe640 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f152effd640 1 -- 172.21.15.19:0/322434193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bd620 msgr2=0x55bf9f9c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f152effd640 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bd620 0x55bf9f9c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f152effd640 1 -- 172.21.15.19:0/322434193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 msgr2=0x55bf9f9c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f152effd640 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f152effd640 1 -- 172.21.15.19:0/322434193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9f701620 con 0x55bf9f9c8800 2024-04-23T18:45:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f152effd640 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9c5940 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1520009e00 tx=0x7f1520009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f4c97b13dddb47 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f1516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f1516ffd640 1 -- 172.21.15.19:0/322434193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520005230 con 0x55bf9f9c8800 2024-04-23T18:45:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f1540f92740 1 -- 172.21.15.19:0/322434193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f152000f7c0 con 0x55bf9f9c8800 2024-04-23T18:45:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f1516ffd640 1 -- 172.21.15.19:0/322434193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7f1520028040 con 0x55bf9f9c8800 2024-04-23T18:45:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f1516ffd640 1 -- 172.21.15.19:0/322434193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f15200053d0 con 0x55bf9f9c8800 2024-04-23T18:45:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f1540f92740 10 client.17793 fetch_fsmap learned FSMap version 31 2024-04-23T18:45:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.697+0000 7f1540f92740 1 -- 172.21.15.19:0/322434193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bf9f9d7330 con 0x55bf9f9c8800 2024-04-23T18:45:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.698+0000 7f1516ffd640 1 -- 172.21.15.19:0/322434193 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f152000f7c0 con 0x55bf9f9c8800 2024-04-23T18:45:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.698+0000 7f1516ffd640 1 -- 172.21.15.19:0/322434193 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f152001f510 con 0x55bf9f9c8800 2024-04-23T18:45:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.698+0000 7f1540f92740 10 client.17793 fetch_fsmap finished waiting for FSMap version 31 2024-04-23T18:45:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.698+0000 7f1540f92740 10 client.17793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.698+0000 7f1540f92740 20 client.17793 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.698+0000 7f1540f92740 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bf9f9d8680 0x55bf9f9daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.698+0000 7f1540f92740 4 client.17793 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.698+0000 7f1540f92740 1 -- 172.21.15.19:0/322434193 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf9f930390 con 0x55bf9f9d8680 2024-04-23T18:45:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.698+0000 7f15377fe640 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bf9f9d8680 0x55bf9f9daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.699+0000 7f15377fe640 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bf9f9d8680 0x55bf9f9daa70 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b98e56af798d25c server_cookie=34dc0e83e30c01a in_seq=0 out_seq=0 2024-04-23T18:45:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.699+0000 7f1516ffd640 10 client.17793 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.699+0000 7f1516ffd640 1 -- 172.21.15.19:0/322434193 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf9f930390 con 0x55bf9f9d8680 2024-04-23T18:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.699+0000 7f1516ffd640 10 client.17793 handle_command_reply: tid=0 2024-04-23T18:45:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.699+0000 7f1540f92740 1 client.17793 shutdown 2024-04-23T18:45:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 1 -- 172.21.15.19:0/322434193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 msgr2=0x55bf9f9c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9c5940 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1520009e00 tx=0x7f1520009d10 comp rx=0 tx=0).stop 2024-04-23T18:45:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 1 -- 172.21.15.19:0/322434193 shutdown_connections 2024-04-23T18:45:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bf9f9d8680 0x55bf9f9daa70 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9c8800 0x55bf9f9c5940 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1516ffd640 0 client.17793 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 1 --2- 172.21.15.19:0/322434193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bd620 0x55bf9f9c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 1 -- 172.21.15.19:0/322434193 >> 172.21.15.19:0/322434193 conn(0x55bf9f9b9b30 msgr2=0x55bf9f9bf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 1 -- 172.21.15.19:0/322434193 shutdown_connections 2024-04-23T18:45:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 1 -- 172.21.15.19:0/322434193 wait complete. 2024-04-23T18:45:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.700+0000 7f1540f92740 20 client.17793 trim_cache size 0 max 16384 2024-04-23T18:45:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f1540f92740 1 Processor -- start 2024-04-23T18:45:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f1540f92740 1 -- start start 2024-04-23T18:45:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f1540f92740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9b9b30 0x55bf9f9d42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f1540f92740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9bd620 0x55bf9f9b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f1540f92740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9b9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f1540f92740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15381afc50 con 0x55bf9f9bd620 2024-04-23T18:45:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f1540f92740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15381b1790 con 0x55bf9f9bdf40 2024-04-23T18:45:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f1540f92740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15381b0320 con 0x55bf9f9b9b30 2024-04-23T18:45:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f15377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9bd620 0x55bf9f9b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f15377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9bd620 0x55bf9f9b8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60088/0 (socket says 172.21.15.19:60088) 2024-04-23T18:45:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f15377fe640 1 -- 172.21.15.19:0/1047999151 learned_addr learned my addr 172.21.15.19:0/1047999151 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.715+0000 7f152effd640 1 --2- 172.21.15.19:0/1047999151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9b9b30 0x55bf9f9d42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.716+0000 7f15377fe640 1 -- 172.21.15.19:0/1047999151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9b9b30 msgr2=0x55bf9f9d42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.716+0000 7f15377fe640 1 --2- 172.21.15.19:0/1047999151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9b9b30 0x55bf9f9d42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.716+0000 7f15377fe640 1 -- 172.21.15.19:0/1047999151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 msgr2=0x55bf9f9b9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.716+0000 7f15377fe640 1 --2- 172.21.15.19:0/1047999151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9b9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.716+0000 7f15377fe640 1 -- 172.21.15.19:0/1047999151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9f9dade0 con 0x55bf9f9bd620 2024-04-23T18:45:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.716+0000 7f15377fe640 1 --2- 172.21.15.19:0/1047999151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9bd620 0x55bf9f9b8b30 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1528031440 tx=0x7f1528009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d797bac31a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.716+0000 7f152d7fa640 1 -- 172.21.15.19:0/1047999151 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528035040 con 0x55bf9f9bd620 2024-04-23T18:45:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.716+0000 7f152d7fa640 1 -- 172.21.15.19:0/1047999151 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1528044dc0 con 0x55bf9f9bd620 2024-04-23T18:45:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.716+0000 7f152d7fa640 1 -- 172.21.15.19:0/1047999151 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528004920 con 0x55bf9f9bd620 2024-04-23T18:45:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 -- 172.21.15.19:0/1047999151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9bd620 msgr2=0x55bf9f9b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 --2- 172.21.15.19:0/1047999151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9bd620 0x55bf9f9b8b30 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1528031440 tx=0x7f1528009560 comp rx=0 tx=0).stop 2024-04-23T18:45:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 -- 172.21.15.19:0/1047999151 shutdown_connections 2024-04-23T18:45:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 --2- 172.21.15.19:0/1047999151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9b9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 --2- 172.21.15.19:0/1047999151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9bd620 0x55bf9f9b8b30 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 --2- 172.21.15.19:0/1047999151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9b9b30 0x55bf9f9d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 -- 172.21.15.19:0/1047999151 >> 172.21.15.19:0/1047999151 conn(0x55bf9f9c5aa0 msgr2=0x55bf9f9b8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 -- 172.21.15.19:0/1047999151 shutdown_connections 2024-04-23T18:45:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 -- 172.21.15.19:0/1047999151 wait complete. 2024-04-23T18:45:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 -- start start 2024-04-23T18:45:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.717+0000 7f1540f92740 1 Processor -- start 2024-04-23T18:45:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1540f92740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9b9b30 0x55bf9f9d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1540f92740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bd620 0x55bf9f9cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1540f92740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1540f92740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15381b10c0 con 0x55bf9f9b9b30 2024-04-23T18:45:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1540f92740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15381b09f0 con 0x55bf9f9bdf40 2024-04-23T18:45:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1540f92740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15381adc30 con 0x55bf9f9bd620 2024-04-23T18:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1540f92740 10 client.0 _ll_register_callbacks cb 0x7f15200050d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f152effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9b9b30 0x55bf9f9d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f152effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9b9b30 0x55bf9f9d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60100/0 (socket says 172.21.15.19:60100) 2024-04-23T18:45:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f152effd640 1 -- 172.21.15.19:0/1230511936 learned_addr learned my addr 172.21.15.19:0/1230511936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f15377fe640 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bd620 0x55bf9f9cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f152effd640 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f152effd640 1 -- 172.21.15.19:0/1230511936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bd620 msgr2=0x55bf9f9cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f152effd640 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bd620 0x55bf9f9cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f152effd640 1 -- 172.21.15.19:0/1230511936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 msgr2=0x55bf9f9ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f152effd640 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f152effd640 1 -- 172.21.15.19:0/1230511936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9f9ce880 con 0x55bf9f9b9b30 2024-04-23T18:45:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f152effd640 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9b9b30 0x55bf9f9d0950 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1520009cb0 tx=0x7f1520000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4336140499e16257 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1516ffd640 1 -- 172.21.15.19:0/1230511936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152001fd30 con 0x55bf9f9b9b30 2024-04-23T18:45:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1516ffd640 1 -- 172.21.15.19:0/1230511936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7f152001f3f0 con 0x55bf9f9b9b30 2024-04-23T18:45:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1540f92740 1 -- 172.21.15.19:0/1230511936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1528044dc0 con 0x55bf9f9b9b30 2024-04-23T18:45:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.718+0000 7f1516ffd640 1 -- 172.21.15.19:0/1230511936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f152000f790 con 0x55bf9f9b9b30 2024-04-23T18:45:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f1540f92740 10 client.17799 fetch_fsmap learned FSMap version 31 2024-04-23T18:45:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f1540f92740 1 -- 172.21.15.19:0/1230511936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bf9fa3e840 con 0x55bf9f9b9b30 2024-04-23T18:45:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f1516ffd640 1 -- 172.21.15.19:0/1230511936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1528044dc0 con 0x55bf9f9b9b30 2024-04-23T18:45:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f1516ffd640 1 -- 172.21.15.19:0/1230511936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f152002d430 con 0x55bf9f9b9b30 2024-04-23T18:45:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f1540f92740 10 client.17799 fetch_fsmap finished waiting for FSMap version 31 2024-04-23T18:45:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f1540f92740 10 client.17799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f1540f92740 20 client.17799 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f1540f92740 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bf9f9c8800 0x55bf9fa40fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f1540f92740 4 client.17799 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f1540f92740 1 -- 172.21.15.19:0/1230511936 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf9f92a200 con 0x55bf9f9c8800 2024-04-23T18:45:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.719+0000 7f15377fe640 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bf9f9c8800 0x55bf9fa40fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.720+0000 7f15377fe640 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bf9f9c8800 0x55bf9fa40fa0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ce2b57d7c95824 server_cookie=233749617888c21b in_seq=0 out_seq=0 2024-04-23T18:45:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.720+0000 7f1516ffd640 10 client.17799 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1516ffd640 1 -- 172.21.15.19:0/1230511936 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf9f92a200 con 0x55bf9f9c8800 2024-04-23T18:45:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1516ffd640 10 client.17799 handle_command_reply: tid=0 2024-04-23T18:45:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 client.17799 shutdown 2024-04-23T18:45:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 -- 172.21.15.19:0/1230511936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9b9b30 msgr2=0x55bf9f9d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9b9b30 0x55bf9f9d0950 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1520009cb0 tx=0x7f1520000bc0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 -- 172.21.15.19:0/1230511936 shutdown_connections 2024-04-23T18:45:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x55bf9f9c8800 0x55bf9fa40fa0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf9f9bdf40 0x55bf9f9ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf9f9bd620 0x55bf9f9cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1516ffd640 0 client.17799 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 --2- 172.21.15.19:0/1230511936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf9f9b9b30 0x55bf9f9d0950 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 -- 172.21.15.19:0/1230511936 >> 172.21.15.19:0/1230511936 conn(0x55bf9f9c5aa0 msgr2=0x55bf9f9b8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 -- 172.21.15.19:0/1230511936 shutdown_connections 2024-04-23T18:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 1 -- 172.21.15.19:0/1230511936 wait complete. 2024-04-23T18:45:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.724+0000 7f1540f92740 20 client.17799 trim_cache size 0 max 16384 2024-04-23T18:45:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-23T18:45:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-23T18:45:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-23T18:45:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-23T18:45:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 -- 172.21.15.19:0/3499616181 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f15240a3e40 msgr2=0x7f15240a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f15240a3e40 0x7f15240a6300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f153009cb00 tx=0x7f153000ff70 comp rx=0 tx=0).stop 2024-04-23T18:45:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 -- 172.21.15.19:0/3499616181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f153810c5d0 msgr2=0x7f15381018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f153810c5d0 0x7f15381018d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f153000ccb0 tx=0x7f153000a620 comp rx=0 tx=0).stop 2024-04-23T18:45:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 -- 172.21.15.19:0/3499616181 shutdown_connections 2024-04-23T18:45:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f15240a3e40 0x7f15240a6300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f153810c5d0 0x7f15381018d0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1538103b20 0x7f1538101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 --2- 172.21.15.19:0/3499616181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1538103200 0x7f1538100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 -- 172.21.15.19:0/3499616181 >> 172.21.15.19:0/3499616181 conn(0x7f15380771d0 msgr2=0x7f15380fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 -- 172.21.15.19:0/3499616181 shutdown_connections 2024-04-23T18:45:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-23T18:45:44.725+0000 7f153e19b640 1 -- 172.21.15.19:0/3499616181 wait complete. 2024-04-23T18:45:49.757 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:45 smithi019 ceph-mon[29215]: pgmap v620: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:49.767 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:47 smithi155 ceph-mon[33468]: pgmap v621: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:49.773 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:47 smithi044 ceph-mon[32458]: pgmap v621: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.724+0000 7fbf95474640 1 -- 172.21.15.19:0/320026284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf90101a50 msgr2=0x7fbf90105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:49.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.724+0000 7fbf95474640 1 --2- 172.21.15.19:0/320026284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf90101a50 0x7fbf90105a90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009900 tx=0x7fbf7802a410 comp rx=0 tx=0).stop 2024-04-23T18:45:49.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.724+0000 7fbf95474640 1 -- 172.21.15.19:0/320026284 shutdown_connections 2024-04-23T18:45:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.724+0000 7fbf95474640 1 --2- 172.21.15.19:0/320026284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf90106170 0x7fbf9010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.724+0000 7fbf95474640 1 --2- 172.21.15.19:0/320026284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf90101a50 0x7fbf90105a90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.724+0000 7fbf95474640 1 --2- 172.21.15.19:0/320026284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf901010a0 0x7fbf90101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.724+0000 7fbf95474640 1 -- 172.21.15.19:0/320026284 >> 172.21.15.19:0/320026284 conn(0x7fbf90077eb0 msgr2=0x7fbf900782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:49.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.724+0000 7fbf95474640 1 -- 172.21.15.19:0/320026284 shutdown_connections 2024-04-23T18:45:49.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.724+0000 7fbf95474640 1 -- 172.21.15.19:0/320026284 wait complete. 2024-04-23T18:45:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.725+0000 7fbf95474640 1 Processor -- start 2024-04-23T18:45:49.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.725+0000 7fbf95474640 1 -- start start 2024-04-23T18:45:49.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.725+0000 7fbf95474640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf901010a0 0x7fbf9019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:49.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.725+0000 7fbf95474640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf90106170 0x7fbf9019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.725+0000 7fbf95474640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf901985a0 0x7fbf90198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:49.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.725+0000 7fbf95474640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901110b0 con 0x7fbf901985a0 2024-04-23T18:45:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.725+0000 7fbf95474640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90110f30 con 0x7fbf90106170 2024-04-23T18:45:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.725+0000 7fbf95474640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90111230 con 0x7fbf901010a0 2024-04-23T18:45:50.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.725+0000 7fbf8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf901985a0 0x7fbf90198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.725+0000 7fbf8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf901985a0 0x7fbf90198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60120/0 (socket says 172.21.15.19:60120) 2024-04-23T18:45:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8f7fe640 1 -- 172.21.15.19:0/1930732279 learned_addr learned my addr 172.21.15.19:0/1930732279 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:50.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8effd640 1 --2- 172.21.15.19:0/1930732279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf901010a0 0x7fbf9019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf867fc640 1 --2- 172.21.15.19:0/1930732279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf90106170 0x7fbf9019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8f7fe640 1 -- 172.21.15.19:0/1930732279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf901010a0 msgr2=0x7fbf9019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:50.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8f7fe640 1 --2- 172.21.15.19:0/1930732279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf901010a0 0x7fbf9019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8f7fe640 1 -- 172.21.15.19:0/1930732279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf90106170 msgr2=0x7fbf9019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:50.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8f7fe640 1 --2- 172.21.15.19:0/1930732279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf90106170 0x7fbf9019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8f7fe640 1 -- 172.21.15.19:0/1930732279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf901990c0 con 0x7fbf901985a0 2024-04-23T18:45:50.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8f7fe640 1 --2- 172.21.15.19:0/1930732279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf901985a0 0x7fbf90198a50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009fd0 tx=0x7fbf8800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8cff9640 1 -- 172.21.15.19:0/1930732279 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88017070 con 0x7fbf901985a0 2024-04-23T18:45:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf95474640 1 -- 172.21.15.19:0/1930732279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf901993b0 con 0x7fbf901985a0 2024-04-23T18:45:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8cff9640 1 -- 172.21.15.19:0/1930732279 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf88003e20 con 0x7fbf901985a0 2024-04-23T18:45:50.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf8cff9640 1 -- 172.21.15.19:0/1930732279 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88010460 con 0x7fbf901985a0 2024-04-23T18:45:50.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.726+0000 7fbf95474640 1 -- 172.21.15.19:0/1930732279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf901a55b0 con 0x7fbf901985a0 2024-04-23T18:45:50.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.729+0000 7fbf8cff9640 1 -- 172.21.15.19:0/1930732279 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbf88003970 con 0x7fbf901985a0 2024-04-23T18:45:50.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.729+0000 7fbf98181740 1 Processor -- start 2024-04-23T18:45:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.729+0000 7fbf98181740 1 -- start start 2024-04-23T18:45:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf98181740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d7550 0x56048e8d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:50.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf98181740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 0x56048e8d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:50.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf98181740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d3770 0x56048e8df4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:50.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf98181740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56048e82e6d0 con 0x56048e8d3770 2024-04-23T18:45:50.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf98181740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56048e7cb1a0 con 0x56048e8d2db0 2024-04-23T18:45:50.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf98181740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56048e82a690 con 0x56048e8d7550 2024-04-23T18:45:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d3770 0x56048e8df4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d3770 0x56048e8df4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60122/0 (socket says 172.21.15.19:60122) 2024-04-23T18:45:50.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf867fc640 1 -- 172.21.15.19:0/3366442399 learned_addr learned my addr 172.21.15.19:0/3366442399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:50.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf8cff9640 1 --2- 172.21.15.19:0/1930732279 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbf7c0a3d20 0x7fbf7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf867fc640 1 --2- 172.21.15.19:0/3366442399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d7550 0x56048e8d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf8effd640 1 --2- 172.21.15.19:0/3366442399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 0x56048e8d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf8f7fe640 1 --2- 172.21.15.19:0/1930732279 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbf7c0a3d20 0x7fbf7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf867fc640 1 -- 172.21.15.19:0/3366442399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d7550 msgr2=0x56048e8d7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:50.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf867fc640 1 --2- 172.21.15.19:0/3366442399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d7550 0x56048e8d7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf867fc640 1 -- 172.21.15.19:0/3366442399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 msgr2=0x56048e8d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf867fc640 1 --2- 172.21.15.19:0/3366442399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 0x56048e8d3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf867fc640 1 -- 172.21.15.19:0/3366442399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56048e7d55b0 con 0x56048e8d3770 2024-04-23T18:45:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf8effd640 1 --2- 172.21.15.19:0/3366442399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 0x56048e8d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf8cff9640 1 -- 172.21.15.19:0/1930732279 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7fbf880d0460 con 0x7fbf901985a0 2024-04-23T18:45:50.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.730+0000 7fbf867fc640 1 --2- 172.21.15.19:0/3366442399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d3770 0x56048e8df4a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009900 tx=0x7fbf78005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36af26e16f875d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:50.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf8f7fe640 1 --2- 172.21.15.19:0/1930732279 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fbf7c0a3d20 0x7fbf7c0a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf88098eb0 tx=0x7fbf88087010 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:50.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf857fa640 1 -- 172.21.15.19:0/3366442399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78035070 con 0x56048e8d3770 2024-04-23T18:45:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf857fa640 1 -- 172.21.15.19:0/3366442399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf780035f0 con 0x56048e8d3770 2024-04-23T18:45:50.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf857fa640 1 -- 172.21.15.19:0/3366442399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf780044e0 con 0x56048e8d3770 2024-04-23T18:45:50.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf98181740 1 -- 172.21.15.19:0/3366442399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d3770 msgr2=0x56048e8df4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:50.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf98181740 1 --2- 172.21.15.19:0/3366442399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d3770 0x56048e8df4a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009900 tx=0x7fbf78005780 comp rx=0 tx=0).stop 2024-04-23T18:45:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf98181740 1 -- 172.21.15.19:0/3366442399 shutdown_connections 2024-04-23T18:45:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf98181740 1 --2- 172.21.15.19:0/3366442399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 0x56048e8d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf98181740 1 --2- 172.21.15.19:0/3366442399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d3770 0x56048e8df4a0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf98181740 1 --2- 172.21.15.19:0/3366442399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d7550 0x56048e8d7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.731+0000 7fbf98181740 1 -- 172.21.15.19:0/3366442399 >> 172.21.15.19:0/3366442399 conn(0x56048e8c5260 msgr2=0x56048e8c5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:50.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 1 -- 172.21.15.19:0/3366442399 shutdown_connections 2024-04-23T18:45:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 1 -- 172.21.15.19:0/3366442399 wait complete. 2024-04-23T18:45:50.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 1 -- start start 2024-04-23T18:45:50.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 1 Processor -- start 2024-04-23T18:45:50.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 0x56048e8df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d3770 0x56048e8dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:50.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:50.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56048e8dfae0 con 0x56048e8d7550 2024-04-23T18:45:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56048e7c8b20 con 0x56048e8d2db0 2024-04-23T18:45:50.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56048e8dfc60 con 0x56048e8d3770 2024-04-23T18:45:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf98181740 10 client.0 _ll_register_callbacks cb 0x56048e7c8530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8dcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60124/0 (socket says 172.21.15.19:60124) 2024-04-23T18:45:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf867fc640 1 -- 172.21.15.19:0/458135079 learned_addr learned my addr 172.21.15.19:0/458135079 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf8effd640 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d3770 0x56048e8dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.732+0000 7fbf867fc640 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 0x56048e8df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf867fc640 1 -- 172.21.15.19:0/458135079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d3770 msgr2=0x56048e8dc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:50.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf867fc640 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d3770 0x56048e8dc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf867fc640 1 -- 172.21.15.19:0/458135079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 msgr2=0x56048e8df3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf867fc640 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 0x56048e8df3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf867fc640 1 -- 172.21.15.19:0/458135079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048e540920 con 0x56048e8d7550 2024-04-23T18:45:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf867fc640 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8dcce0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fbf78005d00 tx=0x7fbf78032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3b801c51c50f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:50.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf767fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:50.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf767fc640 1 -- 172.21.15.19:0/458135079 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78035040 con 0x56048e8d7550 2024-04-23T18:45:50.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf767fc640 1 -- 172.21.15.19:0/458135079 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7fbf78004320 con 0x56048e8d7550 2024-04-23T18:45:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf98181740 1 -- 172.21.15.19:0/458135079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf780035f0 con 0x56048e8d7550 2024-04-23T18:45:50.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf767fc640 1 -- 172.21.15.19:0/458135079 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbf780037a0 con 0x56048e8d7550 2024-04-23T18:45:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf98181740 10 client.17811 fetch_fsmap learned FSMap version 31 2024-04-23T18:45:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf98181740 1 -- 172.21.15.19:0/458135079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56048e60cd30 con 0x56048e8d7550 2024-04-23T18:45:51.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf767fc640 1 -- 172.21.15.19:0/458135079 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf780035f0 con 0x56048e8d7550 2024-04-23T18:45:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf767fc640 1 -- 172.21.15.19:0/458135079 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fbf7803d890 con 0x56048e8d7550 2024-04-23T18:45:51.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf98181740 10 client.17811 fetch_fsmap finished waiting for FSMap version 31 2024-04-23T18:45:51.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf98181740 10 client.17811 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:45:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.733+0000 7fbf98181740 20 client.17811 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.734+0000 7fbf98181740 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56048e8eccc0 0x56048e8ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:51.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.734+0000 7fbf98181740 4 client.17811 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.734+0000 7fbf98181740 1 -- 172.21.15.19:0/458135079 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56048e7ec500 con 0x56048e8eccc0 2024-04-23T18:45:51.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.734+0000 7fbf8effd640 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56048e8eccc0 0x56048e8ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.734+0000 7fbf8effd640 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56048e8eccc0 0x56048e8ef0b0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d23c6ea1b2180752 server_cookie=ff30e219c7d8816a in_seq=0 out_seq=0 2024-04-23T18:45:51.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.734+0000 7fbf767fc640 10 client.17811 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:51.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf767fc640 1 -- 172.21.15.19:0/458135079 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56048e7ec500 con 0x56048e8eccc0 2024-04-23T18:45:51.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf767fc640 10 client.17811 handle_command_reply: tid=0 2024-04-23T18:45:51.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 client.17811 shutdown 2024-04-23T18:45:51.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 -- 172.21.15.19:0/458135079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 msgr2=0x56048e8dcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8dcce0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fbf78005d00 tx=0x7fbf78032e10 comp rx=0 tx=0).stop 2024-04-23T18:45:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 -- 172.21.15.19:0/458135079 shutdown_connections 2024-04-23T18:45:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56048e8eccc0 0x56048e8ef0b0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8dcce0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf767fc640 0 client.17811 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d3770 0x56048e8dc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 --2- 172.21.15.19:0/458135079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d2db0 0x56048e8df3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 -- 172.21.15.19:0/458135079 >> 172.21.15.19:0/458135079 conn(0x56048e8c5260 msgr2=0x56048e8d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 -- 172.21.15.19:0/458135079 shutdown_connections 2024-04-23T18:45:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 1 -- 172.21.15.19:0/458135079 wait complete. 2024-04-23T18:45:51.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.735+0000 7fbf98181740 20 client.17811 trim_cache size 0 max 16384 2024-04-23T18:45:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf98181740 1 Processor -- start 2024-04-23T18:45:51.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf98181740 1 -- start start 2024-04-23T18:45:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf98181740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d2db0 0x56048e8dc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:51.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf98181740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 0x56048e8dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf98181740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8d6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:51.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf98181740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901ab820 con 0x56048e8d7550 2024-04-23T18:45:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf98181740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901ad360 con 0x56048e8d3770 2024-04-23T18:45:51.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf98181740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901abef0 con 0x56048e8d2db0 2024-04-23T18:45:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8d6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8d6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60138/0 (socket says 172.21.15.19:60138) 2024-04-23T18:45:51.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.753+0000 7fbf867fc640 1 -- 172.21.15.19:0/2622156307 learned_addr learned my addr 172.21.15.19:0/2622156307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:51.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf867fc640 1 --2- 172.21.15.19:0/2622156307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d2db0 0x56048e8dc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf867fc640 1 -- 172.21.15.19:0/2622156307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d2db0 msgr2=0x56048e8dc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf8effd640 1 --2- 172.21.15.19:0/2622156307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 0x56048e8dcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf867fc640 1 --2- 172.21.15.19:0/2622156307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d2db0 0x56048e8dc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf867fc640 1 -- 172.21.15.19:0/2622156307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 msgr2=0x56048e8dcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf867fc640 1 --2- 172.21.15.19:0/2622156307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 0x56048e8dcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf867fc640 1 -- 172.21.15.19:0/2622156307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56048e8d7ca0 con 0x56048e8d7550 2024-04-23T18:45:51.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf8effd640 1 --2- 172.21.15.19:0/2622156307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 0x56048e8dcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:51.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf867fc640 1 --2- 172.21.15.19:0/2622156307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8d6810 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fbf7803dcb0 tx=0x7fbf78041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50561262444b10fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf84ff9640 1 -- 172.21.15.19:0/2622156307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78035070 con 0x56048e8d7550 2024-04-23T18:45:51.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf84ff9640 1 -- 172.21.15.19:0/2622156307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf78032e80 con 0x56048e8d7550 2024-04-23T18:45:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.754+0000 7fbf84ff9640 1 -- 172.21.15.19:0/2622156307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7803d6f0 con 0x56048e8d7550 2024-04-23T18:45:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 -- 172.21.15.19:0/2622156307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 msgr2=0x56048e8d6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 --2- 172.21.15.19:0/2622156307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8d6810 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fbf7803dcb0 tx=0x7fbf78041040 comp rx=0 tx=0).stop 2024-04-23T18:45:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 -- 172.21.15.19:0/2622156307 shutdown_connections 2024-04-23T18:45:51.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 --2- 172.21.15.19:0/2622156307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 0x56048e8dcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 --2- 172.21.15.19:0/2622156307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d7550 0x56048e8d6810 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 --2- 172.21.15.19:0/2622156307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d2db0 0x56048e8dc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 -- 172.21.15.19:0/2622156307 >> 172.21.15.19:0/2622156307 conn(0x56048e8c5260 msgr2=0x56048e8d8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:51.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 -- 172.21.15.19:0/2622156307 shutdown_connections 2024-04-23T18:45:51.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 -- 172.21.15.19:0/2622156307 wait complete. 2024-04-23T18:45:51.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 -- start start 2024-04-23T18:45:51.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 Processor -- start 2024-04-23T18:45:51.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d2db0 0x56048e8f2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:51.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 0x56048e8ed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:51.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d7550 0x56048e8edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:51.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901acc90 con 0x56048e8d2db0 2024-04-23T18:45:51.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901ac5c0 con 0x56048e8d3770 2024-04-23T18:45:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901a9840 con 0x56048e8d7550 2024-04-23T18:45:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.755+0000 7fbf98181740 10 client.0 _ll_register_callbacks cb 0x7fbf78031770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d2db0 0x56048e8f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d2db0 0x56048e8f2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60150/0 (socket says 172.21.15.19:60150) 2024-04-23T18:45:51.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf867fc640 1 -- 172.21.15.19:0/4095758577 learned_addr learned my addr 172.21.15.19:0/4095758577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf8effd640 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 0x56048e8ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf867fc640 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d7550 0x56048e8edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf867fc640 1 -- 172.21.15.19:0/4095758577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d7550 msgr2=0x56048e8edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:51.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf867fc640 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d7550 0x56048e8edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:52.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf867fc640 1 -- 172.21.15.19:0/4095758577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 msgr2=0x56048e8ed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:52.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf867fc640 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 0x56048e8ed780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:52.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf867fc640 1 -- 172.21.15.19:0/4095758577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048e8ee4c0 con 0x56048e8d2db0 2024-04-23T18:45:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf867fc640 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d2db0 0x56048e8f2410 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fbf78003730 tx=0x7fbf78009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297bdcd3dfceff22 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:52.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf767fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf767fc640 1 -- 172.21.15.19:0/4095758577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78035040 con 0x56048e8d2db0 2024-04-23T18:45:52.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf767fc640 1 -- 172.21.15.19:0/4095758577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7fbf78031a50 con 0x56048e8d2db0 2024-04-23T18:45:52.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf98181740 1 -- 172.21.15.19:0/4095758577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf78032e80 con 0x56048e8d2db0 2024-04-23T18:45:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.756+0000 7fbf767fc640 1 -- 172.21.15.19:0/4095758577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbf7803d450 con 0x56048e8d2db0 2024-04-23T18:45:52.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf98181740 10 client.17817 fetch_fsmap learned FSMap version 31 2024-04-23T18:45:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf98181740 1 -- 172.21.15.19:0/4095758577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56048e952db0 con 0x56048e8d2db0 2024-04-23T18:45:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf767fc640 1 -- 172.21.15.19:0/4095758577 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf78032e80 con 0x56048e8d2db0 2024-04-23T18:45:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf767fc640 1 -- 172.21.15.19:0/4095758577 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fbf780046c0 con 0x56048e8d2db0 2024-04-23T18:45:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf98181740 10 client.17817 fetch_fsmap finished waiting for FSMap version 31 2024-04-23T18:45:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf98181740 10 client.17817 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:45:52.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf98181740 20 client.17817 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:52.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf98181740 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56048e953360 0x56048e955780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf98181740 4 client.17817 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "46eff5eb", "release": true, "await": true}] 2024-04-23T18:45:52.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf98181740 1 -- 172.21.15.19:0/4095758577 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "set_id": "46eff5eb", "release": true, "await": true}) -- 0x56048e83c0f0 con 0x56048e953360 2024-04-23T18:45:52.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.757+0000 7fbf8effd640 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56048e953360 0x56048e955780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:52.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf767fc640 10 client.17817 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf8effd640 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56048e953360 0x56048e955780 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=715fb571bf85857e server_cookie=ddb430c34b0d96ea in_seq=0 out_seq=0 2024-04-23T18:45:52.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf767fc640 1 -- 172.21.15.19:0/4095758577 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: -1 ) ==== 8+0+592 (crc 0 0 0) 0x56048e83c0f0 con 0x56048e953360 2024-04-23T18:45:52.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf767fc640 10 client.17817 handle_command_reply: tid=0 2024-04-23T18:45:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 client.17817 shutdown 2024-04-23T18:45:52.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 -- 172.21.15.19:0/4095758577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d2db0 msgr2=0x56048e8f2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:52.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d2db0 0x56048e8f2410 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fbf78003730 tx=0x7fbf78009650 comp rx=0 tx=0).stop 2024-04-23T18:45:52.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 -- 172.21.15.19:0/4095758577 shutdown_connections 2024-04-23T18:45:52.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x56048e953360 0x56048e955780 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:52.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56048e8d3770 0x56048e8ed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:52.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56048e8d7550 0x56048e8edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:52.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 --2- 172.21.15.19:0/4095758577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56048e8d2db0 0x56048e8f2410 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf767fc640 0 client.17817 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:52.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 -- 172.21.15.19:0/4095758577 >> 172.21.15.19:0/4095758577 conn(0x56048e8c5260 msgr2=0x56048e8d6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:52.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 -- 172.21.15.19:0/4095758577 shutdown_connections 2024-04-23T18:45:52.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 1 -- 172.21.15.19:0/4095758577 wait complete. 2024-04-23T18:45:52.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:46.758+0000 7fbf98181740 20 client.17817 trim_cache size 0 max 16384 2024-04-23T18:45:52.531 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:45:52.543 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 31, 2024-04-23T18:45:52.553 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:45:52.562 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 47, 2024-04-23T18:45:52.570 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-04-23T18:45:52.580 INFO:teuthology.orchestra.run.smithi019.stdout: "46eff5eb": { 2024-04-23T18:45:52.608 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 47, 2024-04-23T18:45:52.615 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 216.2, 2024-04-23T18:45:52.622 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:45:52.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "EXPIRED", 2024-04-23T18:45:52.638 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-04-23T18:45:52.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:45:52.657 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 60.0, 2024-04-23T18:45:52.667 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 91.5, 2024-04-23T18:45:52.747 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-04-23T18:45:52.755 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-04-23T18:45:52.782 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-04-23T18:45:52.791 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-04-23T18:45:52.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-04-23T18:45:52.809 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 91.5 2024-04-23T18:45:52.818 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:45:52.827 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:45:52.833 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:45:52.877 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:45:52.886 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:45:52.893 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:45:52.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:45:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:Error EPERM: 2024-04-23T18:45:52.923 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:49 smithi155 ceph-mon[33468]: pgmap v622: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:52.931 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:51 smithi155 ceph-mon[33468]: pgmap v623: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:52.938 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:49 smithi044 ceph-mon[32458]: pgmap v622: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:53.001 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:51 smithi044 ceph-mon[32458]: pgmap v623: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:53.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:47 smithi019 ceph-mon[29215]: pgmap v621: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:53.022 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:49 smithi019 ceph-mon[29215]: pgmap v622: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:53.030 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:51 smithi019 ceph-mon[29215]: pgmap v623: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:45:53.036 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-23T18:45:53.042 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-23T18:45:53.058 INFO:tasks.quiescer:joining Quiescers 2024-04-23T18:45:53.066 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-23T18:45:53.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:45:53.129 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't release set '46eff5eb' with rc: 1 (EPERM), stdout: { "epoch": 31, "leader": 24521, "set_version": 47, "sets": { "46eff5eb": { "version": 47, "age_ref": 216.2, "state": { "name": "EXPIRED", "age": 0.0 }, "timeout": 60.0, "expiration": 91.5, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCED", "age": 91.5 } } } } } } 2024-04-23T18:45:53.137 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 337, in _run self.do_quiesce(d) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 313, in do_quiesce raise RuntimeError(f"Error releasing set '{set_id}': {rcinfo}") RuntimeError: Error releasing set '46eff5eb': 1 (EPERM) 2024-04-23T18:45:53.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.269+0000 7f490d291640 1 -- 172.21.15.19:0/3302565269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4908107280 msgr2=0x7f490810ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.269+0000 7f490d291640 1 --2- 172.21.15.19:0/3302565269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4908107280 0x7f490810ee50 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f4900009ca0 tx=0x7f490002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:45:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.269+0000 7f490d291640 1 -- 172.21.15.19:0/3302565269 shutdown_connections 2024-04-23T18:45:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.269+0000 7f490d291640 1 --2- 172.21.15.19:0/3302565269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4908102bf0 0x7f4908106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.269+0000 7f490d291640 1 --2- 172.21.15.19:0/3302565269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4908107280 0x7f490810ee50 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.269+0000 7f490d291640 1 --2- 172.21.15.19:0/3302565269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4908102240 0x7f4908102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.269+0000 7f490d291640 1 -- 172.21.15.19:0/3302565269 >> 172.21.15.19:0/3302565269 conn(0x7f49080790e0 msgr2=0x7f49080794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.269+0000 7f490d291640 1 -- 172.21.15.19:0/3302565269 shutdown_connections 2024-04-23T18:45:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.269+0000 7f490d291640 1 -- 172.21.15.19:0/3302565269 wait complete. 2024-04-23T18:45:53.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f490d291640 1 Processor -- start 2024-04-23T18:45:53.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f490d291640 1 -- start start 2024-04-23T18:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f490d291640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4908102240 0x7f49081986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f490d291640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4908107280 0x7f4908198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f490d291640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490819cfa0 0x7f490819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f490d291640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49081121e0 con 0x7f490819cfa0 2024-04-23T18:45:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f490d291640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4908112060 con 0x7f4908107280 2024-04-23T18:45:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f490d291640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4908112360 con 0x7f4908102240 2024-04-23T18:45:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f49077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490819cfa0 0x7f490819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:53.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f49077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490819cfa0 0x7f490819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36022/0 (socket says 172.21.15.19:36022) 2024-04-23T18:45:53.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f49077fe640 1 -- 172.21.15.19:0/2167104877 learned_addr learned my addr 172.21.15.19:0/2167104877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f4906ffd640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4908102240 0x7f49081986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:53.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.270+0000 7f49067fc640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4908107280 0x7f4908198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:53.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f49077fe640 1 -- 172.21.15.19:0/2167104877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4908102240 msgr2=0x7f49081986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:53.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f49077fe640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4908102240 0x7f49081986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:53.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f49077fe640 1 -- 172.21.15.19:0/2167104877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4908107280 msgr2=0x7f4908198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:53.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f49077fe640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4908107280 0x7f4908198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:53.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f49077fe640 1 -- 172.21.15.19:0/2167104877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490819da90 con 0x7f490819cfa0 2024-04-23T18:45:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f49067fc640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4908107280 0x7f4908198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:53.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f49077fe640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490819cfa0 0x7f490819d450 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4900005cc0 tx=0x7f490002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:53.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f48effff640 1 -- 172.21.15.19:0/2167104877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900003bf0 con 0x7f490819cfa0 2024-04-23T18:45:53.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f490d291640 1 -- 172.21.15.19:0/2167104877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490819dd20 con 0x7f490819cfa0 2024-04-23T18:45:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f48effff640 1 -- 172.21.15.19:0/2167104877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4900003d90 con 0x7f490819cfa0 2024-04-23T18:45:53.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f48effff640 1 -- 172.21.15.19:0/2167104877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490002fbc0 con 0x7f490819cfa0 2024-04-23T18:45:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.271+0000 7f490d291640 1 -- 172.21.15.19:0/2167104877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49081a5540 con 0x7f490819cfa0 2024-04-23T18:45:53.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.272+0000 7f490d291640 1 -- 172.21.15.19:0/2167104877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49080691c0 con 0x7f490819cfa0 2024-04-23T18:45:53.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.274+0000 7f48effff640 1 -- 172.21.15.19:0/2167104877 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f490002fd60 con 0x7f490819cfa0 2024-04-23T18:45:53.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.275+0000 7f48effff640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f48f00a3d20 0x7f48f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.275+0000 7f4906ffd640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f48f00a3d20 0x7f48f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.275+0000 7f48effff640 1 -- 172.21.15.19:0/2167104877 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7f49000f2070 con 0x7f490819cfa0 2024-04-23T18:45:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.275+0000 7f4906ffd640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f48f00a3d20 0x7f48f00a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f48f80058c0 tx=0x7f48f800a4c0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:54.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.279+0000 7f48effff640 1 -- 172.21.15.19:0/2167104877 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49000be850 con 0x7f490819cfa0 2024-04-23T18:45:54.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.503+0000 7f490d291640 1 -- 172.21.15.19:0/2167104877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4908199a30 con 0x7f490819cfa0 2024-04-23T18:45:54.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.503+0000 7f48effff640 1 -- 172.21.15.19:0/2167104877 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11914 (secure 0 0 0) 0x7f4908199a30 con 0x7f490819cfa0 2024-04-23T18:45:54.021 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:45:54.029 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:45:17.914370+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26843":{"gid":26843,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/332625709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":332625709},{"type":"v1","addr":"172.21.15.44:6835","nonce":332625709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:45:54.036 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 31 2024-04-23T18:45:54.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.505+0000 7f490d291640 1 -- 172.21.15.19:0/2167104877 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f48f00a3d20 msgr2=0x7f48f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:54.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.505+0000 7f490d291640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f48f00a3d20 0x7f48f00a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f48f80058c0 tx=0x7f48f800a4c0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.505+0000 7f490d291640 1 -- 172.21.15.19:0/2167104877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490819cfa0 msgr2=0x7f490819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.505+0000 7f490d291640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490819cfa0 0x7f490819d450 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4900005cc0 tx=0x7f490002f380 comp rx=0 tx=0).stop 2024-04-23T18:45:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.505+0000 7f490d291640 1 -- 172.21.15.19:0/2167104877 shutdown_connections 2024-04-23T18:45:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.505+0000 7f490d291640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f48f00a3d20 0x7f48f00a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.505+0000 7f490d291640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4908107280 0x7f4908198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.505+0000 7f490d291640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490819cfa0 0x7f490819d450 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.505+0000 7f490d291640 1 --2- 172.21.15.19:0/2167104877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4908102240 0x7f49081986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.505+0000 7f490d291640 1 -- 172.21.15.19:0/2167104877 >> 172.21.15.19:0/2167104877 conn(0x7f49080790e0 msgr2=0x7f49081002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:54.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.506+0000 7f490d291640 1 -- 172.21.15.19:0/2167104877 shutdown_connections 2024-04-23T18:45:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:53.506+0000 7f490d291640 1 -- 172.21.15.19:0/2167104877 wait complete. 2024-04-23T18:45:54.233 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24521 quiesce db --cancel --all' 2024-04-23T18:45:54.242 DEBUG:teuthology.orchestra.run.smithi019:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24521 quiesce db --cancel --all 2024-04-23T18:45:54.258 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:53 smithi019 ceph-mon[29215]: pgmap v624: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:54.264 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:53 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2167104877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:45:54.311 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:53 smithi044 ceph-mon[32458]: pgmap v624: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:54.317 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:53 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2167104877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:45:54.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:53 smithi155 ceph-mon[33468]: pgmap v624: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:45:54.330 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:53 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2167104877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:45:54.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.378+0000 7fcc75c9e640 1 -- 172.21.15.19:0/1451533035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc70101a40 msgr2=0x7fcc70105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:54.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.378+0000 7fcc75c9e640 1 --2- 172.21.15.19:0/1451533035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc70101a40 0x7fcc70105a80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009900 tx=0x7fcc6402a410 comp rx=0 tx=0).stop 2024-04-23T18:45:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.378+0000 7fcc75c9e640 1 -- 172.21.15.19:0/1451533035 shutdown_connections 2024-04-23T18:45:54.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.378+0000 7fcc75c9e640 1 --2- 172.21.15.19:0/1451533035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc70106160 0x7fcc7010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.378+0000 7fcc75c9e640 1 --2- 172.21.15.19:0/1451533035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc70101a40 0x7fcc70105a80 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.378+0000 7fcc75c9e640 1 --2- 172.21.15.19:0/1451533035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc70101090 0x7fcc70101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.378+0000 7fcc75c9e640 1 -- 172.21.15.19:0/1451533035 >> 172.21.15.19:0/1451533035 conn(0x7fcc70077f50 msgr2=0x7fcc70078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.378+0000 7fcc75c9e640 1 -- 172.21.15.19:0/1451533035 shutdown_connections 2024-04-23T18:45:54.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.378+0000 7fcc75c9e640 1 -- 172.21.15.19:0/1451533035 wait complete. 2024-04-23T18:45:54.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.378+0000 7fcc75c9e640 1 Processor -- start 2024-04-23T18:45:54.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc75c9e640 1 -- start start 2024-04-23T18:45:54.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc75c9e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc70101090 0x7fcc7019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:54.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc75c9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc70106160 0x7fcc7019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:54.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc75c9e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc70198590 0x7fcc70198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:54.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc75c9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701110d0 con 0x7fcc70106160 2024-04-23T18:45:54.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc75c9e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70110f50 con 0x7fcc70101090 2024-04-23T18:45:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc75c9e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70111250 con 0x7fcc70198590 2024-04-23T18:45:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc6effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc70106160 0x7fcc7019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc6effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc70106160 0x7fcc7019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36034/0 (socket says 172.21.15.19:36034) 2024-04-23T18:45:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc6effd640 1 -- 172.21.15.19:0/8411823 learned_addr learned my addr 172.21.15.19:0/8411823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc70101090 0x7fcc7019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc6ffff640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc70198590 0x7fcc70198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:54.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc6effd640 1 -- 172.21.15.19:0/8411823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc70198590 msgr2=0x7fcc70198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc6effd640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc70198590 0x7fcc70198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc6effd640 1 -- 172.21.15.19:0/8411823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc70101090 msgr2=0x7fcc7019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:54.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc6effd640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc70101090 0x7fcc7019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.379+0000 7fcc6effd640 1 -- 172.21.15.19:0/8411823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc701990b0 con 0x7fcc70106160 2024-04-23T18:45:54.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.380+0000 7fcc6effd640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc70106160 0x7fcc7019e920 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005ab0 tx=0x7fcc64003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:54.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.380+0000 7fcc6cff9640 1 -- 172.21.15.19:0/8411823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6403f070 con 0x7fcc70106160 2024-04-23T18:45:54.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.380+0000 7fcc75c9e640 1 -- 172.21.15.19:0/8411823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc70199340 con 0x7fcc70106160 2024-04-23T18:45:54.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.380+0000 7fcc6cff9640 1 -- 172.21.15.19:0/8411823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc64031d60 con 0x7fcc70106160 2024-04-23T18:45:54.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.380+0000 7fcc6cff9640 1 -- 172.21.15.19:0/8411823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64035030 con 0x7fcc70106160 2024-04-23T18:45:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.380+0000 7fcc75c9e640 1 -- 172.21.15.19:0/8411823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc701a5540 con 0x7fcc70106160 2024-04-23T18:45:54.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.382+0000 7fcc6cff9640 1 -- 172.21.15.19:0/8411823 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcc64042070 con 0x7fcc70106160 2024-04-23T18:45:54.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.382+0000 7fcc789b0740 1 Processor -- start 2024-04-23T18:45:54.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.382+0000 7fcc789b0740 1 -- start start 2024-04-23T18:45:54.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e943740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:54.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e943d10 0x556d5e94ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:54.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e94e380 0x556d5e9507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc789b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556d5e8a9130 con 0x556d5e943360 2024-04-23T18:45:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc789b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556d5e845c50 con 0x556d5e943d10 2024-04-23T18:45:54.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc789b0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556d5e8a50f0 con 0x556d5e94e380 2024-04-23T18:45:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e943740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:55.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6cff9640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc580a3d20 0x7fcc580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e943740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36046/0 (socket says 172.21.15.19:36046) 2024-04-23T18:45:55.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6ffff640 1 -- 172.21.15.19:0/2981685964 learned_addr learned my addr 172.21.15.19:0/2981685964 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:55.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/2981685964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e943d10 0x556d5e94ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:55.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6ffff640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc580a3d20 0x7fcc580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6cff9640 1 -- 172.21.15.19:0/8411823 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7fcc640f10c0 con 0x7fcc70106160 2024-04-23T18:45:55.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6effd640 1 --2- 172.21.15.19:0/2981685964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e94e380 0x556d5e9507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:55.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6ffff640 1 -- 172.21.15.19:0/2981685964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e94e380 msgr2=0x556d5e9507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6ffff640 1 --2- 172.21.15.19:0/2981685964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e94e380 0x556d5e9507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6ffff640 1 -- 172.21.15.19:0/2981685964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e943d10 msgr2=0x556d5e94ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:55.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6ffff640 1 --2- 172.21.15.19:0/2981685964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e943d10 0x556d5e94ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6ffff640 1 -- 172.21.15.19:0/2981685964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d5e85ffd0 con 0x556d5e943360 2024-04-23T18:45:55.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6effd640 1 --2- 172.21.15.19:0/2981685964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e94e380 0x556d5e9507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.383+0000 7fcc6ffff640 1 --2- 172.21.15.19:0/2981685964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e943740 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fcc680139f0 tx=0x7fcc68013eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3720c140b25cdba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc55ffb640 1 -- 172.21.15.19:0/2981685964 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc680134c0 con 0x556d5e943360 2024-04-23T18:45:55.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc55ffb640 1 -- 172.21.15.19:0/2981685964 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc68022070 con 0x556d5e943360 2024-04-23T18:45:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc55ffb640 1 -- 172.21.15.19:0/2981685964 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6801cbf0 con 0x556d5e943360 2024-04-23T18:45:55.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc789b0740 1 -- 172.21.15.19:0/2981685964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 msgr2=0x556d5e943740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:55.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc6ffff640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc580a3d20 0x7fcc580a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc68021000 tx=0x7fcc6801f000 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:55.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc789b0740 1 --2- 172.21.15.19:0/2981685964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e943740 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fcc680139f0 tx=0x7fcc68013eb0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc789b0740 1 -- 172.21.15.19:0/2981685964 shutdown_connections 2024-04-23T18:45:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc789b0740 1 --2- 172.21.15.19:0/2981685964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e943d10 0x556d5e94ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc789b0740 1 --2- 172.21.15.19:0/2981685964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e94e380 0x556d5e9507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc789b0740 1 --2- 172.21.15.19:0/2981685964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e943740 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc789b0740 1 -- 172.21.15.19:0/2981685964 >> 172.21.15.19:0/2981685964 conn(0x556d5e93f620 msgr2=0x556d5e93fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc789b0740 1 -- 172.21.15.19:0/2981685964 shutdown_connections 2024-04-23T18:45:55.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.384+0000 7fcc789b0740 1 -- 172.21.15.19:0/2981685964 wait complete. 2024-04-23T18:45:55.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc789b0740 1 -- start start 2024-04-23T18:45:55.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc789b0740 1 Processor -- start 2024-04-23T18:45:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e94b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:55.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943d10 0x556d5e94b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e94bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc789b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556d5e9512f0 con 0x556d5e943360 2024-04-23T18:45:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc789b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556d5e843640 con 0x556d5e94e380 2024-04-23T18:45:55.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc789b0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556d5e951470 con 0x556d5e943d10 2024-04-23T18:45:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc789b0740 10 client.0 _ll_register_callbacks cb 0x556d5e843030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc6f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e94b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc6f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e94b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36062/0 (socket says 172.21.15.19:36062) 2024-04-23T18:45:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.385+0000 7fcc6f7fe640 1 -- 172.21.15.19:0/4249862223 learned_addr learned my addr 172.21.15.19:0/4249862223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:55.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc6effd640 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943d10 0x556d5e94b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc6ffff640 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e94bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:55.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc6f7fe640 1 -- 172.21.15.19:0/4249862223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943d10 msgr2=0x556d5e94b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943d10 0x556d5e94b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc6f7fe640 1 -- 172.21.15.19:0/4249862223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 msgr2=0x556d5e94bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:55.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e94bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc6f7fe640 1 -- 172.21.15.19:0/4249862223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d5e687a20 con 0x556d5e943360 2024-04-23T18:45:55.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e94b2b0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000be80 tx=0x7fcc60012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377f3bc3265d0d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:55.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc4effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:55.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc4effd640 1 -- 172.21.15.19:0/4249862223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6001f060 con 0x556d5e943360 2024-04-23T18:45:55.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc4effd640 1 -- 172.21.15.19:0/4249862223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7fcc600108a0 con 0x556d5e943360 2024-04-23T18:45:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc789b0740 1 -- 172.21.15.19:0/4249862223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc68022070 con 0x556d5e943360 2024-04-23T18:45:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc4effd640 1 -- 172.21.15.19:0/4249862223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcc60010aa0 con 0x556d5e943360 2024-04-23T18:45:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc789b0740 10 client.17835 fetch_fsmap learned FSMap version 31 2024-04-23T18:45:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc789b0740 1 -- 172.21.15.19:0/4249862223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556d5e6e81b0 con 0x556d5e943360 2024-04-23T18:45:55.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.386+0000 7fcc4effd640 1 -- 172.21.15.19:0/4249862223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc68022070 con 0x556d5e943360 2024-04-23T18:45:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.387+0000 7fcc4effd640 1 -- 172.21.15.19:0/4249862223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fcc60019030 con 0x556d5e943360 2024-04-23T18:45:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.387+0000 7fcc789b0740 10 client.17835 fetch_fsmap finished waiting for FSMap version 31 2024-04-23T18:45:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.387+0000 7fcc789b0740 10 client.17835 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:45:55.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.387+0000 7fcc789b0740 20 client.17835 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.387+0000 7fcc789b0740 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556d5e95e880 0x556d5e960cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.387+0000 7fcc789b0740 4 client.17835 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:55.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.387+0000 7fcc789b0740 1 -- 172.21.15.19:0/4249862223 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d5e8b6b50 con 0x556d5e95e880 2024-04-23T18:45:55.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.387+0000 7fcc6effd640 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556d5e95e880 0x556d5e960cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.388+0000 7fcc4effd640 10 client.17835 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:55.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.388+0000 7fcc6effd640 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556d5e95e880 0x556d5e960cd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41728f728aba670 server_cookie=b2de700a87a48733 in_seq=0 out_seq=0 2024-04-23T18:45:55.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.388+0000 7fcc4effd640 1 -- 172.21.15.19:0/4249862223 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d5e8b6b50 con 0x556d5e95e880 2024-04-23T18:45:55.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.388+0000 7fcc4effd640 10 client.17835 handle_command_reply: tid=0 2024-04-23T18:45:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.388+0000 7fcc789b0740 1 client.17835 shutdown 2024-04-23T18:45:55.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.388+0000 7fcc789b0740 1 -- 172.21.15.19:0/4249862223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 msgr2=0x556d5e94b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.388+0000 7fcc789b0740 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e94b2b0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000be80 tx=0x7fcc60012820 comp rx=0 tx=0).stop 2024-04-23T18:45:55.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.389+0000 7fcc789b0740 1 -- 172.21.15.19:0/4249862223 shutdown_connections 2024-04-23T18:45:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.389+0000 7fcc789b0740 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556d5e95e880 0x556d5e960cd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.389+0000 7fcc789b0740 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e94bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.389+0000 7fcc789b0740 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943d10 0x556d5e94b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.389+0000 7fcc789b0740 1 --2- 172.21.15.19:0/4249862223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e94b2b0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.389+0000 7fcc789b0740 1 -- 172.21.15.19:0/4249862223 >> 172.21.15.19:0/4249862223 conn(0x556d5e93f620 msgr2=0x556d5e93e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.389+0000 7fcc4effd640 0 client.17835 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:56.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.389+0000 7fcc789b0740 1 -- 172.21.15.19:0/4249862223 shutdown_connections 2024-04-23T18:45:56.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.389+0000 7fcc789b0740 1 -- 172.21.15.19:0/4249862223 wait complete. 2024-04-23T18:45:56.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.389+0000 7fcc789b0740 20 client.17835 trim_cache size 0 max 16384 2024-04-23T18:45:56.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.406+0000 7fcc789b0740 1 Processor -- start 2024-04-23T18:45:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.406+0000 7fcc789b0740 1 -- start start 2024-04-23T18:45:56.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.406+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e93e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:56.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.406+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943d10 0x556d5e949ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:56.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.406+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e945a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.406+0000 7fcc789b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701adb90 con 0x556d5e943360 2024-04-23T18:45:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.406+0000 7fcc789b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701af6d0 con 0x556d5e94e380 2024-04-23T18:45:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.406+0000 7fcc789b0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701ae260 con 0x556d5e943d10 2024-04-23T18:45:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc6f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e93e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc6f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e93e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36070/0 (socket says 172.21.15.19:36070) 2024-04-23T18:45:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc6f7fe640 1 -- 172.21.15.19:0/535953021 learned_addr learned my addr 172.21.15.19:0/535953021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:56.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc6effd640 1 --2- 172.21.15.19:0/535953021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943d10 0x556d5e949ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc6f7fe640 1 -- 172.21.15.19:0/535953021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943d10 msgr2=0x556d5e949ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/535953021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943d10 0x556d5e949ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:56.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc6f7fe640 1 -- 172.21.15.19:0/535953021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 msgr2=0x556d5e945a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/535953021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e945a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc6f7fe640 1 -- 172.21.15.19:0/535953021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d5e57b310 con 0x556d5e943360 2024-04-23T18:45:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/535953021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e93e720 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fcc60005fa0 tx=0x7fcc60005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50dd46449b14766 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:56.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc557fa640 1 -- 172.21.15.19:0/535953021 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6001f020 con 0x556d5e943360 2024-04-23T18:45:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc557fa640 1 -- 172.21.15.19:0/535953021 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc60010c40 con 0x556d5e943360 2024-04-23T18:45:56.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.407+0000 7fcc557fa640 1 -- 172.21.15.19:0/535953021 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc600224f0 con 0x556d5e943360 2024-04-23T18:45:56.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.408+0000 7fcc789b0740 1 -- 172.21.15.19:0/535953021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 msgr2=0x556d5e93e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.408+0000 7fcc789b0740 1 --2- 172.21.15.19:0/535953021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e93e720 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fcc60005fa0 tx=0x7fcc60005710 comp rx=0 tx=0).stop 2024-04-23T18:45:56.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.408+0000 7fcc789b0740 1 -- 172.21.15.19:0/535953021 shutdown_connections 2024-04-23T18:45:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.408+0000 7fcc789b0740 1 --2- 172.21.15.19:0/535953021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e945a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:56.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.408+0000 7fcc789b0740 1 --2- 172.21.15.19:0/535953021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943d10 0x556d5e949ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.408+0000 7fcc789b0740 1 --2- 172.21.15.19:0/535953021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943360 0x556d5e93e720 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:56.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.408+0000 7fcc789b0740 1 -- 172.21.15.19:0/535953021 >> 172.21.15.19:0/535953021 conn(0x556d5e93f620 msgr2=0x556d5e94bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.408+0000 7fcc789b0740 1 -- 172.21.15.19:0/535953021 shutdown_connections 2024-04-23T18:45:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.408+0000 7fcc789b0740 1 -- 172.21.15.19:0/535953021 wait complete. 2024-04-23T18:45:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.408+0000 7fcc789b0740 1 -- start start 2024-04-23T18:45:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc789b0740 1 Processor -- start 2024-04-23T18:45:56.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943360 0x556d5e9566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:56.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943d10 0x556d5e956c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc789b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e953690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:56.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc789b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701af000 con 0x556d5e943d10 2024-04-23T18:45:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc789b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701ae930 con 0x556d5e94e380 2024-04-23T18:45:56.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc789b0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701abb90 con 0x556d5e943360 2024-04-23T18:45:56.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc789b0740 10 client.0 _ll_register_callbacks cb 0x7fcc6000ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:56.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943d10 0x556d5e956c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:56.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943d10 0x556d5e956c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36072/0 (socket says 172.21.15.19:36072) 2024-04-23T18:45:56.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6effd640 1 -- 172.21.15.19:0/2824675104 learned_addr learned my addr 172.21.15.19:0/2824675104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:56.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6ffff640 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e953690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:56.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943360 0x556d5e9566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6effd640 1 -- 172.21.15.19:0/2824675104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943360 msgr2=0x556d5e9566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:56.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6effd640 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943360 0x556d5e9566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:56.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6effd640 1 -- 172.21.15.19:0/2824675104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 msgr2=0x556d5e953690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6effd640 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e953690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6effd640 1 -- 172.21.15.19:0/2824675104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d5e953c80 con 0x556d5e943d10 2024-04-23T18:45:56.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6ffff640 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e953690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:56.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc6effd640 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943d10 0x556d5e956c00 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fcc640ee8a0 tx=0x7fcc6402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d2978f406ea14d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:56.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.409+0000 7fcc4effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-23T18:45:56.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc4effd640 1 -- 172.21.15.19:0/2824675104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64049070 con 0x556d5e943d10 2024-04-23T18:45:56.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc4effd640 1 -- 172.21.15.19:0/2824675104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7fcc640eeac0 con 0x556d5e943d10 2024-04-23T18:45:56.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc789b0740 1 -- 172.21.15.19:0/2824675104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc60010c40 con 0x556d5e943d10 2024-04-23T18:45:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc4effd640 1 -- 172.21.15.19:0/2824675104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcc640c1440 con 0x556d5e943d10 2024-04-23T18:45:56.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc789b0740 10 client.17841 fetch_fsmap learned FSMap version 31 2024-04-23T18:45:56.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc789b0740 1 -- 172.21.15.19:0/2824675104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556d5e9c5760 con 0x556d5e943d10 2024-04-23T18:45:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc4effd640 1 -- 172.21.15.19:0/2824675104 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc60010c40 con 0x556d5e943d10 2024-04-23T18:45:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc4effd640 1 -- 172.21.15.19:0/2824675104 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fcc6403b4f0 con 0x556d5e943d10 2024-04-23T18:45:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc789b0740 10 client.17841 fetch_fsmap finished waiting for FSMap version 31 2024-04-23T18:45:56.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc789b0740 10 client.17841 resolve_mds: validated gid 24521 aka daemon mds.b 2024-04-23T18:45:56.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc789b0740 20 client.17841 populate_metadata read hostname 'smithi019' 2024-04-23T18:45:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc789b0740 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556d5e9c5e40 0x556d5e9c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc789b0740 4 client.17841 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-23T18:45:56.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc789b0740 1 -- 172.21.15.19:0/2824675104 --> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x556d5e8b09c0 con 0x556d5e9c5e40 2024-04-23T18:45:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.410+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556d5e9c5e40 0x556d5e9c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc4effd640 10 client.17841 ms_handle_connect on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc6f7fe640 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556d5e9c5e40 0x556d5e9c8260 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67e24186cf254ad server_cookie=832169f780e49a18 in_seq=0 out_seq=0 2024-04-23T18:45:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc4effd640 1 -- 172.21.15.19:0/2824675104 <== mds.0 v2:172.21.15.44:6838/2221834529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x556d5e8b09c0 con 0x556d5e9c5e40 2024-04-23T18:45:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc4effd640 10 client.17841 handle_command_reply: tid=0 2024-04-23T18:45:57.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 client.17841 shutdown 2024-04-23T18:45:57.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 -- 172.21.15.19:0/2824675104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943d10 msgr2=0x556d5e956c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:57.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943d10 0x556d5e956c00 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fcc640ee8a0 tx=0x7fcc6402a410 comp rx=0 tx=0).stop 2024-04-23T18:45:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 -- 172.21.15.19:0/2824675104 shutdown_connections 2024-04-23T18:45:57.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.44:6838/2221834529,v1:172.21.15.44:6839/2221834529] conn(0x556d5e9c5e40 0x556d5e9c8260 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d5e94e380 0x556d5e953690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc4effd640 0 client.17841 ms_handle_reset on v2:172.21.15.44:6838/2221834529 2024-04-23T18:45:57.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d5e943d10 0x556d5e956c00 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 --2- 172.21.15.19:0/2824675104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d5e943360 0x556d5e9566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 -- 172.21.15.19:0/2824675104 >> 172.21.15.19:0/2824675104 conn(0x556d5e93f620 msgr2=0x556d5e94cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 -- 172.21.15.19:0/2824675104 shutdown_connections 2024-04-23T18:45:57.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 1 -- 172.21.15.19:0/2824675104 wait complete. 2024-04-23T18:45:57.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.411+0000 7fcc789b0740 20 client.17841 trim_cache size 0 max 16384 2024-04-23T18:45:57.246 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:45:57.253 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 31, 2024-04-23T18:45:57.262 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24521, 2024-04-23T18:45:57.271 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 47, 2024-04-23T18:45:57.339 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": {} 2024-04-23T18:45:57.350 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:45:57.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 -- 172.21.15.19:0/8411823 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc580a3d20 msgr2=0x7fcc580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc580a3d20 0x7fcc580a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc68021000 tx=0x7fcc6801f000 comp rx=0 tx=0).stop 2024-04-23T18:45:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 -- 172.21.15.19:0/8411823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc70106160 msgr2=0x7fcc7019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc70106160 0x7fcc7019e920 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005ab0 tx=0x7fcc64003cd0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 -- 172.21.15.19:0/8411823 shutdown_connections 2024-04-23T18:45:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fcc580a3d20 0x7fcc580a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc70198590 0x7fcc70198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc70106160 0x7fcc7019e920 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 --2- 172.21.15.19:0/8411823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc70101090 0x7fcc7019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 -- 172.21.15.19:0/8411823 >> 172.21.15.19:0/8411823 conn(0x7fcc70077f50 msgr2=0x7fcc70100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:57.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 -- 172.21.15.19:0/8411823 shutdown_connections 2024-04-23T18:45:57.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:54.412+0000 7fcc75c9e640 1 -- 172.21.15.19:0/8411823 wait complete. 2024-04-23T18:45:57.471 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 408, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 361, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error releasing set '46eff5eb': 1 (EPERM) 2024-04-23T18:45:57.479 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-23T18:45:57.496 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:45:57.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:55 smithi019 ceph-mon[29215]: pgmap v625: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T18:45:57.591 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:55 smithi155 ceph-mon[33468]: pgmap v625: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T18:45:57.597 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:55 smithi044 ceph-mon[32458]: pgmap v625: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T18:45:57.605 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-23T18:45:57.622 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-23T18:45:57.630 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:45:57.636 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:45:57.642 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-23T18:45:57.681 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-23T18:45:57.752 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T18:45:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.909+0000 7fc7306a0640 1 -- 172.21.15.19:0/1248400042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7281072a0 msgr2=0x7fc72810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:57.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.909+0000 7fc7306a0640 1 --2- 172.21.15.19:0/1248400042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7281072a0 0x7fc72810ee70 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc720009ca0 tx=0x7fc72002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.909+0000 7fc7306a0640 1 -- 172.21.15.19:0/1248400042 shutdown_connections 2024-04-23T18:45:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.909+0000 7fc7306a0640 1 --2- 172.21.15.19:0/1248400042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc728102b80 0x7fc728106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.909+0000 7fc7306a0640 1 --2- 172.21.15.19:0/1248400042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7281072a0 0x7fc72810ee70 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.909+0000 7fc7306a0640 1 --2- 172.21.15.19:0/1248400042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7281021d0 0x7fc7281025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.909+0000 7fc7306a0640 1 -- 172.21.15.19:0/1248400042 >> 172.21.15.19:0/1248400042 conn(0x7fc7280793e0 msgr2=0x7fc7280797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:58.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.909+0000 7fc7306a0640 1 -- 172.21.15.19:0/1248400042 shutdown_connections 2024-04-23T18:45:58.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.909+0000 7fc7306a0640 1 -- 172.21.15.19:0/1248400042 wait complete. 2024-04-23T18:45:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc7306a0640 1 Processor -- start 2024-04-23T18:45:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc7306a0640 1 -- start start 2024-04-23T18:45:58.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc7306a0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7281021d0 0x7fc7281a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:58.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc7306a0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7281072a0 0x7fc7281a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc7306a0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc72819a670 0x7fc72819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:58.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc7306a0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc728112210 con 0x7fc72819a670 2024-04-23T18:45:58.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc7306a0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc728112090 con 0x7fc7281021d0 2024-04-23T18:45:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc7306a0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc728112390 con 0x7fc7281072a0 2024-04-23T18:45:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc72ec3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc72819a670 0x7fc72819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:58.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc72ec3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc72819a670 0x7fc72819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36080/0 (socket says 172.21.15.19:36080) 2024-04-23T18:45:58.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.910+0000 7fc72ec3d640 1 -- 172.21.15.19:0/446886389 learned_addr learned my addr 172.21.15.19:0/446886389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:58.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc72dc3b640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7281072a0 0x7fc7281a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc72e43c640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7281021d0 0x7fc7281a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc72ec3d640 1 -- 172.21.15.19:0/446886389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7281072a0 msgr2=0x7fc7281a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc72ec3d640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7281072a0 0x7fc7281a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:58.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc72ec3d640 1 -- 172.21.15.19:0/446886389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7281021d0 msgr2=0x7fc7281a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc72ec3d640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7281021d0 0x7fc7281a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc72ec3d640 1 -- 172.21.15.19:0/446886389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72819b270 con 0x7fc72819a670 2024-04-23T18:45:58.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc72e43c640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7281021d0 0x7fc7281a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:58.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc72ec3d640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc72819a670 0x7fc72819aaf0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc7200098d0 tx=0x7fc7200317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc71f7fe640 1 -- 172.21.15.19:0/446886389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720003b70 con 0x7fc72819a670 2024-04-23T18:45:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc71f7fe640 1 -- 172.21.15.19:0/446886389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc720003d10 con 0x7fc72819a670 2024-04-23T18:45:58.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc7306a0640 1 -- 172.21.15.19:0/446886389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72819b500 con 0x7fc72819a670 2024-04-23T18:45:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc71f7fe640 1 -- 172.21.15.19:0/446886389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72002fa20 con 0x7fc72819a670 2024-04-23T18:45:58.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.911+0000 7fc7306a0640 1 -- 172.21.15.19:0/446886389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7281a76a0 con 0x7fc72819a670 2024-04-23T18:45:58.447 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:57 smithi155 ceph-mon[33468]: pgmap v626: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:45:58.510 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:57 smithi044 ceph-mon[32458]: pgmap v626: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:45:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.912+0000 7fc7306a0640 1 -- 172.21.15.19:0/446886389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7281036a0 con 0x7fc72819a670 2024-04-23T18:45:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.914+0000 7fc71f7fe640 1 -- 172.21.15.19:0/446886389 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc72002fc20 con 0x7fc72819a670 2024-04-23T18:45:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.915+0000 7fc71f7fe640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7140a3dc0 0x7fc7140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.915+0000 7fc71f7fe640 1 -- 172.21.15.19:0/446886389 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7fc7200f1e70 con 0x7fc72819a670 2024-04-23T18:45:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.915+0000 7fc72e43c640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7140a3dc0 0x7fc7140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:58.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.916+0000 7fc72e43c640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7140a3dc0 0x7fc7140a6280 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc724009b40 tx=0x7fc724005e50 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:57.919+0000 7fc71f7fe640 1 -- 172.21.15.19:0/446886389 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7200be5f0 con 0x7fc72819a670 2024-04-23T18:45:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.131+0000 7fc7306a0640 1 -- 172.21.15.19:0/446886389 --> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fc728102c90 con 0x7fc7140a3dc0 2024-04-23T18:45:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.139+0000 7fc71f7fe640 1 -- 172.21.15.19:0/446886389 <== mgr.25004 v2:172.21.15.19:6800/1177101685 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fc728102c90 con 0x7fc7140a3dc0 2024-04-23T18:45:58.652 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T18:26:09", "first": "2024-04-23T18:27:00", "last": "2024-04-23T18:45:00", "last_pruned": "2024-04-23T18:45:00", "created_count": 19, "pruned_count": 13, "active": true} 2024-04-23T18:45:58.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 -- 172.21.15.19:0/446886389 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7140a3dc0 msgr2=0x7fc7140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:58.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7140a3dc0 0x7fc7140a6280 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc724009b40 tx=0x7fc724005e50 comp rx=0 tx=0).stop 2024-04-23T18:45:58.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 -- 172.21.15.19:0/446886389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc72819a670 msgr2=0x7fc72819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc72819a670 0x7fc72819aaf0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc7200098d0 tx=0x7fc7200317b0 comp rx=0 tx=0).stop 2024-04-23T18:45:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 -- 172.21.15.19:0/446886389 shutdown_connections 2024-04-23T18:45:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc7140a3dc0 0x7fc7140a6280 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc72819a670 0x7fc72819aaf0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7281072a0 0x7fc7281a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 --2- 172.21.15.19:0/446886389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7281021d0 0x7fc7281a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 -- 172.21.15.19:0/446886389 >> 172.21.15.19:0/446886389 conn(0x7fc7280793e0 msgr2=0x7fc7280fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:58.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 -- 172.21.15.19:0/446886389 shutdown_connections 2024-04-23T18:45:58.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:58.141+0000 7fc7306a0640 1 -- 172.21.15.19:0/446886389 wait complete. 2024-04-23T18:45:58.903 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T18:45:58.910 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:57 smithi019 ceph-mon[29215]: pgmap v626: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:45:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.042+0000 7efdadfdb640 1 -- 172.21.15.19:0/1308043491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda81011a0 msgr2=0x7efda8101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:59.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.042+0000 7efdadfdb640 1 --2- 172.21.15.19:0/1308043491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda81011a0 0x7efda8101600 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c02a400 comp rx=0 tx=0).stop 2024-04-23T18:45:59.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.042+0000 7efdadfdb640 1 -- 172.21.15.19:0/1308043491 shutdown_connections 2024-04-23T18:45:59.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.042+0000 7efdadfdb640 1 --2- 172.21.15.19:0/1308043491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8101b40 0x7efda810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.042+0000 7efdadfdb640 1 --2- 172.21.15.19:0/1308043491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda81011a0 0x7efda8101600 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.042+0000 7efdadfdb640 1 --2- 172.21.15.19:0/1308043491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efda8105e20 0x7efda8106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.042+0000 7efdadfdb640 1 -- 172.21.15.19:0/1308043491 >> 172.21.15.19:0/1308043491 conn(0x7efda8076c50 msgr2=0x7efda8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:59.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.042+0000 7efdadfdb640 1 -- 172.21.15.19:0/1308043491 shutdown_connections 2024-04-23T18:45:59.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.042+0000 7efdadfdb640 1 -- 172.21.15.19:0/1308043491 wait complete. 2024-04-23T18:45:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.043+0000 7efdadfdb640 1 Processor -- start 2024-04-23T18:45:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.043+0000 7efdadfdb640 1 -- start start 2024-04-23T18:45:59.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.043+0000 7efdadfdb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efda81011a0 0x7efda8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.043+0000 7efdadfdb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8101b40 0x7efda8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:59.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.043+0000 7efdadfdb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda8105e20 0x7efda819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:59.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.043+0000 7efdadfdb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efda81122b0 con 0x7efda8105e20 2024-04-23T18:45:59.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.043+0000 7efdadfdb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efda8112130 con 0x7efda81011a0 2024-04-23T18:45:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.043+0000 7efdadfdb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efda8112430 con 0x7efda8101b40 2024-04-23T18:45:59.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda8105e20 0x7efda819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda8105e20 0x7efda819cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36096/0 (socket says 172.21.15.19:36096) 2024-04-23T18:45:59.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda7fff640 1 -- 172.21.15.19:0/4189655565 learned_addr learned my addr 172.21.15.19:0/4189655565 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:45:59.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda77fe640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efda81011a0 0x7efda8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:59.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda6ffd640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8101b40 0x7efda8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:59.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda7fff640 1 -- 172.21.15.19:0/4189655565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8101b40 msgr2=0x7efda8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda7fff640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8101b40 0x7efda8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda7fff640 1 -- 172.21.15.19:0/4189655565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efda81011a0 msgr2=0x7efda8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:59.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda7fff640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efda81011a0 0x7efda8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda7fff640 1 -- 172.21.15.19:0/4189655565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda819d720 con 0x7efda8105e20 2024-04-23T18:45:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda7fff640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda8105e20 0x7efda819cff0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7efda000e960 tx=0x7efda000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda4ff9640 1 -- 172.21.15.19:0/4189655565 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0004d20 con 0x7efda8105e20 2024-04-23T18:45:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda4ff9640 1 -- 172.21.15.19:0/4189655565 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda0005020 con 0x7efda8105e20 2024-04-23T18:45:59.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efdadfdb640 1 -- 172.21.15.19:0/4189655565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda819da10 con 0x7efda8105e20 2024-04-23T18:45:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efda4ff9640 1 -- 172.21.15.19:0/4189655565 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda00116d0 con 0x7efda8105e20 2024-04-23T18:45:59.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.044+0000 7efdadfdb640 1 -- 172.21.15.19:0/4189655565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda81a52a0 con 0x7efda8105e20 2024-04-23T18:45:59.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.045+0000 7efdadfdb640 1 -- 172.21.15.19:0/4189655565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda8102570 con 0x7efda8105e20 2024-04-23T18:45:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.047+0000 7efda4ff9640 1 -- 172.21.15.19:0/4189655565 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efda00051d0 con 0x7efda8105e20 2024-04-23T18:45:59.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.047+0000 7efda4ff9640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efd900a3dc0 0x7efd900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.048+0000 7efda77fe640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efd900a3dc0 0x7efd900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.048+0000 7efda4ff9640 1 -- 172.21.15.19:0/4189655565 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7efda00d0c60 con 0x7efda8105e20 2024-04-23T18:45:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.048+0000 7efda77fe640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efd900a3dc0 0x7efd900a6280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efd9800bbe0 tx=0x7efd9800b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.052+0000 7efda4ff9640 1 -- 172.21.15.19:0/4189655565 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efda0016030 con 0x7efda8105e20 2024-04-23T18:45:59.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.262+0000 7efdadfdb640 1 -- 172.21.15.19:0/4189655565 --> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7efda81093a0 con 0x7efd900a3dc0 2024-04-23T18:45:59.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.270+0000 7efda4ff9640 1 -- 172.21.15.19:0/4189655565 <== mgr.25004 v2:172.21.15.19:6800/1177101685 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7efd98010c30 con 0x7efd900a3dc0 2024-04-23T18:45:59.649 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-04-23T18:45:59.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 -- 172.21.15.19:0/4189655565 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efd900a3dc0 msgr2=0x7efd900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:59.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efd900a3dc0 0x7efd900a6280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efd9800bbe0 tx=0x7efd9800b390 comp rx=0 tx=0).stop 2024-04-23T18:45:59.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 -- 172.21.15.19:0/4189655565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda8105e20 msgr2=0x7efda819cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda8105e20 0x7efda819cff0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7efda000e960 tx=0x7efda000ee20 comp rx=0 tx=0).stop 2024-04-23T18:45:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 -- 172.21.15.19:0/4189655565 shutdown_connections 2024-04-23T18:45:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7efd900a3dc0 0x7efd900a6280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda8105e20 0x7efda819cff0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8101b40 0x7efda8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 --2- 172.21.15.19:0/4189655565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efda81011a0 0x7efda8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 -- 172.21.15.19:0/4189655565 >> 172.21.15.19:0/4189655565 conn(0x7efda8076c50 msgr2=0x7efda80feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 -- 172.21.15.19:0/4189655565 shutdown_connections 2024-04-23T18:45:59.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.272+0000 7efdadfdb640 1 -- 172.21.15.19:0/4189655565 wait complete. 2024-04-23T18:45:59.843 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-23T18:45:59.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:45:59.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.989+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2825329606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac106190 msgr2=0x7fc0ac10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:59.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.989+0000 7fc0b37ad640 1 --2- 172.21.15.19:0/2825329606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac106190 0x7fc0ac10dd60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009ca0 tx=0x7fc0a402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.989+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2825329606 shutdown_connections 2024-04-23T18:46:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.989+0000 7fc0b37ad640 1 --2- 172.21.15.19:0/2825329606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac101ac0 0x7fc0ac105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:00.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.989+0000 7fc0b37ad640 1 --2- 172.21.15.19:0/2825329606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac106190 0x7fc0ac10dd60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.989+0000 7fc0b37ad640 1 --2- 172.21.15.19:0/2825329606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ac101110 0x7fc0ac1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:00.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.989+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2825329606 >> 172.21.15.19:0/2825329606 conn(0x7fc0ac077fb0 msgr2=0x7fc0ac0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.989+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2825329606 shutdown_connections 2024-04-23T18:46:00.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.989+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2825329606 wait complete. 2024-04-23T18:46:00.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b37ad640 1 Processor -- start 2024-04-23T18:46:00.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b37ad640 1 -- start start 2024-04-23T18:46:00.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b37ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac101110 0x7fc0ac1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b37ad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac106190 0x7fc0ac198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:00.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b37ad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ac19cfa0 0x7fc0ac19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:00.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b37ad640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1110e0 con 0x7fc0ac101110 2024-04-23T18:46:00.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b37ad640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac110f60 con 0x7fc0ac106190 2024-04-23T18:46:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b37ad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac111260 con 0x7fc0ac19cfa0 2024-04-23T18:46:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b1549640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac101110 0x7fc0ac1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b1549640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac101110 0x7fc0ac1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48376/0 (socket says 172.21.15.19:48376) 2024-04-23T18:46:00.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.990+0000 7fc0b1549640 1 -- 172.21.15.19:0/2377249183 learned_addr learned my addr 172.21.15.19:0/2377249183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:46:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b0d48640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac106190 0x7fc0ac198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:00.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b1d4a640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ac19cfa0 0x7fc0ac19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b1549640 1 -- 172.21.15.19:0/2377249183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ac19cfa0 msgr2=0x7fc0ac19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:00.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b1549640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ac19cfa0 0x7fc0ac19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:00.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b1549640 1 -- 172.21.15.19:0/2377249183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac106190 msgr2=0x7fc0ac198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:00.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b1549640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac106190 0x7fc0ac198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b1549640 1 -- 172.21.15.19:0/2377249183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac19da90 con 0x7fc0ac101110 2024-04-23T18:46:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b0d48640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac106190 0x7fc0ac198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b1549640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac101110 0x7fc0ac1986e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800c890 tx=0x7fc0a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:00.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0a27fc640 1 -- 172.21.15.19:0/2377249183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8004220 con 0x7fc0ac101110 2024-04-23T18:46:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2377249183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac19dd80 con 0x7fc0ac101110 2024-04-23T18:46:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0a27fc640 1 -- 172.21.15.19:0/2377249183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a80043c0 con 0x7fc0ac101110 2024-04-23T18:46:00.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0a27fc640 1 -- 172.21.15.19:0/2377249183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8004a00 con 0x7fc0ac101110 2024-04-23T18:46:00.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.991+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2377249183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac1a5570 con 0x7fc0ac101110 2024-04-23T18:46:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.992+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2377249183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ac0691c0 con 0x7fc0ac101110 2024-04-23T18:46:00.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.994+0000 7fc0a27fc640 1 -- 172.21.15.19:0/2377249183 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc0a80079b0 con 0x7fc0ac101110 2024-04-23T18:46:00.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.995+0000 7fc0a27fc640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc0980a3d20 0x7fc0980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.995+0000 7fc0a27fc640 1 -- 172.21.15.19:0/2377249183 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7fc0a80d09e0 con 0x7fc0ac101110 2024-04-23T18:46:00.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.995+0000 7fc0b0d48640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc0980a3d20 0x7fc0980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:00.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:45:59.996+0000 7fc0b0d48640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc0980a3d20 0x7fc0980a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005d90 tx=0x7fc09c005d20 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.001+0000 7fc0a27fc640 1 -- 172.21.15.19:0/2377249183 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0a8014030 con 0x7fc0ac101110 2024-04-23T18:46:00.666 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:59 smithi155 ceph-mon[33468]: from='client.17847 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:00.673 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:45:59 smithi155 ceph-mon[33468]: pgmap v627: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:46:00.767 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:59 smithi044 ceph-mon[32458]: from='client.17847 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:00.775 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:45:59 smithi044 ceph-mon[32458]: pgmap v627: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:46:00.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.190+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2377249183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0ac199a30 con 0x7fc0ac101110 2024-04-23T18:46:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.192+0000 7fc0a27fc640 1 -- 172.21.15.19:0/2377249183 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+23044 (secure 0 0 0) 0x7fc0a809d0b0 con 0x7fc0ac101110 2024-04-23T18:46:00.839 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:46:00.848 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":118,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:45:17.617554+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/304874449":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1108384152":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6801/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6800/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/983971095":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2556512605":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:46:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2377249183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc0980a3d20 msgr2=0x7fc0980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc0980a3d20 0x7fc0980a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005d90 tx=0x7fc09c005d20 comp rx=0 tx=0).stop 2024-04-23T18:46:00.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2377249183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac101110 msgr2=0x7fc0ac1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac101110 0x7fc0ac1986e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800c890 tx=0x7fc0a800cd50 comp rx=0 tx=0).stop 2024-04-23T18:46:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2377249183 shutdown_connections 2024-04-23T18:46:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc0980a3d20 0x7fc0980a61e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac106190 0x7fc0ac198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:01.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ac19cfa0 0x7fc0ac19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 --2- 172.21.15.19:0/2377249183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac101110 0x7fc0ac1986e0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2377249183 >> 172.21.15.19:0/2377249183 conn(0x7fc0ac077fb0 msgr2=0x7fc0ac06a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2377249183 shutdown_connections 2024-04-23T18:46:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:00.194+0000 7fc0b37ad640 1 -- 172.21.15.19:0/2377249183 wait complete. 2024-04-23T18:46:01.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:59 smithi019 ceph-mon[29215]: from='client.17847 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:01.129 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:45:59 smithi019 ceph-mon[29215]: pgmap v627: 97 pgs: 97 active+clean; 4.3 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:46:01.137 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:23:53.296504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:23:54.573484+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 116, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:46:01.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:46:01.258 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:00 smithi019 ceph-mon[29215]: from='client.17853 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:01.266 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:00 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2377249183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.279+0000 7fa7657e5640 1 -- 172.21.15.19:0/1755415997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760106160 msgr2=0x7fa76010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.279+0000 7fa7657e5640 1 --2- 172.21.15.19:0/1755415997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760106160 0x7fa76010dd30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa758009ca0 tx=0x7fa75802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:46:01.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.279+0000 7fa7657e5640 1 -- 172.21.15.19:0/1755415997 shutdown_connections 2024-04-23T18:46:01.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.279+0000 7fa7657e5640 1 --2- 172.21.15.19:0/1755415997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa760101a40 0x7fa760105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:01.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.279+0000 7fa7657e5640 1 --2- 172.21.15.19:0/1755415997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa760106160 0x7fa76010dd30 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:01.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.279+0000 7fa7657e5640 1 --2- 172.21.15.19:0/1755415997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa760101090 0x7fa760101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.279+0000 7fa7657e5640 1 -- 172.21.15.19:0/1755415997 >> 172.21.15.19:0/1755415997 conn(0x7fa760077f50 msgr2=0x7fa760078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:01.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.279+0000 7fa7657e5640 1 -- 172.21.15.19:0/1755415997 shutdown_connections 2024-04-23T18:46:01.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.279+0000 7fa7657e5640 1 -- 172.21.15.19:0/1755415997 wait complete. 2024-04-23T18:46:01.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.279+0000 7fa7657e5640 1 Processor -- start 2024-04-23T18:46:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa7657e5640 1 -- start start 2024-04-23T18:46:01.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa7657e5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa760101090 0x7fa7601986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:01.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa7657e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa760106160 0x7fa760198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:01.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa7657e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76019cf10 0x7fa76019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa7657e5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601121e0 con 0x7fa76019cf10 2024-04-23T18:46:01.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa7657e5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa760112060 con 0x7fa760106160 2024-04-23T18:46:01.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa7657e5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa760112360 con 0x7fa760101090 2024-04-23T18:46:01.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa75f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76019cf10 0x7fa76019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:01.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa75f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76019cf10 0x7fa76019d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48396/0 (socket says 172.21.15.19:48396) 2024-04-23T18:46:01.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa75f7fe640 1 -- 172.21.15.19:0/3196078101 learned_addr learned my addr 172.21.15.19:0/3196078101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:46:01.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa75e7fc640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa760106160 0x7fa760198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:01.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa75effd640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa760101090 0x7fa7601986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:01.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa75f7fe640 1 -- 172.21.15.19:0/3196078101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa760101090 msgr2=0x7fa7601986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:01.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa75f7fe640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa760101090 0x7fa7601986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:01.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa75f7fe640 1 -- 172.21.15.19:0/3196078101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa760106160 msgr2=0x7fa760198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:01.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa75f7fe640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa760106160 0x7fa760198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.280+0000 7fa75f7fe640 1 -- 172.21.15.19:0/3196078101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76019da00 con 0x7fa76019cf10 2024-04-23T18:46:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.281+0000 7fa75f7fe640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76019cf10 0x7fa76019d3c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa758005cc0 tx=0x7fa75802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.281+0000 7fa757fff640 1 -- 172.21.15.19:0/3196078101 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758003bf0 con 0x7fa76019cf10 2024-04-23T18:46:01.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.281+0000 7fa757fff640 1 -- 172.21.15.19:0/3196078101 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa758003d90 con 0x7fa76019cf10 2024-04-23T18:46:01.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.281+0000 7fa7657e5640 1 -- 172.21.15.19:0/3196078101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76019dc90 con 0x7fa76019cf10 2024-04-23T18:46:01.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.281+0000 7fa757fff640 1 -- 172.21.15.19:0/3196078101 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75802fb20 con 0x7fa76019cf10 2024-04-23T18:46:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.281+0000 7fa7657e5640 1 -- 172.21.15.19:0/3196078101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7601a54b0 con 0x7fa76019cf10 2024-04-23T18:46:01.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.282+0000 7fa7657e5640 1 -- 172.21.15.19:0/3196078101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa760068d70 con 0x7fa76019cf10 2024-04-23T18:46:01.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.283+0000 7fa757fff640 1 -- 172.21.15.19:0/3196078101 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa758004580 con 0x7fa76019cf10 2024-04-23T18:46:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.284+0000 7fa757fff640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa74c0a3d20 0x7fa74c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.284+0000 7fa75effd640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa74c0a3d20 0x7fa74c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:01.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.284+0000 7fa757fff640 1 -- 172.21.15.19:0/3196078101 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7fa7580f1ba0 con 0x7fa76019cf10 2024-04-23T18:46:01.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.284+0000 7fa75effd640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa74c0a3d20 0x7fa74c0a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa748005fd0 tx=0x7fa74800b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:01.932 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:01 smithi155 ceph-mon[33468]: from='client.17853 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:01.940 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:01 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2377249183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:01.951 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:00 smithi044 ceph-mon[32458]: from='client.17853 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:01.958 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:00 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2377249183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:01.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.289+0000 7fa757fff640 1 -- 172.21.15.19:0/3196078101 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7580be320 con 0x7fa76019cf10 2024-04-23T18:46:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.477+0000 7fa7657e5640 1 -- 172.21.15.19:0/3196078101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa760101a40 con 0x7fa76019cf10 2024-04-23T18:46:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.477+0000 7fa757fff640 1 -- 172.21.15.19:0/3196078101 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7fa7580098b0 con 0x7fa76019cf10 2024-04-23T18:46:02.038 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:46:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.479+0000 7fa7657e5640 1 -- 172.21.15.19:0/3196078101 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa74c0a3d20 msgr2=0x7fa74c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:02.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.479+0000 7fa7657e5640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa74c0a3d20 0x7fa74c0a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa748005fd0 tx=0x7fa74800b390 comp rx=0 tx=0).stop 2024-04-23T18:46:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.479+0000 7fa7657e5640 1 -- 172.21.15.19:0/3196078101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76019cf10 msgr2=0x7fa76019d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:02.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.479+0000 7fa7657e5640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76019cf10 0x7fa76019d3c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa758005cc0 tx=0x7fa75802f380 comp rx=0 tx=0).stop 2024-04-23T18:46:02.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.480+0000 7fa7657e5640 1 -- 172.21.15.19:0/3196078101 shutdown_connections 2024-04-23T18:46:02.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.480+0000 7fa7657e5640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa74c0a3d20 0x7fa74c0a61e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.480+0000 7fa7657e5640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa760106160 0x7fa760198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.480+0000 7fa7657e5640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76019cf10 0x7fa76019d3c0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.480+0000 7fa7657e5640 1 --2- 172.21.15.19:0/3196078101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa760101090 0x7fa7601986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.480+0000 7fa7657e5640 1 -- 172.21.15.19:0/3196078101 >> 172.21.15.19:0/3196078101 conn(0x7fa760077f50 msgr2=0x7fa7600fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.480+0000 7fa7657e5640 1 -- 172.21.15.19:0/3196078101 shutdown_connections 2024-04-23T18:46:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:01.480+0000 7fa7657e5640 1 -- 172.21.15.19:0/3196078101 wait complete. 2024-04-23T18:46:02.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:46:02.303 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:02 smithi019 ceph-mon[29215]: pgmap v628: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-04-23T18:46:02.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:02 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/3196078101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:46:02.411 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:02 smithi155 ceph-mon[33468]: pgmap v628: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-04-23T18:46:02.418 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:02 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/3196078101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:46:02.425 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:02 smithi044 ceph-mon[32458]: pgmap v628: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-04-23T18:46:02.432 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:02 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/3196078101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:46:02.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.538+0000 7fa6dcc72640 1 -- 172.21.15.19:0/3589412051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6d81031c0 msgr2=0x7fa6d81035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:02.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.538+0000 7fa6dcc72640 1 --2- 172.21.15.19:0/3589412051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6d81031c0 0x7fa6d81035a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009ca0 tx=0x7fa6c802a9e0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.538+0000 7fa6dcc72640 1 -- 172.21.15.19:0/3589412051 shutdown_connections 2024-04-23T18:46:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.538+0000 7fa6dcc72640 1 --2- 172.21.15.19:0/3589412051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6d8103ae0 0x7fa6d810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.538+0000 7fa6dcc72640 1 --2- 172.21.15.19:0/3589412051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6d810c500 0x7fa6d810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.538+0000 7fa6dcc72640 1 --2- 172.21.15.19:0/3589412051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6d81031c0 0x7fa6d81035a0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.538+0000 7fa6dcc72640 1 -- 172.21.15.19:0/3589412051 >> 172.21.15.19:0/3589412051 conn(0x7fa6d8078000 msgr2=0x7fa6d8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.538+0000 7fa6dcc72640 1 -- 172.21.15.19:0/3589412051 shutdown_connections 2024-04-23T18:46:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.538+0000 7fa6dcc72640 1 -- 172.21.15.19:0/3589412051 wait complete. 2024-04-23T18:46:02.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6dcc72640 1 Processor -- start 2024-04-23T18:46:02.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6dcc72640 1 -- start start 2024-04-23T18:46:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6dcc72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6d81031c0 0x7fa6d81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:02.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6dcc72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6d8103ae0 0x7fa6d81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6dcc72640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6d810c500 0x7fa6d819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6dcc72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8112170 con 0x7fa6d8103ae0 2024-04-23T18:46:02.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6dcc72640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8111ff0 con 0x7fa6d81031c0 2024-04-23T18:46:02.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6dcc72640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81122f0 con 0x7fa6d810c500 2024-04-23T18:46:02.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6d8103ae0 0x7fa6d81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6d8103ae0 0x7fa6d81a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48414/0 (socket says 172.21.15.19:48414) 2024-04-23T18:46:02.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d5d9b640 1 -- 172.21.15.19:0/2141798805 learned_addr learned my addr 172.21.15.19:0/2141798805 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:46:02.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d659c640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6d81031c0 0x7fa6d81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:02.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d6d9d640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6d810c500 0x7fa6d819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d5d9b640 1 -- 172.21.15.19:0/2141798805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6d810c500 msgr2=0x7fa6d819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d5d9b640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6d810c500 0x7fa6d819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d5d9b640 1 -- 172.21.15.19:0/2141798805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6d81031c0 msgr2=0x7fa6d81a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:02.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d5d9b640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6d81031c0 0x7fa6d81a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d5d9b640 1 -- 172.21.15.19:0/2141798805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d819aff0 con 0x7fa6d8103ae0 2024-04-23T18:46:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d659c640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6d81031c0 0x7fa6d81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:46:02.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.539+0000 7fa6d6d9d640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6d810c500 0x7fa6d819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:46:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.540+0000 7fa6d5d9b640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6d8103ae0 0x7fa6d81a0bb0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00d950 tx=0x7fa6cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.540+0000 7fa6c77fe640 1 -- 172.21.15.19:0/2141798805 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc017070 con 0x7fa6d8103ae0 2024-04-23T18:46:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.540+0000 7fa6dcc72640 1 -- 172.21.15.19:0/2141798805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d819b2e0 con 0x7fa6d8103ae0 2024-04-23T18:46:03.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.540+0000 7fa6c77fe640 1 -- 172.21.15.19:0/2141798805 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6cc009c30 con 0x7fa6d8103ae0 2024-04-23T18:46:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.540+0000 7fa6c77fe640 1 -- 172.21.15.19:0/2141798805 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00f620 con 0x7fa6d8103ae0 2024-04-23T18:46:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.540+0000 7fa6dcc72640 1 -- 172.21.15.19:0/2141798805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d81a75f0 con 0x7fa6d8103ae0 2024-04-23T18:46:03.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.541+0000 7fa6dcc72640 1 -- 172.21.15.19:0/2141798805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d80698e0 con 0x7fa6d8103ae0 2024-04-23T18:46:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.542+0000 7fa6c77fe640 1 -- 172.21.15.19:0/2141798805 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa6cc005030 con 0x7fa6d8103ae0 2024-04-23T18:46:03.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.543+0000 7fa6c77fe640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6c00a3d20 0x7fa6c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.543+0000 7fa6d659c640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6c00a3d20 0x7fa6c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:03.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.543+0000 7fa6c77fe640 1 -- 172.21.15.19:0/2141798805 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7fa6cc0d0520 con 0x7fa6d8103ae0 2024-04-23T18:46:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.543+0000 7fa6d659c640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6c00a3d20 0x7fa6c00a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009430 tx=0x7fa6c8005ef0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:03.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.546+0000 7fa6c77fe640 1 -- 172.21.15.19:0/2141798805 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6cc09cb90 con 0x7fa6d8103ae0 2024-04-23T18:46:03.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.733+0000 7fa6dcc72640 1 -- 172.21.15.19:0/2141798805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa6d81a77c0 con 0x7fa6d8103ae0 2024-04-23T18:46:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.733+0000 7fa6c77fe640 1 -- 172.21.15.19:0/2141798805 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7fa6cc009de0 con 0x7fa6d8103ae0 2024-04-23T18:46:03.225 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-23T18:46:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.735+0000 7fa6dcc72640 1 -- 172.21.15.19:0/2141798805 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6c00a3d20 msgr2=0x7fa6c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.735+0000 7fa6dcc72640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6c00a3d20 0x7fa6c00a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009430 tx=0x7fa6c8005ef0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.735+0000 7fa6dcc72640 1 -- 172.21.15.19:0/2141798805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6d8103ae0 msgr2=0x7fa6d81a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:03.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.735+0000 7fa6dcc72640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6d8103ae0 0x7fa6d81a0bb0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00d950 tx=0x7fa6cc00de10 comp rx=0 tx=0).stop 2024-04-23T18:46:03.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.735+0000 7fa6dcc72640 1 -- 172.21.15.19:0/2141798805 shutdown_connections 2024-04-23T18:46:03.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.735+0000 7fa6dcc72640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa6c00a3d20 0x7fa6c00a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.735+0000 7fa6dcc72640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6d810c500 0x7fa6d819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.735+0000 7fa6dcc72640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6d8103ae0 0x7fa6d81a0bb0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.735+0000 7fa6dcc72640 1 --2- 172.21.15.19:0/2141798805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6d81031c0 0x7fa6d81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.735+0000 7fa6dcc72640 1 -- 172.21.15.19:0/2141798805 >> 172.21.15.19:0/2141798805 conn(0x7fa6d8078000 msgr2=0x7fa6d8108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.736+0000 7fa6dcc72640 1 -- 172.21.15.19:0/2141798805 shutdown_connections 2024-04-23T18:46:03.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:02.736+0000 7fa6dcc72640 1 -- 172.21.15.19:0/2141798805 wait complete. 2024-04-23T18:46:03.384 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:46:03.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:03 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2141798805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:46:03.449 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:03 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2141798805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:46:03.456 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:03 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2141798805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:46:03.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.590+0000 7f434a205640 1 -- 172.21.15.19:0/4281340141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441031d0 msgr2=0x7f43441035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:03.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.590+0000 7f434a205640 1 --2- 172.21.15.19:0/4281340141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441031d0 0x7f43441035b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f4334009900 tx=0x7f433402a510 comp rx=0 tx=0).stop 2024-04-23T18:46:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.590+0000 7f434a205640 1 -- 172.21.15.19:0/4281340141 shutdown_connections 2024-04-23T18:46:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.590+0000 7f434a205640 1 --2- 172.21.15.19:0/4281340141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f434410c510 0x7f434410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.590+0000 7f434a205640 1 --2- 172.21.15.19:0/4281340141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4344103af0 0x7f434410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.590+0000 7f434a205640 1 --2- 172.21.15.19:0/4281340141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441031d0 0x7f43441035b0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.590+0000 7f434a205640 1 -- 172.21.15.19:0/4281340141 >> 172.21.15.19:0/4281340141 conn(0x7f4344077f50 msgr2=0x7f4344078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.590+0000 7f434a205640 1 -- 172.21.15.19:0/4281340141 shutdown_connections 2024-04-23T18:46:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.590+0000 7f434a205640 1 -- 172.21.15.19:0/4281340141 wait complete. 2024-04-23T18:46:03.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f434a205640 1 Processor -- start 2024-04-23T18:46:03.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f434a205640 1 -- start start 2024-04-23T18:46:03.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f434a205640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441031d0 0x7f4344100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:03.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f434a205640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4344103af0 0x7f4344101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:03.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f434a205640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f434410c510 0x7f43441018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f434a205640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43441121a0 con 0x7f43441031d0 2024-04-23T18:46:03.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f434a205640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4344112020 con 0x7f434410c510 2024-04-23T18:46:03.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f434a205640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4344112320 con 0x7f4344103af0 2024-04-23T18:46:03.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f43437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441031d0 0x7f4344100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f43437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441031d0 0x7f4344100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48426/0 (socket says 172.21.15.19:48426) 2024-04-23T18:46:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f43437fe640 1 -- 172.21.15.19:0/714584118 learned_addr learned my addr 172.21.15.19:0/714584118 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:46:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f4342ffd640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4344103af0 0x7f4344101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f4343fff640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f434410c510 0x7f43441018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f43437fe640 1 -- 172.21.15.19:0/714584118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4344103af0 msgr2=0x7f4344101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f43437fe640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4344103af0 0x7f4344101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f43437fe640 1 -- 172.21.15.19:0/714584118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f434410c510 msgr2=0x7f43441018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.591+0000 7f43437fe640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f434410c510 0x7f43441018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.592+0000 7f43437fe640 1 -- 172.21.15.19:0/714584118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43441a6ca0 con 0x7f43441031d0 2024-04-23T18:46:03.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.592+0000 7f43437fe640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441031d0 0x7f4344100e20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4334005940 tx=0x7f4334004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:03.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.592+0000 7f4340ff9640 1 -- 172.21.15.19:0/714584118 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433403f030 con 0x7f43441031d0 2024-04-23T18:46:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.592+0000 7f4340ff9640 1 -- 172.21.15.19:0/714584118 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4334003aa0 con 0x7f43441031d0 2024-04-23T18:46:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.592+0000 7f434a205640 1 -- 172.21.15.19:0/714584118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43441a6f30 con 0x7f43441031d0 2024-04-23T18:46:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.592+0000 7f4340ff9640 1 -- 172.21.15.19:0/714584118 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334035070 con 0x7f43441031d0 2024-04-23T18:46:04.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.592+0000 7f434a205640 1 -- 172.21.15.19:0/714584118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441a7390 con 0x7f43441031d0 2024-04-23T18:46:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.593+0000 7f434a205640 1 -- 172.21.15.19:0/714584118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4344068d70 con 0x7f43441031d0 2024-04-23T18:46:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.594+0000 7f4340ff9640 1 -- 172.21.15.19:0/714584118 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4334042030 con 0x7f43441031d0 2024-04-23T18:46:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.595+0000 7f4340ff9640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f432c0a3d20 0x7f432c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:04.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.595+0000 7f4342ffd640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f432c0a3d20 0x7f432c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:04.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.595+0000 7f4340ff9640 1 -- 172.21.15.19:0/714584118 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7f43340f1090 con 0x7f43441031d0 2024-04-23T18:46:04.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.595+0000 7f4342ffd640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f432c0a3d20 0x7f432c0a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f43441028b0 tx=0x7f4338005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:04.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.598+0000 7f4340ff9640 1 -- 172.21.15.19:0/714584118 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43340bd760 con 0x7f43441031d0 2024-04-23T18:46:04.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.786+0000 7f434a205640 1 -- 172.21.15.19:0/714584118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4344107100 con 0x7f43441031d0 2024-04-23T18:46:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.786+0000 7f4340ff9640 1 -- 172.21.15.19:0/714584118 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7f43340bd060 con 0x7f43441031d0 2024-04-23T18:46:04.195 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-23T18:46:04.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 -- 172.21.15.19:0/714584118 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f432c0a3d20 msgr2=0x7f432c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:04.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f432c0a3d20 0x7f432c0a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f43441028b0 tx=0x7f4338005f90 comp rx=0 tx=0).stop 2024-04-23T18:46:04.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 -- 172.21.15.19:0/714584118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441031d0 msgr2=0x7f4344100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:04.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441031d0 0x7f4344100e20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4334005940 tx=0x7f4334004270 comp rx=0 tx=0).stop 2024-04-23T18:46:04.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 -- 172.21.15.19:0/714584118 shutdown_connections 2024-04-23T18:46:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f432c0a3d20 0x7f432c0a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:04.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f434410c510 0x7f43441018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:04.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4344103af0 0x7f4344101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:04.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 --2- 172.21.15.19:0/714584118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441031d0 0x7f4344100e20 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:04.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 -- 172.21.15.19:0/714584118 >> 172.21.15.19:0/714584118 conn(0x7f4344077f50 msgr2=0x7f43440fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 -- 172.21.15.19:0/714584118 shutdown_connections 2024-04-23T18:46:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:03.788+0000 7f434a205640 1 -- 172.21.15.19:0/714584118 wait complete. 2024-04-23T18:46:04.429 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-23T18:46:04.440 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:04 smithi155 ceph-mon[33468]: pgmap v629: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-04-23T18:46:04.446 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:04 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/714584118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:46:04.452 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:04 smithi044 ceph-mon[32458]: pgmap v629: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-04-23T18:46:04.459 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:04 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/714584118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:46:04.470 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:04 smithi019 ceph-mon[29215]: pgmap v629: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-04-23T18:46:04.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:04 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/714584118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:46:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.625+0000 7f270168a640 1 -- 172.21.15.19:0/3098410278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc103b00 msgr2=0x7f26fc10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:04.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.625+0000 7f270168a640 1 --2- 172.21.15.19:0/3098410278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc103b00 0x7f26fc10bfe0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009900 tx=0x7f26f002a510 comp rx=0 tx=0).stop 2024-04-23T18:46:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.625+0000 7f270168a640 1 -- 172.21.15.19:0/3098410278 shutdown_connections 2024-04-23T18:46:04.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.625+0000 7f270168a640 1 --2- 172.21.15.19:0/3098410278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc10c520 0x7f26fc10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.625+0000 7f270168a640 1 --2- 172.21.15.19:0/3098410278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc103b00 0x7f26fc10bfe0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:04.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.625+0000 7f270168a640 1 --2- 172.21.15.19:0/3098410278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26fc1031e0 0x7f26fc1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:04.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.625+0000 7f270168a640 1 -- 172.21.15.19:0/3098410278 >> 172.21.15.19:0/3098410278 conn(0x7f26fc077f50 msgr2=0x7f26fc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:04.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.625+0000 7f270168a640 1 -- 172.21.15.19:0/3098410278 shutdown_connections 2024-04-23T18:46:04.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.625+0000 7f270168a640 1 -- 172.21.15.19:0/3098410278 wait complete. 2024-04-23T18:46:04.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.625+0000 7f270168a640 1 Processor -- start 2024-04-23T18:46:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f270168a640 1 -- start start 2024-04-23T18:46:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f270168a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1031e0 0x7f26fc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f270168a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc103b00 0x7f26fc198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:04.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f270168a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26fc10c520 0x7f26fc19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:04.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f270168a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1121b0 con 0x7f26fc1031e0 2024-04-23T18:46:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f270168a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc112030 con 0x7f26fc10c520 2024-04-23T18:46:04.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f270168a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc112330 con 0x7f26fc103b00 2024-04-23T18:46:04.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1031e0 0x7f26fc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:04.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1031e0 0x7f26fc198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48442/0 (socket says 172.21.15.19:48442) 2024-04-23T18:46:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26faffd640 1 -- 172.21.15.19:0/4009267241 learned_addr learned my addr 172.21.15.19:0/4009267241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:46:04.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26fb7fe640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26fc10c520 0x7f26fc19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:04.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26fa7fc640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc103b00 0x7f26fc198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:04.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26faffd640 1 -- 172.21.15.19:0/4009267241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc103b00 msgr2=0x7f26fc198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:04.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26faffd640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc103b00 0x7f26fc198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:04.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26faffd640 1 -- 172.21.15.19:0/4009267241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26fc10c520 msgr2=0x7f26fc19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:05.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26faffd640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26fc10c520 0x7f26fc19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:05.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26faffd640 1 -- 172.21.15.19:0/4009267241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc19d720 con 0x7f26fc1031e0 2024-04-23T18:46:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.626+0000 7f26faffd640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1031e0 0x7f26fc198730 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00e9f0 tx=0x7f26ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.627+0000 7f26e3fff640 1 -- 172.21.15.19:0/4009267241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00cd80 con 0x7f26fc1031e0 2024-04-23T18:46:05.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.627+0000 7f26e3fff640 1 -- 172.21.15.19:0/4009267241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26ec0108e0 con 0x7f26fc1031e0 2024-04-23T18:46:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.627+0000 7f270168a640 1 -- 172.21.15.19:0/4009267241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc19da10 con 0x7f26fc1031e0 2024-04-23T18:46:05.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.627+0000 7f26e3fff640 1 -- 172.21.15.19:0/4009267241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec018980 con 0x7f26fc1031e0 2024-04-23T18:46:05.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.627+0000 7f270168a640 1 -- 172.21.15.19:0/4009267241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc1a52a0 con 0x7f26fc1031e0 2024-04-23T18:46:05.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.628+0000 7f270168a640 1 -- 172.21.15.19:0/4009267241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26fc068d70 con 0x7f26fc1031e0 2024-04-23T18:46:05.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.629+0000 7f26e3fff640 1 -- 172.21.15.19:0/4009267241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f26ec010430 con 0x7f26fc1031e0 2024-04-23T18:46:05.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.630+0000 7f26e3fff640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26e40a3d20 0x7f26e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:05.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.630+0000 7f26fa7fc640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26e40a3d20 0x7f26e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:05.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.630+0000 7f26e3fff640 1 -- 172.21.15.19:0/4009267241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8469+0+0 (secure 0 0 0) 0x7f26ec0d0bf0 con 0x7f26fc1031e0 2024-04-23T18:46:05.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.630+0000 7f26fa7fc640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26e40a3d20 0x7f26e40a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f26f0031440 tx=0x7f26f00095f0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:05.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.634+0000 7f26e3fff640 1 -- 172.21.15.19:0/4009267241 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26ec09d2c0 con 0x7f26fc1031e0 2024-04-23T18:46:05.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.874+0000 7f270168a640 1 -- 172.21.15.19:0/4009267241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f26fc107110 con 0x7f26fc1031e0 2024-04-23T18:46:05.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.882+0000 7f26e3fff640 1 -- 172.21.15.19:0/4009267241 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746062 (secure 0 0 0) 0x7f26ec09cbc0 con 0x7f26fc1031e0 2024-04-23T18:46:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:46:05.308 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:46:05.316 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 26, 2024-04-23T18:46:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:05.332 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 25004, 2024-04-23T18:46:05.341 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2024-04-23T18:46:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2024-04-23T18:46:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-23T18:46:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-23T18:46:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-04-23T18:46:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1177101685 2024-04-23T18:46:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-04-23T18:46:05.488 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-04-23T18:46:05.497 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1177101685 2024-04-23T18:46:05.546 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:05.553 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:46:05.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/1177101685", 2024-04-23T18:46:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2024-04-23T18:26:08.117188+0000", 2024-04-23T18:46:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: "active_mgr_features": 4540701547738038271, 2024-04-23T18:46:05.586 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2024-04-23T18:46:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [ 2024-04-23T18:46:05.599 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 15309, 2024-04-23T18:46:05.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "y", 2024-04-23T18:46:05.638 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-04-23T18:46:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-23T18:46:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:05.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-23T18:46:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:05.711 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:46:05.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:46:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:05.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:05.805 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:05.813 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:46:05.820 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:05.826 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:05.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T18:46:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:05.941 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:06.081 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:06.090 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:06.125 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:06.137 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:06.203 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:06.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:06.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:06.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:06.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:06.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:06.288 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:06.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:06.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:06.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:06.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:06.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:06.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:06.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:06.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:06.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:06.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:06.454 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:06.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:06.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:06.570 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:06.578 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:06.610 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:06.619 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:06.627 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:06.688 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:06.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:06.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:06.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:06.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:06.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:06.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:06.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:06.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-23T18:46:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-23T18:46:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:07.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-23T18:46:07.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:07.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:07.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:07.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:07.172 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-23T18:46:07.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-23T18:46:07.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:07.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:07.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:07.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-23T18:46:07.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:07.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:07.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:07.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-23T18:46:07.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:07.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:07.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:07.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:07.378 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-23T18:46:07.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-23T18:46:07.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:07.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:07.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:07.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:07.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:07.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:07.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-23T18:46:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:07.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:07.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:07.601 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-23T18:46:07.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-23T18:46:07.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:07.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:07.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:07.730 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:07.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:07.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:07.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-23T18:46:07.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:07.800 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:07.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-23T18:46:07.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-23T18:46:07.839 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:07.845 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:07.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:07.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-23T18:46:07.900 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:07.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:07.914 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:08.691 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:46:09.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:46:09.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:09.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:09.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-23T18:46:09.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:46:09.489 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-23T18:46:09.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T18:46:09.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:09.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:09.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:09.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:09.610 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-23T18:46:09.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-23T18:46:09.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:09.631 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:09.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:09.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T18:46:09.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:09.693 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:09.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:09.789 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T18:46:09.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T18:46:09.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:09.858 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:09.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:09.874 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-23T18:46:09.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-23T18:46:09.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:46:09.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:09.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-23T18:46:09.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-23T18:46:09.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-23T18:46:09.996 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:10.006 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-23T18:46:10.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T18:46:10.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:10.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:10.095 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-23T18:46:10.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-23T18:46:10.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:10.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:10.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:10.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-23T18:46:10.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:10.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:10.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:10.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T18:46:10.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:46:10.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:10.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:10.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:10.349 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-23T18:46:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-23T18:46:10.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:46:10.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:10.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:10.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:46:10.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-23T18:46:10.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-23T18:46:10.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:10.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T18:46:10.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:46:10.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:10.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:10.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:10.580 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:10.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:10.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:10.658 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:10.665 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:10.672 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:10.679 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:10.686 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:10.693 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:10.718 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:10.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:10.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:10.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:10.778 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:10.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:10.830 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:10.840 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:10.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:10.891 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:10.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:10.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:10.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:10.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:10.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:10.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:10.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:10.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:10.995 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:11.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:11.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:11.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:11.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:11.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:11.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:11.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:11.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:11.144 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:11.153 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:11.162 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:11.170 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:11.181 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:11.190 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:11.198 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:11.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:11.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:11.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:11.330 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:11.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:11.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:11.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:11.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:11.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:11.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:11.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:11.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:11.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:11.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:11.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-23T18:46:11.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-23T18:46:11.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:46:11.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:11.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:11.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:46:11.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:11.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:11.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:11.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T18:46:11.719 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:11.728 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:11.736 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:11.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:11.811 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-23T18:46:11.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-23T18:46:11.827 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:11.836 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-23T18:46:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-23T18:46:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-23T18:46:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-23T18:46:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-23T18:46:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-23T18:46:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-23T18:46:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-23T18:46:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-23T18:46:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T18:46:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:46:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:46:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:46:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T18:46:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-23T18:46:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-23T18:46:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:46:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:46:12.821 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:12.827 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:12.834 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T18:46:12.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T18:46:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-23T18:46:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-23T18:46:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:46:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:46:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T18:46:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-23T18:46:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-23T18:46:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-23T18:46:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:46:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-23T18:46:13.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:13.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:13.369 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:13.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T18:46:13.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:13.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:13.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:13.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:13.455 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-23T18:46:13.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-23T18:46:13.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:13.474 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:13.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:13.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-23T18:46:13.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:13.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:13.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:13.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T18:46:13.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:13.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:13.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:13.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:13.683 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-23T18:46:13.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-23T18:46:13.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:13.703 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:13.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:13.714 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-23T18:46:13.724 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:13.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:13.751 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:13.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T18:46:13.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:13.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:13.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:13.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:13.855 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-23T18:46:13.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-23T18:46:13.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:13.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:13.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:13.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:13.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:13.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T18:46:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T18:46:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-23T18:46:14.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-23T18:46:14.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:14.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:14.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:46:14.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:14.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:14.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:14.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T18:46:14.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:14.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:14.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:14.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:14.248 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-23T18:46:14.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T18:46:14.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:46:14.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:14.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:14.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-23T18:46:14.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:14.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T18:46:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:14.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:14.509 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-23T18:46:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-23T18:46:14.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:14.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:14.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:14.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:14.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-23T18:46:14.641 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-23T18:46:14.706 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-23T18:46:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T18:46:14.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:14.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:14.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:14.754 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-23T18:46:14.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-23T18:46:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:14.827 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:14.841 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:14.900 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:14.907 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:14.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:14.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T18:46:14.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:14.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:14.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:15.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:15.011 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-23T18:46:15.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-23T18:46:15.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:15.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:15.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:15.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:15.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:15.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:15.159 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T18:46:15.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:15.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:15.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:15.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:15.249 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-23T18:46:15.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-23T18:46:15.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:15.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:15.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:15.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:15.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:15.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T18:46:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:15.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:15.466 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-23T18:46:15.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-23T18:46:15.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:15.553 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:15.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T18:46:15.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:15.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:15.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:15.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:15.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:15.660 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-23T18:46:15.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-23T18:46:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:15.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:15.765 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T18:46:15.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:15.778 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:15.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:15.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-23T18:46:15.796 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:15.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-23T18:46:15.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-23T18:46:15.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:15.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T18:46:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:16.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-23T18:46:16.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:16.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:16.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:16.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-23T18:46:16.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-23T18:46:16.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T18:46:16.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:16.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:16.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:16.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:16.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:16.321 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-23T18:46:16.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-23T18:46:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T18:46:16.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-23T18:46:16.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:16.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:16.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:16.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:16.553 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-23T18:46:16.562 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-23T18:46:16.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:16.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:16.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:16.595 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T18:46:16.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:16.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:16.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:16.651 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-23T18:46:16.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:16.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:16.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:16.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:16.759 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-23T18:46:16.766 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-23T18:46:16.774 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:16.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:16.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:16.862 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T18:46:16.868 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:16.875 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:16.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:16.893 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-23T18:46:16.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:16.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:16.936 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:16.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:16.950 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-23T18:46:16.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-23T18:46:17.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:17.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:17.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:17.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T18:46:17.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:17.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:17.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:17.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-23T18:46:17.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:17.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:17.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:17.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:17.151 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-23T18:46:17.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-23T18:46:17.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:17.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:17.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:17.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T18:46:17.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:17.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:17.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:17.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-23T18:46:17.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:17.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:17.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:17.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:17.370 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-23T18:46:17.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-23T18:46:17.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T18:46:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:17.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:17.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-23T18:46:17.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:17.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:17.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:17.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:17.611 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-23T18:46:17.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-23T18:46:17.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:17.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:17.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:17.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T18:46:17.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:17.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:17.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:17.727 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:17.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:17.786 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:17.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:17.809 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-23T18:46:17.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-23T18:46:17.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:17.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:17.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:17.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T18:46:17.907 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:17.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-23T18:46:17.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:17.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:18.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:18.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:18.053 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-23T18:46:18.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-23T18:46:18.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:18.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:18.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:18.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T18:46:18.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:18.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:18.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:18.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:18.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:18.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:18.230 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:18.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-23T18:46:18.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-23T18:46:18.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:18.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T18:46:18.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:18.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:18.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:18.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-23T18:46:18.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:18.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:18.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:18.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:18.445 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-23T18:46:18.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-23T18:46:18.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:18.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:18.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:18.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T18:46:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:18.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:18.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:18.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-23T18:46:18.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:18.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:18.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-23T18:46:18.688 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-23T18:46:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:18.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:18.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T18:46:18.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:18.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:18.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:18.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-23T18:46:18.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:18.834 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:18.851 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:18.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:18.914 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-23T18:46:18.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-23T18:46:18.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:18.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:18.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:18.952 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:18.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:18.966 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:18.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:18.999 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T18:46:19.008 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:19.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:19.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:19.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:19.086 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-23T18:46:19.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-23T18:46:19.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:19.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:19.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:19.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:46:19.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:19.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:19.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:19.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T18:46:19.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:19.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-23T18:46:19.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T18:46:19.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:19.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:19.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-23T18:46:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:19.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:19.554 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:19.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T18:46:19.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:19.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:19.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:19.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:19.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-23T18:46:19.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-23T18:46:19.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:19.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:19.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:19.718 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-23T18:46:19.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:19.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:19.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:19.828 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T18:46:19.834 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:19.840 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:19.846 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:19.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:19.861 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-23T18:46:19.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-23T18:46:19.876 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:19.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:19.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:19.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-23T18:46:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:20.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:20.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:20.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-23T18:46:20.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:20.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:20.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:20.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:20.129 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-23T18:46:20.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-23T18:46:20.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:20.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:20.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:20.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:20.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:20.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:20.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T18:46:20.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:20.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:20.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:20.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:20.343 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-23T18:46:20.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-23T18:46:20.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:20.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:20.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:20.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:46:20.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:20.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:20.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-23T18:46:20.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:20.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:20.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:20.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:20.534 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-23T18:46:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-23T18:46:20.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:20.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:20.617 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:20.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T18:46:20.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:20.639 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:20.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:20.653 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T18:46:20.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:20.668 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:20.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:20.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:20.709 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-23T18:46:20.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-23T18:46:20.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:20.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:20.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:20.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:46:20.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:20.854 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:20.861 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:20.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-23T18:46:20.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:20.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:20.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:20.906 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-23T18:46:20.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-23T18:46:20.940 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:20.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:20.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:21.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:21.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:21.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:21.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:21.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T18:46:21.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:21.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:21.108 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:21.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:21.126 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-23T18:46:21.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-23T18:46:21.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:21.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:21.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:21.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:21.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:21.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:21.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:21.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T18:46:21.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:21.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:21.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:21.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:21.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:21.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:21.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:21.493 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:21.502 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:21.561 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:21.570 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:21.580 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:21.625 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:21.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:21.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:21.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:21.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:21.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:21.676 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-23T18:46:21.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-23T18:46:21.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:21.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:21.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:21.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:21.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:21.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:21.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T18:46:21.821 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:21.871 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:21.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:21.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:21.891 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:21.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:21.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:21.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:21.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:21.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:21.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:21.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:21.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:21.971 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T18:46:22.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:22.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:22.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:22.129 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:22.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:22.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:22.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:22.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:22.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:22.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:22.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:22.241 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:22.249 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:22.256 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:22.262 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:22.317 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:22.324 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:22.331 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:22.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:22.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:22.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:22.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:22.424 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:22.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:22.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:22.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:22.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:22.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:22.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:22.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:22.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:22.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:22.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:22.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:22.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:22.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:22.630 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T18:46:22.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T18:46:22.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:22.654 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:22.662 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:22.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:22.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:22.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:22.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:22.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T18:46:22.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:22.739 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:22.808 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T18:46:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T18:46:22.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:22.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:22.876 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:22.884 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-23T18:46:22.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:22.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:22.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:22.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T18:46:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:22.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-23T18:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-23T18:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-23T18:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-23T18:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-23T18:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T18:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-23T18:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-23T18:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-23T18:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-23T18:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-23T18:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-23T18:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-23T18:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-23T18:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T18:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-23T18:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-23T18:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-23T18:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T18:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-23T18:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-23T18:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T18:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T18:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-23T18:46:24.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-23T18:46:24.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:24.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:24.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:24.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T18:46:24.651 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:24.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:24.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:24.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-23T18:46:24.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-23T18:46:24.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:24.829 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:24.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:24.848 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:24.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:24.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:24.923 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:24.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T18:46:24.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:24.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:24.953 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:24.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:24.966 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-23T18:46:24.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-23T18:46:24.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:25.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:25.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:25.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:25.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:25.099 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:25.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:25.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T18:46:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:25.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:25.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:25.193 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-23T18:46:25.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-23T18:46:25.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:25.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:25.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:25.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:25.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:25.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:25.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:25.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T18:46:25.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:25.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:25.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:25.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:25.422 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-23T18:46:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-23T18:46:25.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:25.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:25.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:25.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:25.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T18:46:25.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:25.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:25.672 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-23T18:46:25.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-23T18:46:25.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:25.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:25.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:25.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-23T18:46:25.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:25.779 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:25.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:25.793 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-23T18:46:25.844 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:25.851 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:25.858 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:25.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:25.926 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-23T18:46:25.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-23T18:46:25.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:25.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:26.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:26.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:26.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:26.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:26.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T18:46:26.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:26.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:26.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:26.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:26.139 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-23T18:46:26.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-23T18:46:26.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:26.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:26.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:26.231 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:26.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:26.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:26.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:26.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T18:46:26.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:26.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:26.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:26.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:26.357 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-23T18:46:26.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-23T18:46:26.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:26.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:26.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:26.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:26.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:26.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:26.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:26.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T18:46:26.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:26.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:26.605 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:26.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:26.624 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-23T18:46:26.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-23T18:46:26.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:26.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:26.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:26.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T18:46:26.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:26.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:26.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:26.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:26.887 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-23T18:46:26.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-23T18:46:26.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:26.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:26.925 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:26.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:26.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:26.945 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:26.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:26.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T18:46:27.001 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:27.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:27.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:27.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:27.084 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-23T18:46:27.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-23T18:46:27.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:27.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:27.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:27.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:27.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:27.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:27.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:27.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T18:46:27.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:27.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:27.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:27.275 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:27.282 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:27.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:27.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-23T18:46:27.386 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:27.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:27.559 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:27.566 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:27.614 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:27.621 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:27.629 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:27.636 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:27.682 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:27.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:27.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:27.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:27.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:27.722 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:27.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:27.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:27.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:27.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:27.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:27.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:27.838 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:27.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:27.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:27.864 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:27.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:27.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:27.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:27.945 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:27.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:27.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:27.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:28.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:28.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:28.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:28.080 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:28.126 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:28.137 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:28.144 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:28.153 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:28.217 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:28.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:28.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:28.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:28.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:28.267 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:28.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:28.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:28.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:28.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:28.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:28.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:28.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:28.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:28.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:28.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:28.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-23T18:46:28.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:28.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:28.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:28.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T18:46:28.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:28.682 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:28.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:28.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:28.745 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-23T18:46:28.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-23T18:46:28.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:28.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:28.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:28.786 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-23T18:46:28.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:28.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:28.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:28.879 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T18:46:28.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:28.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:28.960 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:28.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:28.982 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:28.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:29.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:29.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-23T18:46:29.066 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:29.073 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:29.082 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:29.097 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T18:46:29.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T18:46:29.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:29.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:29.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:29.172 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:46:29.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:29.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:29.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:29.362 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T18:46:29.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T18:46:29.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:29.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:29.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:29.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:29.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:29.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:29.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:29.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:29.523 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:29.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:29.538 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T18:46:29.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T18:46:29.601 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:29.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:29.619 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:29.627 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:29.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:29.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:29.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:29.666 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:29.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:29.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:29.721 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:29.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:29.783 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-23T18:46:29.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T18:46:29.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:29.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:29.871 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:29.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:29.894 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:29.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:29.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:29.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:29.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:29.932 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:29.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:29.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:29.989 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T18:46:29.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T18:46:30.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:30.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:30.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T18:46:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T18:46:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:30.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:30.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:30.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:30.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:30.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:30.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:30.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:30.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:30.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:30.370 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T18:46:30.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T18:46:30.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:30.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:30.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:30.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:30.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:30.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:30.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:30.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:30.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:30.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:30.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:30.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:30.686 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T18:46:30.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T18:46:30.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:30.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:30.728 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:30.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:30.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:30.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:30.756 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:30.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:30.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:30.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:30.908 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:30.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:30.950 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T18:46:30.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T18:46:31.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:31.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:31.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:31.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:31.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:31.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:31.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:31.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:31.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:31.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:31.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:31.190 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T18:46:31.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T18:46:31.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:31.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:31.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:31.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:31.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:31.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:31.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:31.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:31.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:31.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:31.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:31.474 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T18:46:31.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T18:46:31.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:31.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:31.567 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:31.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:31.625 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:31.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:31.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:31.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:31.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:31.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:31.722 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:31.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:31.742 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T18:46:31.762 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T18:46:31.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:31.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:31.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:31.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:31.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:31.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:31.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:31.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:31.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:32.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:32.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:32.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:32.033 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T18:46:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T18:46:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:32.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:32.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:32.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:32.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:32.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:32.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:32.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:32.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:32.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:32.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:32.396 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T18:46:32.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T18:46:32.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:32.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:32.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:32.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:32.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:32.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:32.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:32.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:32.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:32.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:32.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:32.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:32.576 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T18:46:32.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T18:46:32.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:32.770 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:32.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:32.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-23T18:46:32.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:32.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:32.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:32.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:32.824 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:32.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:32.837 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:32.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:32.895 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T18:46:32.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T18:46:32.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:32.920 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:32.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:32.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:33.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:33.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:33.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:33.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:33.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:33.093 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-23T18:46:33.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-23T18:46:33.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:33.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:33.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:33.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:33.170 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:33.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:33.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:33.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:33.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:33.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:33.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:33.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:33.320 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T18:46:33.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T18:46:33.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:33.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:33.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-23T18:46:33.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:33.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:33.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:33.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:33.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:33.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:33.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:33.567 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T18:46:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T18:46:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:33.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:33.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:33.604 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:33.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:33.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:33.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:33.659 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:33.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:33.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:33.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:33.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:33.765 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T18:46:33.827 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T18:46:33.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:33.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:33.848 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:33.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:33.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:33.871 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:33.878 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:33.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:33.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:33.921 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:33.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:33.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:33.944 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T18:46:33.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T18:46:33.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:34.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:34.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:34.068 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:34.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:34.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:34.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:34.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:34.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:34.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:34.180 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T18:46:34.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T18:46:34.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:34.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:34.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:34.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:34.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:34.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:34.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:34.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:34.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:34.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:34.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:34.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:34.431 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T18:46:34.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T18:46:34.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-23T18:46:34.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:34.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:34.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:34.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:34.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:34.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:34.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:34.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:34.702 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T18:46:34.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T18:46:34.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:34.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:34.736 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:34.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-23T18:46:34.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:34.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:34.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:34.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:34.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:34.865 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:34.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:34.976 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T18:46:34.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T18:46:34.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:34.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:35.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:35.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:35.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:35.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:35.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:35.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:35.089 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:35.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:35.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:35.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:35.161 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T18:46:35.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T18:46:35.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:35.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:35.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:35.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:35.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:35.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:35.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:35.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:35.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:35.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:35.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:35.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T18:46:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T18:46:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:35.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:35.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:35.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:35.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:35.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:35.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:35.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:35.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:35.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:35.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:35.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:35.610 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T18:46:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T18:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:35.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:35.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:35.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:35.756 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:35.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:35.836 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:35.847 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:35.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:35.863 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:35.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:35.881 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T18:46:35.889 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T18:46:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:35.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:35.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:35.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:35.966 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:35.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:36.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:36.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:36.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:36.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T18:46:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T18:46:36.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:36.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:36.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:36.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:36.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:36.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:36.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:36.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:36.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:36.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:36.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:36.339 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T18:46:36.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T18:46:36.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:36.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:36.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:36.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:36.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:36.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:36.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:36.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:36.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:36.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:36.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:36.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:36.518 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T18:46:36.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T18:46:36.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:36.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:36.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:36.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:36.664 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:36.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:36.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:36.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:36.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:36.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:36.738 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T18:46:36.747 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T18:46:36.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:36.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:36.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:36.826 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:36.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:36.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:36.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:36.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:36.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:36.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:36.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:36.961 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T18:46:37.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T18:46:37.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:37.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:37.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:37.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:37.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:37.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:37.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:37.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:37.184 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T18:46:37.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T18:46:37.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:37.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:37.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:37.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T18:46:37.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:37.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:37.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:37.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:37.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:37.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:37.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:37.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:37.457 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T18:46:37.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T18:46:37.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:37.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:37.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:37.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:46:37.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:37.554 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:37.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:37.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:37.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:37.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:37.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:37.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:37.650 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T18:46:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T18:46:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:37.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-23T18:46:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:37.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:37.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:37.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:37.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:37.825 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T18:46:37.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T18:46:37.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:37.892 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-23T18:46:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:38.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:38.030 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:38.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:38.059 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T18:46:38.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T18:46:38.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:38.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:38.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:38.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:38.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:38.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:38.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:38.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:38.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:38.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:38.236 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:38.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:38.250 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T18:46:38.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T18:46:38.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:38.298 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:38.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:38.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-23T18:46:38.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:38.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:38.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:38.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:38.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:38.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:38.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:38.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:38.479 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T18:46:38.486 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T18:46:38.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:38.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:38.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:38.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:38.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:38.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:38.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:38.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:38.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:38.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:38.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:38.699 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T18:46:38.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T18:46:38.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:38.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:38.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:38.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:38.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:38.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:38.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:38.807 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:38.821 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:38.829 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:38.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:38.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:38.890 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T18:46:38.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T18:46:38.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:38.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:38.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:38.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:38.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:38.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:38.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:39.006 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:39.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:39.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:39.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T18:46:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T18:46:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:46:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T18:46:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T18:46:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:46:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T18:46:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T18:46:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:46:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-23T18:46:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-23T18:46:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-23T18:46:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-23T18:46:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-23T18:46:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-23T18:46:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-23T18:46:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-23T18:46:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:40.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:40.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:40.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:40.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:40.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:40.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-23T18:46:40.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-23T18:46:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:40.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:40.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:40.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:40.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:40.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:40.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:40.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:40.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:40.730 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:40.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:40.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:40.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:40.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:40.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:40.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:40.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:40.859 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:40.866 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:40.926 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:40.934 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:40.943 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:40.973 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:40.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:40.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:40.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:41.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:41.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:41.017 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:41.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:41.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:41.066 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:41.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:41.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:41.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:41.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:41.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:41.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:41.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:41.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:41.244 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:41.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:41.262 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:41.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:41.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:41.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:41.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:41.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:41.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:41.400 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:41.408 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:41.469 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:41.477 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:41.498 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:41.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:41.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:41.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:41.759 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:41.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:41.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:41.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:41.793 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-23T18:46:41.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-23T18:46:41.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:41.840 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:41.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:41.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:41.861 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:41.870 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:41.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:41.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-23T18:46:41.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:41.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:41.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:42.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-23T18:46:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T18:46:42.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:42.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:42.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:42.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:42.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:42.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:42.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:42.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:42.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:42.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:42.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:42.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:42.282 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-23T18:46:42.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-23T18:46:42.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:42.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:42.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-23T18:46:42.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:42.464 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-23T18:46:42.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-23T18:46:42.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:42.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:42.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:42.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-23T18:46:42.592 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:42.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:42.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:42.613 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:42.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:42.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:42.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:42.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:42.683 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-23T18:46:42.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-23T18:46:42.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:42.733 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:42.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:42.783 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:42.790 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:42.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:42.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:42.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:42.820 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:42.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:42.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:42.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:42.857 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-23T18:46:42.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-23T18:46:42.870 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:42.876 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:42.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:42.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-23T18:46:42.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:42.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:43.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:43.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:43.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:43.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:43.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:43.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:43.109 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-23T18:46:43.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-23T18:46:43.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:43.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:43.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:43.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-23T18:46:43.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:43.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:43.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:43.262 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:43.271 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-23T18:46:43.350 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:43.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:43.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:43.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:43.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:43.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:43.397 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-23T18:46:43.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-23T18:46:43.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:43.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:43.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:43.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-23T18:46:43.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-23T18:46:43.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-23T18:46:43.512 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:43.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:43.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:43.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:43.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:43.692 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-23T18:46:43.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-23T18:46:43.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:43.733 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:43.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:43.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:43.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:43.825 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:43.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:43.841 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:43.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:43.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:43.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:43.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-23T18:46:44.005 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:44.013 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:44.019 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:44.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-23T18:46:44.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-23T18:46:44.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:44.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:44.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:44.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:44.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:44.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:44.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-23T18:46:44.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:44.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:44.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:44.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:44.328 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:44.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:44.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:44.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:44.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:44.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:44.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:44.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:44.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:44.439 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:44.448 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:44.564 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:44.570 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:44.625 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:44.633 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:44.642 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:44.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:44.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:44.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:44.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:44.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:44.692 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:44.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:44.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:44.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:44.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:44.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:44.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:44.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:44.857 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:44.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:44.908 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:44.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:44.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:44.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:44.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:44.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:45.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:45.035 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:45.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:45.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:45.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:45.062 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:45.099 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:45.130 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:45.186 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:45.193 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:45.200 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:45.207 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:45.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:45.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:45.230 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:45.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:45.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:45.276 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:45.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:45.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:45.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:45.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:45.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:45.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:45.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:45.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:45.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:45.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:45.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:45.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:45.461 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-23T18:46:45.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-23T18:46:45.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:45.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:45.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:45.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-23T18:46:45.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:45.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:45.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:45.630 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T18:46:45.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:45.691 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:45.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:45.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:45.715 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-23T18:46:45.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-23T18:46:45.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:45.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:45.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-23T18:46:45.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:45.825 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:45.834 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:45.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T18:46:45.852 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:45.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:45.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:45.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-23T18:46:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-23T18:46:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-23T18:46:46.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:46.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:46.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:46.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-23T18:46:46.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:46.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:46.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:46.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-23T18:46:46.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-23T18:46:46.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:46.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:46.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:46.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-23T18:46:46.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:46.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:46.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:46.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T18:46:46.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:46.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-23T18:46:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-23T18:46:46.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:46.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:46.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:46.627 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:46.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:46.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T18:46:46.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:46.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:46.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:46.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:46.670 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:46:46.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:46:46.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:46.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:46.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:46.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:46:46.734 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:46.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:46.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:46.890 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T18:46:46.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:46.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:46.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:46.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:46.933 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-23T18:46:46.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-23T18:46:46.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:46.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:47.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:47.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-23T18:46:47.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:47.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:47.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:47.065 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T18:46:47.149 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:47.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:47.230 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:47.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:47.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-23T18:46:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:47.282 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:47.289 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:47.298 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:47.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:47.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:47.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:47.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:47.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:47.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:47.501 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:47.514 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:47.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:47.592 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:47.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:47.649 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:47.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:47.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:47.797 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:47.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:47.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:47.829 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:47.836 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:47.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:47.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:47.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:47.916 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:47.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:47.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:48.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:48.111 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:48.129 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:48.138 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:48.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:48.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:48.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:48.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:48.207 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:48.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:48.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:48.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:48.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:48.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:48.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:48.461 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-23T18:46:48.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-23T18:46:48.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:48.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:48.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-23T18:46:48.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:48.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:48.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:48.665 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:48.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:48.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:48.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:48.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-23T18:46:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-23T18:46:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-23T18:46:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:48.863 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:48.869 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:48.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:48.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:48.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:48.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:48.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:48.953 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:46:48.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:46:48.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:49.004 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:49.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:46:49.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:49.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:49.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:49.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:49.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:49.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:49.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:49.189 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:49.198 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:49.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:49.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:49.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-23T18:46:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-23T18:46:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:49.255 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-23T18:46:49.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-23T18:46:49.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:49.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:49.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:49.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-23T18:46:49.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:49.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:49.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:49.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T18:46:49.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:49.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:49.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:49.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:49.496 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-23T18:46:49.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-23T18:46:49.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:49.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:49.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:49.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-23T18:46:49.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:49.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:49.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T18:46:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:49.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:49.743 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:49.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:49.762 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-23T18:46:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-23T18:46:49.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:49.786 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:49.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:49.802 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:49.848 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:49.856 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:49.862 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:49.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-23T18:46:49.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:49.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:49.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:49.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:49.981 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:46:50.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:46:50.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:46:50.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:50.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:50.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-23T18:46:50.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-23T18:46:50.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:50.072 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:50.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T18:46:50.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:50.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:50.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:50.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:50.136 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:50.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:50.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:50.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:50.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:50.425 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:50.432 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:50.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:50.446 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:50.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:50.517 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:50.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:50.599 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:50.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:50.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:50.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:50.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:50.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:50.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:50.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:50.689 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:50.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:50.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:50.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:50.725 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:50.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:50.818 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:50.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:50.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:50.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:50.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:50.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:50.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:50.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:50.963 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:50.972 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:50.980 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:51.035 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:51.045 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:51.054 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:51.062 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:51.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:51.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:51.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:51.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:51.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:51.260 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:51.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:51.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:51.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:51.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:51.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:51.305 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:51.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:51.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:51.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:51.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:51.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:51.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:51.458 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-23T18:46:51.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-23T18:46:51.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:51.557 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:51.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:51.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:51.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:51.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:51.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:51.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-23T18:46:51.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:51.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:51.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:51.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:51.680 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-23T18:46:51.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-23T18:46:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-23T18:46:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-23T18:46:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:51.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:51.916 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-23T18:46:51.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-23T18:46:51.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:51.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:52.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-23T18:46:52.024 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:52.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:52.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:52.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:46:52.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:52.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:52.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:52.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:52.128 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-23T18:46:52.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-23T18:46:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:46:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:46:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-23T18:46:52.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:52.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T18:46:52.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:52.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:52.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:52.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:52.420 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-23T18:46:52.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-23T18:46:52.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:52.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:52.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:52.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:52.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:52.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:52.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:52.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-23T18:46:52.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:52.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:52.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:52.676 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-23T18:46:52.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-23T18:46:52.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:52.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:52.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:52.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-23T18:46:52.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:52.723 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:52.775 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:46:52.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:52.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:52.793 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:52.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:52.891 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:52.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:52.953 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:52.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-23T18:46:52.965 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:52.974 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:52.982 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:52.990 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:52.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:53.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:53.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:53.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:53.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:53.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:53.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:53.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:53.084 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:53.142 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:53.148 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:53.247 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:53.255 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:53.263 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:53.275 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:53.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:53.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:53.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:53.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:53.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:53.330 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:53.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:53.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:53.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:53.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:53.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:53.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:53.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:53.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:53.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:53.566 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:53.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:53.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:53.586 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:53.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:53.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:53.627 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:53.634 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:53.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:53.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:53.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:53.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:53.815 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:53.838 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:53.845 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:53.852 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:53.859 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:53.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:53.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:53.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:53.941 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:53.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:53.954 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:53.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:53.969 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:54.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:54.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:54.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:54.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:54.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:54.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:54.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:54.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:54.161 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:54.167 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:54.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:54.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:54.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-23T18:46:54.222 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:54.231 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:54.240 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:54.246 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:54.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:54.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:54.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:54.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:54.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:54.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:54.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:54.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:54.462 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:54.471 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:54.479 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:54.505 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:54.513 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:54.522 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:54.528 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:54.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:54.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:54.626 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:54.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:54.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:54.700 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:54.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:54.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:54.720 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:54.728 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:54.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:54.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:54.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:54.802 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:54.810 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:54.817 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:54.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:54.883 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:54.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:54.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:54.940 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:54.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:54.962 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:54.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:54.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:54.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:54.995 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:55.004 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:55.040 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:55.046 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:55.053 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:55.061 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:55.068 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:55.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:55.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:55.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:55.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:55.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:55.209 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:55.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:55.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:55.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:55.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:55.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:55.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:55.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:55.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:55.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:55.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:55.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:55.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:55.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:55.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-23T18:46:55.505 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:55.512 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:55.520 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:55.529 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-23T18:46:55.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-23T18:46:55.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:55.576 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:55.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:55.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-23T18:46:55.602 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:55.611 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:55.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:55.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T18:46:55.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:55.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:55.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:55.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:55.817 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-23T18:46:55.826 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-23T18:46:55.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:55.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:55.848 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:46:55.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:46:55.862 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-23T18:46:55.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:55.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:55.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T18:46:55.950 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:55.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:56.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:56.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:56.093 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:56.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:56.109 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:56.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:56.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:56.157 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:56.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:56.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:56.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:56.209 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:56.226 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:56.309 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:56.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:56.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:56.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:56.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:56.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:56.442 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:56.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:56.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:56.462 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:56.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:56.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:56.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:56.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:56.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:56.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:56.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:56.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:56.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:56.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:56.760 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:56.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:56.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:56.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:56.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:56.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:56.807 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:56.816 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:56.878 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:56.887 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:56.896 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:56.907 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:56.916 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:56.924 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:56.986 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:56.992 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:57.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:57.067 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:57.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:57.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:57.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:57.099 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:57.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:57.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:57.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:57.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:57.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:57.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:57.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:57.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:57.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:57.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:57.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:57.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:57.354 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:57.361 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:46:57.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:57.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:46:57.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-23T18:46:57.387 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:46:57.394 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:46:57.400 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:46:57.419 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-23T18:46:57.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-23T18:46:57.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:57.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:57.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:57.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-23T18:46:57.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:57.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:57.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:57.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-23T18:46:57.674 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T18:46:57.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:57.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:57.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:57.701 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:46:57.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:46:57.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:57.738 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:57.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:57.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:57.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:57.765 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:57.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:57.860 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:57.867 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:57.930 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:57.937 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:57.945 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:57.954 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:57.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:57.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:57.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:58.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:58.064 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:46:58.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:46:58.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:58.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:58.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:58.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:58.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:58.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:58.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:58.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:58.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:58.278 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:58.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:58.298 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:46:58.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:58.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:58.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:58.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:46:58.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:58.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:58.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:46:58.442 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:46:58.494 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:46:58.500 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:46:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:46:58.514 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:46:58.523 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:46:58.530 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:46:58.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:58.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:58.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:58.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:58.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:58.596 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:46:58.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:46:58.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:46:58.617 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:58.680 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:58.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:46:58.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:58.767 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:58.775 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:46:58.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:58.789 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:58.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:58.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:58.811 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-23T18:46:58.845 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-23T18:46:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:58.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:58.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:58.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:58.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:58.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:58.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:59.083 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-23T18:46:59.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T18:46:59.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:59.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:59.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:59.117 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-23T18:46:59.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-23T18:46:59.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:59.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-23T18:46:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-23T18:46:59.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:59.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:59.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:59.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:59.387 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-23T18:46:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-23T18:46:59.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:46:59.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:59.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:59.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-23T18:46:59.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:59.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:59.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:59.496 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T18:46:59.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:59.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:59.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:59.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:59.595 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-23T18:46:59.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-23T18:46:59.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:59.662 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:59.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:59.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:46:59.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-23T18:46:59.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:46:59.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:46:59.758 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:46:59.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:46:59.775 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-23T18:46:59.783 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-23T18:46:59.843 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:46:59.851 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:46:59.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:46:59.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-23T18:46:59.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:46:59.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:46:59.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:46:59.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T18:47:00.001 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T18:47:00.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:00.054 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:00.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:00.075 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:00.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-23T18:47:00.089 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:00.160 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:00.168 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:00.246 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:00.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:00.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:00.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:00.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:00.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:00.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:00.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:00.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:00.333 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:00.341 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:00.348 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:00.356 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:00.363 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:00.369 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:00.448 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:00.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:00.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:00.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:00.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:00.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:00.551 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:00.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:00.565 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:00.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:00.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:00.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:00.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:00.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:00.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:00.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:00.648 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:00.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:00.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:00.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:00.772 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:00.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:00.787 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:00.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:00.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:00.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:00.813 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:00.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:00.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:00.851 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:00.857 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:00.865 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:00.874 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:00.884 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:00.934 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:00.940 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:00.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:01.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:01.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:01.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:01.029 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:01.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:01.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:01.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:01.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:01.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:01.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:01.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:01.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:01.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:01.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:01.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:01.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:01.285 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:01.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-23T18:47:01.332 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:01.340 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:01.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-23T18:47:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:02.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:02.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:02.767 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:02.775 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:02.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:02.788 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:02.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:02.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:02.844 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:02.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:02.858 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:02.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:02.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:02.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:02.937 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:02.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:03.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:03.011 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:03.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:03.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:03.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:03.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:03.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:03.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:03.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:03.120 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:03.128 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:03.136 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:03.144 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:03.290 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:03.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:03.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:03.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:03.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:03.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:03.350 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:03.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:03.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:03.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:03.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:03.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:03.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:03.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:03.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:03.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:03.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:03.596 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:03.603 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:03.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:03.617 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:03.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-23T18:47:03.633 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:03.642 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:03.677 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:03.684 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-23T18:47:03.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-23T18:47:03.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:03.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:03.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:03.775 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:03.782 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:03.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:03.849 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:03.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T18:47:03.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:03.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:03.875 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:03.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:03.890 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:03.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:03.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:03.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:03.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:03.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:03.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:03.953 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:04.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:04.068 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:04.112 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:04.119 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:04.127 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:04.136 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:04.144 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:04.150 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:04.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:04.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:04.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:04.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:04.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:04.217 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:04.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:04.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:04.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:04.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:04.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:04.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:04.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:04.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:04.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:04.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:04.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:04.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:04.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:04.433 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:04.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:04.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:04.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:04.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:04.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:04.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:04.598 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:04.669 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:04.676 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:04.687 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:04.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:04.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:04.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:04.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:04.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:04.743 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:04.758 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:04.834 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:04.897 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:04.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:04.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:04.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:04.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:04.981 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-23T18:47:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-23T18:47:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:05.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:05.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:05.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:05.161 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:05.169 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-23T18:47:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-23T18:47:05.183 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-23T18:47:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:05.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-23T18:47:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:05.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:05.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:05.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:05.272 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-23T18:47:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:05.339 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:05.473 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:05.483 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:05.513 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:05.522 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:05.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:05.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:05.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:05.674 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:05.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:05.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:05.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:05.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:05.866 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:05.879 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:05.939 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:05.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:05.952 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:05.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:06.098 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:06.110 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:06.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:06.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:06.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:06.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:06.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:06.199 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:06.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:06.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:06.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:06.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:06.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:06.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:06.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:06.404 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:06.411 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:06.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:06.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-23T18:47:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:06.458 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:06.467 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:06.474 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:06.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:06.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:06.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:06.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:06.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:06.714 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:06.720 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:06.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:06.800 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:06.822 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:06.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:07.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:07.111 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:07.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:07.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:07.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:07.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:07.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:07.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:07.305 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:07.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:07.319 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:07.326 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:07.334 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:07.340 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:07.346 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:07.354 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:07.361 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:07.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:07.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:07.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:07.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:07.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:07.470 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:07.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:07.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:07.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:07.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:07.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:07.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:07.616 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:07.682 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:07.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:07.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:07.745 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:07.753 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:07.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:07.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:07.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-23T18:47:07.785 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:07.849 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:07.901 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:07.908 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:07.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:07.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:07.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:07.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:07.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:08.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:08.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:08.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:08.160 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:08.166 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:08.172 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:08.179 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:08.185 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:08.191 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:08.233 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:08.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:08.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:08.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:08.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:08.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:08.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:08.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:08.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:08.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:08.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:08.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:08.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:08.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:08.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:08.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:08.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:08.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:08.676 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:08.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:08.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:08.703 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:08.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:08.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:08.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:08.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:08.852 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:08.861 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:08.870 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:08.881 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:08.940 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:08.946 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:09.006 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:09.014 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:09.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:09.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:09.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:09.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:09.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:09.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:09.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:09.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:09.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:09.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:09.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:09.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:09.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:09.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:09.372 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:47:09.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:47:09.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:47:09.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:09.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:09.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:47:09.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:09.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:09.552 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:09.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:09.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:09.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:09.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:09.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:09.678 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-23T18:47:09.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-23T18:47:09.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:47:09.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:09.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:09.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-23T18:47:09.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-23T18:47:09.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:09.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:09.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-23T18:47:09.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T18:47:09.840 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:09.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:09.858 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:09.866 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:09.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:09.917 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:09.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-23T18:47:09.994 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:10.003 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:10.010 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:10.018 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-23T18:47:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-23T18:47:10.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:10.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:10.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:10.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:10.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:10.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:10.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:10.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T18:47:10.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:10.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:10.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:10.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:10.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-23T18:47:10.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-23T18:47:10.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:10.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:10.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:10.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:10.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:10.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:10.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:10.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:10.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:10.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:10.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:10.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:10.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:10.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:10.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:10.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:10.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:10.667 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:10.676 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:10.709 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:10.716 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:10.722 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:10.728 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:10.737 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:10.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:10.786 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:10.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:10.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:10.881 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:10.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:10.899 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:10.907 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:10.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:10.980 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:11.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:11.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:11.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:11.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:11.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:11.128 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:11.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:11.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:11.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:11.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:11.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:11.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:11.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:11.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:11.284 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:11.345 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:11.379 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:11.387 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:11.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:11.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:11.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:11.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:11.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:11.533 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:11.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:11.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:11.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:11.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:11.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:11.664 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:11.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:11.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:11.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:11.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:11.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:11.737 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:11.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:11.818 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-23T18:47:11.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-23T18:47:11.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:11.842 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:11.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:11.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-23T18:47:11.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-23T18:47:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:47:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:47:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:47:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:47:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-23T18:47:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-23T18:47:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-23T18:47:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-23T18:47:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-23T18:47:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-23T18:47:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T18:47:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T18:47:12.844 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:12.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:12.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:12.870 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:13.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:13.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:13.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:13.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:13.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:13.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:13.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:13.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:13.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:13.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:13.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:13.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:13.533 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:13.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:13.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:13.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:13.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:13.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:13.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:13.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:13.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:13.728 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:13.737 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:13.775 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:13.783 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:13.791 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:13.845 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:13.854 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:13.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:13.871 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:13.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:13.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:13.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:14.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:14.066 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:14.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:14.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:14.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:14.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:14.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:14.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:14.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:14.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:14.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:14.231 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-23T18:47:14.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-23T18:47:14.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:14.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:14.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:14.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:14.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:14.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:14.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:14.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T18:47:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T18:47:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:14.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:14.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-23T18:47:14.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:14.618 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:14.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:14.683 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-23T18:47:14.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-23T18:47:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:47:14.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:14.733 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:14.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-23T18:47:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:14.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:14.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:14.855 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:14.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:14.901 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-23T18:47:14.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-23T18:47:14.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:14.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:14.983 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-23T18:47:14.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:14.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:15.006 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T18:47:15.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:15.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:15.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:15.126 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-23T18:47:15.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-23T18:47:15.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:15.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:15.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:15.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-23T18:47:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:15.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:15.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:15.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T18:47:15.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:15.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:15.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:15.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-23T18:47:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-23T18:47:15.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:15.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:15.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-23T18:47:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:15.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:15.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:15.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:15.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:15.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:15.569 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-23T18:47:15.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-23T18:47:15.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:15.627 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:15.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:15.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-23T18:47:15.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:15.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-23T18:47:15.751 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-23T18:47:15.810 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:15.871 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:15.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:15.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:15.896 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-23T18:47:15.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-23T18:47:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-23T18:47:16.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-23T18:47:16.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-23T18:47:16.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:16.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:16.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:16.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:16.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:16.168 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:16.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:16.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:16.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-23T18:47:16.199 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:16.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:16.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:16.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:16.442 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:16.451 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:16.458 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:16.465 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:16.474 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:16.483 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:16.518 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:16.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:16.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:16.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:16.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:16.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:16.648 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:16.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:16.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:16.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:16.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:16.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:16.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:16.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:16.778 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:16.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:16.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:16.852 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:16.859 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:16.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:16.912 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:16.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:16.925 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:16.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:16.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:16.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:16.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:16.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:17.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:17.025 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:17.031 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:17.036 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:17.043 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:17.050 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:17.058 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:17.064 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:17.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:17.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:17.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:17.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:17.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:17.161 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:17.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:17.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:17.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:17.231 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:17.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:17.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:17.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:17.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:17.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:17.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-23T18:47:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-23T18:47:17.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:47:17.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:17.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:17.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:17.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:17.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:17.564 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:17.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:17.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:17.625 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-23T18:47:17.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T18:47:17.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:17.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:17.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:17.704 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:17.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:17.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:17.759 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:17.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:17.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:17.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:17.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:17.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:17.830 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-23T18:47:17.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-23T18:47:17.876 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:17.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:17.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:17.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:17.961 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:17.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:17.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:17.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:17.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:18.080 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:18.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:18.141 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:18.147 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:18.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:18.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:18.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-23T18:47:18.175 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:18.226 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:18.233 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:18.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-23T18:47:18.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-23T18:47:18.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:18.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:18.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:18.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:18.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:18.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:18.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:18.429 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-23T18:47:18.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-23T18:47:18.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:18.643 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:18.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:18.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:18.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:18.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:18.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:18.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:18.853 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:18.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:18.869 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:18.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:18.885 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:18.892 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:18.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:18.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:19.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:19.043 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:19.050 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:19.058 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:19.066 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:19.160 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:19.166 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:19.174 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:19.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:19.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:19.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:19.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:19.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:19.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:19.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:19.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:19.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:19.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:19.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:19.557 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:19.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:19.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:19.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:19.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:19.705 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:19.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:19.718 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:19.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:19.843 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:19.850 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:19.858 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:19.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:19.870 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:19.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:19.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:19.941 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:20.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:20.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:20.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:20.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:20.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:20.065 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:20.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:20.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:20.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:20.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:20.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:20.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:20.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:20.272 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-23T18:47:20.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-23T18:47:20.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:20.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:20.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:20.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:20.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:20.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:20.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:20.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:20.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:20.486 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:20.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:20.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:20.559 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-23T18:47:20.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-23T18:47:20.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:20.580 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:20.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:20.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:20.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:20.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:20.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:20.718 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:20.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:20.732 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:20.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:20.744 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:20.786 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:20.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:20.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:20.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-23T18:47:20.844 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:20.853 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:20.861 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:20.867 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:20.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:20.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:20.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:20.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:20.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:20.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:21.008 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:21.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:21.027 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:21.074 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:21.083 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:21.115 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:21.122 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:21.130 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:21.139 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:21.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:21.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:21.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:21.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:21.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:21.287 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:21.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:21.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:21.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:21.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:21.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:21.385 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:21.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:21.455 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:21.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:21.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:21.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:21.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:21.542 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:21.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:21.617 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:21.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:21.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:21.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:21.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:21.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:21.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:21.690 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:21.698 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:21.734 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:21.742 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:21.748 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:21.754 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:21.761 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:21.767 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:21.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:21.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:21.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:21.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:21.908 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:21.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:21.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:21.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:21.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:21.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:22.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:22.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:22.065 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:22.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:22.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:22.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:22.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:22.149 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:22.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:22.235 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:22.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-23T18:47:22.251 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:22.260 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:22.267 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:22.275 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:22.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:22.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:22.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:22.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:22.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:22.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:22.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:22.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:22.477 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:22.485 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:22.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:22.548 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:22.556 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:22.565 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:22.573 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:22.582 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:22.589 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:22.660 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:22.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:22.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:22.728 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:22.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:22.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:22.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:22.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:22.890 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:22.898 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:22.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:22.912 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:22.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:22.982 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:22.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:23.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-23T18:47:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-23T18:47:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-23T18:47:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T18:47:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-23T18:47:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:24.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:24.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:24.508 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:24.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:24.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:24.572 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:24.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:24.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:24.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:24.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:24.668 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:24.677 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:24.714 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:24.722 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:24.780 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:24.790 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:24.799 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:24.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:24.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:24.819 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:24.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:24.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:24.940 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:24.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:24.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:24.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:24.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:24.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:24.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:25.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:25.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:25.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:25.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:25.073 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:25.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:25.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:25.094 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-23T18:47:25.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-23T18:47:25.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:25.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:25.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:25.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:25.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:25.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:25.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:25.304 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:25.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:25.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:25.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:25.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-23T18:47:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-23T18:47:25.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:25.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:25.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-23T18:47:25.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:25.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:25.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:25.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:25.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:25.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:25.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:25.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-23T18:47:25.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-23T18:47:25.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:25.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:25.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:25.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:25.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:25.807 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:25.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:25.878 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:25.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:25.891 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:25.898 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:25.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:25.912 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-23T18:47:25.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-23T18:47:25.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:25.963 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:25.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:26.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:26.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:26.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:26.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:26.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:26.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:26.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:26.153 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-23T18:47:26.160 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-23T18:47:26.169 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:47:26.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:26.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:26.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:26.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:26.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:26.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:26.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:26.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:26.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:26.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:26.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:26.382 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-23T18:47:26.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-23T18:47:26.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:26.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:26.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:26.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:26.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:26.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:26.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:26.548 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:26.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:26.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:26.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:26.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:26.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-23T18:47:26.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-23T18:47:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:26.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:26.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:26.814 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:26.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:26.830 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:26.837 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:26.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:26.884 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-23T18:47:26.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-23T18:47:26.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:26.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:26.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:26.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:26.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:27.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:27.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:27.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:27.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:27.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:27.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:27.164 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-23T18:47:27.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-23T18:47:27.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:27.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:27.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:27.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:47:27.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-23T18:47:27.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:27.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:27.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:27.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:27.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:27.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:27.402 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-23T18:47:27.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-23T18:47:27.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:27.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:27.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:27.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:27.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:27.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:27.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:27.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:27.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:27.689 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-23T18:47:27.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-23T18:47:27.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:27.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:27.720 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:27.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:27.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:27.787 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:27.795 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:27.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:27.858 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:27.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:27.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:27.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:27.899 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-23T18:47:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-23T18:47:28.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:28.050 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:28.058 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:28.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:28.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:28.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:28.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:28.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:28.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:28.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:28.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:28.206 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:28.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:28.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:28.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-23T18:47:28.282 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:28.346 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:28.352 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-23T18:47:28.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-23T18:47:28.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:28.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:28.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:28.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:28.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:28.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:28.446 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T18:47:28.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:28.460 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:28.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:28.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:28.523 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-23T18:47:28.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-23T18:47:28.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:28.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:28.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:28.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:28.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:28.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:28.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:28.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-23T18:47:28.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:28.701 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:28.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:28.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:28.719 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:28.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:28.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:28.861 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:28.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:28.929 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:28.938 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:28.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:28.957 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:28.968 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:29.026 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:29.032 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:29.040 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:29.104 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:29.113 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:29.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:29.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:29.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:29.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:29.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:29.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:29.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:29.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:29.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:29.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:29.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:29.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:29.494 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:29.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:29.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:29.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:29.623 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:29.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:29.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:29.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:29.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:29.656 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:29.700 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:29.706 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:29.734 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:29.741 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:29.748 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:29.755 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:29.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:29.766 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:29.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:29.809 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:29.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:29.889 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:29.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:29.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:29.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:29.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:29.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:29.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:29.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:30.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:30.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:30.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:30.034 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:30.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:30.081 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:30.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-23T18:47:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:30.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:30.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:30.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:30.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:30.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:30.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:30.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:30.328 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:30.356 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:30.362 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:30.433 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:30.440 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:30.447 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:30.454 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:30.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:30.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:30.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:30.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:30.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:30.595 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:30.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:30.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:30.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:30.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:30.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:30.665 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:30.671 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:30.724 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:30.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:30.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:30.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:30.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:30.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:30.762 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:30.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:30.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:30.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:30.849 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:30.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:30.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:30.873 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:30.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:30.940 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:30.947 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:31.025 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:31.030 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:31.036 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:31.068 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:31.076 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:31.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:31.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:31.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:31.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:31.214 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:31.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:31.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:31.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:31.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:31.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:31.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:31.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:31.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:31.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:31.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:31.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:31.416 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:31.508 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:31.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:31.561 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-23T18:47:31.577 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:31.584 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:31.592 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:31.600 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:31.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:31.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:31.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:31.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:31.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:31.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:31.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:31.772 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:31.816 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:31.823 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:31.890 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:31.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:31.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:31.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:31.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:32.057 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:32.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:32.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:32.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:32.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:32.096 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:32.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:32.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:32.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:32.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:32.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:32.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:32.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:32.281 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:32.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:32.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:32.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:32.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:32.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:32.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:32.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:32.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:32.480 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:32.487 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:32.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:32.553 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:32.562 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:32.569 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:32.576 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:32.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:32.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:32.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:32.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:32.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:32.726 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:32.732 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:32.741 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:32.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:32.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:32.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:32.829 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:32.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:32.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:32.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:32.896 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:32.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:32.912 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:32.964 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:32.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:33.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:33.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-23T18:47:33.068 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:33.074 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:33.080 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:33.089 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-23T18:47:33.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-23T18:47:33.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:33.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:33.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:33.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T18:47:33.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:33.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:33.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:33.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:33.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:33.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:33.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:33.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:33.410 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:47:33.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:47:33.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:47:33.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:33.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:33.506 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-23T18:47:33.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:33.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:33.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:33.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:33.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:33.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:33.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:33.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:33.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:33.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:33.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:33.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:33.805 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:33.846 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:33.854 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:33.861 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:33.869 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:33.878 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:33.885 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:33.964 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:33.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:34.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:34.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:34.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:34.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:34.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:34.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:34.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:34.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:34.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:34.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:34.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:34.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:34.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:34.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:34.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:34.278 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:34.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:34.333 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:34.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:34.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:34.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:34.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:34.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:34.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:34.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:34.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:34.568 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:34.577 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:34.586 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:34.594 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:34.601 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:34.647 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:34.655 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:34.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:34.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:34.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:34.747 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:34.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:34.764 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:34.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:34.826 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:34.834 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:34.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:34.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:34.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:34.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:34.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:34.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:34.912 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:34.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:34.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:34.992 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:35.000 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:35.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:35.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:35.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-23T18:47:35.029 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:35.098 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:35.104 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:35.139 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-23T18:47:35.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-23T18:47:35.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:35.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:35.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:35.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:35.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:35.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:35.315 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:35.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T18:47:35.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:35.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:35.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-23T18:47:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-23T18:47:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:35.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:35.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:35.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:35.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T18:47:35.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:35.591 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:35.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:35.609 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-23T18:47:35.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-23T18:47:35.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:35.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:35.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:35.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:35.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:35.764 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:35.773 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:35.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T18:47:35.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:35.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:35.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:35.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-23T18:47:35.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-23T18:47:35.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:35.920 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:35.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:35.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:35.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:35.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:36.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T18:47:36.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:36.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:36.030 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:36.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:36.044 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-23T18:47:36.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-23T18:47:36.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:36.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:36.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:36.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:36.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:36.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:36.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:36.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T18:47:36.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:36.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:36.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:36.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:36.317 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-23T18:47:36.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-23T18:47:36.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:36.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:36.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:36.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:36.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:36.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:36.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:36.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:36.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:36.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:36.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:36.569 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-23T18:47:36.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-23T18:47:36.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:36.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:36.654 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:36.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:36.706 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:36.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:36.775 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:36.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:36.791 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-23T18:47:36.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-23T18:47:36.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:36.861 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:36.900 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T18:47:36.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:36.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:36.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:36.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:36.971 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:36.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:37.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-23T18:47:37.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-23T18:47:37.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:37.080 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:37.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:37.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:37.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:37.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:37.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:37.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:37.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:37.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:37.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:37.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:37.396 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:47:37.447 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:47:37.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:37.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:37.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-23T18:47:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-23T18:47:37.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:37.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:37.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:37.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:37.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:37.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:37.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:37.578 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-23T18:47:37.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-23T18:47:37.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:37.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:37.738 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-23T18:47:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:37.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:37.772 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:37.780 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:37.787 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:37.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:37.847 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-23T18:47:37.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-23T18:47:37.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:37.868 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:37.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:38.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:38.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:38.043 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:38.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:38.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:38.064 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:38.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:38.078 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-23T18:47:38.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-23T18:47:38.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:38.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:38.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:38.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:38.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:38.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:38.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:38.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:38.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:38.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:38.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:38.375 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:38.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:38.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:38.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:38.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:38.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:38.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:38.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:38.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:38.498 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:38.508 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:38.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:38.613 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:38.663 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:38.673 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:38.682 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:38.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:38.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:38.717 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:38.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:38.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:38.745 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:38.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:38.818 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:38.826 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:38.833 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:38.841 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:38.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:38.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:38.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:39.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:39.021 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:39.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:39.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:39.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-23T18:47:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-23T18:47:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-23T18:47:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-23T18:47:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-23T18:47:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-23T18:47:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T18:47:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-23T18:47:40.477 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:40.492 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:40.498 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:40.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:40.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:40.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:40.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:40.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:40.602 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:40.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:40.623 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:40.658 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:40.728 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:40.796 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:40.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:40.853 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:40.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:40.903 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:40.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:40.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:40.927 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:40.935 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:40.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:41.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:41.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:41.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:41.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:41.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:41.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:41.118 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:41.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:41.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:41.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:41.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:41.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:41.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:41.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:41.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:41.225 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:41.254 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:41.314 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:41.382 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:41.388 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:41.395 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:41.402 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:41.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:41.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:41.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:41.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:41.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:41.498 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:41.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:41.704 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:41.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:41.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:41.728 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:41.737 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:41.745 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:41.753 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:41.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:41.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:41.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-23T18:47:41.829 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:41.836 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:41.843 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:41.849 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:41.858 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:41.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:41.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:41.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:41.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:41.996 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:42.005 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:42.026 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:42.034 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:42.043 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:42.070 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:42.079 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:42.086 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:42.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:42.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:42.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:42.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:42.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:42.323 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:42.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:42.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:42.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:42.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:42.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:42.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:42.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:42.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:42.669 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:42.677 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:42.684 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:42.691 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:42.698 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:42.738 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:42.745 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:42.751 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:42.758 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:42.764 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:42.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:42.811 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:42.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:42.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:42.937 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:42.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:42.952 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:42.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:42.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:42.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:42.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:42.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:43.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:43.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:43.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:43.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:43.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:43.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:43.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-23T18:47:43.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-23T18:47:43.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:43.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:43.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:43.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-23T18:47:43.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:43.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:43.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:43.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T18:47:43.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:43.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:43.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:43.290 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-23T18:47:43.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-23T18:47:43.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:43.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:43.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:43.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:43.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:43.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:43.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:43.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-23T18:47:43.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:43.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:43.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:43.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:43.621 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-23T18:47:43.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-23T18:47:43.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:43.646 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:43.660 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:43.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:47:43.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:43.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:43.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:43.853 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T18:47:43.860 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:43.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:43.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:43.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:43.888 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-23T18:47:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T18:47:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:43.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:43.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:43.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:43.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:43.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:43.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:44.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T18:47:44.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:44.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:44.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:44.373 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:44.380 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:44.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:44.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:44.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-23T18:47:44.406 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:44.413 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:44.474 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:44.484 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-23T18:47:44.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-23T18:47:44.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:47:44.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:44.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:44.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-23T18:47:44.599 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:44.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:44.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:44.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:44.712 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:44.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:44.729 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:44.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:44.747 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-23T18:47:44.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-23T18:47:44.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:44.826 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:44.834 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:44.843 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:44.852 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:44.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:44.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:44.993 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:45.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:45.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:45.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:45.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:45.089 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:47:45.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:47:45.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:47:45.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:45.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:45.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:47:45.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:45.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:45.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:45.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:45.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:45.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:45.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:45.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:45.435 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:45.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:45.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:45.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:45.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:45.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:45.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:45.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:45.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:45.542 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:45.549 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:45.556 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:45.563 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:45.571 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:45.619 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:45.675 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:45.728 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:45.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:45.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:45.780 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:45.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:45.798 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:45.807 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:45.865 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:45.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:45.882 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:45.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:45.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:45.908 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:45.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:46.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:46.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:46.081 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:46.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:46.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:46.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:46.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:46.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:46.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:46.140 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:46.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:46.205 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:46.217 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:46.226 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:46.235 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:46.285 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:46.364 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:46.422 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:46.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:46.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:46.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:46.555 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:46.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:46.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:46.626 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:46.729 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:46.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:46.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:46.823 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-23T18:47:46.830 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-23T18:47:46.837 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:46.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:46.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:46.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:46.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:46.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:46.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:46.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:46.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:46.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:46.960 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:47.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:47.081 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-23T18:47:47.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-23T18:47:47.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:47.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:47.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:47.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-23T18:47:47.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:47.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:47.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:47.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:47.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:47.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-23T18:47:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-23T18:47:47.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:47.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:47.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:47.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T18:47:47.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:47.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:47.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:47.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:47.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:47.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:47.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:47.571 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:47.587 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:47:47.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:47.650 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24991, 2024-04-23T18:47:47.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "z", 2024-04-23T18:47:47.744 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-04-23T18:47:47.752 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-23T18:47:47.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:47.766 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-23T18:47:47.773 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:47.781 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:47.789 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:47.796 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:47:47.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:47:47.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:47:47.870 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:47.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:47:47.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:47.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T18:47:48.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:48.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:48.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:48.113 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:48.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:48.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:48.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:48.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:48.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:48.235 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:48.244 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:48.298 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:48.364 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:48.373 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:48.381 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:48.390 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:48.480 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:48.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:48.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:48.506 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:48.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:48.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:48.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:48.615 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:48.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:48.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:48.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:48.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:48.807 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:48.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:48.820 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:48.849 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:48.890 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:48.935 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:48.942 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:48.953 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:48.959 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:48.966 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:48.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:48.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:49.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:49.024 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:49.032 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:49.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:49.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:49.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:49.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:49.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:49.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:49.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:49.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:49.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:49.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:49.271 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-23T18:47:49.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-23T18:47:49.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:49.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:49.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:49.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:49.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:49.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:49.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-23T18:47:49.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:49.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:49.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:49.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:49.558 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-23T18:47:49.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-23T18:47:49.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:49.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:49.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:49.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-23T18:47:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:49.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-23T18:47:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:49.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:49.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:49.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:49.823 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-23T18:47:49.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-23T18:47:49.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:49.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:49.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:49.867 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:49.876 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:49.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:49.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:49.932 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-23T18:47:49.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:49.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:49.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:49.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:49.999 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-23T18:47:50.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-23T18:47:50.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:50.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:50.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:50.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:50.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:50.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:50.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:50.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-23T18:47:50.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:50.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:50.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:50.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:50.289 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-23T18:47:50.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-23T18:47:50.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-23T18:47:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-23T18:47:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:50.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:50.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:50.544 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-23T18:47:50.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-23T18:47:50.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:50.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:50.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:50.580 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:50.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:50.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:50.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:50.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-23T18:47:50.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:50.767 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:50.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:50.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:50.790 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-23T18:47:50.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-23T18:47:50.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:50.849 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:50.856 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:50.862 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:50.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:50.876 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:50.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:50.916 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T18:47:50.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:51.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:51.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:51.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:51.053 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-23T18:47:51.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-23T18:47:51.066 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:51.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:51.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:51.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:51.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:51.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:51.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:51.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-23T18:47:51.153 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:51.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:51.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:51.274 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:51.338 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:51.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:51.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:51.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-23T18:47:51.371 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:51.380 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:51.388 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:51.395 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-23T18:47:51.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-23T18:47:51.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:51.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:51.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:51.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:51.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:51.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T18:47:51.674 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:51.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:51.690 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:51.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:51.706 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-23T18:47:51.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-23T18:47:51.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:51.733 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-23T18:47:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T18:47:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:47:51.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:51.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:52.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:52.010 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-23T18:47:52.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-23T18:47:52.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:47:52.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:52.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:52.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:47:52.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-23T18:47:52.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-23T18:47:52.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:52.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T18:47:52.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:47:52.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:52.125 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:52.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:52.182 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-23T18:47:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-23T18:47:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:47:52.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:52.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:52.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-23T18:47:52.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:47:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-23T18:47:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:52.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T18:47:52.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:52.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:52.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:52.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:52.468 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-23T18:47:52.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-23T18:47:52.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:52.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:52.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T18:47:52.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:52.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:52.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:52.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T18:47:52.640 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T18:47:52.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:52.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:52.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:52.689 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-23T18:47:52.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-23T18:47:52.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:47:52.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:52.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:52.733 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-23T18:47:52.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-23T18:47:52.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-23T18:47:52.868 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:52.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-23T18:47:52.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T18:47:52.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:52.899 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:52.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:52.914 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-23T18:47:52.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-23T18:47:52.928 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:52.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:52.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:52.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-23T18:47:52.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:52.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:52.987 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:53.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T18:47:53.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:47:53.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:53.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:53.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:53.155 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-23T18:47:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-23T18:47:53.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:47:53.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:53.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:53.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:47:53.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-23T18:47:53.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-23T18:47:53.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:53.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T18:47:53.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:47:53.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:53.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:53.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:53.384 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:47:53.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:47:53.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:53.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:53.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:53.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:53.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:53.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:53.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:53.490 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:53.499 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:53.509 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:53.517 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:53.525 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:53.565 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:53.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:53.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:53.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:53.727 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:47:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:47:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:53.840 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:53.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:53.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:53.862 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:53.869 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:53.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:53.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:53.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:54.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:54.026 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:47:54.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:47:54.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:54.047 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:54.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:54.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:47:54.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:54.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:54.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:54.119 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:47:54.127 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:47:54.134 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:47:54.141 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:47:54.148 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:47:54.198 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:47:54.255 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:54.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:54.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:54.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:54.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:54.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:54.362 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:47:54.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:47:54.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:54.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:54.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:54.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:54.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:54.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:54.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:54.444 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:47:54.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:54.626 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:54.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:54.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-23T18:47:54.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-23T18:47:54.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:47:54.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:54.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:54.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:47:54.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:54.763 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:54.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:54.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T18:47:54.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:54.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:54.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:54.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:54.922 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-23T18:47:54.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-23T18:47:54.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:54.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:55.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:55.010 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-23T18:47:55.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:55.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:55.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:55.046 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-23T18:47:55.102 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-23T18:47:55.111 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-23T18:47:55.119 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-23T18:47:55.128 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-23T18:47:55.138 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:55.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-23T18:47:55.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:55.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:55.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-23T18:47:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-23T18:47:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:55.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:55.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:55.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T18:47:55.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:55.483 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:55.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:55.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:55.613 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:47:55.667 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:47:55.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:47:55.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:55.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:55.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:47:55.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:55.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:55.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:55.727 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T18:47:55.755 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:55.762 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:55.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:55.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:55.787 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-23T18:47:55.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-23T18:47:55.857 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:55.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:55.983 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:55.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:47:55.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:47:56.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:56.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:56.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T18:47:56.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T18:47:56.034 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:56.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:56.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:56.108 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-23T18:47:56.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-23T18:47:56.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:47:56.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:56.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:56.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:47:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:56.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:56.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:56.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T18:47:56.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:56.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:56.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:56.315 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:56.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:47:56.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:56.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:47:56.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-23T18:47:56.373 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:47:56.382 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:47:56.390 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:47:56.398 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-23T18:47:56.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-23T18:47:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:47:56.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:56.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:56.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-23T18:47:56.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:56.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:56.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:56.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T18:47:56.592 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:56.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:56.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:56.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:56.675 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-23T18:47:56.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-23T18:47:56.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:47:56.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:56.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:56.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-23T18:47:56.845 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:56.853 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:56.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:56.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T18:47:56.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:56.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:56.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:56.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:56.904 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-23T18:47:56.934 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-23T18:47:56.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:47:56.954 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:56.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:56.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-23T18:47:56.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:57.015 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:57.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:57.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T18:47:57.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:57.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:57.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:57.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:57.143 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-23T18:47:57.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-23T18:47:57.155 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:57.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:57.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:57.204 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:57.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:57.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:57.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T18:47:57.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T18:47:57.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:57.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:57.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:57.401 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-23T18:47:57.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-23T18:47:57.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:47:57.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:57.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:57.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:47:57.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:57.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:57.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:57.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T18:47:57.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:57.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:57.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:57.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:57.704 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-23T18:47:57.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T18:47:57.760 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:47:57.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:57.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:57.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-23T18:47:57.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:57.802 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:57.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:57.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T18:47:57.823 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:57.875 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:57.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:57.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:57.896 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-23T18:47:57.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-23T18:47:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:58.015 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:58.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:58.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:47:58.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:58.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:58.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:47:58.104 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-23T18:47:58.112 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-23T18:47:58.118 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-23T18:47:58.124 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:47:58.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T18:47:58.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:58.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:58.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:58.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:58.228 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-23T18:47:58.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-23T18:47:58.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:58.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:58.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:58.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:58.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:58.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:58.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:58.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T18:47:58.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:58.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:58.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:58.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:58.500 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-23T18:47:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-23T18:47:58.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:58.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:58.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:58.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:47:58.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:58.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:58.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:58.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T18:47:58.734 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:58.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:58.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:58.770 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-23T18:47:58.833 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-23T18:47:58.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:47:58.851 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:58.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:58.867 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:47:58.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:58.923 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:59.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:59.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T18:47:59.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:59.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:59.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:59.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:59.136 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-23T18:47:59.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-23T18:47:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T18:47:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:47:59.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:59.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:59.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-23T18:47:59.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-23T18:47:59.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:59.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:59.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:47:59.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T18:47:59.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:59.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:59.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:59.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-23T18:47:59.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:59.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:59.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:59.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:59.667 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-23T18:47:59.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-23T18:47:59.725 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:59.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:47:59.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:47:59.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T18:47:59.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:47:59.759 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:47:59.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:47:59.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-23T18:47:59.782 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:47:59.845 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:47:59.855 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:47:59.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:47:59.873 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-23T18:47:59.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-23T18:47:59.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:47:59.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:00.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:00.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T18:48:00.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:00.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:00.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:00.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:48:00.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:00.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:00.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:00.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:00.187 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-23T18:48:00.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-23T18:48:00.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:00.211 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:00.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:00.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T18:48:00.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:00.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:00.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:00.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-23T18:48:00.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:00.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:00.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:00.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:00.427 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-23T18:48:00.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-23T18:48:00.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:00.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:00.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:00.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T18:48:00.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:00.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:00.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:00.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-23T18:48:00.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:00.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:00.680 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:00.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:00.694 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-23T18:48:00.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-23T18:48:00.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:00.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:00.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:00.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T18:48:00.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:00.804 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:00.812 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:00.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-23T18:48:00.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:00.864 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:00.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:00.973 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-23T18:48:00.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-23T18:48:00.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:00.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:01.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:01.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T18:48:01.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:01.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:01.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:01.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-23T18:48:01.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:01.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:01.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:01.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:01.214 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-23T18:48:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-23T18:48:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T18:48:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-23T18:48:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-23T18:48:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-23T18:48:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T18:48:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-23T18:48:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-23T18:48:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-23T18:48:01.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T18:48:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:02.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:48:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-23T18:48:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-23T18:48:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T18:48:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-23T18:48:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-23T18:48:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-23T18:48:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T18:48:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:48:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-23T18:48:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-23T18:48:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:02.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T18:48:02.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:02.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:02.761 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:02.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-23T18:48:02.826 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:02.835 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:02.844 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:02.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:02.861 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-23T18:48:02.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-23T18:48:02.952 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:03.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:03.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:03.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T18:48:03.035 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:03.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:03.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:03.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-23T18:48:03.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:03.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:03.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:03.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:03.137 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-23T18:48:03.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-23T18:48:03.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:03.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T18:48:03.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:03.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:03.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:03.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-23T18:48:03.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:03.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:03.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:03.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:03.374 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-23T18:48:03.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-23T18:48:03.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:03.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:03.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:03.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:03.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:03.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:03.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T18:48:03.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:03.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:03.615 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:03.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:03.633 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-23T18:48:03.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-23T18:48:03.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:03.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:03.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:03.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:48:03.726 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:03.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:03.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:03.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T18:48:03.789 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:03.859 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:03.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:03.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:03.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-23T18:48:03.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T18:48:03.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:03.954 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:03.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:03.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-23T18:48:03.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:03.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:04.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:04.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T18:48:04.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:04.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:04.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:04.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:04.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-23T18:48:04.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-23T18:48:04.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:04.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:04.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:04.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-23T18:48:04.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:04.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:04.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:04.310 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T18:48:04.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:04.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:04.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:04.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:04.387 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-23T18:48:04.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-23T18:48:04.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:04.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:04.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:04.578 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-23T18:48:04.585 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:04.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:04.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:04.608 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-23T18:48:04.615 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:04.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:04.694 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-23T18:48:04.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-23T18:48:04.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:04.720 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:04.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:04.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:04.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:04.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T18:48:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:04.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-23T18:48:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-23T18:48:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:05.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:05.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:05.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:48:05.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:05.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:05.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-23T18:48:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:05.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:05.304 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-23T18:48:05.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-23T18:48:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T18:48:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T18:48:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:05.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:05.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-23T18:48:05.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-23T18:48:05.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:05.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:05.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:05.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:48:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:05.726 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:05.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-23T18:48:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:05.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:05.806 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-23T18:48:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-23T18:48:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T18:48:06.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:06.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:06.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:06.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-23T18:48:06.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-23T18:48:06.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:06.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:06.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:06.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:06.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T18:48:06.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:06.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:06.487 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:06.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:06.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:06.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:06.748 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:06.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:06.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:06.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:06.873 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-23T18:48:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-23T18:48:06.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:07.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:07.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:07.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:07.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:07.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:07.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T18:48:07.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:07.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:07.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:07.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:07.187 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:07.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:07.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:07.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:07.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:07.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:07.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:07.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:07.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:07.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T18:48:07.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:07.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:07.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:07.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:07.511 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:07.521 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:07.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:07.592 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:07.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:07.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:07.622 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:07.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:07.731 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:07.790 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:07.837 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:07.847 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:07.854 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:07.862 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:07.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:07.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:07.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:07.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:07.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:07.942 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:07.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:07.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:07.969 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:08.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:08.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:08.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:08.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:08.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:08.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:08.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:08.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:08.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:08.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:08.308 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T18:48:08.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T18:48:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:08.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:08.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:08.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:08.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:08.446 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:08.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:08.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T18:48:08.665 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:08.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:08.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:08.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:08.692 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T18:48:08.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T18:48:08.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:08.786 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:08.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-23T18:48:08.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:08.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:08.815 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:08.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T18:48:08.872 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:08.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:09.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-23T18:48:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-23T18:48:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:09.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:09.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:09.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:48:09.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:09.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:09.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:09.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-23T18:48:09.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:09.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:09.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:09.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:09.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-23T18:48:09.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-23T18:48:09.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:09.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:09.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:48:09.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:09.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:09.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:09.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T18:48:09.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:09.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:09.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:09.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:09.642 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-23T18:48:09.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-23T18:48:09.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:09.766 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:09.828 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:09.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:09.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:09.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:09.927 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:09.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-23T18:48:09.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:10.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-23T18:48:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-23T18:48:10.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:10.049 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:10.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:10.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-23T18:48:10.170 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:10.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:10.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-23T18:48:10.331 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-23T18:48:10.337 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:10.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T18:48:10.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:10.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:10.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:10.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:10.426 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-23T18:48:10.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-23T18:48:10.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:10.445 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:10.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:10.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-23T18:48:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:10.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:10.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:10.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T18:48:10.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:10.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:10.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:10.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:10.765 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-23T18:48:10.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-23T18:48:10.833 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:10.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:10.845 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T18:48:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:10.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:10.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T18:48:11.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:11.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:11.145 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:11.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:11.162 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-23T18:48:11.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-23T18:48:11.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:11.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:11.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:11.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:11.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:11.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T18:48:11.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:11.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:11.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:11.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:11.521 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-23T18:48:11.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-23T18:48:11.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:11.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:11.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:11.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:11.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:11.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:11.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:11.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T18:48:11.662 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:11.670 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:11.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:11.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:11.729 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-23T18:48:11.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-23T18:48:11.836 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:11.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T18:48:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-23T18:48:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-23T18:48:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T18:48:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-23T18:48:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-23T18:48:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T18:48:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-23T18:48:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-23T18:48:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-23T18:48:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:12.828 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-23T18:48:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-23T18:48:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-23T18:48:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T18:48:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-23T18:48:13.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-23T18:48:13.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:13.426 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:13.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:13.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:13.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:13.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:13.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:13.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T18:48:13.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:13.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:13.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:13.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:13.574 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-23T18:48:13.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-23T18:48:13.676 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:13.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:13.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:13.895 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:13.904 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:13.914 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:13.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:13.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T18:48:13.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:13.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-23T18:48:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-23T18:48:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:14.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:14.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:14.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:14.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T18:48:14.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:14.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:14.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:14.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:14.358 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-23T18:48:14.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-23T18:48:14.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:14.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:14.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 -- 172.21.15.19:0/4009267241 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26e40a3d20 msgr2=0x7f26e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:14.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26e40a3d20 0x7f26e40a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f26f0031440 tx=0x7f26f00095f0 comp rx=0 tx=0).stop 2024-04-23T18:48:14.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 -- 172.21.15.19:0/4009267241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1031e0 msgr2=0x7f26fc198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1031e0 0x7f26fc198730 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00e9f0 tx=0x7f26ec00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:48:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 -- 172.21.15.19:0/4009267241 shutdown_connections 2024-04-23T18:48:14.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f26e40a3d20 0x7f26e40a61e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:14.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26fc10c520 0x7f26fc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:14.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc103b00 0x7f26fc198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:14.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 --2- 172.21.15.19:0/4009267241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1031e0 0x7f26fc198730 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 -- 172.21.15.19:0/4009267241 >> 172.21.15.19:0/4009267241 conn(0x7f26fc077f50 msgr2=0x7f26fc0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:14.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 -- 172.21.15.19:0/4009267241 shutdown_connections 2024-04-23T18:48:14.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:46:04.888+0000 7f270168a640 1 -- 172.21.15.19:0/4009267241 wait complete. 2024-04-23T18:48:14.763 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2024-04-23T18:48:14.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:14.831 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:14.838 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:14.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:14.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T18:48:14.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:15.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:15.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:15.230 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-23T18:48:15.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-23T18:48:15.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:15.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:15.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:15.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:15.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:15.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T18:48:15.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:15.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:15.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-23T18:48:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:48:15.618 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:48:15.629 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:15.638 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:15.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:15.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:15.667 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:15.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:15.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:15.756 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:15.765 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:15.775 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:15.811 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:15.869 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:15.916 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:16.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:16.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:16.078 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:16.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:16.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:16.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:16.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:16.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:16.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:16.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:16.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:16.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:16.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:16.494 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:16.552 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:16.608 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:16.665 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:16.672 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:16.679 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:16.685 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:16.691 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:16.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:16.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:16.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:16.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:16.785 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:16.791 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:16.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:16.803 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:16.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:16.852 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:16.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:16.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:17.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:17.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:17.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:17.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:17.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:17.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:17.054 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-23T18:48:17.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-23T18:48:17.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:17.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:17.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:17.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-23T18:48:17.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:17.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:17.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:17.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T18:48:17.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:17.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:17.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:17.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-23T18:48:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-23T18:48:17.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-23T18:48:17.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:17.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:17.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:17.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T18:48:17.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:17.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:17.622 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:17.676 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:17.737 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:17.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:17.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:17.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-23T18:48:17.764 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:48:17.772 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:48:17.779 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:17.785 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T18:48:17.833 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T18:48:17.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:17.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:17.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:17.861 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:48:17.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:17.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:17.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:18.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:18.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:18.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:18.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:18.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:18.124 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T18:48:18.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T18:48:18.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:18.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:18.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:18.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:18.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:18.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:18.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:18.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:18.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:18.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:18.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:18.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:18.465 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T18:48:18.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T18:48:18.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:18.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:18.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:18.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:18.506 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:18.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:18.554 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:18.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:18.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:18.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:18.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:18.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-23T18:48:18.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T18:48:18.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:18.808 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:18.816 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:18.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:18.829 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:18.836 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:18.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:18.855 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:18.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:18.918 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:18.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:18.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:18.939 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T18:48:18.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T18:48:18.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:18.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:19.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:19.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:19.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:19.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:19.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:19.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:19.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:19.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:19.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:19.213 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T18:48:19.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T18:48:19.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:19.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:19.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:19.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T18:48:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T18:48:19.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:19.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:19.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:19.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:19.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:19.611 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:19.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:19.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:19.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:19.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:19.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:19.847 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T18:48:19.854 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T18:48:19.864 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:19.874 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:19.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:19.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:19.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:19.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:19.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:19.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:20.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:20.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:20.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:20.153 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T18:48:20.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T18:48:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:20.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:20.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:20.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:20.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:20.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:20.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:20.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:20.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:20.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:20.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:20.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:20.342 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T18:48:20.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T18:48:20.474 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:20.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:20.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:20.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:20.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:20.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:20.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:20.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:20.592 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:20.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:20.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:20.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:20.676 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T18:48:20.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T18:48:20.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:20.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:20.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:20.845 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:20.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:20.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:20.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:20.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:20.937 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:20.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:20.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:21.008 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T18:48:21.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T18:48:21.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:21.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:21.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:21.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:21.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:21.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:21.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:21.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:21.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:21.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:21.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:21.316 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T18:48:21.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T18:48:21.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:21.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:21.521 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:21.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:21.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:21.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:21.674 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T18:48:21.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T18:48:21.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:21.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:21.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:21.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:21.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:21.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:21.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:21.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:21.814 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:21.822 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:21.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:21.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:21.969 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T18:48:22.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T18:48:22.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:22.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-23T18:48:22.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:22.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:22.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:22.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:22.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:22.145 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:22.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:22.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:22.167 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T18:48:22.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T18:48:22.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:22.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:22.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:22.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:22.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:22.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:22.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:22.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:22.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:22.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:22.461 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-23T18:48:22.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-23T18:48:22.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:22.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:22.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:22.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:22.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:22.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:22.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:22.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:22.753 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T18:48:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T18:48:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:22.855 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:22.864 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:22.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-23T18:48:22.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:22.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:22.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T18:48:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T18:48:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T18:48:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T18:48:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T18:48:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T18:48:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T18:48:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T18:48:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T18:48:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T18:48:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-23T18:48:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:24.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:24.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:24.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:24.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:24.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:24.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:24.612 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T18:48:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T18:48:24.670 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:24.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:24.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:24.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-23T18:48:24.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:24.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:24.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:24.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:24.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:24.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:24.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:24.971 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T18:48:24.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T18:48:24.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:24.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:24.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:25.002 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:25.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:25.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:25.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:25.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:25.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:25.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:25.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:25.309 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T18:48:25.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T18:48:25.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:25.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:25.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:25.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:25.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:25.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:25.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:25.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:25.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:25.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:25.516 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T18:48:25.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T18:48:25.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:25.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:25.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:25.718 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:25.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:25.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:25.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:25.749 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:25.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:25.810 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:25.816 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:25.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:25.832 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T18:48:25.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T18:48:25.846 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:25.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:25.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:26.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:26.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:26.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:26.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:26.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:26.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:26.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:26.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:26.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:26.207 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T18:48:26.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T18:48:26.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:26.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:26.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:26.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:26.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:26.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:26.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:26.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:26.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:26.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:26.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:26.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:26.538 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T18:48:26.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T18:48:26.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:26.565 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:26.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:26.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:26.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:26.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:26.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:26.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:26.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:26.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:26.905 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T18:48:26.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T18:48:26.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:26.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:26.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:26.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:26.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:27.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:27.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:27.037 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:27.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:27.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:27.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:27.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:27.214 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T18:48:27.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T18:48:27.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:27.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:27.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:27.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:27.279 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:27.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:27.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:27.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:27.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:27.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:27.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:27.378 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T18:48:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T18:48:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:27.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:27.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:27.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:27.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:27.683 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:27.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:27.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:27.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:27.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:27.766 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T18:48:27.774 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T18:48:27.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:27.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:27.861 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:27.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:28.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:28.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:28.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:28.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:28.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:28.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:28.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:28.082 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T18:48:28.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T18:48:28.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:28.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:28.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:28.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T18:48:28.172 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:28.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:28.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:28.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:28.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:28.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:28.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:28.403 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T18:48:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T18:48:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:28.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:28.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:48:28.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:28.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:28.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:28.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:28.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:28.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:28.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:28.740 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T18:48:28.747 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T18:48:28.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:28.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:28.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:28.776 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-23T18:48:28.786 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:28.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:28.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:28.857 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:28.866 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:28.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:28.884 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:28.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:28.987 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T18:48:29.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T18:48:29.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:29.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:29.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:29.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-23T18:48:29.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:29.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:29.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:29.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T18:48:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T18:48:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:29.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:29.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:29.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:29.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:29.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:29.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:29.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:29.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:29.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:29.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:29.601 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T18:48:29.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T18:48:29.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:29.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:29.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:29.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-23T18:48:29.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:29.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:29.842 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:29.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:29.854 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:29.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:29.866 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:29.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:29.880 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T18:48:29.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T18:48:29.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:29.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:29.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:29.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:29.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:29.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:30.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:30.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:30.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:30.236 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T18:48:30.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T18:48:30.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:30.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:30.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:30.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:30.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:30.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:30.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:30.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:30.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:30.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:30.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:30.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:30.600 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T18:48:30.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T18:48:30.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:30.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:30.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:30.654 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:30.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:30.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:30.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:30.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:30.741 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:30.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:30.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:30.814 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T18:48:30.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T18:48:30.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:30.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:30.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:30.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:48:30.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:31.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:31.023 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:31.030 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:31.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:31.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:31.100 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T18:48:31.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T18:48:31.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:31.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:31.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:31.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:48:31.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:31.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:31.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:31.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:31.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:31.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:31.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:31.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:31.348 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T18:48:31.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T18:48:31.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:31.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:31.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:31.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:48:31.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:31.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:31.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:31.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:31.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:31.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:31.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:31.626 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:31.635 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-23T18:48:31.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-23T18:48:31.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:31.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:31.730 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:31.738 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:31.746 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:31.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:31.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:31.837 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:31.921 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:31.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:31.938 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-23T18:48:31.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-23T18:48:31.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:31.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:31.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:31.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:31.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:32.017 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:32.026 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:32.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:32.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:32.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:32.187 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-23T18:48:32.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-23T18:48:32.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:32.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:32.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:32.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:32.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:32.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:32.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:32.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-23T18:48:32.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:32.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:32.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:32.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:32.369 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-23T18:48:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-23T18:48:32.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:32.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:32.549 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:32.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-23T18:48:32.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:32.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:32.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:32.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:32.592 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:32.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:32.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:32.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:32.677 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-23T18:48:32.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-23T18:48:32.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:32.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:32.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:32.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:32.831 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:32.882 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:32.890 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:32.898 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:32.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:32.912 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:32.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:32.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:32.937 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:32.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:32.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:32.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:32.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:33.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:33.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:33.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:33.131 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:33.182 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:33.191 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:33.200 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:33.210 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:33.218 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:33.227 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:33.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:33.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:33.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:33.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:33.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:33.334 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:33.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:33.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:33.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:33.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:33.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:33.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:33.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:33.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:33.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:33.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:33.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:33.649 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:33.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:33.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:33.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:33.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:33.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:33.730 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:33.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:33.789 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:33.796 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:33.801 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:33.806 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:33.812 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:33.818 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:33.825 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:33.866 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:33.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:33.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:33.912 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:33.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:33.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:33.933 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:33.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:34.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:34.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:34.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:34.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:34.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:34.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:34.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:34.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:34.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:34.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:34.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:34.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:34.249 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-23T18:48:34.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-23T18:48:34.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:34.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:34.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:34.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:34.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:34.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:34.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:34.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-23T18:48:34.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:34.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:34.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:34.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:34.493 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-23T18:48:34.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T18:48:34.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:34.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:34.562 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:34.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:34.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:34.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:34.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:34.682 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:34.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:34.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:34.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:34.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:34.824 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-23T18:48:34.832 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-23T18:48:34.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:34.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:34.858 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:34.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-23T18:48:34.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:34.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:34.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:34.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:35.021 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:35.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:35.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:35.137 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-23T18:48:35.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-23T18:48:35.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:35.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:35.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:35.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-23T18:48:35.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:35.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:35.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:35.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:35.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:35.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:35.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:35.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:35.334 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-23T18:48:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-23T18:48:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:35.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:35.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:35.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:35.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:35.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:35.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:35.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:35.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:35.553 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-23T18:48:35.562 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-23T18:48:35.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:35.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:35.697 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-23T18:48:35.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:35.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:35.823 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:35.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:35.839 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:35.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:35.919 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-23T18:48:35.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-23T18:48:35.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:35.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:35.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-23T18:48:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:36.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:36.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:36.123 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:36.130 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-23T18:48:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:36.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:36.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:36.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:36.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:36.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:36.173 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-23T18:48:36.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-23T18:48:36.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:36.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:36.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:36.260 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-23T18:48:36.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-23T18:48:36.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-23T18:48:36.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:36.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:36.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:36.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:36.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:36.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:36.494 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-23T18:48:36.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-23T18:48:36.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:36.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:36.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:36.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:36.590 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:36.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:36.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:36.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:36.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:36.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:36.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:36.844 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:36.852 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:36.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:36.866 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-23T18:48:36.886 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:48:36.895 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:48:36.959 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:36.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-23T18:48:36.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-23T18:48:36.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:36.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:36.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:37.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:37.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:37.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:37.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:37.262 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-23T18:48:37.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:37.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:37.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:37.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:37.299 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:37.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:37.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:37.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:37.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:37.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:37.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:37.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:37.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:37.500 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:37.550 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:37.606 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:37.612 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:37.619 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:37.625 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:37.631 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:37.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:37.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:37.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:37.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:37.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:37.821 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:37.849 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:37.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:38.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:38.031 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:38.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:38.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:38.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:38.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:38.090 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:38.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:38.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:38.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:38.249 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:38.360 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:38.366 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:38.374 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:38.381 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:38.388 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:38.394 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:38.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:38.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:38.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:38.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:38.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:38.484 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:38.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:38.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:38.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:38.601 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:38.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:38.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:38.745 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:38.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:38.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:38.764 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:38.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:38.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:38.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:38.843 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-23T18:48:38.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-23T18:48:38.856 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:38.863 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:38.870 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:38.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-23T18:48:38.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:38.991 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T18:48:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-23T18:48:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-23T18:48:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-23T18:48:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T18:48:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-23T18:48:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-23T18:48:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-23T18:48:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-23T18:48:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-23T18:48:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-23T18:48:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-23T18:48:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T18:48:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-23T18:48:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-23T18:48:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:48:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T18:48:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:48:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:48:40.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:40.503 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:40.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:48:40.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:40.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:40.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T18:48:40.655 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:40.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:40.674 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:40.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:40.690 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-23T18:48:40.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-23T18:48:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:40.765 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:40.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:40.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-23T18:48:40.787 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:40.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:40.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T18:48:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:41.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:41.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:41.060 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:41.067 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:41.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:41.099 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:41.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-23T18:48:41.134 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:48:41.142 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:48:41.151 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:41.158 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:41.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:41.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:41.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:41.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:41.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:41.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:41.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:41.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:41.396 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:41.402 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:41.409 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:41.416 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:41.423 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:41.471 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:41.496 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:41.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:41.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:41.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:41.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:41.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:41.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:41.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:41.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:41.859 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:41.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:41.875 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:41.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:41.893 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:41.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:41.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:41.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:42.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:42.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:42.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:42.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:42.079 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:42.086 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:42.093 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:42.105 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:42.157 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:42.165 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:42.171 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:42.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:42.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:42.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:42.244 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:42.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:42.328 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:42.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:42.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:42.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:42.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:42.533 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-23T18:48:42.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-23T18:48:42.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:42.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:42.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:42.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-23T18:48:42.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:42.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:42.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:42.729 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:42.737 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:42.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:42.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:42.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:42.810 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-23T18:48:42.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-23T18:48:42.824 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:42.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:42.895 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:42.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-23T18:48:42.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:42.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:42.991 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:42.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:43.005 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:43.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:43.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:43.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:43.030 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:48:43.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:48:43.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:43.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:43.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:43.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:48:43.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:43.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:43.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:43.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:43.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:43.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:43.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:43.406 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:43.414 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:43.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:43.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:43.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-23T18:48:43.494 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-23T18:48:43.560 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-23T18:48:43.570 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-23T18:48:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-23T18:48:43.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:43.636 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:43.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:43.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-23T18:48:43.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:43.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:43.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:43.788 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T18:48:43.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:43.804 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:43.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:43.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:43.826 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-23T18:48:43.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-23T18:48:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:43.903 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:43.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-23T18:48:43.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:44.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:44.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T18:48:44.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:44.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:44.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:44.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:44.133 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-23T18:48:44.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-23T18:48:44.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:44.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:44.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:44.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:44.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:44.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:44.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:44.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-23T18:48:44.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:44.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:44.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:44.441 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:48:44.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:48:44.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:48:44.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:44.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:44.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-23T18:48:44.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-23T18:48:44.554 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:44.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:44.573 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T18:48:44.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:44.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:44.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:44.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:44.777 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:44.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:44.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:44.798 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:44.805 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:44.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:44.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:44.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:44.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:44.892 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:44.945 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:44.953 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:44.963 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:44.971 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:45.018 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:45.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:45.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:45.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:45.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:45.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:45.136 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:45.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:45.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:45.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:45.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:45.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:45.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:45.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:45.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:45.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:45.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:45.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:45.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:45.572 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:45.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:45.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:45.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:45.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:45.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:45.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:45.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:45.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:45.743 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:45.754 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:45.772 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:45.782 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:45.884 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:45.922 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:46.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:46.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:46.070 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:46.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:46.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:46.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:46.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:46.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:46.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:46.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:46.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:46.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:46.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:46.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:46.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:46.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:46.417 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-23T18:48:46.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-23T18:48:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:46.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:46.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-23T18:48:46.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:46.551 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:46.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:46.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:46.622 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-23T18:48:46.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-23T18:48:46.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:46.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:46.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:46.730 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-23T18:48:46.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:46.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:46.756 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:46.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-23T18:48:46.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:46.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:46.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:46.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:46.854 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-23T18:48:46.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-23T18:48:46.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:46.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:46.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:47.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-23T18:48:47.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:47.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:47.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:47.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:48:47.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:47.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:47.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:47.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:47.187 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-23T18:48:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-23T18:48:47.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:47.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:47.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:48:47.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:48:47.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-23T18:48:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T18:48:47.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:47.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:47.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:47.445 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-23T18:48:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-23T18:48:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:47.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:47.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:47.633 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:47.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:47.724 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-23T18:48:47.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:47.793 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-23T18:48:47.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-23T18:48:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:47.900 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-23T18:48:47.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:47.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:48:47.990 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:48.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:48.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:48.140 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:48.148 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:48.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:48.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:48.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-23T18:48:48.186 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:48:48.195 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:48:48.203 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:48.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:48.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:48.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:48.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:48.311 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:48.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:48.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:48.458 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:48.466 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:48.484 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:48.495 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:48.505 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:48.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:48.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:48.605 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:48.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:48.622 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:48.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:48.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:48.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:48.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:48.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:48.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:48.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:48.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:48.853 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:48.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:48.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:48.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:48.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:48.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:48.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:49.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:49.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:49.085 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:49.092 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:49.099 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:49.106 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:49.113 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:49.122 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:49.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:49.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:49.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:49.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:49.275 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:49.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:49.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:49.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:49.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:49.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:49.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:49.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:49.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:49.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:49.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:49.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:49.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:49.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:49.619 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:49.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-23T18:48:49.656 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:48:49.709 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:48:49.740 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:49.787 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:49.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:49.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:49.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:49.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:49.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:49.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:49.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:49.846 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:49.854 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:49.911 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:49.919 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:49.935 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:49.944 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:49.953 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:50.009 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:50.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:50.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:50.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:50.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:50.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:50.118 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:50.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:50.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:50.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:50.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:50.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:50.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:50.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:50.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:50.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:50.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:50.423 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:50.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:50.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:50.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:50.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:50.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:50.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:50.591 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:50.599 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:50.608 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:50.617 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:50.626 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:50.692 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:50.763 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:50.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:50.828 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:50.835 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:50.842 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:50.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:50.860 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:50.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:50.876 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:50.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:50.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:51.013 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:51.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:51.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:51.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:51.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:51.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:51.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:51.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:51.268 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:51.275 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:51.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:51.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:51.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-23T18:48:51.309 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:48:51.318 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:48:51.326 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:51.370 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-23T18:48:51.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-23T18:48:51.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:51.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:51.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-23T18:48:51.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:51.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:51.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:51.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T18:48:51.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:51.660 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:51.668 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:51.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:51.684 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-23T18:48:51.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-23T18:48:51.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:48:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:48:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-23T18:48:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:51.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T18:48:51.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:51.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:52.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:52.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:52.017 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:52.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:52.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:52.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:52.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:52.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:52.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:52.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:52.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:52.197 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:52.341 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:52.383 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:52.392 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:52.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:52.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:52.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:52.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:52.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:52.445 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:52.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:52.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:52.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:52.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:52.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:52.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:52.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:52.666 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:52.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:52.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:52.788 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:52.798 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:52.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:52.814 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:52.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:52.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:52.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:52.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:52.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:52.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:52.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:52.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:52.946 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:52.954 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:53.025 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:53.086 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:53.144 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:53.151 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:53.159 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:53.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:53.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:53.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:53.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:53.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:53.208 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:53.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:53.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:53.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:53.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:53.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:53.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:53.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:53.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:53.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:53.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:53.586 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:53.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:53.600 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:53.608 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:53.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:53.622 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:53.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-23T18:48:53.684 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:48:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:48:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-23T18:48:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-23T18:48:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:53.862 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:53.911 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:53.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-23T18:48:53.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:53.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:53.994 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:54.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-23T18:48:54.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T18:48:54.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:54.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:54.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:54.054 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:54.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:54.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:54.148 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:54.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:54.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:54.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:54.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:54.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:54.337 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:54.346 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:54.355 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:54.364 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:54.372 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:54.381 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:54.391 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:54.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:54.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:54.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:54.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:54.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:54.499 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:54.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:54.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:54.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:54.726 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:54.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:54.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:54.764 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:54.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:54.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:54.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:54.909 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:54.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:54.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:54.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:54.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:54.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:55.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:55.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:55.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:55.118 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:55.125 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:55.133 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:55.142 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:55.151 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:55.159 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:55.166 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:55.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:55.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:55.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:55.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:55.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:55.259 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:55.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:55.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:55.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:55.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:55.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:55.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:55.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:55.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:55.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:55.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:55.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:55.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-23T18:48:55.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-23T18:48:55.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:55.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:55.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:55.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:55.876 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:55.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:55.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-23T18:48:55.912 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T18:48:55.921 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:55.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:55.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:55.946 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-23T18:48:55.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-23T18:48:56.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:56.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:56.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:56.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-23T18:48:56.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:56.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-23T18:48:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:56.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:56.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:56.290 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-23T18:48:56.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-23T18:48:56.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:48:56.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:56.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:56.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-23T18:48:56.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:56.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:56.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:56.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T18:48:56.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:56.586 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:56.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:56.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:56.696 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-23T18:48:56.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-23T18:48:56.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:56.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:56.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:56.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:56.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:56.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:56.871 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:56.879 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-23T18:48:56.886 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:56.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:56.898 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:56.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:57.015 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-23T18:48:57.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-23T18:48:57.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:57.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:57.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:57.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-23T18:48:57.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:57.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:57.134 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:57.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T18:48:57.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T18:48:57.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:57.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:57.247 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:57.254 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:57.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:57.306 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:57.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-23T18:48:57.409 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:48:57.416 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:48:57.423 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:57.430 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:57.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:57.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:57.449 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:57.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:57.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:57.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:57.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:57.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:57.574 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:57.580 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:57.588 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:57.640 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:57.698 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:57.751 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:57.758 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:57.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:57.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:57.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:57.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:57.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:57.806 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:57.814 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:57.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:57.885 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:57.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:57.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:57.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:57.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:57.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:58.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:58.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:58.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:58.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:58.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:58.133 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:58.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:58.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:58.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:58.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:58.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:48:58.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:58.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:58.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:58.266 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:58.275 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:58.284 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:58.348 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:58.401 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:58.408 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:58.416 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:58.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:58.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:58.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:58.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:58.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:58.466 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:48:58.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:48:58.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:58.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:58.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:58.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:58.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:58.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:58.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:58.826 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:58.834 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:58.841 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:58.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:58.856 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:58.864 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:48:58.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:58.878 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:48:58.885 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-23T18:48:58.953 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:48:59.025 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:48:59.033 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:48:59.040 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:48:59.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:48:59.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:59.131 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:48:59.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:59.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:59.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:48:59.271 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:48:59.278 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:48:59.284 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:48:59.291 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:48:59.299 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:48:59.364 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:48:59.404 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:48:59.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:59.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:59.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:59.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:59.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:59.576 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:48:59.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:48:59.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:48:59.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:48:59.688 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:48:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:48:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:48:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:48:59.724 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:48:59.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:48:59.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:48:59.820 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:48:59.829 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:48:59.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:48:59.848 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:48:59.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:48:59.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:48:59.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:00.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:00.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:00.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:00.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:00.072 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:00.081 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:00.089 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:00.098 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:00.107 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:00.167 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:00.212 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:00.220 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:00.230 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:00.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:00.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:00.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:00.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:00.442 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:00.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:00.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:00.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:00.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:00.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:00.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:00.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:00.501 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:00.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:00.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:00.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:00.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:00.582 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:00.589 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:00.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:00.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:00.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-23T18:49:00.899 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:00.927 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:00.933 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:00.938 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:00.944 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:00.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:00.959 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:00.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:01.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:01.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:01.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:01.051 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:01.057 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:01.122 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:01.179 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:01.258 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:01.265 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:01.273 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:01.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:01.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-23T18:49:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-23T18:49:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-23T18:49:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:02.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:02.740 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:02.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T18:49:02.755 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:02.764 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:02.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:02.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:02.913 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:02.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:02.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:02.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:02.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:02.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:03.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:03.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:03.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:03.141 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:03.158 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:03.165 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:03.173 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:03.180 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:03.188 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:03.196 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:03.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:03.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:03.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:03.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:03.332 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:03.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:03.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:03.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:03.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:03.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:03.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:03.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:03.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:03.688 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:03.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:03.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:03.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:03.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:03.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:03.876 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:03.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:03.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:03.903 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:03.912 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:03.922 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:03.931 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:03.940 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:03.949 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:03.998 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:04.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:04.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:04.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:04.064 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:04.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:04.113 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:04.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:04.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:04.231 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:04.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:04.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:04.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:04.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:04.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:04.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:04.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:04.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:04.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:04.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:04.400 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-23T18:49:04.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-23T18:49:04.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:04.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:04.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:04.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:04.615 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-23T18:49:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-23T18:49:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-23T18:49:04.666 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:04.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-23T18:49:04.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:04.737 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:04.760 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:04.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:04.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:04.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-23T18:49:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:04.952 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:04.961 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:05.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:05.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:05.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:05.166 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:05.216 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:05.317 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:05.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:05.352 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:05.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:05.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:05.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:05.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:05.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:05.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:05.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:05.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:05.732 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:05.830 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:05.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:05.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:05.910 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:05.932 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:05.973 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:06.092 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:06.101 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:06.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:06.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:06.171 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:06.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:06.186 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:06.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:06.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:06.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:06.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:06.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:06.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:06.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-23T18:49:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:06.575 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:06.588 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:06.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:06.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:06.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:06.803 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:06.812 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:06.834 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:06.897 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:06.965 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:07.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:07.091 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:07.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:07.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:07.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:07.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:07.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:07.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:07.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:07.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:07.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:07.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:07.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:07.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:07.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:07.358 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:07.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:07.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:07.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:07.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:07.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:07.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:07.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:07.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:07.508 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:07.517 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:07.525 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:07.534 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:07.592 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:07.643 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:07.649 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:07.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:07.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:07.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:07.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:07.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:07.890 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:07.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:07.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:07.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:07.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:07.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:07.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:07.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:07.956 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:08.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:08.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:08.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:08.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:08.080 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:08.087 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:08.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:08.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-23T18:49:08.224 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:08.233 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:08.241 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:08.253 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:08.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:08.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:08.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:08.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:08.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:08.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:08.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:08.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:08.376 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:08.420 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:08.446 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:08.513 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:08.523 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:08.530 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:08.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:08.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:08.551 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:08.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:08.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:08.575 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:08.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:08.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:08.672 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:08.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:08.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:08.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:08.812 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:08.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:08.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:08.908 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:08.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:08.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:08.929 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:08.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:08.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:08.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:08.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:09.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:09.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:09.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:09.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:09.207 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:09.216 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:09.224 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:09.233 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:09.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:09.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:09.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:09.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:09.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:09.309 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:09.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:09.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:09.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:09.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:09.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:09.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:09.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:09.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:09.640 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:09.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:09.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:09.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:09.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:09.684 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:49:09.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:49:09.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:49:09.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:09.830 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:09.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:49:09.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:09.850 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:09.856 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:09.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:10.005 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:10.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:10.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:10.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:10.120 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-23T18:49:10.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-23T18:49:10.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:49:10.142 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:10.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:10.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-23T18:49:10.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-23T18:49:10.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:10.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:10.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-23T18:49:10.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T18:49:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:10.382 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:10.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:10.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:10.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:10.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-23T18:49:10.470 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:10.479 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:10.488 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:10.496 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-23T18:49:10.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-23T18:49:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:10.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:10.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:10.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:10.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:10.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:10.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T18:49:10.826 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:10.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:10.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:10.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-23T18:49:10.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-23T18:49:10.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:10.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:10.895 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:10.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:10.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:10.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:11.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:11.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:11.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:11.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:11.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:11.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:11.218 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:11.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:11.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:11.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:11.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:11.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:11.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:11.343 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:11.350 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:11.421 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:11.501 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:11.575 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:11.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:11.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:11.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:11.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:11.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:11.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:11.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:11.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:11.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:11.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:11.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:11.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:11.796 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:11.844 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-23T18:49:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-23T18:49:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:12.800 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:12.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:12.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-23T18:49:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-23T18:49:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:49:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:49:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:49:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:49:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-23T18:49:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:13.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:13.324 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:13.331 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:13.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:13.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:13.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-23T18:49:13.496 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:13.503 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:13.511 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:13.518 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-23T18:49:13.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-23T18:49:13.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:13.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:13.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:13.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:13.652 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:13.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:13.666 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:13.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:13.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:13.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:13.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:13.858 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:13.865 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-23T18:49:13.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-23T18:49:13.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:13.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:13.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:13.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:13.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:13.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:13.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T18:49:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T18:49:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:14.093 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:14.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:14.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:14.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:14.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:14.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:14.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:14.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:14.317 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:14.326 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:14.335 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:14.390 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:14.397 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:14.406 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:14.415 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:14.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:14.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:14.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:14.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:14.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:14.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:14.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:14.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:14.734 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:14.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:14.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:14.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:14.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:14.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:14.920 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:14.928 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:14.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:14.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:14.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:14.973 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:15.001 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:15.056 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:15.063 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:15.070 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:15.077 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:15.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:15.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:15.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:15.284 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:15.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:15.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:15.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:15.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:15.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:15.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:15.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-23T18:49:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-23T18:49:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:15.625 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:15.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:15.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:15.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:15.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:15.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:15.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:15.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:15.767 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:15.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:15.789 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T18:49:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T18:49:15.897 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-23T18:49:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:16.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:16.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:16.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:16.116 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-23T18:49:16.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-23T18:49:16.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:49:16.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:16.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-23T18:49:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:16.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:16.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:16.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:16.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:16.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-23T18:49:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-23T18:49:16.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:16.490 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:16.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:16.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-23T18:49:16.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:16.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:16.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:16.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T18:49:16.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:16.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:16.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:16.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:16.687 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-23T18:49:16.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-23T18:49:16.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:16.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:16.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:16.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-23T18:49:16.789 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:16.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:16.829 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T18:49:16.884 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:16.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:16.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:16.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:16.995 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-23T18:49:17.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-23T18:49:17.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:17.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:17.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:17.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-23T18:49:17.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:17.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:17.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:17.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:17.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:17.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:17.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:17.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:17.264 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-23T18:49:17.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-23T18:49:17.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:17.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:17.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:17.353 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-23T18:49:17.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:17.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:17.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-23T18:49:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-23T18:49:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:17.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:17.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:17.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:17.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:17.630 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-23T18:49:17.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-23T18:49:17.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:17.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:17.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:17.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-23T18:49:17.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-23T18:49:17.776 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-23T18:49:17.786 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:17.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:17.853 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:17.862 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:17.871 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:17.880 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:17.888 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:17.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:17.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:18.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-23T18:49:18.050 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:18.059 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:18.093 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:18.103 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:18.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:18.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:18.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:18.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:18.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:18.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:18.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:18.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:18.229 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:18.237 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:18.243 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:18.380 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:18.451 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:18.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:18.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:18.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:18.505 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:18.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:18.519 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:18.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:18.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:18.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:18.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:18.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:18.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:18.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:18.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:18.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:18.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:18.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:18.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:18.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:18.865 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:18.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:18.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:18.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:18.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:18.945 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:18.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:18.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:18.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:18.975 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:18.982 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:19.042 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:19.088 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:19.126 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:19.135 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:19.173 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:19.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:19.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:19.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:19.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:19.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:19.229 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:19.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:19.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:19.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:19.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:19.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:19.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:19.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-23T18:49:19.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-23T18:49:19.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:19.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:19.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:49:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:19.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:19.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:19.867 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:19.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:19.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:20.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:20.056 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-23T18:49:20.066 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T18:49:20.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:20.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:20.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:20.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:20.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:20.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:20.178 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:20.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:20.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:20.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:20.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:20.313 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-23T18:49:20.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-23T18:49:20.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:20.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:20.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:20.499 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:20.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:20.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:20.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:20.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:20.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:20.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:20.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:20.592 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:20.599 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:20.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:20.666 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:20.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-23T18:49:20.799 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:20.807 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:20.857 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:20.864 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-23T18:49:20.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-23T18:49:20.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:20.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:20.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:20.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:20.961 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:20.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:20.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:20.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:20.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:21.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:21.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:21.190 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-23T18:49:21.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-23T18:49:21.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:21.329 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:21.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:21.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:21.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:21.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:21.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:21.529 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:21.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:21.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:21.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:21.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:21.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:21.776 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:21.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:21.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:21.802 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:21.811 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:21.839 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:21.887 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:21.895 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:21.904 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:21.913 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:21.922 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:21.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:21.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:22.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:22.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:22.099 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:22.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:22.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:22.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:22.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:22.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:22.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:22.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:22.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:22.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:22.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:22.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:22.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:22.308 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:22.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:22.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:22.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:22.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:22.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:22.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:22.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:22.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:22.532 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:22.538 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:22.592 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:22.599 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:22.609 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:22.618 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:22.625 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:22.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:22.670 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:22.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:22.858 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:22.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:22.880 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:22.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:22.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:22.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:22.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:22.971 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:22.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:22.985 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:22.993 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:23.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-23T18:49:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-23T18:49:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-23T18:49:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-23T18:49:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-23T18:49:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:24.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:24.551 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:24.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:24.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:24.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:24.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:24.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:24.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:24.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:24.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:24.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:24.755 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:24.770 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:24.778 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:24.787 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:24.817 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:24.862 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:24.908 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:24.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:24.961 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:24.968 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:24.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:24.983 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:24.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:24.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:25.004 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:25.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:25.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:25.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:25.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:25.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:25.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:25.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:25.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:25.240 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:25.294 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:25.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:25.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:25.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-23T18:49:25.319 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:25.328 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:25.336 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:25.345 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:25.395 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:25.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:25.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:25.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:25.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:25.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:25.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:25.577 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:25.669 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:25.679 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:25.688 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:25.697 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:25.705 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:25.714 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:25.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:25.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:25.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:25.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:25.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:25.812 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:25.819 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:25.867 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:25.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:25.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:25.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:26.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:26.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:26.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:26.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:26.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:26.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:26.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:26.160 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:26.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:26.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:26.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:26.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:26.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:26.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:26.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:26.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:26.418 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:26.425 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:26.432 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:26.439 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:26.446 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:26.453 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:26.461 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:26.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:26.523 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:26.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:26.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:26.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:26.551 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:26.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:26.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:26.791 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:26.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:26.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:26.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:26.825 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:26.831 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:26.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:26.895 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-23T18:49:26.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-23T18:49:26.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:26.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:26.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:27.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-23T18:49:27.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:27.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:27.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T18:49:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:27.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:27.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:27.155 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:27.164 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:27.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:27.229 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:27.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-23T18:49:27.247 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:27.265 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:27.297 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:27.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:27.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:27.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:27.559 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:27.568 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:27.578 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:27.586 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:27.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:27.622 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:27.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:27.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:27.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:27.788 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:27.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:27.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:27.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:27.818 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:27.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:27.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:27.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:27.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:27.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:27.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:27.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:27.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:28.032 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:28.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:28.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:28.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:28.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:28.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:28.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:28.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:28.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:28.174 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:28.235 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:28.242 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:28.249 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:28.256 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:28.263 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:28.272 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:28.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:28.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:28.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:28.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:28.485 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:28.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:28.499 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:28.506 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:28.513 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:28.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:28.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:28.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:28.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:28.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:28.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:28.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:28.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:28.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:28.714 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-23T18:49:28.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-23T18:49:28.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:28.791 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:28.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:28.805 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:28.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:28.826 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:28.853 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:28.863 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:28.871 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:28.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:28.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:28.891 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-23T18:49:28.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-23T18:49:28.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:28.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:29.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:29.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-23T18:49:29.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:29.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:29.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:29.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:29.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:29.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:29.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:29.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:29.177 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-23T18:49:29.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-23T18:49:29.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:29.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:29.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:29.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:29.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:29.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:29.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-23T18:49:29.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-23T18:49:29.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:29.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:29.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:29.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:29.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:29.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:29.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:29.668 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:29.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:29.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:29.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:29.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:29.760 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-23T18:49:29.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-23T18:49:29.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:49:29.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:29.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:29.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:29.870 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:29.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:29.890 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:29.898 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:29.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:29.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:30.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:30.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:30.063 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-23T18:49:30.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-23T18:49:30.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:30.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:30.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:30.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:30.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:30.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:30.328 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-23T18:49:30.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-23T18:49:30.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:30.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:30.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:30.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:30.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:30.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:30.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:30.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:30.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:30.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:30.560 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:30.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:30.574 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-23T18:49:30.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-23T18:49:30.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:30.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:30.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:30.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:30.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:30.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:30.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:30.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:30.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:30.825 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:30.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:30.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:30.873 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-23T18:49:30.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-23T18:49:30.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:30.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:30.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:30.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:30.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:49:31.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-23T18:49:31.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:31.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:31.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:31.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:31.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:31.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:31.127 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-23T18:49:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-23T18:49:31.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:31.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:31.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:31.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:31.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:31.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:31.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:31.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:31.354 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:31.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:31.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:31.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:31.450 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-23T18:49:31.460 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-23T18:49:31.495 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:31.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:31.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:31.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:31.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:31.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:31.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:31.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:31.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:31.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:31.737 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-23T18:49:31.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-23T18:49:31.803 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:31.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:31.818 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:31.826 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:31.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:31.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:31.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:31.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:31.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:31.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:31.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:31.949 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:31.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:32.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:32.080 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:32.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-23T18:49:32.148 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:32.162 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:32.170 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:32.177 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-23T18:49:32.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-23T18:49:32.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:32.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:32.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:32.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:32.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:32.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:32.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:32.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T18:49:32.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:32.398 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:32.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:32.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:32.520 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-23T18:49:32.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-23T18:49:32.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:32.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:32.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:32.562 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:32.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:32.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:32.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:32.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-23T18:49:32.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:32.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:32.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:32.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:32.727 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:32.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:32.805 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:32.868 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:32.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:32.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:32.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:32.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:32.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:32.925 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:32.933 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:32.989 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:33.068 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:33.077 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:33.084 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:33.092 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:33.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:33.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:33.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:33.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:33.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:33.351 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:33.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:33.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:33.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:33.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:33.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:33.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:33.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:33.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:33.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:33.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:33.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:33.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:33.662 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:33.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:33.786 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:33.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:33.801 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:33.808 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:33.815 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:33.823 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:33.831 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:33.886 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:33.900 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:33.907 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:33.915 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:33.922 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:33.928 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:33.953 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:34.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:34.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:34.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:34.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:34.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:34.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:34.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:34.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:34.191 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:34.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:34.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:34.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:34.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:34.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:34.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:34.318 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:34.349 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:34.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:34.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:34.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:34.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-23T18:49:34.430 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:34.437 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:34.445 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:34.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:34.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:34.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:34.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:34.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:34.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:34.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:34.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:34.641 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:34.648 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:34.702 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:34.720 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:34.730 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:34.739 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:34.747 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:34.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:34.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:34.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:34.827 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:34.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:34.845 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:34.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:34.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:34.895 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:34.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:35.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:35.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:35.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:35.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:35.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:35.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:35.125 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:35.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:35.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:35.198 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:35.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:35.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:35.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:35.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:35.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:35.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:35.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:35.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:35.412 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:35.479 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:35.486 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:35.493 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:35.499 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:35.506 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:35.513 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:35.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:35.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:35.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:35.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:35.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:35.604 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:35.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:35.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:35.684 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:35.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:35.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:35.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:35.807 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:35.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:35.831 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:35.837 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:35.869 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:35.876 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:35.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:35.890 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:35.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-23T18:49:35.904 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:35.975 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:36.033 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:36.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:36.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:36.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:36.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:36.157 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:36.170 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:36.177 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:36.183 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:36.189 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:36.224 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:36.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:36.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:36.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:36.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:36.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:36.401 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:36.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:36.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:36.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:36.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:36.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:36.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:36.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:36.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:36.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:36.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:36.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:36.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:36.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:36.684 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:36.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:36.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:36.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:36.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:36.808 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:36.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:36.826 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:36.852 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:36.860 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:36.869 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:36.881 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:36.897 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:36.941 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:36.975 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:37.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:37.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:37.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:37.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:37.144 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:37.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:37.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:37.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:37.231 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:37.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:37.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:37.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:37.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:37.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:37.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:37.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:37.515 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:37.583 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:37.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:37.604 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:37.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-23T18:49:37.616 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:37.622 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:37.630 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:37.694 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-23T18:49:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-23T18:49:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T18:49:37.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:37.828 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:49:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:49:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:49:38.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:38.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:38.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-23T18:49:38.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:38.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:38.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:38.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:38.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:38.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:38.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:38.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:38.327 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:38.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:38.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:38.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:38.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:38.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:38.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:38.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:38.441 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:38.448 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:38.455 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:38.485 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:38.534 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:38.602 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:38.609 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:38.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:38.682 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:38.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:38.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:38.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:38.715 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:38.725 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:38.733 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:38.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:38.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:38.808 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:38.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:38.826 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:38.847 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:38.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:38.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:38.942 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:39.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:39.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:39.028 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:39.036 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:39.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-23T18:49:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-23T18:49:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-23T18:49:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T18:49:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-23T18:49:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-23T18:49:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T18:49:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-23T18:49:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-23T18:49:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:40.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:40.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:40.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:40.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:40.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:40.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T18:49:40.616 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:40.739 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:40.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-23T18:49:40.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-23T18:49:40.827 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:40.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:40.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:40.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:40.929 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:40.939 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T18:49:40.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:40.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:40.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:41.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:41.066 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-23T18:49:41.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-23T18:49:41.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:41.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:41.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:41.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:41.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:41.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:41.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:41.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T18:49:41.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:41.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:41.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:41.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:41.268 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-23T18:49:41.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-23T18:49:41.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:41.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:41.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:41.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:41.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-23T18:49:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-23T18:49:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:41.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:41.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:41.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:41.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:41.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:41.820 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:41.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:41.833 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-23T18:49:41.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-23T18:49:41.868 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:41.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:41.882 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:41.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T18:49:41.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:41.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:41.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:42.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:42.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:42.082 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:42.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:42.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-23T18:49:42.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-23T18:49:42.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:42.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:42.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:42.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:42.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:42.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:42.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:42.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:42.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:42.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:42.428 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:49:42.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:49:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-23T18:49:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-23T18:49:42.540 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:42.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:42.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:42.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:42.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:42.607 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:42.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:42.690 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-23T18:49:42.735 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-23T18:49:42.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:42.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:42.791 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:42.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-23T18:49:42.808 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:42.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:42.825 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:42.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:42.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:42.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:42.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:42.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:42.925 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-23T18:49:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-23T18:49:42.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:42.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:43.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:43.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:43.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:43.140 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:43.147 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:43.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:43.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:43.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:43.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:43.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:43.239 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-23T18:49:43.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-23T18:49:43.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:43.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:43.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:43.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:43.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:43.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:43.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:43.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:43.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:43.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:43.512 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:43.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:43.525 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:43.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:43.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:43.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:43.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:43.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:43.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:43.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:43.809 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:43.817 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:43.881 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:43.918 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:43.926 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:43.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:43.950 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:44.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:44.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:44.035 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:44.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:44.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:44.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:44.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:44.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:44.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:44.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:44.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:44.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:44.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:44.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:44.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:44.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:44.315 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:44.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:44.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:44.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:44.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:44.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:44.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:44.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:44.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:44.571 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:44.579 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:44.639 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:44.648 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:44.656 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:44.664 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:44.673 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:44.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:44.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:44.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:44.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:44.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:44.783 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:44.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:44.803 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:44.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:44.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:45.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:45.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:45.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:45.076 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:45.083 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:45.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:45.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:45.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:45.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:45.118 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-23T18:49:45.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-23T18:49:45.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:45.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:45.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:45.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:45.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:45.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:45.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:45.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:45.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:45.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:45.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:45.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:45.456 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-23T18:49:45.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-23T18:49:45.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:45.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:45.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:45.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:45.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:45.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:45.566 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:45.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:45.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:45.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:45.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:45.783 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-23T18:49:45.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-23T18:49:45.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:45.804 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:45.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:45.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T18:49:45.826 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:45.900 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:45.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:45.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:45.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:45.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:46.104 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:46.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:46.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-23T18:49:46.187 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:46.196 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:46.205 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:46.214 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:46.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:46.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:46.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:46.298 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:46.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:46.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:46.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:46.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:46.403 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:46.463 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:46.533 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:46.539 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:46.547 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:46.556 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:46.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:46.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:46.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:46.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:46.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:46.669 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:46.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:46.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:46.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:46.797 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:46.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:46.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:47.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:47.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:47.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:47.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:47.149 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:47.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:47.164 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:47.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:47.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:47.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:47.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:47.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:47.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:47.432 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:47.500 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:47.515 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:47.522 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:47.528 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:47.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:47.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:47.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:47.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:47.616 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:47.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:47.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:47.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:47.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:47.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:47.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:47.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:47.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:47.864 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:47.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:47.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:47.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:47.961 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:47.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:47.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-23T18:49:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:48.041 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:48.089 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:48.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:48.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:48.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:48.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:48.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:48.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:48.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:48.261 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:48.324 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:48.335 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:48.344 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:48.352 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:48.360 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:48.369 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:48.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:48.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:48.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:48.588 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:48.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:48.606 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:48.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:48.623 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:48.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:48.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:48.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:48.948 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:48.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:49.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:49.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:49.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:49.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:49.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:49.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:49.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:49.145 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:49.154 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:49.161 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:49.170 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:49.177 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:49.207 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:49.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:49.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:49.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:49.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:49.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:49.385 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:49.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:49.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:49.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:49.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:49.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:49.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:49.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:49.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:49.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:49.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:49.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:49.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-23T18:49:49.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-23T18:49:49.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:49.774 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:49.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:49.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-23T18:49:49.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:49.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:49.857 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:49.866 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T18:49:49.875 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:49.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:49.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:49.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:49.926 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-23T18:49:49.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-23T18:49:50.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:50.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:50.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:50.078 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:50.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:50.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:50.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:50.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-23T18:49:50.122 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:50.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:50.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:50.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:50.206 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-23T18:49:50.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-23T18:49:50.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:50.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:50.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:49:50.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:50.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:50.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:50.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T18:49:50.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:50.536 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-23T18:49:50.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T18:49:50.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:50.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:50.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:50.576 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:50.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:50.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:50.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:50.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T18:49:50.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:50.788 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:50.795 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:50.800 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:50.807 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:50.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:50.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:50.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-23T18:49:50.884 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:50.890 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:50.896 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:50.903 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-23T18:49:50.911 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-23T18:49:50.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:49:50.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:51.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:51.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-23T18:49:51.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:51.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:51.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:51.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:51.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:51.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:51.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:51.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:51.245 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-23T18:49:51.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-23T18:49:51.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:51.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:51.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:51.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:51.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:51.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:51.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:51.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:51.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:51.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:51.491 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:49:51.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:49:51.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:49:51.559 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:51.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:51.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:49:51.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:51.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:51.623 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:51.655 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:51.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:51.923 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:51.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:51.961 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:52.007 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:52.071 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:52.135 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:52.159 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:52.166 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:52.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:52.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:52.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:52.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:52.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:52.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:52.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:52.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:52.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:52.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:52.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:52.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:52.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:52.594 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:52.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:52.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:52.614 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:52.666 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:52.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:52.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:52.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:52.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:52.751 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:52.776 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:52.813 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:52.875 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:52.883 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:52.889 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:52.896 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:52.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:52.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:52.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:52.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:52.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:52.990 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:52.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:53.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:53.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:53.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:53.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:53.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:53.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:53.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:53.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:53.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:53.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:53.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:53.274 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-23T18:49:53.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-23T18:49:53.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:53.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:53.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:53.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:53.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:53.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:53.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:53.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:53.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:53.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:53.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:53.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:53.597 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-23T18:49:53.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-23T18:49:53.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:53.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:53.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:53.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-23T18:49:53.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:53.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:53.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:53.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:53.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:53.907 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-23T18:49:53.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-23T18:49:53.922 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:53.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:53.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:49:53.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T18:49:53.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:53.966 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:53.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:54.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:54.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:54.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:54.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:54.160 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:54.167 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:54.199 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:54.238 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:49:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:54.311 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2024-04-23T18:49:54.321 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-23T18:49:54.329 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard", 2024-04-23T18:49:54.336 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2024-04-23T18:49:54.342 INFO:teuthology.orchestra.run.smithi019.stdout: "nfs", 2024-04-23T18:49:54.349 INFO:teuthology.orchestra.run.smithi019.stdout: "restful", 2024-04-23T18:49:54.356 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_schedule" 2024-04-23T18:49:54.410 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:54.418 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-23T18:49:54.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:54.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-23T18:49:54.502 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:54.509 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:54.540 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:54.616 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:49:54.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:49:54.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:49:54.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:54.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:54.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:49:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:54.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:54.778 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:54.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T18:49:54.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:54.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:54.905 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:54.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:54.920 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:49:54.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:49:54.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:55.012 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:55.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:55.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:55.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:55.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:55.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:55.129 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:55.138 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:55.147 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:55.156 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:55.232 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:55.241 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:55.309 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:55.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:55.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:55.477 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:49:55.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:49:55.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:55.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:55.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:55.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:55.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:55.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:55.546 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:55.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:55.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:55.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:55.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:55.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:55.667 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:49:55.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:49:55.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:55.811 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:55.818 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:55.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:49:55.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:55.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:55.847 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:49:55.856 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:49:55.863 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:49:55.869 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:49:55.919 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:49:55.927 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:49:55.976 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:49:55.983 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:49:55.991 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:55.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:56.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:56.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:56.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:56.182 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:49:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:49:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:56.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:49:56.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:56.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:56.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:49:56.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:56.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:56.389 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-23T18:49:56.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-23T18:49:56.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:56.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:56.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:56.524 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:56.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:56.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:56.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:56.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-23T18:49:56.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:56.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:56.622 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:56.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:56.721 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-23T18:49:56.728 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-23T18:49:56.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:56.774 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:56.828 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:56.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-23T18:49:56.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:56.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:56.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:56.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-23T18:49:56.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:56.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:56.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:56.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:57.030 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-23T18:49:57.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-23T18:49:57.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:57.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:57.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:57.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:57.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:57.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:57.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:57.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-23T18:49:57.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:57.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:57.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:57.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:57.342 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-23T18:49:57.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-23T18:49:57.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:57.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:57.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:57.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:57.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:57.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:57.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:57.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-23T18:49:57.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:57.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:57.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:57.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:57.693 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-23T18:49:57.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-23T18:49:57.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:49:57.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:57.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:57.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-23T18:49:57.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:57.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:57.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:57.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-23T18:49:57.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:57.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:57.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:57.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:58.066 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-23T18:49:58.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-23T18:49:58.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:58.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:58.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:58.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:58.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:58.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:58.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:58.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-23T18:49:58.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:58.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:58.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:58.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:58.229 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-23T18:49:58.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-23T18:49:58.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:58.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:58.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:58.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:58.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:58.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:58.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:58.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T18:49:58.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:58.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:58.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:58.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:58.522 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-23T18:49:58.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-23T18:49:58.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:58.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:58.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:58.623 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:49:58.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:58.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:58.726 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:58.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-23T18:49:58.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:58.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:58.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:58.789 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:58.795 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:49:58.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:58.837 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:49:58.846 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-23T18:49:58.857 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:49:58.922 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:49:58.932 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:49:58.990 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-23T18:49:59.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-23T18:49:59.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:49:59.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:59.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:49:59.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:59.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:59.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:59.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T18:49:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:49:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:59.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:59.291 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-23T18:49:59.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-23T18:49:59.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:49:59.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:59.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:59.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-23T18:49:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:49:59.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:49:59.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:59.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T18:49:59.558 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:49:59.566 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:49:59.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:49:59.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:49:59.585 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-23T18:49:59.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-23T18:49:59.753 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:49:59.760 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:49:59.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:49:59.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:49:59.809 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-23T18:49:59.858 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-23T18:49:59.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:49:59.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T18:49:59.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:50:00.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:00.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:00.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:00.024 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-23T18:50:00.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-23T18:50:00.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:50:00.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:00.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:00.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-23T18:50:00.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:50:00.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-23T18:50:00.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:00.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T18:50:00.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:00.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:00.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:00.402 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-23T18:50:00.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-23T18:50:00.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:00.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:00.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:00.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T18:50:00.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:00.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:00.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:00.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T18:50:00.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T18:50:00.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:00.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:00.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:00.680 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-23T18:50:00.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-23T18:50:00.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:50:00.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:00.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:00.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-23T18:50:00.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-23T18:50:00.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-23T18:50:00.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:00.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-23T18:50:00.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T18:50:00.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:00.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:00.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:00.842 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-23T18:50:00.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-23T18:50:00.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:00.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:00.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:00.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-23T18:50:01.004 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:01.015 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:01.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:01.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T18:50:01.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:50:01.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:01.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:01.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:01.175 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-23T18:50:01.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-23T18:50:01.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:50:01.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:01.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:01.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:50:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-23T18:50:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-23T18:50:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T18:50:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:50:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:01.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:50:01.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:50:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:50:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:50:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:50:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:50:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:50:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:50:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:50:02.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:02.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:02.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:02.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:02.867 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:02.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:02.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:02.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:02.902 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:02.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:02.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:02.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:02.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-23T18:50:03.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-23T18:50:03.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:50:03.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:03.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:03.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:50:03.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:03.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:03.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:03.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T18:50:03.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:03.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:03.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:03.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-23T18:50:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-23T18:50:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:03.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:03.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-23T18:50:03.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:03.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:03.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:03.383 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-23T18:50:03.429 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-23T18:50:03.435 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-23T18:50:03.442 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-23T18:50:03.449 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-23T18:50:03.457 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:03.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-23T18:50:03.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:03.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:03.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-23T18:50:03.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-23T18:50:03.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:03.592 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:03.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:03.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:03.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:03.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:03.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:03.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T18:50:03.767 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:03.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:03.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:03.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:03.801 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:50:03.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:50:03.874 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:03.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:03.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:03.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:50:03.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:03.994 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:04.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:04.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T18:50:04.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:04.132 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:04.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:04.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:04.158 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-23T18:50:04.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-23T18:50:04.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:04.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:04.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:04.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:50:04.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:50:04.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:04.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:04.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T18:50:04.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T18:50:04.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:04.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:04.524 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-23T18:50:04.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-23T18:50:04.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:50:04.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:04.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:04.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:50:04.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:04.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T18:50:04.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:04.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:04.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:04.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:04.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:04.836 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:50:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-23T18:50:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:50:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:50:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:50:04.870 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-23T18:50:04.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-23T18:50:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:50:04.935 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:04.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:05.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-23T18:50:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:05.048 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:05.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T18:50:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:05.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:05.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:05.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-23T18:50:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-23T18:50:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:05.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-23T18:50:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T18:50:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-23T18:50:05.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-23T18:50:05.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:05.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:05.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-23T18:50:05.557 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:05.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T18:50:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:05.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:05.680 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-23T18:50:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-23T18:50:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:05.846 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:05.864 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:05.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T18:50:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T18:50:05.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:05.901 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:05.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-23T18:50:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-23T18:50:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:50:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:06.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:06.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T18:50:06.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:06.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:06.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:06.224 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-23T18:50:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T18:50:06.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:50:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:06.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-23T18:50:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:06.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T18:50:06.565 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:06.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:06.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:06.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:06.623 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-23T18:50:06.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-23T18:50:06.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:06.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:06.652 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-23T18:50:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-23T18:50:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-23T18:50:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:06.862 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T18:50:06.871 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:06.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:06.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-23T18:50:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-23T18:50:06.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T18:50:07.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:07.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:07.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:07.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:07.208 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-23T18:50:07.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-23T18:50:07.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:07.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:07.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:07.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:07.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:07.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:07.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:07.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T18:50:07.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:07.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:07.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:07.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:07.465 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-23T18:50:07.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-23T18:50:07.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:07.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:07.592 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:07.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:07.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:07.613 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T18:50:07.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:07.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:07.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:07.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:07.729 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-23T18:50:07.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-23T18:50:07.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:07.871 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:07.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T18:50:07.885 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:07.899 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:07.906 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:07.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:50:07.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:07.934 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:07.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:07.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:08.007 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-23T18:50:08.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-23T18:50:08.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:08.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:08.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:08.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T18:50:08.167 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:08.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:08.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:08.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-23T18:50:08.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:08.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:08.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:08.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:08.300 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-23T18:50:08.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-23T18:50:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:08.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:08.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:08.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T18:50:08.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:08.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:08.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:08.586 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-23T18:50:08.622 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:08.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:08.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:08.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:08.655 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-23T18:50:08.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-23T18:50:08.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:08.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:08.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:08.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T18:50:08.805 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:08.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:08.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:08.834 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:50:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:08.879 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:08.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:08.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:08.985 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-23T18:50:08.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-23T18:50:09.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:09.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:09.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:09.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T18:50:09.035 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:09.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-23T18:50:09.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:09.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:09.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:09.220 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-23T18:50:09.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-23T18:50:09.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:09.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:09.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:09.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T18:50:09.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:09.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:09.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:09.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-23T18:50:09.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:09.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:09.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:09.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:09.494 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-23T18:50:09.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-23T18:50:09.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:09.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:09.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:09.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T18:50:09.656 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:09.666 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:09.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:09.682 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-23T18:50:09.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:09.701 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:09.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:09.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:09.783 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-23T18:50:09.840 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-23T18:50:09.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:09.857 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:09.865 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:09.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T18:50:09.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:10.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:10.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-23T18:50:10.021 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:10.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:10.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:10.057 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-23T18:50:10.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-23T18:50:10.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:10.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:10.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:10.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T18:50:10.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:10.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:10.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:10.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-23T18:50:10.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:10.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:10.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:10.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:10.350 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-23T18:50:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-23T18:50:10.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:10.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:10.385 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T18:50:10.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:10.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:10.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:10.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-23T18:50:10.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:10.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:10.671 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-23T18:50:10.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-23T18:50:10.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:10.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:10.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:10.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T18:50:10.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:10.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:10.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:10.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:50:10.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:10.825 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:10.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:10.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:10.844 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-23T18:50:10.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-23T18:50:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:10.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:10.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T18:50:10.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:11.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:11.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:11.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-23T18:50:11.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:11.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:11.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:11.178 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-23T18:50:11.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-23T18:50:11.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:11.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:11.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:11.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T18:50:11.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:11.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-23T18:50:11.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:11.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:11.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:11.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:11.426 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-23T18:50:11.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-23T18:50:11.567 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:11.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:11.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:11.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T18:50:11.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:11.678 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:11.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:11.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-23T18:50:11.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:11.800 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:11.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:11.819 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-23T18:50:11.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-23T18:50:11.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:11.838 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:11.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T18:50:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-23T18:50:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-23T18:50:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-23T18:50:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T18:50:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-23T18:50:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-23T18:50:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-23T18:50:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T18:50:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-23T18:50:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-23T18:50:12.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:50:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T18:50:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-23T18:50:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T18:50:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-23T18:50:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T18:50:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:13.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:13.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-23T18:50:13.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-23T18:50:13.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:13.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:13.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:13.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-23T18:50:13.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:13.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:13.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:13.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T18:50:13.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:13.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:13.586 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:13.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:13.602 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-23T18:50:13.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-23T18:50:13.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:13.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:13.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:13.742 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-23T18:50:13.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:13.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:13.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:13.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-23T18:50:13.782 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:13.875 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:13.937 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:13.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-23T18:50:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-23T18:50:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:14.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:14.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T18:50:14.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:14.132 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:14.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:14.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:14.242 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-23T18:50:14.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-23T18:50:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:14.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:14.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:14.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:50:14.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:14.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:14.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:14.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-23T18:50:14.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-23T18:50:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-23T18:50:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:14.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T18:50:14.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:14.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:14.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T18:50:14.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:14.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:14.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:14.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:14.798 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-23T18:50:14.805 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-23T18:50:14.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:14.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:14.826 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:14.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:50:14.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:14.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:14.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:14.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-23T18:50:14.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:15.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:15.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:15.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:15.030 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-23T18:50:15.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-23T18:50:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:15.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:15.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:15.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:15.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:15.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:15.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:15.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T18:50:15.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:15.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-23T18:50:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-23T18:50:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:15.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:15.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:15.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:15.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:15.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:15.559 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T18:50:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:15.752 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:50:15.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:50:15.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:15.771 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:15.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:15.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:15.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:15.912 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:15.919 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:15.926 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:16.032 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:16.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:16.089 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:16.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:16.126 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-23T18:50:16.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-23T18:50:16.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:16.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:16.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:16.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:16.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T18:50:16.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:16.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:50:16.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:50:16.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:16.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:16.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:16.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:16.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:16.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:16.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:16.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T18:50:16.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:16.568 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:16.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:16.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:16.698 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:50:16.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:50:16.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:16.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:16.786 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:16.794 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:50:16.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:16.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:16.877 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:16.905 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:16.912 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:16.921 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:16.933 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:16.943 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:16.976 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:17.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:17.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:17.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:17.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:17.124 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:50:17.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:50:17.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:17.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:17.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:17.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:17.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:17.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:17.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:17.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:17.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:17.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:17.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:17.527 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T18:50:17.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T18:50:17.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:17.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:17.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:17.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:17.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:17.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:17.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:17.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T18:50:17.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:17.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:17.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:17.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:17.716 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T18:50:17.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T18:50:17.868 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:17.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:17.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:17.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-23T18:50:17.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:17.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:17.975 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:17.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T18:50:17.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:17.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:18.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:18.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-23T18:50:18.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-23T18:50:18.091 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:18.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:18.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:18.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:50:18.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:18.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:18.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:18.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-23T18:50:18.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:18.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:18.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-23T18:50:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-23T18:50:18.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:18.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:18.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:18.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:50:18.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:18.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:18.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:18.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T18:50:18.554 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:18.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:18.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:18.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:18.582 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-23T18:50:18.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-23T18:50:18.602 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:18.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:18.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:18.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:18.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:18.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:18.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:18.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-23T18:50:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:18.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:18.828 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:18.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:18.844 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-23T18:50:18.853 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-23T18:50:18.860 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:18.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:18.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:18.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-23T18:50:18.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:18.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:18.991 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:18.999 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-23T18:50:19.009 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-23T18:50:19.017 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:19.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T18:50:19.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:19.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:19.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:19.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:19.159 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-23T18:50:19.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-23T18:50:19.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:19.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:19.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:19.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-23T18:50:19.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:19.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T18:50:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:19.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:19.423 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-23T18:50:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-23T18:50:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T18:50:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:19.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:19.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T18:50:19.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:19.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:19.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:19.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:19.652 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-23T18:50:19.662 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-23T18:50:19.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:19.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:19.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:19.815 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:19.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T18:50:19.835 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:19.846 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:19.855 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:19.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:19.926 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-23T18:50:19.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-23T18:50:19.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:19.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:20.008 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:20.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:20.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:20.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:20.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T18:50:20.134 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:20.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:20.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:20.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:20.178 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-23T18:50:20.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-23T18:50:20.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:20.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:20.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:20.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:20.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:20.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:20.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:20.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T18:50:20.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:20.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:20.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:20.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:20.504 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-23T18:50:20.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-23T18:50:20.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:20.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:20.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:20.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:20.611 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:20.622 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:20.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:20.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T18:50:20.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:20.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:20.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:20.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:20.801 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-23T18:50:20.811 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-23T18:50:20.821 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:20.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:20.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:20.852 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:20.859 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:20.867 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:20.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:20.964 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T18:50:20.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:20.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:20.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:21.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:21.041 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-23T18:50:21.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-23T18:50:21.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:21.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:21.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:21.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-23T18:50:21.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:21.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:21.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:21.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-23T18:50:21.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:21.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:21.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:21.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:21.328 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-23T18:50:21.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-23T18:50:21.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:21.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:21.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:21.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:21.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:21.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:21.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:21.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T18:50:21.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:21.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:21.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:21.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:21.573 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-23T18:50:21.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-23T18:50:21.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:21.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:21.625 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:21.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:21.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:21.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:21.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:21.770 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T18:50:21.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:21.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:21.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:21.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:21.809 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-23T18:50:21.818 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-23T18:50:21.826 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:21.845 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:21.900 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:21.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:21.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:21.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:21.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:21.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T18:50:21.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:22.030 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:22.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:22.058 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-23T18:50:22.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-23T18:50:22.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:22.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:22.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:22.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:22.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:22.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:22.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:22.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T18:50:22.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:22.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:22.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:22.478 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-23T18:50:22.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-23T18:50:22.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:22.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:22.530 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:22.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:22.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:22.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:22.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:22.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T18:50:22.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:22.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:22.714 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:22.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:22.735 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-23T18:50:22.768 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-23T18:50:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:22.889 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:22.896 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:22.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:22.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:22.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:22.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T18:50:22.955 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:22.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:50:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-23T18:50:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:50:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:50:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:50:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:50:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:50:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:50:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:50:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:50:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:50:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:50:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:24.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:24.503 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:50:24.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:50:24.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:24.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:24.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:24.648 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:24.659 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:24.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:24.677 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:24.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:24.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:24.859 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:24.871 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:24.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:24.897 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-23T18:50:24.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-23T18:50:24.920 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:24.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:24.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:24.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-23T18:50:25.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:25.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:25.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:25.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T18:50:25.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:25.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:25.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:25.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:25.269 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-23T18:50:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-23T18:50:25.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:25.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:25.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:25.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-23T18:50:25.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:25.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:25.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T18:50:25.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:25.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:25.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:25.487 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:25.500 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:25.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:25.589 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:50:25.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-23T18:50:25.663 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:50:25.672 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:50:25.682 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:50:25.690 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T18:50:25.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T18:50:25.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:25.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:25.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:25.830 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:50:25.839 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:25.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:25.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:25.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:25.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:25.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:25.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:26.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:26.034 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T18:50:26.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T18:50:26.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:26.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:26.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:26.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:26.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:26.153 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:26.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:26.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:26.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:26.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:26.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:26.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:26.286 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T18:50:26.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T18:50:26.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:26.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:26.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:26.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:26.446 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:26.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:26.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:26.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:26.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:26.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:26.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:26.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:26.644 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-23T18:50:26.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T18:50:26.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:26.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:26.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:26.841 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:26.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:26.857 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:26.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:26.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:26.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:26.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:26.989 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T18:50:27.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T18:50:27.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:27.020 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:27.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:27.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:27.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:27.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:27.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:27.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:27.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:27.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:27.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:27.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:27.214 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T18:50:27.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T18:50:27.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:27.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:27.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:27.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:27.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:27.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:27.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:27.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:27.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:27.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:27.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T18:50:27.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T18:50:27.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:27.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:27.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:27.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:27.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:27.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:27.859 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:27.871 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:27.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:27.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:27.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:28.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:28.100 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T18:50:28.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T18:50:28.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:28.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:28.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:28.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:28.157 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:28.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:28.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:28.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:28.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:28.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:28.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:28.324 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T18:50:28.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T18:50:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:28.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:28.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:28.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:28.520 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:28.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:28.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:28.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:28.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:28.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:28.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:28.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:28.664 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T18:50:28.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T18:50:28.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:28.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:28.733 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:28.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:28.857 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:28.869 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:28.880 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:28.892 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:28.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:28.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:28.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:28.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:28.983 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T18:50:28.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T18:50:29.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:29.049 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:29.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:29.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:29.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:29.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:29.170 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:29.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T18:50:29.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T18:50:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:29.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:29.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:29.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:29.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:29.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:29.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:29.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:29.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:29.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:29.663 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T18:50:29.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T18:50:29.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:29.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:29.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:29.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:29.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:29.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:29.763 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:29.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:29.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:29.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:29.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:29.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:29.920 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T18:50:29.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T18:50:29.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:30.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:30.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:30.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:30.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:30.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:30.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:30.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:30.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:30.250 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T18:50:30.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T18:50:30.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:30.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:30.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:30.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-23T18:50:30.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:30.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:30.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:30.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:30.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:30.505 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:30.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:30.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:30.592 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T18:50:30.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T18:50:30.612 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:30.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:30.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:30.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:30.720 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:30.823 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:30.838 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:30.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:30.866 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:30.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:30.885 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:30.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:30.905 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-23T18:50:30.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-23T18:50:31.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:31.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:31.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:31.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:31.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:31.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:31.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:31.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:31.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:31.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:31.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:31.257 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T18:50:31.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T18:50:31.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:31.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:31.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:31.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-23T18:50:31.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:31.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:31.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:31.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:31.455 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:31.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:31.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:31.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:31.652 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T18:50:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T18:50:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:31.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:31.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:31.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:31.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:31.720 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:31.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:31.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:31.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:31.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:31.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:31.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:31.837 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T18:50:31.886 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T18:50:31.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:31.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:31.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:31.980 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:31.988 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:31.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:32.006 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:32.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:32.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:32.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:32.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:32.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:32.168 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T18:50:32.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T18:50:32.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:32.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:32.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:32.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:32.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:32.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:32.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:32.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:32.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:32.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:32.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:32.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:32.492 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T18:50:32.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T18:50:32.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:32.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:32.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:32.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:32.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:32.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:32.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:32.635 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:32.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:32.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:32.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:32.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:32.677 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T18:50:32.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T18:50:32.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:32.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:32.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:32.794 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-23T18:50:32.802 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:32.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:32.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:32.845 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:32.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:32.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:32.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:33.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:33.009 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T18:50:33.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T18:50:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:33.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-23T18:50:33.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:33.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:33.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:33.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:33.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:33.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:33.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:33.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:33.337 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T18:50:33.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T18:50:33.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:33.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:33.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:33.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:33.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:33.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:33.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:33.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:33.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:33.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:33.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T18:50:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T18:50:33.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:33.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:33.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:33.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:33.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:33.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:33.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:33.734 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:33.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:33.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:33.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:33.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:33.890 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T18:50:33.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T18:50:33.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:33.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:33.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:34.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:34.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:34.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:34.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:34.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:34.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:34.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:34.201 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T18:50:34.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T18:50:34.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:34.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:34.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:34.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:34.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:34.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:34.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:34.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:34.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:34.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:34.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:34.490 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T18:50:34.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T18:50:34.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:34.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:34.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:34.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:34.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:34.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:34.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:34.696 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:34.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:34.859 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:34.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:34.872 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T18:50:34.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T18:50:34.889 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:34.895 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:34.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:34.909 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:34.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:35.008 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:35.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:35.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:35.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:35.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:35.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:35.141 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T18:50:35.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T18:50:35.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:35.211 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:35.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:35.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:35.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:35.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:35.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:35.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T18:50:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T18:50:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:35.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:35.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:35.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:35.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:35.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:35.585 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:35.592 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:35.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:35.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:35.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:35.709 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T18:50:35.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T18:50:35.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:35.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:35.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:35.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:35.852 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:35.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:35.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:35.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:35.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:35.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:35.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:35.954 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T18:50:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T18:50:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:36.058 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:36.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:36.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:36.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:36.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:36.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:36.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:36.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:36.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:36.322 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T18:50:36.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T18:50:36.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:36.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:36.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:36.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T18:50:36.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:36.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:36.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:36.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:36.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:36.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:36.610 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:36.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:36.695 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T18:50:36.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T18:50:36.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:36.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:36.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:36.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:50:36.738 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:36.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:36.811 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:36.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:36.879 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:36.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:36.892 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:36.942 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T18:50:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T18:50:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:37.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:37.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:37.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-23T18:50:37.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:37.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:37.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:37.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:37.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:37.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:37.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:37.252 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T18:50:37.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T18:50:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:37.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:37.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:37.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-23T18:50:37.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:37.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:37.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:37.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:37.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:37.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:37.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:37.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:37.525 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T18:50:37.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T18:50:37.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:37.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:37.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:37.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:37.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:37.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:37.737 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T18:50:37.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T18:50:37.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:37.838 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:37.882 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:37.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-23T18:50:37.894 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:38.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:38.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:38.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:38.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:38.088 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T18:50:38.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T18:50:38.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:38.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:38.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:38.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:38.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:38.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:38.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:38.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:38.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:38.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:38.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:38.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:38.435 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T18:50:38.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T18:50:38.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:38.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:38.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:38.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:38.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:38.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:38.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:38.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:38.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:38.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:38.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:38.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:38.641 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T18:50:38.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T18:50:38.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:38.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:38.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:38.815 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:38.822 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:38.829 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:38.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:38.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:38.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:38.887 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:38.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:38.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:38.954 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T18:50:38.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T18:50:38.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:38.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:39.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:50:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T18:50:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T18:50:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:50:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T18:50:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T18:50:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:50:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-23T18:50:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-23T18:50:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-23T18:50:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-23T18:50:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:40.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:40.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:40.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:40.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:40.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:40.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:40.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:40.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:40.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:40.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-23T18:50:40.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-23T18:50:40.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:40.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:40.811 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:40.825 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-23T18:50:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:40.855 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:40.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:40.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:40.984 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-23T18:50:40.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-23T18:50:41.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:41.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:41.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:41.058 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-23T18:50:41.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:41.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:41.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:41.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:41.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:41.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:41.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:41.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:41.222 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-23T18:50:41.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-23T18:50:41.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:41.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:41.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:41.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:41.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:41.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:41.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:50:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:50:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:41.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:41.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:41.756 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:41.765 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:41.774 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:41.785 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:41.833 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:41.895 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:41.951 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:41.958 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:41.965 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:41.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:41.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:41.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:41.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:42.066 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:50:42.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:50:42.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:42.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:42.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:42.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:42.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:42.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:42.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:50:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:50:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:42.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:42.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:50:42.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:42.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:42.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:42.577 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:42.633 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:42.690 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:42.738 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:42.749 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:42.756 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:42.762 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:42.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:42.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:42.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:42.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:42.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:42.900 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:50:42.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:50:42.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:42.920 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:42.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:42.961 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:43.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:43.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:43.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:43.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:43.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:43.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:43.120 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-23T18:50:43.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-23T18:50:43.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:43.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:43.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:43.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:43.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:43.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:43.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-23T18:50:43.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:43.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:43.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:43.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:43.441 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-23T18:50:43.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T18:50:43.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:43.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:43.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:43.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:43.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:43.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:43.613 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:43.622 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:43.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:43.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:43.674 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:43.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:43.771 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-23T18:50:43.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-23T18:50:43.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:43.798 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:43.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:43.846 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-23T18:50:43.856 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:43.864 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:43.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:43.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:43.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:43.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:43.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:43.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:43.982 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-23T18:50:44.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-23T18:50:44.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:44.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:44.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:44.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-23T18:50:44.162 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:44.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:44.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:44.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:44.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:44.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:44.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:44.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:44.304 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-23T18:50:44.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-23T18:50:44.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:44.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:44.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:44.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:44.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:44.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:44.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:44.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:44.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:44.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:44.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:44.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:44.549 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-23T18:50:44.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-23T18:50:44.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:44.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:44.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:44.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-23T18:50:44.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:44.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:44.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:44.787 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:44.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:44.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:44.809 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:44.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:44.823 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-23T18:50:44.837 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-23T18:50:44.843 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:44.900 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:44.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:44.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-23T18:50:44.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:44.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:44.982 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:45.024 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-23T18:50:45.069 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:45.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:45.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:45.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:45.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:45.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:45.138 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-23T18:50:45.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-23T18:50:45.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:45.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:45.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:45.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-23T18:50:45.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-23T18:50:45.292 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-23T18:50:45.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:45.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:45.352 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:45.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:45.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:45.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:45.466 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-23T18:50:45.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-23T18:50:45.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:45.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:45.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:45.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:45.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:45.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:45.637 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:45.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:45.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:45.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:45.699 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:45.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:45.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:50:45.815 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-23T18:50:45.822 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:50:45.829 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:50:45.836 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:50:45.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-23T18:50:45.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-23T18:50:45.889 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:45.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-23T18:50:46.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:46.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:46.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:46.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:46.157 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:50:46.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:50:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:46.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:46.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:46.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:46.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:46.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:46.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:46.305 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:46.313 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:46.321 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:46.373 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:46.505 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:46.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:46.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:46.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:46.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:46.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:46.568 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:50:46.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:50:46.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:46.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:46.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:46.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:46.726 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:46.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:46.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:46.879 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:46.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:46.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:46.954 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:46.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:46.972 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:50:46.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:50:46.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:47.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:47.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:47.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:50:47.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:47.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:47.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:47.127 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:47.139 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:47.148 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:47.261 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:47.270 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:47.278 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:47.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:47.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:47.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:47.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:47.362 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:50:47.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:50:47.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:47.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:47.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:47.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:47.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:47.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:47.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:47.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:47.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:47.632 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-23T18:50:47.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-23T18:50:47.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:47.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:47.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-23T18:50:47.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:47.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:47.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:47.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T18:50:47.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:47.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:47.934 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:47.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:47.950 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-23T18:50:47.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-23T18:50:47.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:47.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:47.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:47.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-23T18:50:48.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:48.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:48.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:48.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T18:50:48.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:48.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:48.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:48.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:48.265 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-23T18:50:48.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-23T18:50:48.378 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:48.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-23T18:50:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-23T18:50:48.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:48.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:48.594 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:48.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:48.612 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-23T18:50:48.621 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-23T18:50:48.676 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:48.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:48.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:48.807 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-23T18:50:48.815 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:48.824 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:48.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:48.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T18:50:48.853 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:48.861 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:48.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:48.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-23T18:50:48.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-23T18:50:48.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:49.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:49.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:49.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:50:49.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:49.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:49.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T18:50:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:49.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:49.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:49.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:49.272 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:50:49.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:50:49.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:49.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:50:49.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:49.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:49.439 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:49.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T18:50:49.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:49.573 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:49.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:49.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:49.597 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-23T18:50:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-23T18:50:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:49.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-23T18:50:49.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:49.746 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:49.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:49.761 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T18:50:49.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:49.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:49.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:49.876 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:49.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:49.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:49.907 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:50:49.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-23T18:50:49.921 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:50:49.928 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:50:49.937 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:50:49.991 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:50:50.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:50:50.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:50.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:50.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:50.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:50.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:50.090 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:50.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:50.201 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:50.222 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:50.232 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:50.240 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:50.249 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:50.258 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:50.289 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:50.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:50.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:50.433 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:50:50.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:50:50.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:50.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:50.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:50.678 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:50.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:50.696 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:50.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:50.714 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:50.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:50.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:50.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:50.814 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:50:50.822 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:50:50.830 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:50.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:50.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:50.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:50:50.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:50.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:51.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:51.008 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:51.019 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:51.030 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:51.038 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:51.046 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:51.054 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:51.062 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:51.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:51.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:51.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:51.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:51.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:51.194 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:50:51.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:50:51.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:51.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:51.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:51.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:51.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:51.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:51.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:51.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:51.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:51.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:51.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:51.526 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-23T18:50:51.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-23T18:50:51.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:51.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:51.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:51.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-23T18:50:51.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:51.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:51.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:51.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:51.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:51.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:51.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:51.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:51.733 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-23T18:50:51.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-23T18:50:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-23T18:50:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:51.923 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:51.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:52.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:52.034 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:52.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:52.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:52.055 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:50:52.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:50:52.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:52.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:52.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:52.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-23T18:50:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:52.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:52.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:52.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:52.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:52.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:52.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:50:52.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-23T18:50:52.462 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-23T18:50:52.470 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-23T18:50:52.479 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:50:52.522 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-23T18:50:52.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-23T18:50:52.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:52.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:52.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:52.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-23T18:50:52.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:52.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:52.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:52.796 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T18:50:52.804 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:52.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:52.820 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:52.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:52.836 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-23T18:50:52.845 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-23T18:50:52.854 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:52.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:52.959 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:52.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-23T18:50:52.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:52.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:52.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:53.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T18:50:53.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:53.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:53.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:53.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:53.186 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-23T18:50:53.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-23T18:50:53.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:53.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:53.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:53.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:53.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:53.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:53.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:53.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-23T18:50:53.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:53.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:53.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:53.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:53.408 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:50:53.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:50:53.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:50:53.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:53.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:53.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-23T18:50:53.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-23T18:50:53.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:53.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:53.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T18:50:53.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:53.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:53.607 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:53.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:53.622 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:50:53.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:50:53.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:53.730 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:53.813 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:53.836 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:53.844 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:53.852 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:53.859 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:53.939 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:53.999 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:54.006 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:54.012 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:54.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:54.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:54.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:54.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:54.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:54.168 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:50:54.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:50:54.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:54.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:54.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:54.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:54.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:54.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:54.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:54.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:54.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:54.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:54.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:54.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:54.364 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:50:54.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:50:54.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:54.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:54.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:50:54.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:54.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:54.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:54.599 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:54.609 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:54.666 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:54.753 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:54.761 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:54.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:54.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:54.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:54.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:54.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:54.935 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:50:54.944 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:50:54.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:54.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:54.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:54.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:54.985 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:55.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:55.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:55.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:55.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:55.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:55.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:55.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:55.172 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-23T18:50:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-23T18:50:55.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:55.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:55.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:55.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:55.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-23T18:50:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:55.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:55.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:55.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:55.475 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-23T18:50:55.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-23T18:50:55.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:55.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:55.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:55.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-23T18:50:55.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:55.693 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:55.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:55.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-23T18:50:55.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:55.722 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:55.729 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:55.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:55.824 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-23T18:50:55.830 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-23T18:50:55.836 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:55.842 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:55.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:55.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-23T18:50:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:55.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:55.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:56.001 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:50:56.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:56.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:56.028 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:56.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:56.044 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-23T18:50:56.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-23T18:50:56.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:50:56.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:56.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:56.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:50:56.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:50:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-23T18:50:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T18:50:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:56.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:56.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:56.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:56.340 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-23T18:50:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-23T18:50:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:56.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:56.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:56.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:56.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:56.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:56.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:56.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-23T18:50:56.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:56.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:56.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:56.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:56.701 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-23T18:50:56.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-23T18:50:56.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:56.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:56.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:50:56.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-23T18:50:56.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:56.763 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:56.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:56.830 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:50:56.837 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:56.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:56.851 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:56.858 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:56.865 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:56.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:56.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:50:57.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-23T18:50:57.030 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:50:57.036 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:50:57.043 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:50:57.050 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:50:57.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:50:57.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:57.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:57.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:57.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:57.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:57.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:57.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:57.209 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:57.216 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:57.263 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:57.315 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:57.321 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:57.384 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:57.391 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:57.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:57.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:57.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:57.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:57.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:57.505 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:50:57.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:50:57.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:57.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:57.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:57.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:57.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:57.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:57.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:57.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:57.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:57.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:57.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:57.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:57.807 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:50:57.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:50:57.824 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:57.833 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:57.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:57.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:50:57.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:57.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:57.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:57.936 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:57.990 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:58.066 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:58.153 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:58.180 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:58.189 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:58.200 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:58.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:58.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:58.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:58.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:58.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:58.298 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:50:58.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:50:58.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:58.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:58.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:58.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:58.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:58.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:58.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:58.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:58.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:58.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:58.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:58.566 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:58.575 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:58.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:58.590 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:50:58.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-23T18:50:58.647 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:50:58.653 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:50:58.661 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:50:58.668 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:50:58.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:50:58.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:58.801 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:58.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:58.847 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:50:58.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:58.864 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:58.871 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:58.878 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:58.885 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:58.892 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:58.899 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:58.934 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:58.942 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:58.948 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:58.955 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:58.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:58.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:59.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:59.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:59.130 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:50:59.138 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:50:59.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:59.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:59.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:59.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:59.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:59.277 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:50:59.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:50:59.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:50:59.426 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:59.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:59.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:50:59.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:59.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:59.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:50:59.478 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:50:59.487 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:50:59.496 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:50:59.554 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:50:59.563 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:50:59.572 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:50:59.581 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:50:59.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:59.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:59.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:59.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:50:59.773 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:50:59.781 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:50:59.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:50:59.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:50:59.805 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:50:59.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:50:59.820 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:50:59.828 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:50:59.836 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:50:59.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:50:59.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:50:59.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:50:59.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:50:59.936 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:50:59.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:00.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:00.065 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:00.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-23T18:51:00.080 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:00.086 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:00.093 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:00.100 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-23T18:51:00.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-23T18:51:00.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:00.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:00.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:00.191 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-23T18:51:00.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:00.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:00.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:00.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T18:51:00.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:00.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:00.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:00.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:00.371 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-23T18:51:00.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-23T18:51:00.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:00.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:00.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:00.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:51:00.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-23T18:51:00.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:00.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:00.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T18:51:00.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:00.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:00.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:00.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:00.530 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:00.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:00.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:00.602 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:00.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:00.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:00.623 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:00.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:00.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:00.644 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:00.680 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:00.687 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:00.696 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:00.704 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:00.713 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:00.722 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:00.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:00.827 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:00.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:00.896 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:00.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:00.912 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:00.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:00.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:00.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:00.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:00.956 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:01.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:01.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:01.027 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:01.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:01.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:01.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:01.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:01.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:01.192 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:01.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:01.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:01.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:01.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:01.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:01.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:01.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:01.275 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:01.284 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:01.292 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:01.360 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-23T18:51:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-23T18:51:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-23T18:51:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-23T18:51:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-23T18:51:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T18:51:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:02.736 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:02.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:02.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:02.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:02.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:02.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:02.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:02.868 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:02.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:02.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:02.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:02.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:02.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:02.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:02.967 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:02.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:02.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:03.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:03.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:03.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:03.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:03.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:03.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:03.134 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:03.175 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:03.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:03.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:03.320 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:03.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:03.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:03.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:03.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:03.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:03.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:03.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:03.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:03.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:03.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-23T18:51:03.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-23T18:51:03.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:03.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:03.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:03.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:03.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:03.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:03.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:03.717 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-23T18:51:03.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T18:51:03.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:03.770 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:03.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-23T18:51:03.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-23T18:51:03.805 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:03.813 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:03.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:03.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-23T18:51:03.867 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:03.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:03.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:03.890 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-23T18:51:03.899 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:03.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:03.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:03.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:04.001 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-23T18:51:04.007 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-23T18:51:04.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:04.022 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:04.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:04.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-23T18:51:04.047 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:04.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:04.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:04.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T18:51:04.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:04.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:04.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:04.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:04.141 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-23T18:51:04.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-23T18:51:04.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:04.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:04.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:04.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:04.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:04.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:04.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:04.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-23T18:51:04.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:04.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:04.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:04.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:04.422 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-23T18:51:04.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-23T18:51:04.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:04.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:04.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:04.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-23T18:51:04.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:04.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:04.587 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:04.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T18:51:04.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T18:51:04.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:04.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:04.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:04.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:04.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-23T18:51:04.687 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:04.692 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:04.724 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:04.773 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:04.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:04.900 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:04.907 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:04.915 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:04.969 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:04.978 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:05.006 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:05.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:05.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:05.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:05.174 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:05.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:05.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:05.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:05.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:05.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:05.314 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:05.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:05.460 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:05.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:05.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:05.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:05.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:05.563 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:05.572 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:05.595 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:05.601 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:05.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:05.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:05.774 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:05.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:05.786 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:05.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:05.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:05.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:05.831 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:05.844 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:05.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:06.084 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:06.091 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:06.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-23T18:51:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:06.134 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:06.183 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:06.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:06.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:06.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:06.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:06.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:06.392 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:06.407 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:06.414 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:06.430 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:06.436 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:06.488 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:06.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:06.522 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:06.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:06.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:06.843 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:06.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:06.861 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:06.870 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:06.888 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:07.070 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:07.078 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:07.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:07.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:07.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:07.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:07.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:07.177 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:07.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:07.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:07.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:07.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:07.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:07.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:07.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:07.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:07.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:07.396 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:07.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:07.455 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:07.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:07.470 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:07.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:07.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:07.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-23T18:51:07.499 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:07.542 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:07.605 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:07.659 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:07.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:07.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:07.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:07.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:07.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:07.700 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:07.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:07.759 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:07.766 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:07.773 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:07.779 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:07.786 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:07.793 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:07.800 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:07.835 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:07.883 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:07.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:07.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:07.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:07.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:07.985 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:07.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:07.998 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:08.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:08.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:08.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:08.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:08.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:08.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:08.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:08.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:08.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:08.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:08.263 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:08.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:08.282 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:08.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:08.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:08.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:08.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:08.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:08.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:08.366 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:08.374 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:08.382 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:08.390 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:08.399 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:08.435 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:08.483 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:08.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:08.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:08.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:08.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:08.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:08.592 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:08.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:08.659 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:08.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:08.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:08.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:08.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:08.702 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:08.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:08.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:08.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:08.871 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:08.880 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:08.887 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:08.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:08.904 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:08.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-23T18:51:08.921 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:08.933 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:08.941 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:08.992 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-23T18:51:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-23T18:51:09.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:09.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:09.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:09.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:09.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:09.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:09.203 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T18:51:09.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:09.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:09.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:09.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:09.236 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:09.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:09.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:09.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:09.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:09.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:09.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:09.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:09.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:09.364 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:09.416 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:09.478 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:09.484 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:09.491 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:09.498 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:09.505 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:09.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:09.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:09.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:09.568 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:09.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:09.581 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:09.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:09.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:09.601 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:09.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:09.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:09.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:09.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:09.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:09.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:09.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:09.990 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:09.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:10.004 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:10.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:10.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:10.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:10.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:10.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:10.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:10.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:10.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:10.191 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:10.269 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:10.277 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:10.286 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:10.297 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:10.314 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:10.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:10.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:10.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:10.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:10.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:10.413 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:10.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:10.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:10.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:10.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:10.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:10.622 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:10.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:10.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:10.700 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-23T18:51:10.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-23T18:51:10.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:10.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:10.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:10.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:10.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:10.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:10.832 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-23T18:51:10.886 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-23T18:51:10.894 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-23T18:51:10.902 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:10.910 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-23T18:51:10.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:10.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:10.937 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:10.945 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:10.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:10.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:11.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-23T18:51:11.012 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:11.021 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:11.059 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:11.149 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:11.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:11.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:11.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:11.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:11.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:11.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:11.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:11.317 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:11.325 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:11.332 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:11.339 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:11.346 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:11.353 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:11.360 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:11.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:11.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:11.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:11.545 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:11.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:11.557 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:11.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:11.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:11.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:11.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:11.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:11.620 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:11.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:11.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:11.640 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:11.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:11.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:11.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:11.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:11.803 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:11.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:11.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:11.824 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:11.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:11.838 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:11.845 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:11.852 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-23T18:51:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:12.814 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:12.823 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:12.831 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:12.839 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:12.847 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:12.856 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:12.864 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:13.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:13.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:13.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:13.476 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:13.483 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:13.489 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:13.498 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:13.504 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:13.511 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:13.519 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:13.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:13.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:13.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:13.591 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:13.604 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:13.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:13.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:13.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:13.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:13.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:13.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:13.802 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:13.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:13.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:13.823 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:13.829 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:13.896 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:13.905 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:13.914 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:13.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:13.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:13.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-23T18:51:13.948 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:14.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:14.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:14.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:14.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:14.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:14.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:14.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:14.133 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:14.139 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:14.148 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:14.157 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:14.166 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:14.174 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:14.242 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:14.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:14.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:14.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:14.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:14.410 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:14.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:14.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:14.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:14.548 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:14.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:14.704 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:14.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:14.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:14.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:14.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:14.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:14.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:14.825 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:14.841 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:14.847 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:14.854 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:14.925 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:14.991 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:15.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:15.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:15.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:15.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:15.109 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:15.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:15.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:15.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:15.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:15.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:15.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:15.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:15.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:15.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:15.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:51:15.443 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:51:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:51:15.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:15.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:15.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:51:15.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:15.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:15.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:15.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:15.648 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-23T18:51:15.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-23T18:51:15.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:51:15.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:15.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:15.726 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-23T18:51:15.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-23T18:51:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:15.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-23T18:51:15.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T18:51:15.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:15.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:15.824 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:15.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:15.867 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:15.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-23T18:51:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-23T18:51:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-23T18:51:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:16.058 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:16.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:16.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:16.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:16.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T18:51:16.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:16.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:16.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:16.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-23T18:51:16.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-23T18:51:16.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:16.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:16.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:51:16.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:16.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:16.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:16.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:16.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:16.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:16.505 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:16.557 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:16.606 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:16.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:16.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:16.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:16.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:16.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:16.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:16.695 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:16.701 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:16.757 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:16.763 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:16.770 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:16.778 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:16.784 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:16.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:16.796 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:16.837 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:16.889 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:16.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:16.951 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:16.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:16.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:16.972 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:16.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:16.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:16.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:17.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:17.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:17.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:17.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:17.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:17.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:17.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:17.099 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:17.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:17.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:17.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:17.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:17.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:17.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:17.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:17.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:17.255 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:17.293 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:17.300 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:17.307 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:17.315 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:17.322 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:17.330 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:17.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:17.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:17.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:17.494 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:17.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:17.506 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:17.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:17.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:17.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:17.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:17.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:17.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:17.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:17.594 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:17.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:17.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:17.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:17.689 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-23T18:51:17.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-23T18:51:17.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:17.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:17.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:17.763 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-23T18:51:17.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:17.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:17.788 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:17.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-23T18:51:17.831 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:17.839 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:17.847 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:17.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:17.865 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-23T18:51:17.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-23T18:51:17.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:51:17.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:18.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:18.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-23T18:51:18.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:18.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:18.083 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:18.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-23T18:51:18.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:18.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:18.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:18.176 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:18.185 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:18.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:18.201 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:18.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-23T18:51:18.218 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:18.249 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:18.306 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:18.364 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-23T18:51:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-23T18:51:18.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:18.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:18.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:18.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:51:18.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:18.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:18.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:18.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:18.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:18.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:18.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:18.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:18.529 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-23T18:51:18.562 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-23T18:51:18.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:18.662 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:51:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:18.691 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:18.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T18:51:18.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T18:51:18.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:18.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:18.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:18.763 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:18.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:18.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:18.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:18.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:19.018 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:19.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:19.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:19.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:19.160 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:19.168 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:19.174 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:19.181 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:19.187 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:19.250 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:19.361 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:19.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:19.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:19.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:19.550 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:19.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:19.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:19.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:19.635 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:19.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:19.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:19.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:19.670 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:19.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:19.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:19.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:19.828 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:19.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:19.846 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:19.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:19.861 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:19.868 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:19.892 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:19.899 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:19.932 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:19.939 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:19.946 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:19.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:19.960 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:20.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:20.113 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:20.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:20.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:20.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:20.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:20.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:20.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:20.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:20.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:20.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:20.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:20.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:20.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:20.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:20.335 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-23T18:51:20.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-23T18:51:20.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:20.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:20.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:20.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:20.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:20.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:20.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:20.522 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:20.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:20.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:20.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:20.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:20.600 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T18:51:20.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T18:51:20.617 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:20.625 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:20.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:20.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-23T18:51:20.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:20.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:20.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:20.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:20.823 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:20.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:20.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:20.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:20.860 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-23T18:51:20.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-23T18:51:20.922 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:51:20.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:20.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:20.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-23T18:51:20.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:20.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:21.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:21.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:21.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:21.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:21.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:21.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:21.167 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-23T18:51:21.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-23T18:51:21.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:21.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:21.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:21.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-23T18:51:21.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:21.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:21.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:21.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T18:51:21.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:21.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:21.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:21.530 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-23T18:51:21.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-23T18:51:21.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:21.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:21.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-23T18:51:21.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:21.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:21.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:21.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T18:51:21.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:21.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:21.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:21.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:21.744 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-23T18:51:21.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-23T18:51:21.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:21.874 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:21.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:21.895 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-23T18:51:21.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:21.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:21.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:21.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:21.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:21.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:21.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:22.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:22.012 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-23T18:51:22.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-23T18:51:22.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:22.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:22.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:22.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-23T18:51:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:22.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:22.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:22.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-23T18:51:22.226 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-23T18:51:22.238 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:22.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:22.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:22.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:22.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-23T18:51:22.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-23T18:51:22.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:22.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:22.489 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:22.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-23T18:51:22.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-23T18:51:22.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-23T18:51:22.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:22.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:22.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:22.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:22.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:22.655 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:22.663 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:22.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:22.688 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:22.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-23T18:51:22.707 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:22.717 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:22.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:22.871 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:22.880 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:22.890 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:22.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:22.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:22.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:22.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:22.937 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:23.000 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:23.010 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:23.020 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-23T18:51:24.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-23T18:51:24.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:24.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:24.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:24.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-23T18:51:24.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:24.613 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:24.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:24.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:24.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:24.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:24.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:24.728 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-23T18:51:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T18:51:24.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:24.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:24.868 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:24.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:24.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:24.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:24.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:24.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:24.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:24.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:24.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:25.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:25.033 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-23T18:51:25.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-23T18:51:25.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:25.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:25.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:25.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:25.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:25.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:25.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:25.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:25.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:25.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:25.259 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:25.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:25.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:25.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-23T18:51:25.349 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:25.363 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:25.374 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:25.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-23T18:51:25.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-23T18:51:25.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:25.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:25.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:25.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:51:25.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:25.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:25.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:25.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:25.613 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:25.626 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:25.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:25.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:25.674 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-23T18:51:25.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-23T18:51:25.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:25.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:25.838 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:25.848 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:25.856 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:25.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:25.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:25.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:25.890 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:25.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:25.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:25.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:25.961 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:26.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:26.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:26.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:26.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:26.078 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:26.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:26.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:26.166 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:26.174 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:26.182 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:26.190 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:26.197 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:26.205 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:26.234 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:26.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:26.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:26.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:26.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:26.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:26.301 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:26.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:26.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:26.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:26.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:26.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:26.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:26.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:26.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:26.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:26.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:26.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:26.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:26.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:26.548 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:26.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:26.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:26.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:26.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:26.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:26.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:26.816 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:26.828 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:26.835 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:26.854 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:26.862 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:26.911 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:26.921 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:26.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:26.961 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:26.971 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:26.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:26.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:26.997 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:27.036 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:27.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:27.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:27.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:27.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:27.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:27.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:27.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:27.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:27.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:27.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:27.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:27.312 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-23T18:51:27.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-23T18:51:27.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:27.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:27.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:27.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:27.589 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-23T18:51:27.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-23T18:51:27.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:27.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:27.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:27.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:27.805 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:27.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:27.820 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:27.828 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:27.837 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:27.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:27.851 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:27.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:27.884 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:27.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-23T18:51:27.900 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:27.947 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:27.955 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:27.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:27.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:28.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:28.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:28.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:28.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:28.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:28.120 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:28.127 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:28.135 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:28.142 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:28.165 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:28.171 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:28.179 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:28.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:28.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:28.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:28.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:28.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:28.367 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:28.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:28.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:28.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:28.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:28.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:28.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:28.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:28.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:28.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:28.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:28.555 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:28.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:28.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:28.693 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:28.705 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:28.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:28.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:28.745 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:28.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:28.769 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:28.778 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:28.787 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:28.820 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:28.832 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:28.880 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:28.889 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:28.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:28.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:28.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:28.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:29.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:29.051 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:29.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:29.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:29.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:29.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:29.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:29.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:29.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:29.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:29.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:29.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:29.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-23T18:51:29.450 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:29.458 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:29.469 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:29.480 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:29.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:29.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:29.562 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:29.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:29.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:29.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:29.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:29.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:29.671 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:29.681 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:29.719 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:29.772 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:29.780 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:29.791 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:29.802 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:29.814 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:29.834 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:29.844 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:29.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:29.909 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:29.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:29.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:29.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:29.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:29.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:29.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:30.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:30.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:30.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:30.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:30.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:30.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:30.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:30.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:30.271 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:30.320 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:30.374 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:30.386 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:30.398 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:30.416 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:30.427 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:30.436 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:30.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:30.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:30.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:30.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:30.531 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:30.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:30.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:30.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:30.601 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:30.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:30.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:30.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:30.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:30.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:30.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:30.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:30.778 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:30.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:30.798 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-23T18:51:30.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-23T18:51:30.870 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:30.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:30.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:30.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-23T18:51:30.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:30.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:30.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T18:51:31.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:31.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:31.147 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:31.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:31.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:31.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-23T18:51:31.184 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:31.249 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:31.293 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:31.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:31.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:31.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:31.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:31.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:31.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:31.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:31.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:31.489 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:31.496 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:31.503 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:31.509 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:31.518 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:31.578 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:31.587 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:31.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:31.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:31.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:31.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:31.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:31.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:31.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:31.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:31.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:31.785 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:31.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:31.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:31.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:31.823 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:31.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:31.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:31.926 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:31.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:31.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:31.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:31.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:32.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:32.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:32.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:32.076 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:32.083 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:32.090 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:32.099 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:32.125 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:32.132 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:32.139 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:32.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:32.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:32.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:32.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:32.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:32.251 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:32.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:32.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:32.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:32.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:32.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:32.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:32.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:32.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:32.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:32.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:32.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:32.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:32.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:32.506 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-23T18:51:32.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-23T18:51:32.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:32.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:32.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:32.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:32.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:32.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:32.658 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:32.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:32.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:32.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:32.744 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:32.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:32.799 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-23T18:51:32.837 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-23T18:51:32.847 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:32.864 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:32.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-23T18:51:32.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:32.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:32.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:32.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:32.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:33.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:33.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-23T18:51:33.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-23T18:51:33.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:33.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:33.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:33.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:33.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:33.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:33.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:33.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:33.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:33.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:33.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:33.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:33.349 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-23T18:51:33.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-23T18:51:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:33.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:33.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:33.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:33.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:33.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:33.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:33.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:33.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:33.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:33.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:33.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:33.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-23T18:51:33.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-23T18:51:33.670 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-23T18:51:33.676 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:33.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:33.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:33.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:33.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:33.825 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:33.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:33.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:33.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:33.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:33.896 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-23T18:51:33.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-23T18:51:34.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:34.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:34.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:34.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:34.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:34.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:34.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:34.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:34.073 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:34.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:34.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:34.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:34.246 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-23T18:51:34.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-23T18:51:34.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:34.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:34.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:34.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:34.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:34.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:34.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:34.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:34.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:34.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:34.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:34.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:34.459 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-23T18:51:34.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-23T18:51:34.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:34.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:34.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:34.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:51:34.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:34.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:34.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:34.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:34.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:34.695 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:34.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:34.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:34.721 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-23T18:51:34.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-23T18:51:34.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:34.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:34.870 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:34.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:34.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-23T18:51:34.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-23T18:51:34.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:34.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:34.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:35.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:35.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:35.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-23T18:51:35.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-23T18:51:35.097 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:35.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:35.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:35.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:35.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:35.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:35.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:35.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:35.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:35.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:35.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:35.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-23T18:51:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-23T18:51:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:35.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:35.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:35.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:35.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:35.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:35.512 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:35.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:35.611 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-23T18:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-23T18:51:35.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:35.756 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:35.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:35.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:35.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:35.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:35.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:35.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:35.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:35.871 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:35.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:35.932 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:35.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:35.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:35.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-23T18:51:35.997 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:36.036 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:36.079 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-23T18:51:36.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-23T18:51:36.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:36.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:36.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:36.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:36.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T18:51:36.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:36.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:36.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:36.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:36.256 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-23T18:51:36.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-23T18:51:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:36.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:36.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:36.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:36.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:36.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:36.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:36.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-23T18:51:36.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:36.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:36.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:36.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:36.558 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:36.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:36.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:36.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:36.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:36.664 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:36.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:36.683 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:36.691 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:36.699 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:36.707 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:36.716 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:36.723 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:36.753 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:36.793 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:36.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:36.841 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:36.850 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:36.858 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:36.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:36.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:36.998 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:37.009 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:37.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:37.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:37.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:37.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:37.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:37.215 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:37.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:37.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:37.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:37.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:37.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:37.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:37.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:37.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:37.470 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:37.477 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:37.556 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:37.650 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:37.719 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:37.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:37.736 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:37.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:37.871 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:37.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:37.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:37.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:38.042 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:38.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:38.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:38.095 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:38.104 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:38.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:38.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:38.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-23T18:51:38.214 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:38.222 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:38.231 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:38.239 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:38.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:38.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:38.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:38.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:38.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:38.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:38.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:38.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:38.461 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:38.470 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:38.479 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:38.519 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:38.584 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:38.593 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:38.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:38.613 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:38.622 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:38.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:38.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:38.681 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:38.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:38.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:38.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:38.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:38.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:38.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:38.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:38.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:38.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:38.896 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:38.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:38.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:38.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:38.922 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:38.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:38.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:38.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:38.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:39.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-23T18:51:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:40.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:40.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:40.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:40.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:40.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:40.555 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:40.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:40.584 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:40.592 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:40.651 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:40.689 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:40.726 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:40.734 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:40.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:40.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:40.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:40.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:40.855 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:40.862 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:40.870 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:40.876 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:40.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:40.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:40.895 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:40.950 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:40.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:41.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:41.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:41.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:41.049 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:41.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:41.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:41.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-23T18:51:41.143 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:41.149 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:41.157 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:41.164 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-23T18:51:41.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-23T18:51:41.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:41.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:41.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:41.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T18:51:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:41.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:41.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:41.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:41.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:41.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:41.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:51:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:51:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:51:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-23T18:51:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:41.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:41.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:41.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:41.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:41.788 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:41.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:41.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:41.816 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:41.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:41.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:41.842 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:41.873 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:41.883 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:41.912 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:41.943 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:41.951 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:41.960 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:41.969 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:41.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:42.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:42.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:42.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:42.087 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:42.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:42.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:42.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:42.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:42.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:42.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:42.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:42.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:42.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:42.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:42.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:42.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:42.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:42.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:42.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:42.547 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:42.555 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:42.563 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:42.570 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:42.642 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:42.649 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:42.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:42.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:42.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:42.728 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:42.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:42.742 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:42.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:42.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:42.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:42.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:42.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:42.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:42.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:42.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:42.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:42.916 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:42.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:42.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:43.009 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:43.017 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:43.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:43.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:43.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-23T18:51:43.050 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:43.057 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:43.064 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:43.120 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-23T18:51:43.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-23T18:51:43.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:43.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:43.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:43.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:51:43.237 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:43.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:43.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:43.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T18:51:43.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:43.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:43.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:43.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:43.392 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-23T18:51:43.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-23T18:51:43.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:43.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:43.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:43.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:51:43.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:43.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T18:51:43.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:43.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:43.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:43.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:43.700 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-23T18:51:43.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-23T18:51:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:43.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:43.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:43.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:51:43.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:43.776 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:43.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:43.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T18:51:43.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:43.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:43.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:43.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:43.917 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-23T18:51:43.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-23T18:51:43.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:43.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:43.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:44.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:44.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:44.026 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:44.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:44.048 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T18:51:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:44.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:44.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:44.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:44.202 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-23T18:51:44.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-23T18:51:44.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:44.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:44.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:44.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:44.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:44.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:44.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:44.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T18:51:44.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:44.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:44.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:44.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:44.487 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-23T18:51:44.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-23T18:51:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:44.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:44.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:44.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:44.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:44.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:44.691 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:44.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:44.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:44.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:44.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:44.744 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-23T18:51:44.803 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-23T18:51:44.811 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:44.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:44.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:44.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:44.933 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:44.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:45.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:45.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:45.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:45.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:45.077 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-23T18:51:45.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-23T18:51:45.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:45.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:45.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:45.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T18:51:45.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:45.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:45.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:45.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:45.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:45.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:45.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:45.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:45.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-23T18:51:45.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-23T18:51:45.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:45.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:45.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:45.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:45.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:45.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:45.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:45.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:45.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:45.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:45.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:45.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:45.696 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:51:45.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:51:45.786 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:45.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:45.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:45.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-23T18:51:45.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-23T18:51:45.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:45.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:45.869 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:45.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:45.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-23T18:51:46.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-23T18:51:46.066 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:46.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:46.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:46.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-23T18:51:46.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:46.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:46.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:46.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:46.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:46.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:46.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:46.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:46.284 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-23T18:51:46.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-23T18:51:46.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:46.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:46.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:46.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:46.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:46.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:46.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:46.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:46.574 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-23T18:51:46.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-23T18:51:46.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:46.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:46.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:46.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:46.768 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:46.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:46.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:46.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:46.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:46.899 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:46.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:46.914 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:46.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:46.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:46.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:46.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:47.002 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:47.013 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:47.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:47.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:47.121 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:47.131 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:47.138 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:47.148 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:47.192 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:47.255 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:47.264 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:47.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:47.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:47.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:47.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:47.326 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:47.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:47.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:47.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:47.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:47.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:47.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:47.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:47.619 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:47.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:47.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:47.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:47.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:47.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:47.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:47.756 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:47.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:47.850 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:47.871 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:47.880 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:47.889 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:47.938 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:48.024 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:48.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:48.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:48.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:48.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:48.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:48.102 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:48.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:48.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:48.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:48.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:48.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:48.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:48.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:48.460 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-23T18:51:48.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-23T18:51:48.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:48.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:48.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:48.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:48.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:48.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:48.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:48.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:48.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-23T18:51:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-23T18:51:48.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:48.803 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:48.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:48.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:48.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:48.882 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:48.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:48.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:48.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:49.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:49.058 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-23T18:51:49.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-23T18:51:49.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:49.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T18:51:49.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:49.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:49.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:49.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:49.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:49.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:49.319 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:49.329 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:49.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:49.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:49.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-23T18:51:49.453 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:49.462 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:49.469 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:49.476 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:49.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:49.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:49.506 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:49.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:49.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:49.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:49.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:49.703 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:49.711 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:49.730 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:49.823 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:49.894 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:49.908 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:49.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:49.932 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:49.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:49.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:49.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:49.975 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:50.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:50.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:50.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:50.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:50.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:50.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:50.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:50.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:50.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:50.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:50.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:50.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:50.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:50.291 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:50.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:50.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:50.504 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:50.541 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:50.584 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:50.593 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:50.603 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:50.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:50.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:50.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:50.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:50.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:50.703 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:50.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:50.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:50.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:50.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:50.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:50.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:50.832 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:50.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:50.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:50.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:50.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:50.967 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:50.980 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:50.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:51.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:51.064 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-23T18:51:51.075 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:51.111 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:51.154 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:51.161 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:51.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:51.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:51.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:51.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:51.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:51.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:51.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:51.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:51.336 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:51.345 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:51.354 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:51.361 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:51.369 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:51.421 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:51.459 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:51.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:51.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:51.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:51.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:51.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:51.587 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:51.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:51.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:51.636 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:51.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:51.654 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:51.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:51.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:51.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:51.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:51.874 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:51.925 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:51.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:51.945 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:51.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:51.966 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:51.975 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:51.985 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:51.995 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:52.003 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:52.058 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:52.115 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:52.147 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:52.159 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:52.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:52.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:52.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:52.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:52.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:52.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:52.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:52.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:52.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:52.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:52.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:52.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:52.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:52.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:52.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:52.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-23T18:51:52.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-23T18:51:52.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:52.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:52.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:52.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-23T18:51:52.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:52.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:52.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:52.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T18:51:52.727 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:52.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:52.795 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:52.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:52.872 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-23T18:51:52.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-23T18:51:52.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:52.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:52.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:52.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:52.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:52.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:53.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:53.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-23T18:51:53.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:53.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:53.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:53.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:53.053 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-23T18:51:53.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-23T18:51:53.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:53.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:53.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-23T18:51:53.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:53.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:53.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:53.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T18:51:53.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:53.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:53.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:53.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:53.318 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-23T18:51:53.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T18:51:53.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:53.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:53.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:53.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-23T18:51:53.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:53.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:53.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:53.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T18:51:53.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:53.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:53.626 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:53.633 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:53.643 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:53.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:53.663 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:53.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-23T18:51:53.681 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-23T18:51:53.690 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-23T18:51:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-23T18:51:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-23T18:51:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-23T18:51:53.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-23T18:51:53.851 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:53.863 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-23T18:51:53.882 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:53.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:54.008 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:54.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:54.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:54.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:54.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:54.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:54.066 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-23T18:51:54.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-23T18:51:54.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:54.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:54.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:54.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:54.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:54.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:54.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:54.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:54.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:54.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:54.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:54.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:54.389 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-23T18:51:54.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-23T18:51:54.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-23T18:51:54.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:54.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:54.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-23T18:51:54.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:54.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:54.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:54.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:54.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:54.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:54.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:54.754 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-23T18:51:54.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-23T18:51:54.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:54.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:54.797 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:54.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:54.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:54.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:54.900 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:54.995 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:55.029 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:55.039 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:55.049 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:55.058 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:55.071 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:55.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:55.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:55.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:55.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:55.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:55.251 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-23T18:51:55.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-23T18:51:55.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:55.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:55.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:55.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:55.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:55.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:55.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:55.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:55.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:55.571 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-23T18:51:55.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-23T18:51:55.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:55.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:55.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:55.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-23T18:51:55.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:55.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:55.706 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-23T18:51:55.752 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-23T18:51:55.784 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-23T18:51:55.796 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-23T18:51:55.806 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-23T18:51:55.815 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-23T18:51:55.826 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-23T18:51:55.865 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:55.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:55.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:55.982 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:55.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:56.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:56.011 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-23T18:51:56.019 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-23T18:51:56.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-23T18:51:56.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:56.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-23T18:51:56.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-23T18:51:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:56.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:56.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:56.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:56.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:56.366 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-23T18:51:56.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-23T18:51:56.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:56.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:56.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:56.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-23T18:51:56.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:56.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:56.526 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:56.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:56.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:56.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:56.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:56.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:56.622 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-23T18:51:56.676 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-23T18:51:56.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-23T18:51:56.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:56.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:56.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-23T18:51:56.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:56.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:56.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:56.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:56.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:56.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:56.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:56.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:56.910 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-23T18:51:56.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-23T18:51:56.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-23T18:51:56.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-23T18:51:57.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-23T18:51:57.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T18:51:57.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-23T18:51:57.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-23T18:51:57.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-23T18:51:57.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-23T18:51:57.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-23T18:51:57.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-23T18:51:57.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-23T18:51:57.220 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:57.227 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:57.316 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:57.350 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:57.405 INFO:teuthology.orchestra.run.smithi019.stdout: "services": { 2024-04-23T18:51:57.418 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard": "https://172.21.15.19:8443/" 2024-04-23T18:51:57.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:57.540 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2024-04-23T18:51:57.546 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus": [ 2024-04-23T18:51:57.657 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-23T18:51:57.664 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-23T18:51:57.672 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-23T18:51:57.682 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-23T18:51:57.691 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-23T18:51:57.700 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-23T18:51:57.708 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-23T18:51:57.717 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-23T18:51:57.758 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-23T18:51:57.766 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-23T18:51:57.848 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:57.857 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific": [ 2024-04-23T18:51:57.866 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-23T18:51:57.874 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-23T18:51:57.882 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-23T18:51:57.921 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-23T18:51:57.985 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-23T18:51:57.994 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-23T18:51:58.003 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-23T18:51:58.011 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-23T18:51:58.019 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-23T18:51:58.052 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-23T18:51:58.061 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:58.071 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy": [ 2024-04-23T18:51:58.105 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-23T18:51:58.113 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-23T18:51:58.151 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-23T18:51:58.187 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-23T18:51:58.196 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-23T18:51:58.205 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-23T18:51:58.213 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-23T18:51:58.221 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-23T18:51:58.279 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-23T18:51:58.336 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-23T18:51:58.345 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:58.353 INFO:teuthology.orchestra.run.smithi019.stdout: "reef": [ 2024-04-23T18:51:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-23T18:51:58.369 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-23T18:51:58.378 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-23T18:51:58.386 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-23T18:51:58.395 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-23T18:51:58.450 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-23T18:51:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-23T18:51:58.526 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-23T18:51:58.558 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-23T18:51:58.566 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-23T18:51:58.575 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-23T18:51:58.583 INFO:teuthology.orchestra.run.smithi019.stdout: "squid": [ 2024-04-23T18:51:58.590 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-23T18:51:58.631 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-23T18:51:58.684 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-23T18:51:58.692 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-23T18:51:58.701 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-23T18:51:58.711 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-23T18:51:58.719 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-23T18:51:58.728 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-23T18:51:58.736 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-23T18:51:58.742 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-23T18:51:58.781 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:51:58.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:58.855 INFO:teuthology.orchestra.run.smithi019.stdout: "last_failure_osd_epoch": 89, 2024-04-23T18:51:58.887 INFO:teuthology.orchestra.run.smithi019.stdout: "active_clients": [ 2024-04-23T18:51:58.894 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:58.900 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-23T18:51:58.906 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-23T18:51:58.912 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:58.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-23T18:51:59.002 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-23T18:51:59.009 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3462615729 2024-04-23T18:51:59.016 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:59.023 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:51:59.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:59.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:59.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "libcephsqlite", 2024-04-23T18:51:59.051 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-23T18:51:59.110 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:59.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-23T18:51:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-23T18:51:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 434632310 2024-04-23T18:51:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:51:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:59.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-23T18:51:59.293 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-23T18:51:59.300 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:59.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-23T18:51:59.314 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-23T18:51:59.321 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2368736548 2024-04-23T18:51:59.329 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:51:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:59.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-23T18:51:59.441 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-23T18:51:59.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:59.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-23T18:51:59.487 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-23T18:51:59.494 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1740940199 2024-04-23T18:51:59.501 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:59.533 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:51:59.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:59.591 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:59.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-23T18:51:59.605 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-23T18:51:59.612 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:59.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-23T18:51:59.626 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-23T18:51:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2704545171 2024-04-23T18:51:59.642 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:51:59.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:51:59.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:59.786 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-23T18:51:59.795 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-23T18:51:59.804 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-23T18:51:59.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-23T18:51:59.846 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-23T18:51:59.900 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2639927060 2024-04-23T18:51:59.909 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:59.917 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:51:59.925 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:51:59.934 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-23T18:51:59.943 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:51:59.961 DEBUG:tasks.check_counter:Getting stats from a 2024-04-23T18:51:59.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-23T18:52:00.060 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:05 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4009267241' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T18:52:00.067 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:06 smithi155 ceph-mon[33468]: pgmap v630: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-04-23T18:52:00.118 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:07 smithi155 ceph-mon[33468]: pgmap v631: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 28 op/s 2024-04-23T18:52:00.149 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:09 smithi155 ceph-mon[33468]: pgmap v632: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 28 op/s 2024-04-23T18:52:00.159 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:11 smithi155 ceph-mon[33468]: pgmap v633: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-04-23T18:52:00.168 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:11 smithi155 ceph-mon[33468]: osdmap e119: 12 total, 12 up, 12 in 2024-04-23T18:52:00.176 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:13 smithi155 ceph-mon[33468]: pgmap v635: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-04-23T18:52:00.184 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:15 smithi155 ceph-mon[33468]: pgmap v636: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:00.236 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:17 smithi155 ceph-mon[33468]: pgmap v637: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:00.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:18 smithi155 ceph-mon[33468]: osdmap e120: 12 total, 12 up, 12 in 2024-04-23T18:52:00.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:19 smithi155 ceph-mon[33468]: pgmap v639: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:00.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:21 smithi155 ceph-mon[33468]: pgmap v640: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:00.306 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:23 smithi155 ceph-mon[33468]: pgmap v641: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:00.314 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:25 smithi155 ceph-mon[33468]: pgmap v642: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:00.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:27 smithi155 ceph-mon[33468]: pgmap v643: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:00.330 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:29 smithi155 ceph-mon[33468]: pgmap v644: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:00.339 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:31 smithi155 ceph-mon[33468]: pgmap v645: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:00.401 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:31 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:00.409 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:33 smithi155 ceph-mon[33468]: pgmap v646: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:00.474 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:33 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:00.503 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:33 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:00.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:33 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:00.519 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:33 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:00.529 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:33 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:00.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:33 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:00.586 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.695 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.704 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.712 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.721 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.760 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.766 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.772 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.779 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.814 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.820 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.878 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:00.912 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:00.919 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:00.926 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:00.933 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:35 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-04-23T18:52:00.941 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:35 smithi155 ceph-mon[33468]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:52:00.985 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:35 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-23T18:52:01.050 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:35 smithi155 ceph-mon[33468]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:52:01.057 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:35 smithi155 ceph-mon[33468]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-23T18:52:01.066 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:35 smithi155 ceph-mon[33468]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:52:01.073 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:35 smithi155 ceph-mon[33468]: pgmap v647: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:01.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:37 smithi155 ceph-mon[33468]: pgmap v648: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.086 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:39 smithi155 ceph-mon[33468]: pgmap v649: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.093 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:41 smithi155 ceph-mon[33468]: pgmap v650: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:01.101 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:43 smithi155 ceph-mon[33468]: pgmap v651: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.159 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:45 smithi155 ceph-mon[33468]: pgmap v652: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:01.167 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:47 smithi155 ceph-mon[33468]: pgmap v653: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.208 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:49 smithi155 ceph-mon[33468]: pgmap v654: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.282 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:51 smithi155 ceph-mon[33468]: pgmap v655: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:01.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:53 smithi155 ceph-mon[33468]: pgmap v656: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:55 smithi155 ceph-mon[33468]: pgmap v657: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:01.302 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:57 smithi155 ceph-mon[33468]: pgmap v658: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.312 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:46:59 smithi155 ceph-mon[33468]: pgmap v659: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.351 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:02 smithi155 ceph-mon[33468]: pgmap v660: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:01.409 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:04 smithi155 ceph-mon[33468]: pgmap v661: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.416 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:06 smithi155 ceph-mon[33468]: pgmap v662: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:01.423 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:08 smithi155 ceph-mon[33468]: pgmap v663: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:01.429 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:10 smithi155 ceph-mon[33468]: pgmap v664: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:01.437 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:12 smithi155 ceph-mon[33468]: pgmap v665: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:01.443 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:12 smithi155 ceph-mon[33468]: osdmap e121: 12 total, 12 up, 12 in 2024-04-23T18:52:01.450 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:13 smithi155 ceph-mon[33468]: pgmap v667: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:01.456 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:15 smithi155 ceph-mon[33468]: pgmap v668: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:01.524 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:17 smithi155 ceph-mon[33468]: pgmap v669: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:01.530 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:18 smithi155 ceph-mon[33468]: osdmap e122: 12 total, 12 up, 12 in 2024-04-23T18:52:01.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:19 smithi155 ceph-mon[33468]: pgmap v671: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:01.623 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:21 smithi155 ceph-mon[33468]: pgmap v672: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 216 B/s wr, 10 op/s 2024-04-23T18:52:01.630 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:23 smithi155 ceph-mon[33468]: pgmap v673: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:01.637 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:25 smithi155 ceph-mon[33468]: pgmap v674: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:01.645 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:27 smithi155 ceph-mon[33468]: pgmap v675: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:01.651 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:29 smithi155 ceph-mon[33468]: pgmap v676: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:01.697 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:31 smithi155 ceph-mon[33468]: pgmap v677: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:01.748 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:33 smithi155 ceph-mon[33468]: pgmap v678: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.757 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:34 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:01.764 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:36 smithi155 ceph-mon[33468]: pgmap v679: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:01.772 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:01.781 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:01.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:01.797 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:01.806 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:01.865 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:01.874 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:01.933 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:01.976 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:01.982 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:38 smithi155 ceph-mon[33468]: pgmap v680: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:40 smithi155 ceph-mon[33468]: pgmap v681: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:01.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:42 smithi155 ceph-mon[33468]: pgmap v682: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:02.007 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:44 smithi155 ceph-mon[33468]: pgmap v683: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:02.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:45 smithi155 ceph-mon[33468]: pgmap v684: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:02.125 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:47 smithi155 ceph-mon[33468]: pgmap v685: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:02.134 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:49 smithi155 ceph-mon[33468]: pgmap v686: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:02.143 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:51 smithi155 ceph-mon[33468]: pgmap v687: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:02.149 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:53 smithi155 ceph-mon[33468]: pgmap v688: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:02.155 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:55 smithi155 ceph-mon[33468]: pgmap v689: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:02.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:57 smithi155 ceph-mon[33468]: pgmap v690: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:02.167 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:47:59 smithi155 ceph-mon[33468]: pgmap v691: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:02.173 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:01 smithi155 ceph-mon[33468]: pgmap v692: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:02.234 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:03 smithi155 ceph-mon[33468]: pgmap v693: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:02.243 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:05 smithi155 ceph-mon[33468]: pgmap v694: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:02.302 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:07 smithi155 ceph-mon[33468]: pgmap v695: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:02.345 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:09 smithi155 ceph-mon[33468]: pgmap v696: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:02.350 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:11 smithi155 ceph-mon[33468]: pgmap v697: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:02.357 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:11 smithi155 ceph-mon[33468]: osdmap e123: 12 total, 12 up, 12 in 2024-04-23T18:52:02.365 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:13 smithi155 ceph-mon[33468]: pgmap v699: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:02.372 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:15 smithi155 ceph-mon[33468]: pgmap v700: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:02.435 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:17 smithi155 ceph-mon[33468]: pgmap v701: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:02.527 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:18 smithi155 ceph-mon[33468]: osdmap e124: 12 total, 12 up, 12 in 2024-04-23T18:52:02.700 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:19 smithi155 ceph-mon[33468]: pgmap v703: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:02.730 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:21 smithi155 ceph-mon[33468]: pgmap v704: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:02.744 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:23 smithi155 ceph-mon[33468]: pgmap v705: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:02.757 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:25 smithi155 ceph-mon[33468]: pgmap v706: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:02.771 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:27 smithi155 ceph-mon[33468]: pgmap v707: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:02.782 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:29 smithi155 ceph-mon[33468]: pgmap v708: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:02.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:31 smithi155 ceph-mon[33468]: pgmap v709: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:02.912 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:33 smithi155 ceph-mon[33468]: pgmap v710: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:02.921 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:35 smithi155 ceph-mon[33468]: pgmap v711: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:02.930 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:35 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:02.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:02.988 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:02.996 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:36 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:03.003 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:37 smithi155 ceph-mon[33468]: pgmap v712: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.011 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:39 smithi155 ceph-mon[33468]: pgmap v713: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:42 smithi155 ceph-mon[33468]: pgmap v714: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:03.084 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:44 smithi155 ceph-mon[33468]: pgmap v715: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.159 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:46 smithi155 ceph-mon[33468]: pgmap v716: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:03.168 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:48 smithi155 ceph-mon[33468]: pgmap v717: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.178 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:50 smithi155 ceph-mon[33468]: pgmap v718: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.187 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:52 smithi155 ceph-mon[33468]: pgmap v719: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:03.197 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:54 smithi155 ceph-mon[33468]: pgmap v720: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.205 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:56 smithi155 ceph-mon[33468]: pgmap v721: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:03.212 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:48:58 smithi155 ceph-mon[33468]: pgmap v722: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:03.267 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:00 smithi155 ceph-mon[33468]: pgmap v723: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:03.306 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:02 smithi155 ceph-mon[33468]: pgmap v724: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:03.311 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:04 smithi155 ceph-mon[33468]: pgmap v725: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:03.361 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:06 smithi155 ceph-mon[33468]: pgmap v726: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:03.367 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:08 smithi155 ceph-mon[33468]: pgmap v727: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:03.376 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:09 smithi155 ceph-mon[33468]: pgmap v728: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:03.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:11 smithi155 ceph-mon[33468]: osdmap e125: 12 total, 12 up, 12 in 2024-04-23T18:52:03.395 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:11 smithi155 ceph-mon[33468]: pgmap v730: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:03.458 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:13 smithi155 ceph-mon[33468]: pgmap v731: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:03.520 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:15 smithi155 ceph-mon[33468]: pgmap v732: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:03.529 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:17 smithi155 ceph-mon[33468]: pgmap v733: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:03.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:18 smithi155 ceph-mon[33468]: osdmap e126: 12 total, 12 up, 12 in 2024-04-23T18:52:03.547 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:19 smithi155 ceph-mon[33468]: pgmap v735: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 125 B/s wr, 6 op/s 2024-04-23T18:52:03.555 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:21 smithi155 ceph-mon[33468]: pgmap v736: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:03.564 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:23 smithi155 ceph-mon[33468]: pgmap v737: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:03.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:25 smithi155 ceph-mon[33468]: pgmap v738: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:03.581 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:27 smithi155 ceph-mon[33468]: pgmap v739: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:03.640 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:29 smithi155 ceph-mon[33468]: pgmap v740: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:03.646 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:31 smithi155 ceph-mon[33468]: pgmap v741: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:03.655 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:34 smithi155 ceph-mon[33468]: pgmap v742: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.664 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:36 smithi155 ceph-mon[33468]: pgmap v743: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:03.672 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:37 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:03.771 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:38 smithi155 ceph-mon[33468]: pgmap v744: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.777 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:38 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:03.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:38 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:03.836 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:38 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:03.875 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:40 smithi155 ceph-mon[33468]: pgmap v745: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.883 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:42 smithi155 ceph-mon[33468]: pgmap v746: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:03.891 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:44 smithi155 ceph-mon[33468]: pgmap v747: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.898 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:46 smithi155 ceph-mon[33468]: pgmap v748: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:03.904 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:48 smithi155 ceph-mon[33468]: pgmap v749: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.911 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:50 smithi155 ceph-mon[33468]: pgmap v750: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.918 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:52 smithi155 ceph-mon[33468]: pgmap v751: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:03.925 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:54 smithi155 ceph-mon[33468]: pgmap v752: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:03.970 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:56 smithi155 ceph-mon[33468]: pgmap v753: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:04.002 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:49:58 smithi155 ceph-mon[33468]: pgmap v754: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:04.011 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:00 smithi155 ceph-mon[33468]: pgmap v755: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:04.070 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:00 smithi155 ceph-mon[33468]: overall HEALTH_OK 2024-04-23T18:52:04.078 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:01 smithi155 ceph-mon[33468]: pgmap v756: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:04.086 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:03 smithi155 ceph-mon[33468]: pgmap v757: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:04.093 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:05 smithi155 ceph-mon[33468]: pgmap v758: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:04.102 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:07 smithi155 ceph-mon[33468]: pgmap v759: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:04.135 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:09 smithi155 ceph-mon[33468]: pgmap v760: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:04.204 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:11 smithi155 ceph-mon[33468]: pgmap v761: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:04.211 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:11 smithi155 ceph-mon[33468]: osdmap e127: 12 total, 12 up, 12 in 2024-04-23T18:52:04.218 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:13 smithi155 ceph-mon[33468]: pgmap v763: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:04.226 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:15 smithi155 ceph-mon[33468]: pgmap v764: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:04.233 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:17 smithi155 ceph-mon[33468]: pgmap v765: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:04.241 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:18 smithi155 ceph-mon[33468]: osdmap e128: 12 total, 12 up, 12 in 2024-04-23T18:52:04.249 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:19 smithi155 ceph-mon[33468]: pgmap v767: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:04.256 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:21 smithi155 ceph-mon[33468]: pgmap v768: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:04.316 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:23 smithi155 ceph-mon[33468]: pgmap v769: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:04.322 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:26 smithi155 ceph-mon[33468]: pgmap v770: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:04.329 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:28 smithi155 ceph-mon[33468]: pgmap v771: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:04.375 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:30 smithi155 ceph-mon[33468]: pgmap v772: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 96 B/s wr, 9 op/s 2024-04-23T18:52:04.382 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:32 smithi155 ceph-mon[33468]: pgmap v773: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:04.389 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:34 smithi155 ceph-mon[33468]: pgmap v774: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:04.395 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:36 smithi155 ceph-mon[33468]: pgmap v775: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:04.403 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:38 smithi155 ceph-mon[33468]: pgmap v776: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:04.456 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:38 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:04.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:39 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:04.518 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:39 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:04.526 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:39 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:04.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:40 smithi155 ceph-mon[33468]: pgmap v777: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:04.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:42 smithi155 ceph-mon[33468]: pgmap v778: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:04.547 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:44 smithi155 ceph-mon[33468]: pgmap v779: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:04.554 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:46 smithi155 ceph-mon[33468]: pgmap v780: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:04.562 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:48 smithi155 ceph-mon[33468]: pgmap v781: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:04.615 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:50 smithi155 ceph-mon[33468]: pgmap v782: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:04.624 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:52 smithi155 ceph-mon[33468]: pgmap v783: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:04.631 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:54 smithi155 ceph-mon[33468]: pgmap v784: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:04.673 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:56 smithi155 ceph-mon[33468]: pgmap v785: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:04.680 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:50:58 smithi155 ceph-mon[33468]: pgmap v786: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:04.686 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:00 smithi155 ceph-mon[33468]: pgmap v787: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:04.694 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:02 smithi155 ceph-mon[33468]: pgmap v788: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:04.703 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:04 smithi155 ceph-mon[33468]: pgmap v789: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:04.732 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:06 smithi155 ceph-mon[33468]: pgmap v790: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:04.794 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:07 smithi155 ceph-mon[33468]: pgmap v791: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:04.802 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:09 smithi155 ceph-mon[33468]: pgmap v792: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:04.809 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:11 smithi155 ceph-mon[33468]: pgmap v793: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:04.818 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:11 smithi155 ceph-mon[33468]: osdmap e129: 12 total, 12 up, 12 in 2024-04-23T18:52:04.827 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:13 smithi155 ceph-mon[33468]: pgmap v795: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:04.835 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:15 smithi155 ceph-mon[33468]: pgmap v796: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:04.844 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:17 smithi155 ceph-mon[33468]: pgmap v797: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:04.854 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:19 smithi155 ceph-mon[33468]: osdmap e130: 12 total, 12 up, 12 in 2024-04-23T18:52:04.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:20 smithi155 ceph-mon[33468]: pgmap v799: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:04.915 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:22 smithi155 ceph-mon[33468]: pgmap v800: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:04.923 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:24 smithi155 ceph-mon[33468]: pgmap v801: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:04.964 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:26 smithi155 ceph-mon[33468]: pgmap v802: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:04.973 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:28 smithi155 ceph-mon[33468]: pgmap v803: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:04.982 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:30 smithi155 ceph-mon[33468]: pgmap v804: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:04.991 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:32 smithi155 ceph-mon[33468]: pgmap v805: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:05.000 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:34 smithi155 ceph-mon[33468]: pgmap v806: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:05.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:36 smithi155 ceph-mon[33468]: pgmap v807: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:05.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:38 smithi155 ceph-mon[33468]: pgmap v808: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:05.101 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:39 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:05.111 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:39 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:05.119 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:39 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:05.128 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:39 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:05.137 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:40 smithi155 ceph-mon[33468]: pgmap v809: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:05.146 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:42 smithi155 ceph-mon[33468]: pgmap v810: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:05.154 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:44 smithi155 ceph-mon[33468]: pgmap v811: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:05.208 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:46 smithi155 ceph-mon[33468]: pgmap v812: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:05.217 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:48 smithi155 ceph-mon[33468]: pgmap v813: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:05.226 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:50 smithi155 ceph-mon[33468]: pgmap v814: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:05.273 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:52 smithi155 ceph-mon[33468]: pgmap v815: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:05.281 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:54 smithi155 ceph-mon[33468]: pgmap v816: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:05.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:56 smithi155 ceph-mon[33468]: pgmap v817: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:05.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:51:58 smithi155 ceph-mon[33468]: pgmap v818: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:05.307 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:05 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4009267241' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T18:52:05.336 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:06 smithi044 ceph-mon[32458]: pgmap v630: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-04-23T18:52:05.393 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:07 smithi044 ceph-mon[32458]: pgmap v631: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 28 op/s 2024-04-23T18:52:05.400 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:09 smithi044 ceph-mon[32458]: pgmap v632: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 28 op/s 2024-04-23T18:52:05.410 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:11 smithi044 ceph-mon[32458]: pgmap v633: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-04-23T18:52:05.419 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:11 smithi044 ceph-mon[32458]: osdmap e119: 12 total, 12 up, 12 in 2024-04-23T18:52:05.428 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:13 smithi044 ceph-mon[32458]: pgmap v635: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-04-23T18:52:05.442 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:15 smithi044 ceph-mon[32458]: pgmap v636: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:05.450 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:17 smithi044 ceph-mon[32458]: pgmap v637: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:05.458 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:18 smithi044 ceph-mon[32458]: osdmap e120: 12 total, 12 up, 12 in 2024-04-23T18:52:05.502 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:19 smithi044 ceph-mon[32458]: pgmap v639: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:05.510 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:21 smithi044 ceph-mon[32458]: pgmap v640: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:05.519 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:23 smithi044 ceph-mon[32458]: pgmap v641: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:05.573 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:25 smithi044 ceph-mon[32458]: pgmap v642: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:05.582 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:27 smithi044 ceph-mon[32458]: pgmap v643: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:05.591 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:29 smithi044 ceph-mon[32458]: pgmap v644: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:05.599 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:31 smithi044 ceph-mon[32458]: pgmap v645: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:05.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:31 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:05.637 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:33 smithi044 ceph-mon[32458]: pgmap v646: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:05.682 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:33 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:05.691 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:33 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:05.700 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:33 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:05.709 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:33 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:05.716 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:33 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:05.722 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:33 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:05.730 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.737 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.799 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.806 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.847 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.855 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.861 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.868 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.875 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.904 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.966 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:05.982 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:05.990 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:05.997 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:06.004 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:35 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-04-23T18:52:06.011 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:35 smithi044 ceph-mon[32458]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:52:06.018 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:35 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-23T18:52:06.024 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:35 smithi044 ceph-mon[32458]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:52:06.092 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:35 smithi044 ceph-mon[32458]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-23T18:52:06.099 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:35 smithi044 ceph-mon[32458]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:52:06.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:35 smithi044 ceph-mon[32458]: pgmap v647: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:06.156 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:37 smithi044 ceph-mon[32458]: pgmap v648: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.163 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:39 smithi044 ceph-mon[32458]: pgmap v649: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.170 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:41 smithi044 ceph-mon[32458]: pgmap v650: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:06.177 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:43 smithi044 ceph-mon[32458]: pgmap v651: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.183 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:45 smithi044 ceph-mon[32458]: pgmap v652: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:06.221 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:47 smithi044 ceph-mon[32458]: pgmap v653: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.264 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:49 smithi044 ceph-mon[32458]: pgmap v654: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.271 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:51 smithi044 ceph-mon[32458]: pgmap v655: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:06.277 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:53 smithi044 ceph-mon[32458]: pgmap v656: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.283 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:55 smithi044 ceph-mon[32458]: pgmap v657: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:06.289 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:57 smithi044 ceph-mon[32458]: pgmap v658: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.296 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:46:59 smithi044 ceph-mon[32458]: pgmap v659: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.302 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:02 smithi044 ceph-mon[32458]: pgmap v660: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:06.310 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:04 smithi044 ceph-mon[32458]: pgmap v661: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.369 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:06 smithi044 ceph-mon[32458]: pgmap v662: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:06.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:08 smithi044 ceph-mon[32458]: pgmap v663: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:06.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:10 smithi044 ceph-mon[32458]: pgmap v664: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:06.431 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:12 smithi044 ceph-mon[32458]: pgmap v665: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:06.437 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:12 smithi044 ceph-mon[32458]: osdmap e121: 12 total, 12 up, 12 in 2024-04-23T18:52:06.444 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:13 smithi044 ceph-mon[32458]: pgmap v667: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:06.451 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:15 smithi044 ceph-mon[32458]: pgmap v668: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:06.458 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:17 smithi044 ceph-mon[32458]: pgmap v669: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:06.485 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:18 smithi044 ceph-mon[32458]: osdmap e122: 12 total, 12 up, 12 in 2024-04-23T18:52:06.548 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:19 smithi044 ceph-mon[32458]: pgmap v671: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:06.554 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:21 smithi044 ceph-mon[32458]: pgmap v672: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 216 B/s wr, 10 op/s 2024-04-23T18:52:06.561 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:23 smithi044 ceph-mon[32458]: pgmap v673: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:06.568 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:25 smithi044 ceph-mon[32458]: pgmap v674: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:06.574 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:27 smithi044 ceph-mon[32458]: pgmap v675: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:06.582 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:29 smithi044 ceph-mon[32458]: pgmap v676: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:06.588 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:31 smithi044 ceph-mon[32458]: pgmap v677: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:06.596 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:33 smithi044 ceph-mon[32458]: pgmap v678: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.652 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:34 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:06.659 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:36 smithi044 ceph-mon[32458]: pgmap v679: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:06.667 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:06.714 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:06.726 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:06.734 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:06.742 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:06.750 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:06.776 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:06.844 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:06.853 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:06.861 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:38 smithi044 ceph-mon[32458]: pgmap v680: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.869 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:40 smithi044 ceph-mon[32458]: pgmap v681: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.878 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:42 smithi044 ceph-mon[32458]: pgmap v682: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:06.887 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:44 smithi044 ceph-mon[32458]: pgmap v683: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.895 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:45 smithi044 ceph-mon[32458]: pgmap v684: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:06.904 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:47 smithi044 ceph-mon[32458]: pgmap v685: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.960 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:49 smithi044 ceph-mon[32458]: pgmap v686: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:06.972 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:51 smithi044 ceph-mon[32458]: pgmap v687: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:06.981 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:53 smithi044 ceph-mon[32458]: pgmap v688: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:07.030 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:55 smithi044 ceph-mon[32458]: pgmap v689: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:07.039 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:57 smithi044 ceph-mon[32458]: pgmap v690: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:07.047 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:47:59 smithi044 ceph-mon[32458]: pgmap v691: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:07.055 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:01 smithi044 ceph-mon[32458]: pgmap v692: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:07.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:03 smithi044 ceph-mon[32458]: pgmap v693: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:07.110 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:05 smithi044 ceph-mon[32458]: pgmap v694: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:07.165 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:07 smithi044 ceph-mon[32458]: pgmap v695: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:07.171 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:09 smithi044 ceph-mon[32458]: pgmap v696: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:07.179 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:11 smithi044 ceph-mon[32458]: pgmap v697: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:07.187 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:11 smithi044 ceph-mon[32458]: osdmap e123: 12 total, 12 up, 12 in 2024-04-23T18:52:07.195 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:13 smithi044 ceph-mon[32458]: pgmap v699: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:07.203 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:15 smithi044 ceph-mon[32458]: pgmap v700: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:07.211 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:17 smithi044 ceph-mon[32458]: pgmap v701: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:07.219 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:18 smithi044 ceph-mon[32458]: osdmap e124: 12 total, 12 up, 12 in 2024-04-23T18:52:07.260 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:19 smithi044 ceph-mon[32458]: pgmap v703: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:07.292 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:21 smithi044 ceph-mon[32458]: pgmap v704: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:07.300 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:23 smithi044 ceph-mon[32458]: pgmap v705: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:07.339 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:25 smithi044 ceph-mon[32458]: pgmap v706: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:07.347 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:27 smithi044 ceph-mon[32458]: pgmap v707: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:07.354 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:29 smithi044 ceph-mon[32458]: pgmap v708: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:07.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:31 smithi044 ceph-mon[32458]: pgmap v709: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:07.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:33 smithi044 ceph-mon[32458]: pgmap v710: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:07.418 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:35 smithi044 ceph-mon[32458]: pgmap v711: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:07.522 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:35 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:07.535 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:07.543 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:07.551 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:36 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:07.559 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:37 smithi044 ceph-mon[32458]: pgmap v712: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:07.567 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:39 smithi044 ceph-mon[32458]: pgmap v713: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:07.574 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:42 smithi044 ceph-mon[32458]: pgmap v714: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:07.581 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:44 smithi044 ceph-mon[32458]: pgmap v715: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:07.660 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:46 smithi044 ceph-mon[32458]: pgmap v716: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:07.877 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:48 smithi044 ceph-mon[32458]: pgmap v717: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:08.136 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:50 smithi044 ceph-mon[32458]: pgmap v718: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:08.161 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:52 smithi044 ceph-mon[32458]: pgmap v719: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:08.224 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:54 smithi044 ceph-mon[32458]: pgmap v720: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:08.231 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:56 smithi044 ceph-mon[32458]: pgmap v721: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:08.237 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:48:58 smithi044 ceph-mon[32458]: pgmap v722: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:08.244 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:00 smithi044 ceph-mon[32458]: pgmap v723: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:08.284 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:02 smithi044 ceph-mon[32458]: pgmap v724: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:08.291 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:04 smithi044 ceph-mon[32458]: pgmap v725: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:08.377 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:06 smithi044 ceph-mon[32458]: pgmap v726: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:08.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:08 smithi044 ceph-mon[32458]: pgmap v727: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:08.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:09 smithi044 ceph-mon[32458]: pgmap v728: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:08.398 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:11 smithi044 ceph-mon[32458]: osdmap e125: 12 total, 12 up, 12 in 2024-04-23T18:52:08.405 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:11 smithi044 ceph-mon[32458]: pgmap v730: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:08.412 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:13 smithi044 ceph-mon[32458]: pgmap v731: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:08.418 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:15 smithi044 ceph-mon[32458]: pgmap v732: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:08.516 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:17 smithi044 ceph-mon[32458]: pgmap v733: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:08.523 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:18 smithi044 ceph-mon[32458]: osdmap e126: 12 total, 12 up, 12 in 2024-04-23T18:52:08.530 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:19 smithi044 ceph-mon[32458]: pgmap v735: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 125 B/s wr, 6 op/s 2024-04-23T18:52:08.580 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:21 smithi044 ceph-mon[32458]: pgmap v736: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:08.587 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:23 smithi044 ceph-mon[32458]: pgmap v737: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:08.596 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:25 smithi044 ceph-mon[32458]: pgmap v738: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:08.604 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:27 smithi044 ceph-mon[32458]: pgmap v739: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:08.613 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:29 smithi044 ceph-mon[32458]: pgmap v740: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:08.651 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:31 smithi044 ceph-mon[32458]: pgmap v741: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:08.739 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:34 smithi044 ceph-mon[32458]: pgmap v742: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:08.748 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:36 smithi044 ceph-mon[32458]: pgmap v743: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:08.764 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:37 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:08.774 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:38 smithi044 ceph-mon[32458]: pgmap v744: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:08.782 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:38 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:08.793 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:38 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:08.804 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:38 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:08.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:40 smithi044 ceph-mon[32458]: pgmap v745: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:08.900 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:42 smithi044 ceph-mon[32458]: pgmap v746: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:08.909 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:44 smithi044 ceph-mon[32458]: pgmap v747: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:08.922 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:46 smithi044 ceph-mon[32458]: pgmap v748: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:08.963 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:48 smithi044 ceph-mon[32458]: pgmap v749: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:08.973 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:50 smithi044 ceph-mon[32458]: pgmap v750: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:08.982 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:52 smithi044 ceph-mon[32458]: pgmap v751: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:08.992 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:54 smithi044 ceph-mon[32458]: pgmap v752: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.001 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:56 smithi044 ceph-mon[32458]: pgmap v753: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:09.043 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:49:58 smithi044 ceph-mon[32458]: pgmap v754: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.052 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:00 smithi044 ceph-mon[32458]: pgmap v755: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.126 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:00 smithi044 ceph-mon[32458]: overall HEALTH_OK 2024-04-23T18:52:09.135 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:01 smithi044 ceph-mon[32458]: pgmap v756: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:09.146 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:03 smithi044 ceph-mon[32458]: pgmap v757: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.153 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:05 smithi044 ceph-mon[32458]: pgmap v758: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:09.161 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:07 smithi044 ceph-mon[32458]: pgmap v759: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:09.170 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:09 smithi044 ceph-mon[32458]: pgmap v760: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:09.177 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:11 smithi044 ceph-mon[32458]: pgmap v761: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:09.218 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:11 smithi044 ceph-mon[32458]: osdmap e127: 12 total, 12 up, 12 in 2024-04-23T18:52:09.238 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:13 smithi044 ceph-mon[32458]: pgmap v763: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:09.249 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:15 smithi044 ceph-mon[32458]: pgmap v764: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:09.288 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:17 smithi044 ceph-mon[32458]: pgmap v765: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:09.297 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:18 smithi044 ceph-mon[32458]: osdmap e128: 12 total, 12 up, 12 in 2024-04-23T18:52:09.306 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:19 smithi044 ceph-mon[32458]: pgmap v767: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:09.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:21 smithi044 ceph-mon[32458]: pgmap v768: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:09.321 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:23 smithi044 ceph-mon[32458]: pgmap v769: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:09.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:26 smithi044 ceph-mon[32458]: pgmap v770: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:09.424 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:28 smithi044 ceph-mon[32458]: pgmap v771: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:09.432 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:30 smithi044 ceph-mon[32458]: pgmap v772: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 96 B/s wr, 9 op/s 2024-04-23T18:52:09.439 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:32 smithi044 ceph-mon[32458]: pgmap v773: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:09.447 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:34 smithi044 ceph-mon[32458]: pgmap v774: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.456 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:36 smithi044 ceph-mon[32458]: pgmap v775: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:09.464 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:38 smithi044 ceph-mon[32458]: pgmap v776: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.473 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:38 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:09.481 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:39 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:09.557 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:39 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:09.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:39 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:09.569 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:40 smithi044 ceph-mon[32458]: pgmap v777: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.605 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:42 smithi044 ceph-mon[32458]: pgmap v778: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:09.614 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:44 smithi044 ceph-mon[32458]: pgmap v779: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:09.623 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:46 smithi044 ceph-mon[32458]: pgmap v780: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:09.630 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:48 smithi044 ceph-mon[32458]: pgmap v781: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:09.637 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:50 smithi044 ceph-mon[32458]: pgmap v782: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:09.686 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:52 smithi044 ceph-mon[32458]: pgmap v783: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:09.780 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:54 smithi044 ceph-mon[32458]: pgmap v784: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.787 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:56 smithi044 ceph-mon[32458]: pgmap v785: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:09.794 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:50:58 smithi044 ceph-mon[32458]: pgmap v786: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.801 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:00 smithi044 ceph-mon[32458]: pgmap v787: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.807 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:02 smithi044 ceph-mon[32458]: pgmap v788: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:09.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:04 smithi044 ceph-mon[32458]: pgmap v789: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:09.821 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:06 smithi044 ceph-mon[32458]: pgmap v790: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:09.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:07 smithi044 ceph-mon[32458]: pgmap v791: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:09.897 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:09 smithi044 ceph-mon[32458]: pgmap v792: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:09.904 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:11 smithi044 ceph-mon[32458]: pgmap v793: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:09.912 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:11 smithi044 ceph-mon[32458]: osdmap e129: 12 total, 12 up, 12 in 2024-04-23T18:52:09.973 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:13 smithi044 ceph-mon[32458]: pgmap v795: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:09.980 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:15 smithi044 ceph-mon[32458]: pgmap v796: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:09.987 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:17 smithi044 ceph-mon[32458]: pgmap v797: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:09.993 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:19 smithi044 ceph-mon[32458]: osdmap e130: 12 total, 12 up, 12 in 2024-04-23T18:52:10.002 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:20 smithi044 ceph-mon[32458]: pgmap v799: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:10.041 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:22 smithi044 ceph-mon[32458]: pgmap v800: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:10.088 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:24 smithi044 ceph-mon[32458]: pgmap v801: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:10.095 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:26 smithi044 ceph-mon[32458]: pgmap v802: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:10.104 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:28 smithi044 ceph-mon[32458]: pgmap v803: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:10.112 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:30 smithi044 ceph-mon[32458]: pgmap v804: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:10.118 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:32 smithi044 ceph-mon[32458]: pgmap v805: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:10.125 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:34 smithi044 ceph-mon[32458]: pgmap v806: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.131 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:36 smithi044 ceph-mon[32458]: pgmap v807: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:10.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:38 smithi044 ceph-mon[32458]: pgmap v808: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.195 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:39 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:10.203 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:39 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:10.210 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:39 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:10.263 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:39 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:10.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:40 smithi044 ceph-mon[32458]: pgmap v809: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.279 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:42 smithi044 ceph-mon[32458]: pgmap v810: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:10.287 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:44 smithi044 ceph-mon[32458]: pgmap v811: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.294 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:46 smithi044 ceph-mon[32458]: pgmap v812: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:10.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:48 smithi044 ceph-mon[32458]: pgmap v813: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:50 smithi044 ceph-mon[32458]: pgmap v814: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:52 smithi044 ceph-mon[32458]: pgmap v815: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:10.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:54 smithi044 ceph-mon[32458]: pgmap v816: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.394 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:56 smithi044 ceph-mon[32458]: pgmap v817: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:10.409 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:51:58 smithi044 ceph-mon[32458]: pgmap v818: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.418 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:00 smithi044 ceph-mon[32458]: pgmap v819: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.425 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:01 smithi044 ceph-mon[32458]: pgmap v820: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:10.432 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:03 smithi044 ceph-mon[32458]: pgmap v821: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.497 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:05 smithi044 ceph-mon[32458]: pgmap v822: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:10.508 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:06 smithi044 ceph-mon[32458]: osdmap e131: 12 total, 12 up, 12 in 2024-04-23T18:52:10.517 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:07 smithi044 ceph-mon[32458]: pgmap v824: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:10.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:09 smithi044 ceph-mon[32458]: pgmap v825: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:10.573 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:00 smithi155 ceph-mon[33468]: pgmap v819: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.582 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:01 smithi155 ceph-mon[33468]: pgmap v820: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:10.590 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:03 smithi155 ceph-mon[33468]: pgmap v821: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:10.599 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:05 smithi155 ceph-mon[33468]: pgmap v822: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:10.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:06 smithi155 ceph-mon[33468]: osdmap e131: 12 total, 12 up, 12 in 2024-04-23T18:52:10.738 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:07 smithi155 ceph-mon[33468]: pgmap v824: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:10.745 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:09 smithi155 ceph-mon[33468]: pgmap v825: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:10.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:05 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4009267241' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T18:52:10.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:06 smithi019 ceph-mon[29215]: pgmap v630: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-04-23T18:52:10.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:07 smithi019 ceph-mon[29215]: pgmap v631: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 28 op/s 2024-04-23T18:52:10.781 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:09 smithi019 ceph-mon[29215]: pgmap v632: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 28 op/s 2024-04-23T18:52:10.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:11 smithi019 ceph-mon[29215]: pgmap v633: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-04-23T18:52:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:11 smithi019 ceph-mon[29215]: osdmap e119: 12 total, 12 up, 12 in 2024-04-23T18:52:10.835 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:13 smithi019 ceph-mon[29215]: pgmap v635: 97 pgs: 97 active+clean; 4.4 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-04-23T18:52:10.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:15 smithi019 ceph-mon[29215]: pgmap v636: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:10.866 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:17 smithi019 ceph-mon[29215]: pgmap v637: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:10.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:18 smithi019 ceph-mon[29215]: osdmap e120: 12 total, 12 up, 12 in 2024-04-23T18:52:10.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:19 smithi019 ceph-mon[29215]: pgmap v639: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:10.923 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:21 smithi019 ceph-mon[29215]: pgmap v640: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:10.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:23 smithi019 ceph-mon[29215]: pgmap v641: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:10.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:25 smithi019 ceph-mon[29215]: pgmap v642: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:10.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:27 smithi019 ceph-mon[29215]: pgmap v643: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:11.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:29 smithi019 ceph-mon[29215]: pgmap v644: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:11.057 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:31 smithi019 ceph-mon[29215]: pgmap v645: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:11.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:31 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:11.076 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:33 smithi019 ceph-mon[29215]: pgmap v646: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:11.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:33 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:11.095 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:33 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:11.103 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:33 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:11.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:33 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:11.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:33 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:11.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:33 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:11.200 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.265 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.275 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.390 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:52:11.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:11.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:11.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:11.526 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:35 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-04-23T18:52:11.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:35 smithi019 ceph-mon[29215]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:52:11.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:35 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-23T18:52:11.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:35 smithi019 ceph-mon[29215]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:52:11.593 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:35 smithi019 ceph-mon[29215]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-23T18:52:11.601 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:35 smithi019 ceph-mon[29215]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-23T18:52:11.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:35 smithi019 ceph-mon[29215]: pgmap v647: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:11.618 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:37 smithi019 ceph-mon[29215]: pgmap v648: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:11.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:39 smithi019 ceph-mon[29215]: pgmap v649: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:11.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:41 smithi019 ceph-mon[29215]: pgmap v650: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:11.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:43 smithi019 ceph-mon[29215]: pgmap v651: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:11.744 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:45 smithi019 ceph-mon[29215]: pgmap v652: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:11.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:47 smithi019 ceph-mon[29215]: pgmap v653: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:11.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:49 smithi019 ceph-mon[29215]: pgmap v654: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:11.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:51 smithi019 ceph-mon[29215]: pgmap v655: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:11.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:53 smithi019 ceph-mon[29215]: pgmap v656: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:11.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:55 smithi019 ceph-mon[29215]: pgmap v657: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:11.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:57 smithi019 ceph-mon[29215]: pgmap v658: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:11.848 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:46:59 smithi019 ceph-mon[29215]: pgmap v659: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:11.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:02 smithi019 ceph-mon[29215]: pgmap v660: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:11.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:04 smithi019 ceph-mon[29215]: pgmap v661: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:11.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:06 smithi019 ceph-mon[29215]: pgmap v662: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:11.910 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:08 smithi019 ceph-mon[29215]: pgmap v663: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:11.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:10 smithi019 ceph-mon[29215]: pgmap v664: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:11.925 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:12 smithi019 ceph-mon[29215]: pgmap v665: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:11.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:12 smithi019 ceph-mon[29215]: osdmap e121: 12 total, 12 up, 12 in 2024-04-23T18:52:12.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:13 smithi019 ceph-mon[29215]: pgmap v667: 97 pgs: 97 active+clean; 4.4 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:12.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:15 smithi019 ceph-mon[29215]: pgmap v668: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:12.088 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:17 smithi019 ceph-mon[29215]: pgmap v669: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:12.095 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:18 smithi019 ceph-mon[29215]: osdmap e122: 12 total, 12 up, 12 in 2024-04-23T18:52:12.102 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:19 smithi019 ceph-mon[29215]: pgmap v671: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:12.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:21 smithi019 ceph-mon[29215]: pgmap v672: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 216 B/s wr, 10 op/s 2024-04-23T18:52:12.115 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:23 smithi019 ceph-mon[29215]: pgmap v673: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:12.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:25 smithi019 ceph-mon[29215]: pgmap v674: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:12.177 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:27 smithi019 ceph-mon[29215]: pgmap v675: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:12.185 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:29 smithi019 ceph-mon[29215]: pgmap v676: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:12.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:31 smithi019 ceph-mon[29215]: pgmap v677: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:12.232 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:33 smithi019 ceph-mon[29215]: pgmap v678: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:34 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:12.245 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:36 smithi019 ceph-mon[29215]: pgmap v679: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:12.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:12.258 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:12.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:12.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:12.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:12.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:12.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:12.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:12.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:12.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:38 smithi019 ceph-mon[29215]: pgmap v680: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:40 smithi019 ceph-mon[29215]: pgmap v681: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.467 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:42 smithi019 ceph-mon[29215]: pgmap v682: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:12.473 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:44 smithi019 ceph-mon[29215]: pgmap v683: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.480 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:45 smithi019 ceph-mon[29215]: pgmap v684: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:12.522 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:47 smithi019 ceph-mon[29215]: pgmap v685: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:49 smithi019 ceph-mon[29215]: pgmap v686: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.540 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:51 smithi019 ceph-mon[29215]: pgmap v687: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:12.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:53 smithi019 ceph-mon[29215]: pgmap v688: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.557 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:55 smithi019 ceph-mon[29215]: pgmap v689: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:12.601 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:57 smithi019 ceph-mon[29215]: pgmap v690: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:47:59 smithi019 ceph-mon[29215]: pgmap v691: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:01 smithi019 ceph-mon[29215]: pgmap v692: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:12.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:03 smithi019 ceph-mon[29215]: pgmap v693: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:05 smithi019 ceph-mon[29215]: pgmap v694: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:12.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:07 smithi019 ceph-mon[29215]: pgmap v695: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:12.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:09 smithi019 ceph-mon[29215]: pgmap v696: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:12.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:11 smithi019 ceph-mon[29215]: pgmap v697: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:12.728 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:11 smithi019 ceph-mon[29215]: osdmap e123: 12 total, 12 up, 12 in 2024-04-23T18:52:12.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:13 smithi019 ceph-mon[29215]: pgmap v699: 97 pgs: 97 active+clean; 4.4 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:12.784 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:15 smithi019 ceph-mon[29215]: pgmap v700: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:12.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:17 smithi019 ceph-mon[29215]: pgmap v701: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:12.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:18 smithi019 ceph-mon[29215]: osdmap e124: 12 total, 12 up, 12 in 2024-04-23T18:52:12.841 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:19 smithi019 ceph-mon[29215]: pgmap v703: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:12.850 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:21 smithi019 ceph-mon[29215]: pgmap v704: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:12.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:23 smithi019 ceph-mon[29215]: pgmap v705: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:12.868 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:25 smithi019 ceph-mon[29215]: pgmap v706: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:12.908 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:27 smithi019 ceph-mon[29215]: pgmap v707: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:12.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:29 smithi019 ceph-mon[29215]: pgmap v708: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:12.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:31 smithi019 ceph-mon[29215]: pgmap v709: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:12.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:33 smithi019 ceph-mon[29215]: pgmap v710: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:12.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:35 smithi019 ceph-mon[29215]: pgmap v711: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:12.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:35 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:13.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:13.019 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:13.028 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:36 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:13.091 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:37 smithi019 ceph-mon[29215]: pgmap v712: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.100 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:39 smithi019 ceph-mon[29215]: pgmap v713: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.108 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:42 smithi019 ceph-mon[29215]: pgmap v714: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:44 smithi019 ceph-mon[29215]: pgmap v715: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:46 smithi019 ceph-mon[29215]: pgmap v716: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.175 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:48 smithi019 ceph-mon[29215]: pgmap v717: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:50 smithi019 ceph-mon[29215]: pgmap v718: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:52 smithi019 ceph-mon[29215]: pgmap v719: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:54 smithi019 ceph-mon[29215]: pgmap v720: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:56 smithi019 ceph-mon[29215]: pgmap v721: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:48:58 smithi019 ceph-mon[29215]: pgmap v722: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:13.304 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:00 smithi019 ceph-mon[29215]: pgmap v723: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:13.313 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:02 smithi019 ceph-mon[29215]: pgmap v724: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:04 smithi019 ceph-mon[29215]: pgmap v725: 97 pgs: 97 active+clean; 4.4 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:13.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:06 smithi019 ceph-mon[29215]: pgmap v726: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:13.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:08 smithi019 ceph-mon[29215]: pgmap v727: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:13.349 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:09 smithi019 ceph-mon[29215]: pgmap v728: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:13.409 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:11 smithi019 ceph-mon[29215]: osdmap e125: 12 total, 12 up, 12 in 2024-04-23T18:52:13.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:11 smithi019 ceph-mon[29215]: pgmap v730: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:13.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:13 smithi019 ceph-mon[29215]: pgmap v731: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:13.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:15 smithi019 ceph-mon[29215]: pgmap v732: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:13.495 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:17 smithi019 ceph-mon[29215]: pgmap v733: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:13.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:18 smithi019 ceph-mon[29215]: osdmap e126: 12 total, 12 up, 12 in 2024-04-23T18:52:13.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:19 smithi019 ceph-mon[29215]: pgmap v735: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 125 B/s wr, 6 op/s 2024-04-23T18:52:13.517 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:21 smithi019 ceph-mon[29215]: pgmap v736: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:13.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:23 smithi019 ceph-mon[29215]: pgmap v737: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:13.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:25 smithi019 ceph-mon[29215]: pgmap v738: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:13.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:27 smithi019 ceph-mon[29215]: pgmap v739: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:13.628 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:29 smithi019 ceph-mon[29215]: pgmap v740: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:13.636 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:31 smithi019 ceph-mon[29215]: pgmap v741: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:13.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:34 smithi019 ceph-mon[29215]: pgmap v742: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:36 smithi019 ceph-mon[29215]: pgmap v743: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:37 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:13.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:38 smithi019 ceph-mon[29215]: pgmap v744: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:38 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:13.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:38 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:13.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:38 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:13.755 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:40 smithi019 ceph-mon[29215]: pgmap v745: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:42 smithi019 ceph-mon[29215]: pgmap v746: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:44 smithi019 ceph-mon[29215]: pgmap v747: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:46 smithi019 ceph-mon[29215]: pgmap v748: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.784 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:48 smithi019 ceph-mon[29215]: pgmap v749: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:50 smithi019 ceph-mon[29215]: pgmap v750: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:52 smithi019 ceph-mon[29215]: pgmap v751: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.854 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:54 smithi019 ceph-mon[29215]: pgmap v752: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:56 smithi019 ceph-mon[29215]: pgmap v753: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.868 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:49:58 smithi019 ceph-mon[29215]: pgmap v754: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:00 smithi019 ceph-mon[29215]: pgmap v755: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:00 smithi019 ceph-mon[29215]: overall HEALTH_OK 2024-04-23T18:52:13.891 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:01 smithi019 ceph-mon[29215]: pgmap v756: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:13.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:03 smithi019 ceph-mon[29215]: pgmap v757: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:05 smithi019 ceph-mon[29215]: pgmap v758: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:13.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:07 smithi019 ceph-mon[29215]: pgmap v759: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:13.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:09 smithi019 ceph-mon[29215]: pgmap v760: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:14.029 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:11 smithi019 ceph-mon[29215]: pgmap v761: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:14.035 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:11 smithi019 ceph-mon[29215]: osdmap e127: 12 total, 12 up, 12 in 2024-04-23T18:52:14.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:13 smithi019 ceph-mon[29215]: pgmap v763: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:14.054 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:15 smithi019 ceph-mon[29215]: pgmap v764: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:14.060 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:17 smithi019 ceph-mon[29215]: pgmap v765: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:14.099 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:18 smithi019 ceph-mon[29215]: osdmap e128: 12 total, 12 up, 12 in 2024-04-23T18:52:14.186 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:19 smithi019 ceph-mon[29215]: pgmap v767: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:14.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:21 smithi019 ceph-mon[29215]: pgmap v768: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:14.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:23 smithi019 ceph-mon[29215]: pgmap v769: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:14.209 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:26 smithi019 ceph-mon[29215]: pgmap v770: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:14.216 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:28 smithi019 ceph-mon[29215]: pgmap v771: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:14.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:30 smithi019 ceph-mon[29215]: pgmap v772: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 96 B/s wr, 9 op/s 2024-04-23T18:52:14.230 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:32 smithi019 ceph-mon[29215]: pgmap v773: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:14.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:34 smithi019 ceph-mon[29215]: pgmap v774: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:14.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:36 smithi019 ceph-mon[29215]: pgmap v775: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:14.304 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:38 smithi019 ceph-mon[29215]: pgmap v776: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:14.311 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:38 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:14.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:39 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:14.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:39 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:14.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:39 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:14.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:40 smithi019 ceph-mon[29215]: pgmap v777: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:14.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:42 smithi019 ceph-mon[29215]: pgmap v778: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:14.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:44 smithi019 ceph-mon[29215]: pgmap v779: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:14.471 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:46 smithi019 ceph-mon[29215]: pgmap v780: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:14.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:48 smithi019 ceph-mon[29215]: pgmap v781: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:14.487 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:50 smithi019 ceph-mon[29215]: pgmap v782: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:52:14.496 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:52 smithi019 ceph-mon[29215]: pgmap v783: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:14.505 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:54 smithi019 ceph-mon[29215]: pgmap v784: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:14.514 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:56 smithi019 ceph-mon[29215]: pgmap v785: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:14.523 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:50:58 smithi019 ceph-mon[29215]: pgmap v786: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:14.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:00 smithi019 ceph-mon[29215]: pgmap v787: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:14.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:02 smithi019 ceph-mon[29215]: pgmap v788: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:14.607 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:04 smithi019 ceph-mon[29215]: pgmap v789: 97 pgs: 97 active+clean; 4.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:14.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:06 smithi019 ceph-mon[29215]: pgmap v790: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:14.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:07 smithi019 ceph-mon[29215]: pgmap v791: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:14.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:09 smithi019 ceph-mon[29215]: pgmap v792: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:52:14.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:11 smithi019 ceph-mon[29215]: pgmap v793: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:11 smithi019 ceph-mon[29215]: osdmap e129: 12 total, 12 up, 12 in 2024-04-23T18:52:14.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:13 smithi019 ceph-mon[29215]: pgmap v795: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:14.724 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:15 smithi019 ceph-mon[29215]: pgmap v796: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:14.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:17 smithi019 ceph-mon[29215]: pgmap v797: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:14.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:19 smithi019 ceph-mon[29215]: osdmap e130: 12 total, 12 up, 12 in 2024-04-23T18:52:14.811 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:20 smithi019 ceph-mon[29215]: pgmap v799: 97 pgs: 97 active+clean; 4.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:52:14.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:22 smithi019 ceph-mon[29215]: pgmap v800: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T18:52:14.829 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:24 smithi019 ceph-mon[29215]: pgmap v801: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:14.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:26 smithi019 ceph-mon[29215]: pgmap v802: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:14.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:28 smithi019 ceph-mon[29215]: pgmap v803: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:14.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:30 smithi019 ceph-mon[29215]: pgmap v804: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 97 B/s wr, 9 op/s 2024-04-23T18:52:14.934 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:32 smithi019 ceph-mon[29215]: pgmap v805: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T18:52:14.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:34 smithi019 ceph-mon[29215]: pgmap v806: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:14.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:36 smithi019 ceph-mon[29215]: pgmap v807: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:15.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:38 smithi019 ceph-mon[29215]: pgmap v808: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:15.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:39 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:15.022 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:39 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:15.030 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:39 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:15.039 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:39 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:15.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:40 smithi019 ceph-mon[29215]: pgmap v809: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:15.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:42 smithi019 ceph-mon[29215]: pgmap v810: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:15.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:44 smithi019 ceph-mon[29215]: pgmap v811: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:15.177 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:46 smithi019 ceph-mon[29215]: pgmap v812: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:15.185 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:48 smithi019 ceph-mon[29215]: pgmap v813: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:15.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:50 smithi019 ceph-mon[29215]: pgmap v814: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:15.201 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:52 smithi019 ceph-mon[29215]: pgmap v815: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:15.210 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:54 smithi019 ceph-mon[29215]: pgmap v816: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:15.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:56 smithi019 ceph-mon[29215]: pgmap v817: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:15.278 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:51:58 smithi019 ceph-mon[29215]: pgmap v818: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:15.287 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:00 smithi019 ceph-mon[29215]: pgmap v819: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:15.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:01 smithi019 ceph-mon[29215]: pgmap v820: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:15.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:03 smithi019 ceph-mon[29215]: pgmap v821: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:15.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:05 smithi019 ceph-mon[29215]: pgmap v822: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:15.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:06 smithi019 ceph-mon[29215]: osdmap e131: 12 total, 12 up, 12 in 2024-04-23T18:52:15.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:07 smithi019 ceph-mon[29215]: pgmap v824: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:15.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:09 smithi019 ceph-mon[29215]: pgmap v825: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:15.426 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:52:15.471 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:11 smithi155 ceph-mon[33468]: pgmap v826: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:15.480 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:14 smithi155 ceph-mon[33468]: pgmap v827: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:15.514 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:14 smithi155 ceph-mon[33468]: osdmap e132: 12 total, 12 up, 12 in 2024-04-23T18:52:15.521 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:11 smithi044 ceph-mon[32458]: pgmap v826: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:15.528 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:14 smithi044 ceph-mon[32458]: pgmap v827: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:15.535 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:14 smithi044 ceph-mon[32458]: osdmap e132: 12 total, 12 up, 12 in 2024-04-23T18:52:15.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:11 smithi019 ceph-mon[29215]: pgmap v826: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:15.553 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:52:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:52:15.624 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2207, 2024-04-23T18:52:15.631 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2191, 2024-04-23T18:52:15.687 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 2304240, 2024-04-23T18:52:15.694 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 825664, 2024-04-23T18:52:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-23T18:52:15.709 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-23T18:52:15.715 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.269058626, 2024-04-23T18:52:15.758 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.095049263, 2024-04-23T18:52:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.093116867, 2024-04-23T18:52:15.835 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.067355159, 2024-04-23T18:52:15.845 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:15.851 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2190, 2024-04-23T18:52:15.859 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.103950144, 2024-04-23T18:52:15.870 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000047465 2024-04-23T18:52:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:15.885 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:16.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1737072, 2024-04-23T18:52:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 288880 2024-04-23T18:52:16.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:16.074 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:52:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1571, 2024-04-23T18:52:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1912, 2024-04-23T18:52:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 389002, 2024-04-23T18:52:16.196 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 1276650, 2024-04-23T18:52:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 8, 2024-04-23T18:52:16.215 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-04-23T18:52:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.189900053, 2024-04-23T18:52:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.062476205, 2024-04-23T18:52:16.236 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.044026763, 2024-04-23T18:52:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.066115159, 2024-04-23T18:52:16.323 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:16.330 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1910, 2024-04-23T18:52:16.370 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.068057574, 2024-04-23T18:52:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000035632 2024-04-23T18:52:16.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:16.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:16.397 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:16.436 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:16.512 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:16.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:16.531 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-23T18:52:16.540 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 778912 2024-04-23T18:52:16.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:16.557 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:52:16.566 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1694, 2024-04-23T18:52:16.575 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1696, 2024-04-23T18:52:16.625 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 445832, 2024-04-23T18:52:16.643 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 538693, 2024-04-23T18:52:16.652 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-23T18:52:16.704 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-23T18:52:16.713 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.161584652, 2024-04-23T18:52:16.721 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.051514356, 2024-04-23T18:52:16.730 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.043906246, 2024-04-23T18:52:16.738 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.050312174, 2024-04-23T18:52:16.788 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:16.837 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1694, 2024-04-23T18:52:16.846 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.060172254, 2024-04-23T18:52:16.854 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000035520 2024-04-23T18:52:16.863 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:16.874 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:16.884 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:16.893 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:16.901 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:16.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:16.958 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:52:16.966 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:52:16.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:16.984 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-23T18:52:16.992 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-04-23T18:52:17.000 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-23T18:52:17.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:17.058 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-04-23T18:52:17.103 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-23T18:52:17.112 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-23T18:52:17.121 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 7826, 2024-04-23T18:52:17.129 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.241918763, 2024-04-23T18:52:17.138 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000030912 2024-04-23T18:52:17.146 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:17.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:17.163 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-04-23T18:52:17.208 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-23T18:52:17.217 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-23T18:52:17.227 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:17.305 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:17.314 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:17.323 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:17.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:17.341 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-04-23T18:52:17.373 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-04-23T18:52:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-04-23T18:52:17.445 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-04-23T18:52:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-04-23T18:52:17.500 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-04-23T18:52:17.557 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-04-23T18:52:17.564 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-04-23T18:52:17.572 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-04-23T18:52:17.611 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-04-23T18:52:17.619 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-04-23T18:52:17.626 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 0, 2024-04-23T18:52:17.633 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-04-23T18:52:17.640 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 0, 2024-04-23T18:52:17.669 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-04-23T18:52:17.719 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 0, 2024-04-23T18:52:17.727 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-04-23T18:52:17.735 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-04-23T18:52:17.742 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-04-23T18:52:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-04-23T18:52:17.756 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-04-23T18:52:17.763 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-04-23T18:52:17.770 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-04-23T18:52:17.808 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-04-23T18:52:17.870 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:52:18.152 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-04-23T18:52:18.711 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-04-23T18:52:18.742 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-04-23T18:52:18.775 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-04-23T18:52:18.844 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-04-23T18:52:18.880 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-04-23T18:52:18.940 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-04-23T18:52:18.949 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-04-23T18:52:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-04-23T18:52:19.046 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:52:19.055 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-04-23T18:52:19.064 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-04-23T18:52:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-04-23T18:52:19.082 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-04-23T18:52:19.091 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:52:19.177 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-04-23T18:52:19.185 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:52:19.193 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:52:19.231 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:52:19.240 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:52:19.249 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:52:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-04-23T18:52:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:52:19.365 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:52:19.403 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:52:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:52:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:52:19.427 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:52:19.435 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-04-23T18:52:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:52:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-04-23T18:52:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-04-23T18:52:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-04-23T18:52:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-04-23T18:52:19.563 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-04-23T18:52:19.571 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-04-23T18:52:19.579 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-04-23T18:52:19.587 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-04-23T18:52:19.606 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-04-23T18:52:19.636 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-04-23T18:52:19.685 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:52:19.692 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-04-23T18:52:19.699 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-04-23T18:52:19.705 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-04-23T18:52:19.714 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-04-23T18:52:19.721 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-04-23T18:52:19.728 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:52:19.735 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:52:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-04-23T18:52:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-04-23T18:52:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:52:19.830 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:52:19.836 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:52:19.843 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-04-23T18:52:19.854 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-04-23T18:52:19.873 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:52:19.906 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-04-23T18:52:19.931 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-04-23T18:52:19.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:19.955 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-04-23T18:52:19.963 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-04-23T18:52:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-04-23T18:52:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-04-23T18:52:19.983 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-04-23T18:52:19.990 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 7, 2024-04-23T18:52:20.047 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-04-23T18:52:20.053 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-04-23T18:52:20.062 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-04-23T18:52:20.114 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-04-23T18:52:20.123 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-04-23T18:52:20.134 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-04-23T18:52:20.145 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-04-23T18:52:20.156 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-04-23T18:52:20.189 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-04-23T18:52:20.237 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-04-23T18:52:20.243 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4196347, 2024-04-23T18:52:20.251 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 4196206, 2024-04-23T18:52:20.259 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-04-23T18:52:20.268 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:20.276 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:20.285 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:20.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:20.349 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 7 2024-04-23T18:52:20.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:20.369 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-04-23T18:52:20.422 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 3, 2024-04-23T18:52:20.431 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 3, 2024-04-23T18:52:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-04-23T18:52:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 1, 2024-04-23T18:52:20.465 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 1, 2024-04-23T18:52:20.536 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-04-23T18:52:20.585 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 0, 2024-04-23T18:52:20.592 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 0, 2024-04-23T18:52:20.609 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-04-23T18:52:20.618 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-04-23T18:52:20.627 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-04-23T18:52:20.635 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-04-23T18:52:20.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 52896, 2024-04-23T18:52:20.651 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215416 2024-04-23T18:52:20.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:20.724 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-04-23T18:52:20.733 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-04-23T18:52:20.788 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-04-23T18:52:20.795 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-04-23T18:52:20.803 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-04-23T18:52:20.811 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-04-23T18:52:20.832 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-04-23T18:52:20.894 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:20.935 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:20.942 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:20.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:20.955 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-04-23T18:52:20.965 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:20.972 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:20.979 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:20.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.023 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-04-23T18:52:21.031 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:21.038 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:21.088 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:21.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.103 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-04-23T18:52:21.109 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:21.128 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:21.161 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:21.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.218 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-04-23T18:52:21.226 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:21.233 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:21.240 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:21.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.252 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-04-23T18:52:21.258 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:21.311 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:21.317 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:21.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.379 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-04-23T18:52:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.457 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-04-23T18:52:21.514 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:21.521 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:21.528 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:21.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.542 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-04-23T18:52:21.549 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:21.557 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:21.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.663 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-04-23T18:52:21.670 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:21.676 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:21.683 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:21.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.698 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-04-23T18:52:21.777 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:21.802 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:21.810 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:21.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.906 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-04-23T18:52:21.916 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:21.925 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:21.932 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:21.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:21.950 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-04-23T18:52:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:22.054 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:22.065 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:22.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:22.121 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-04-23T18:52:22.129 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:22.138 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:22.166 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:22.260 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-04-23T18:52:22.268 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:22.277 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:22.286 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:22.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:22.303 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-04-23T18:52:22.311 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:22.320 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:22.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:22.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:22.395 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-04-23T18:52:22.437 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:22.445 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:22.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:22.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:22.491 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-04-23T18:52:22.519 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:22.568 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:22.578 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:22.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:22.604 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-04-23T18:52:22.613 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:22.622 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:22.633 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:22.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:22.693 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-04-23T18:52:22.699 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:22.707 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-04-23T18:52:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:22.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:22.900 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-04-23T18:52:22.907 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:22.914 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:22.921 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:22.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:22.948 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-04-23T18:52:22.954 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:23.007 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:23.014 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:23.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-04-23T18:52:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-04-23T18:52:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-04-23T18:52:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-04-23T18:52:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-04-23T18:52:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-04-23T18:52:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:52:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:52:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-04-23T18:52:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-04-23T18:52:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-04-23T18:52:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 3, 2024-04-23T18:52:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 2, 2024-04-23T18:52:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 3, 2024-04-23T18:52:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-04-23T18:52:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-04-23T18:52:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-04-23T18:52:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 5259, 2024-04-23T18:52:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-04-23T18:52:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-23T18:52:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:52:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-23T18:52:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:52:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:52:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:52:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:52:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:52:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:52:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:52:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:52:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:52:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:52:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:52:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:52:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:52:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-23T18:52:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:52:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-23T18:52:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:52:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:52:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:52:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:52:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:52:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:52:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:52:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-23T18:52:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:52:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:52:24.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:52:24.502 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-23T18:52:24.511 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-23T18:52:24.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-23T18:52:24.577 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:52:24.586 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:52:24.593 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:52:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:52:24.636 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 190155, 2024-04-23T18:52:24.642 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 53, 2024-04-23T18:52:24.649 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:52:24.659 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-23T18:52:24.693 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-23T18:52:24.738 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-23T18:52:24.746 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:52:24.753 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-23T18:52:24.760 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:52:24.768 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-23T18:52:24.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 8152, 2024-04-23T18:52:24.783 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 170, 2024-04-23T18:52:24.790 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:52:24.839 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:52:24.846 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-23T18:52:24.852 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-23T18:52:24.860 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 8512, 2024-04-23T18:52:24.928 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 13, 2024-04-23T18:52:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-23T18:52:24.944 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-23T18:52:24.977 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-23T18:52:25.026 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-23T18:52:25.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:25.088 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-04-23T18:52:25.095 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-04-23T18:52:25.101 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-04-23T18:52:25.108 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 4947, 2024-04-23T18:52:25.115 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-04-23T18:52:25.121 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-04-23T18:52:25.128 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 4947, 2024-04-23T18:52:25.198 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-04-23T18:52:25.205 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4947, 2024-04-23T18:52:25.213 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 3.170966302, 2024-04-23T18:52:25.253 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000640987 2024-04-23T18:52:25.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:25.267 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-04-23T18:52:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-04-23T18:52:25.301 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4947, 2024-04-23T18:52:25.335 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 4948 2024-04-23T18:52:25.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:25.396 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 4947, 2024-04-23T18:52:25.403 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 4947, 2024-04-23T18:52:25.411 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-04-23T18:52:25.418 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-04-23T18:52:25.426 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-04-23T18:52:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 3250, 2024-04-23T18:52:25.439 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-04-23T18:52:25.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 1695, 2024-04-23T18:52:25.504 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-04-23T18:52:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-04-23T18:52:25.519 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-04-23T18:52:25.587 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-04-23T18:52:25.595 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-04-23T18:52:25.604 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-04-23T18:52:25.643 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-04-23T18:52:25.686 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-04-23T18:52:25.726 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-04-23T18:52:25.733 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-04-23T18:52:25.742 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-04-23T18:52:25.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-04-23T18:52:25.759 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:52:25.768 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-04-23T18:52:25.777 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:52:25.786 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-04-23T18:52:25.839 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-04-23T18:52:25.849 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-04-23T18:52:25.858 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:52:25.912 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-04-23T18:52:25.923 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:52:25.932 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-04-23T18:52:25.941 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-04-23T18:52:25.960 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-04-23T18:52:25.993 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-04-23T18:52:26.068 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-04-23T18:52:26.074 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-04-23T18:52:26.083 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-04-23T18:52:26.091 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-04-23T18:52:26.099 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-04-23T18:52:26.108 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-04-23T18:52:26.116 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-04-23T18:52:26.127 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-04-23T18:52:26.180 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-04-23T18:52:26.189 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-04-23T18:52:26.201 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-04-23T18:52:26.237 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-04-23T18:52:26.246 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-04-23T18:52:26.253 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 132, 2024-04-23T18:52:26.261 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-04-23T18:52:26.276 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 46, 2024-04-23T18:52:26.309 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 5, 2024-04-23T18:52:26.354 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 6, 2024-04-23T18:52:26.363 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 1, 2024-04-23T18:52:26.372 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-04-23T18:52:26.380 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-04-23T18:52:26.390 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-04-23T18:52:26.398 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-04-23T18:52:26.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:26.414 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-04-23T18:52:26.481 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-04-23T18:52:26.487 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:52:26.495 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-04-23T18:52:26.553 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-04-23T18:52:26.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:26.567 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-04-23T18:52:26.575 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-04-23T18:52:26.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:26.651 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-04-23T18:52:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-04-23T18:52:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:52:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-04-23T18:52:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-04-23T18:52:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-04-23T18:52:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-04-23T18:52:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-04-23T18:52:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:26.795 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:52:26.803 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:26.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-23T18:52:26.870 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:26.876 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 5468, 2024-04-23T18:52:26.883 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 2617410, 2024-04-23T18:52:26.896 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:26.921 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 5468, 2024-04-23T18:52:26.959 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-23T18:52:27.011 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-23T18:52:27.017 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 5468, 2024-04-23T18:52:27.024 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2617410, 2024-04-23T18:52:27.030 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:27.038 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:27.045 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:27.051 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:27.058 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:27.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:27.117 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-04-23T18:52:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:27.178 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-23T18:52:27.186 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:27.193 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-23T18:52:27.202 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-23T18:52:27.227 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:27.267 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-23T18:52:27.345 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 4947, 2024-04-23T18:52:27.351 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2048, 2024-04-23T18:52:27.357 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 4947, 2024-04-23T18:52:27.364 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2048, 2024-04-23T18:52:27.371 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:27.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:27.386 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:27.393 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-04-23T18:52:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:27.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-04-23T18:52:27.571 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:27.580 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-23T18:52:27.594 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-23T18:52:27.634 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:27.684 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-23T18:52:27.691 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 4947, 2024-04-23T18:52:27.699 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 4947, 2024-04-23T18:52:27.707 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 4947, 2024-04-23T18:52:27.715 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 4947, 2024-04-23T18:52:27.724 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:27.734 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:27.743 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:27.810 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:27.821 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:27.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:27.839 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:52:27.903 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:27.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-23T18:52:27.917 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:27.933 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-23T18:52:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-23T18:52:28.017 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:28.024 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-23T18:52:28.035 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-23T18:52:28.044 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-23T18:52:28.053 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-04-23T18:52:28.061 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-04-23T18:52:28.070 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:28.078 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:28.137 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:28.154 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:28.163 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:28.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:28.211 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x5614802b06e0": { 2024-04-23T18:52:28.219 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-23T18:52:28.252 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:28.292 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-23T18:52:28.341 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-23T18:52:28.348 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:28.354 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-23T18:52:28.361 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-23T18:52:28.370 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-23T18:52:28.377 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-04-23T18:52:28.387 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-04-23T18:52:28.395 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:28.460 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:28.482 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:28.489 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:28.544 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:28.559 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:28.566 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:52:28.586 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-23T18:52:28.611 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-23T18:52:28.650 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-23T18:52:28.696 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-23T18:52:28.703 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-23T18:52:28.710 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:52:28.716 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:52:28.724 DEBUG:tasks.check_counter:Getting stats from d 2024-04-23T18:52:28.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-23T18:52:28.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:14 smithi019 ceph-mon[29215]: pgmap v827: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:52:28.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:14 smithi019 ceph-mon[29215]: osdmap e132: 12 total, 12 up, 12 in 2024-04-23T18:52:28.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:16 smithi019 ceph-mon[29215]: pgmap v829: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-23T18:52:28.811 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:18 smithi019 ceph-mon[29215]: pgmap v830: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:28.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:20 smithi019 ceph-mon[29215]: pgmap v831: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:28.853 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:22 smithi019 ceph-mon[29215]: pgmap v832: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:28.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:24 smithi019 ceph-mon[29215]: pgmap v833: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:28.868 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:26 smithi019 ceph-mon[29215]: pgmap v834: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 88 B/s wr, 13 op/s 2024-04-23T18:52:28.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:28 smithi019 ceph-mon[29215]: pgmap v835: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:28.933 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:16 smithi155 ceph-mon[33468]: pgmap v829: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-23T18:52:28.967 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:18 smithi155 ceph-mon[33468]: pgmap v830: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:28.973 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:20 smithi155 ceph-mon[33468]: pgmap v831: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:29.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:22 smithi155 ceph-mon[33468]: pgmap v832: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:29.046 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:24 smithi155 ceph-mon[33468]: pgmap v833: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:29.053 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:26 smithi155 ceph-mon[33468]: pgmap v834: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 88 B/s wr, 13 op/s 2024-04-23T18:52:29.060 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:28 smithi155 ceph-mon[33468]: pgmap v835: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:29.068 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:16 smithi044 ceph-mon[32458]: pgmap v829: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-23T18:52:29.075 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:18 smithi044 ceph-mon[32458]: pgmap v830: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:29.102 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:20 smithi044 ceph-mon[32458]: pgmap v831: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:52:29.184 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:22 smithi044 ceph-mon[32458]: pgmap v832: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:29.192 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:24 smithi044 ceph-mon[32458]: pgmap v833: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:29.199 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:26 smithi044 ceph-mon[32458]: pgmap v834: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 88 B/s wr, 13 op/s 2024-04-23T18:52:29.237 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:28 smithi044 ceph-mon[32458]: pgmap v835: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:52:30.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:30 smithi044 ceph-mon[32458]: pgmap v836: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:30.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:30 smithi155 ceph-mon[33468]: pgmap v836: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:30 smithi019 ceph-mon[29215]: pgmap v836: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:31.046 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:52:31.060 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:52:31.068 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4, 2024-04-23T18:52:31.077 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4, 2024-04-23T18:52:31.085 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3040, 2024-04-23T18:52:31.093 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 448, 2024-04-23T18:52:31.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-23T18:52:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-23T18:52:31.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001666190, 2024-04-23T18:52:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000043322, 2024-04-23T18:52:31.204 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000601926, 2024-04-23T18:52:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:52:31.261 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:31.270 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-04-23T18:52:31.279 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000726096, 2024-04-23T18:52:31.287 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000242032 2024-04-23T18:52:31.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:31.480 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:31.486 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:31.495 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:31.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:31.513 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-23T18:52:31.520 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T18:52:31.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:31.578 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:52:31.651 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3, 2024-04-23T18:52:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 354, 2024-04-23T18:52:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 480, 2024-04-23T18:52:31.711 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 296880, 2024-04-23T18:52:31.719 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-04-23T18:52:31.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-23T18:52:31.733 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.029241278, 2024-04-23T18:52:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.021889476, 2024-04-23T18:52:31.818 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002323348, 2024-04-23T18:52:31.891 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:52:31.897 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:31.904 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 351, 2024-04-23T18:52:31.912 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.033155838, 2024-04-23T18:52:31.919 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000094461 2024-04-23T18:52:31.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:31.934 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:31.993 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:32.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:32.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-23T18:52:32.109 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 296880 2024-04-23T18:52:32.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:32.123 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:52:32.175 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 474, 2024-04-23T18:52:32.183 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 463, 2024-04-23T18:52:32.237 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1662096, 2024-04-23T18:52:32.246 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 286512, 2024-04-23T18:52:32.253 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-23T18:52:32.260 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-23T18:52:32.267 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.086023167, 2024-04-23T18:52:32.274 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.033323314, 2024-04-23T18:52:32.281 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.046281443, 2024-04-23T18:52:32.288 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.010220967, 2024-04-23T18:52:32.355 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:32.364 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 462, 2024-04-23T18:52:32.370 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.025249825, 2024-04-23T18:52:32.457 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000054653 2024-04-23T18:52:32.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:32.471 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:32.477 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:32.517 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:32.595 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:32.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:32.676 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1662096, 2024-04-23T18:52:32.685 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 286512 2024-04-23T18:52:32.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:32.703 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-23T18:52:32.714 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-23T18:52:32.724 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-23T18:52:32.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:32.847 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-23T18:52:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:52:32.864 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-23T18:52:32.919 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:52:32.925 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:52:32.934 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:52:32.943 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:52:32.952 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:52:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:52:33.078 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:52:33.087 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:52:33.096 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:52:33.104 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:52:33.112 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:52:33.119 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:52:33.128 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:52:33.137 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-23T18:52:33.204 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:52:33.213 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-23T18:52:33.222 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:52:33.261 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:52:33.268 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:52:33.275 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:52:33.281 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:52:33.332 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:52:33.391 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:52:33.450 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-23T18:52:33.466 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:52:33.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:52:33.529 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:52:33.540 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-23T18:52:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-23T18:52:33.595 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-23T18:52:33.629 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:52:33.687 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:52:33.695 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:52:33.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:52:33.762 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148732, 2024-04-23T18:52:33.768 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 31, 2024-04-23T18:52:33.775 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:52:33.783 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-23T18:52:33.822 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-23T18:52:33.848 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-23T18:52:33.910 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:52:33.917 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-23T18:52:33.927 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:52:33.934 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-23T18:52:33.941 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-23T18:52:33.948 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-23T18:52:33.955 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:52:34.001 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:52:34.049 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-23T18:52:34.056 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-23T18:52:34.063 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-23T18:52:34.110 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-23T18:52:34.117 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-23T18:52:34.127 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-23T18:52:34.135 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-23T18:52:34.168 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-23T18:52:34.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:34.250 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:52:34.258 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:34.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-23T18:52:34.274 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:34.282 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 477, 2024-04-23T18:52:34.289 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1595101, 2024-04-23T18:52:34.296 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:34.324 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 477, 2024-04-23T18:52:34.369 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-23T18:52:34.379 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-23T18:52:34.387 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 477, 2024-04-23T18:52:34.426 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1595101, 2024-04-23T18:52:34.435 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:34.445 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:34.551 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:34.582 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:34.625 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:34.634 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:52:34.645 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:32 smithi155 ceph-mon[33468]: pgmap v837: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:34.654 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:34 smithi155 ceph-mon[33468]: pgmap v838: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:34.663 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:32 smithi044 ceph-mon[32458]: pgmap v837: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:34.672 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:34 smithi044 ceph-mon[32458]: pgmap v838: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:34.682 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-23T18:52:34.745 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-23T18:52:34.794 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-23T18:52:34.804 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-23T18:52:34.813 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-23T18:52:34.851 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-23T18:52:34.860 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-23T18:52:34.870 DEBUG:tasks.check_counter:Getting stats from g 2024-04-23T18:52:34.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-23T18:52:34.923 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:32 smithi019 ceph-mon[29215]: pgmap v837: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:34.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:34 smithi019 ceph-mon[29215]: pgmap v838: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:52:36.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:36 smithi044 ceph-mon[32458]: pgmap v839: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:36.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:36 smithi155 ceph-mon[33468]: pgmap v839: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:36.454 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:36 smithi019 ceph-mon[29215]: pgmap v839: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:37.062 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:52:37.078 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:52:37.085 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2238, 2024-04-23T18:52:37.174 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2222, 2024-04-23T18:52:37.283 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 2314208, 2024-04-23T18:52:37.290 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 837776, 2024-04-23T18:52:37.297 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-23T18:52:37.363 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-23T18:52:37.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.293635135, 2024-04-23T18:52:37.376 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.122004782, 2024-04-23T18:52:37.383 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.090080090, 2024-04-23T18:52:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.064868485, 2024-04-23T18:52:37.510 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:37.517 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2221, 2024-04-23T18:52:37.524 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.124756110, 2024-04-23T18:52:37.531 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000056171 2024-04-23T18:52:37.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:37.546 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:37.608 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:37.615 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:37.648 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:37.718 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1738976, 2024-04-23T18:52:37.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 293360 2024-04-23T18:52:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:37.781 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:52:37.789 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3309, 2024-04-23T18:52:37.796 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 3656, 2024-04-23T18:52:37.817 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 845538, 2024-04-23T18:52:37.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 1840075, 2024-04-23T18:52:37.864 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-04-23T18:52:37.874 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-04-23T18:52:37.883 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.254522151, 2024-04-23T18:52:37.892 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.082552898, 2024-04-23T18:52:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.060938483, 2024-04-23T18:52:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.090329193, 2024-04-23T18:52:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3654, 2024-04-23T18:52:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.125822849, 2024-04-23T18:52:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000034434 2024-04-23T18:52:38.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:38.065 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:38.071 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:38.080 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:38.089 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:38.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:38.201 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-23T18:52:38.213 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 789664 2024-04-23T18:52:38.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:38.309 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:52:38.319 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4, 2024-04-23T18:52:38.328 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 6, 2024-04-23T18:52:38.380 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1629, 2024-04-23T18:52:38.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 1284, 2024-04-23T18:52:38.443 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-04-23T18:52:38.451 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-04-23T18:52:38.459 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.002239230, 2024-04-23T18:52:38.467 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000135486, 2024-04-23T18:52:38.518 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.001745979, 2024-04-23T18:52:38.524 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000073694, 2024-04-23T18:52:38.531 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:38.539 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4, 2024-04-23T18:52:38.557 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.002276142, 2024-04-23T18:52:38.777 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000569035 2024-04-23T18:52:38.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:38.808 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:38.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:38.895 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:38.904 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:38.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:38.974 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:52:39.032 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:52:39.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-23T18:52:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-04-23T18:52:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-23T18:52:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-04-23T18:52:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-23T18:52:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-23T18:52:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8430, 2024-04-23T18:52:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.220143555, 2024-04-23T18:52:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000026114 2024-04-23T18:52:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-04-23T18:52:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-23T18:52:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-23T18:52:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-04-23T18:52:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-04-23T18:52:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-04-23T18:52:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-04-23T18:52:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-04-23T18:52:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-04-23T18:52:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-04-23T18:52:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-04-23T18:52:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-04-23T18:52:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-04-23T18:52:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-04-23T18:52:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 0, 2024-04-23T18:52:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-04-23T18:52:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 0, 2024-04-23T18:52:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-04-23T18:52:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 0, 2024-04-23T18:52:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-04-23T18:52:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-04-23T18:52:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-04-23T18:52:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-04-23T18:52:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-04-23T18:52:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-04-23T18:52:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-04-23T18:52:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-04-23T18:52:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:52:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-04-23T18:52:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-04-23T18:52:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-04-23T18:52:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-04-23T18:52:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-04-23T18:52:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-04-23T18:52:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-04-23T18:52:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-04-23T18:52:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-04-23T18:52:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:52:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-04-23T18:52:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-04-23T18:52:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-04-23T18:52:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-04-23T18:52:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:52:40.490 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-04-23T18:52:40.496 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:52:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:52:40.565 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:52:40.573 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:52:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:52:40.589 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-04-23T18:52:40.659 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:52:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:52:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:52:40.732 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:52:40.784 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:52:40.792 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:52:40.801 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-04-23T18:52:40.914 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:52:40.923 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-04-23T18:52:40.932 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-04-23T18:52:40.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:40.998 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-04-23T18:52:41.006 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-04-23T18:52:41.069 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-04-23T18:52:41.077 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-04-23T18:52:41.085 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-04-23T18:52:41.141 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-04-23T18:52:41.149 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-04-23T18:52:41.157 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-04-23T18:52:41.166 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:52:41.226 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-04-23T18:52:41.257 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-04-23T18:52:41.263 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-04-23T18:52:41.270 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-04-23T18:52:41.339 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-04-23T18:52:41.347 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:52:41.354 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:52:41.397 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-04-23T18:52:41.405 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-04-23T18:52:41.464 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:52:41.471 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:52:41.478 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:52:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-04-23T18:52:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-04-23T18:52:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:52:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-04-23T18:52:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-04-23T18:52:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-04-23T18:52:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-04-23T18:52:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-04-23T18:52:41.690 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-04-23T18:52:41.696 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-04-23T18:52:41.703 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 8, 2024-04-23T18:52:41.739 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-04-23T18:52:41.746 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-04-23T18:52:41.803 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-04-23T18:52:41.810 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-04-23T18:52:41.817 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-04-23T18:52:41.824 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-04-23T18:52:41.866 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-04-23T18:52:41.873 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-04-23T18:52:41.884 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-04-23T18:52:41.891 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-04-23T18:52:41.917 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4196488, 2024-04-23T18:52:41.949 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 4196347, 2024-04-23T18:52:41.954 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-04-23T18:52:41.962 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:42.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:42.092 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 8 2024-04-23T18:52:42.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:42.104 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-04-23T18:52:42.166 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 3, 2024-04-23T18:52:42.173 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 3, 2024-04-23T18:52:42.179 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-04-23T18:52:42.213 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 1, 2024-04-23T18:52:42.220 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 1, 2024-04-23T18:52:42.227 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-04-23T18:52:42.233 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 0, 2024-04-23T18:52:42.281 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 0, 2024-04-23T18:52:42.347 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-04-23T18:52:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-04-23T18:52:42.363 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-04-23T18:52:42.424 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-04-23T18:52:42.433 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 50016, 2024-04-23T18:52:42.441 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215416 2024-04-23T18:52:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:42.545 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-04-23T18:52:42.613 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-04-23T18:52:42.622 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-04-23T18:52:42.631 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-04-23T18:52:42.640 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-04-23T18:52:42.749 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-04-23T18:52:42.758 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-04-23T18:52:42.767 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:42.775 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:42.817 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:42.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:42.915 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-04-23T18:52:42.923 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:42.981 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:42.987 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:42.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:43.056 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-04-23T18:52:43.065 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:43.129 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:43.138 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:43.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:43.155 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-04-23T18:52:43.199 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:43.208 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:43.219 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:43.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:43.289 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-04-23T18:52:43.333 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:43.342 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:43.351 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:43.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-04-23T18:52:43.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:43.455 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:43.465 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:43.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:43.545 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-04-23T18:52:43.556 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:43.565 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:43.634 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:43.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:43.649 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-04-23T18:52:43.656 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:43.705 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:43.750 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:43.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:43.847 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-04-23T18:52:43.953 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:43.977 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:43.984 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:44.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:44.055 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-04-23T18:52:44.066 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:44.121 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:44.130 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:44.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:44.174 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-04-23T18:52:44.181 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:44.188 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:44.194 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:44.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:44.283 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-04-23T18:52:44.291 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:44.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:38 smithi155 ceph-mon[33468]: pgmap v840: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:44.372 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:40 smithi155 ceph-mon[33468]: pgmap v841: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:44.378 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:40 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:44.385 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:42 smithi155 ceph-mon[33468]: pgmap v842: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:44.456 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:38 smithi044 ceph-mon[32458]: pgmap v840: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:44.462 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:40 smithi044 ceph-mon[32458]: pgmap v841: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:44.517 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:40 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:44.525 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:42 smithi044 ceph-mon[32458]: pgmap v842: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:44.533 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:44.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:44.590 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-04-23T18:52:44.596 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:44.602 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:44.633 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:44.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:44.674 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-04-23T18:52:44.684 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:44.738 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:44.748 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:44.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:44.855 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-04-23T18:52:44.864 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:44.922 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:44.931 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:44.949 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-04-23T18:52:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:44.998 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:45.007 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:45.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:45.050 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-04-23T18:52:45.091 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:45.102 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:45.112 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:45.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:45.161 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-04-23T18:52:45.170 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:45.231 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:45.240 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:45.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:45.271 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-04-23T18:52:45.280 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:45.289 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:45.332 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:45.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:45.351 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-04-23T18:52:45.360 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:45.397 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:45.441 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:45.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:45.459 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-04-23T18:52:45.503 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:45.511 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:45.520 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:45.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:45.576 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-04-23T18:52:45.635 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:45.644 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:45.653 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:45.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:45.723 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-04-23T18:52:45.732 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:45.743 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:45.755 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:45.799 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-04-23T18:52:45.807 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:45.813 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:45.878 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:45.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:45.893 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-04-23T18:52:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:46.050 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-04-23T18:52:46.059 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:46.099 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:46.106 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:46.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:46.121 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-04-23T18:52:46.142 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:46.191 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:46.198 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:46.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:46.250 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-04-23T18:52:46.258 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:46.266 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:46.321 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:46.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:46.359 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-04-23T18:52:46.367 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:46.374 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:46.381 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:52:46.502 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:52:46.508 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-04-23T18:52:46.542 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:46.620 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:46.627 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:46.635 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:46.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:46.713 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-04-23T18:52:46.720 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 0, 2024-04-23T18:52:46.779 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 3, 2024-04-23T18:52:46.786 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 3, 2024-04-23T18:52:46.840 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 3, 2024-04-23T18:52:46.847 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-04-23T18:52:46.855 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-04-23T18:52:46.862 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-04-23T18:52:46.921 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 1317, 2024-04-23T18:52:46.928 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-04-23T18:52:46.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:46.942 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-23T18:52:46.975 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:52:47.027 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-23T18:52:47.040 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:52:47.049 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:52:47.140 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:52:47.147 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:52:47.155 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:52:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:52:47.247 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:52:47.258 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:52:47.328 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:52:47.337 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:52:47.346 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:52:47.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:52:47.390 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:52:47.399 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-23T18:52:47.408 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:52:47.439 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-23T18:52:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:52:47.481 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:52:47.491 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:52:47.533 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:52:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:52:47.586 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:52:47.595 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:52:47.605 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-23T18:52:47.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:52:47.662 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:52:47.672 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:52:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-23T18:52:47.732 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-23T18:52:47.741 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-23T18:52:47.749 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:52:47.757 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:52:47.784 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:52:47.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:52:47.867 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 198274, 2024-04-23T18:52:47.879 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 58, 2024-04-23T18:52:47.941 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:52:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-23T18:52:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-23T18:52:48.027 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-23T18:52:48.036 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:52:48.114 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-23T18:52:48.123 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:52:48.134 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-23T18:52:48.142 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 8032, 2024-04-23T18:52:48.182 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 167, 2024-04-23T18:52:48.190 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:52:48.198 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:52:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-23T18:52:48.238 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-23T18:52:48.282 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 8512, 2024-04-23T18:52:48.289 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 13, 2024-04-23T18:52:48.297 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-23T18:52:48.349 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-23T18:52:48.358 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-23T18:52:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-23T18:52:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-04-23T18:52:48.494 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-04-23T18:52:48.501 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-04-23T18:52:48.509 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 5019, 2024-04-23T18:52:48.516 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-04-23T18:52:48.556 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-04-23T18:52:48.564 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 5019, 2024-04-23T18:52:48.572 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-04-23T18:52:48.582 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 5019, 2024-04-23T18:52:48.630 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2.630972061, 2024-04-23T18:52:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000524202 2024-04-23T18:52:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-04-23T18:52:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-04-23T18:52:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 5019, 2024-04-23T18:52:48.833 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 5020 2024-04-23T18:52:48.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:48.887 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 5019, 2024-04-23T18:52:49.213 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 5019, 2024-04-23T18:52:50.129 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-04-23T18:52:50.159 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-04-23T18:52:50.216 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-04-23T18:52:50.225 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 3298, 2024-04-23T18:52:50.233 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-04-23T18:52:50.242 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 1719, 2024-04-23T18:52:50.305 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-04-23T18:52:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-04-23T18:52:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-04-23T18:52:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-04-23T18:52:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-04-23T18:52:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-04-23T18:52:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-04-23T18:52:50.574 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-04-23T18:52:50.581 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-04-23T18:52:50.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-04-23T18:52:50.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-04-23T18:52:50.640 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-04-23T18:52:50.654 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:52:50.707 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-04-23T18:52:50.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:52:50.731 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-04-23T18:52:50.745 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-04-23T18:52:50.796 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-04-23T18:52:50.802 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:52:50.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-04-23T18:52:50.830 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:52:50.864 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-04-23T18:52:50.882 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-04-23T18:52:50.943 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-04-23T18:52:50.950 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-04-23T18:52:50.968 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-04-23T18:52:50.994 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-04-23T18:52:51.003 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-04-23T18:52:51.016 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-04-23T18:52:51.026 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-04-23T18:52:51.080 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-04-23T18:52:51.088 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-04-23T18:52:51.096 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-04-23T18:52:51.104 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-04-23T18:52:51.129 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-04-23T18:52:51.164 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-04-23T18:52:51.174 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-04-23T18:52:51.183 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-04-23T18:52:51.257 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 132, 2024-04-23T18:52:51.267 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-04-23T18:52:51.336 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 46, 2024-04-23T18:52:51.345 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 6, 2024-04-23T18:52:51.352 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 8, 2024-04-23T18:52:51.390 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 2, 2024-04-23T18:52:51.400 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-04-23T18:52:51.408 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-04-23T18:52:51.419 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-04-23T18:52:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-04-23T18:52:51.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:51.497 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-04-23T18:52:51.506 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-04-23T18:52:51.553 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:52:51.598 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-04-23T18:52:51.604 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-04-23T18:52:51.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:51.681 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-04-23T18:52:51.689 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-04-23T18:52:51.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-04-23T18:52:51.758 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-04-23T18:52:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:52:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-04-23T18:52:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-04-23T18:52:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-04-23T18:52:51.873 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-04-23T18:52:51.883 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-04-23T18:52:51.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:51.908 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:52:51.955 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:51.987 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-23T18:52:52.000 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:52.014 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 5547, 2024-04-23T18:52:52.065 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 2632237, 2024-04-23T18:52:52.077 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:52.135 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 5547, 2024-04-23T18:52:52.149 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-23T18:52:52.161 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-23T18:52:52.194 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 5547, 2024-04-23T18:52:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2632237, 2024-04-23T18:52:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:52.298 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:52.305 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:52.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:52.321 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-04-23T18:52:52.373 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:52.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-23T18:52:52.395 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:52.474 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-23T18:52:52.489 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-23T18:52:52.550 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:52.559 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-23T18:52:52.568 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 5019, 2024-04-23T18:52:52.577 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2189, 2024-04-23T18:52:52.624 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 5019, 2024-04-23T18:52:52.635 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2189, 2024-04-23T18:52:52.645 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:52.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:52.707 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:52.719 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:52.730 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:52.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:52.796 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-04-23T18:52:52.804 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:52.815 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-04-23T18:52:52.857 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:52.864 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-23T18:52:52.923 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-23T18:52:52.937 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:52.946 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-23T18:52:52.980 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 5019, 2024-04-23T18:52:52.991 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 5019, 2024-04-23T18:52:53.000 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 5019, 2024-04-23T18:52:53.010 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 5019, 2024-04-23T18:52:53.055 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:53.064 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:53.082 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:53.150 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:53.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:53.187 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:52:53.197 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:53.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-23T18:52:53.216 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:53.268 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-23T18:52:53.276 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-23T18:52:53.289 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:53.366 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-23T18:52:53.379 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-23T18:52:53.392 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-23T18:52:53.406 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-04-23T18:52:53.480 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-04-23T18:52:53.488 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:53.498 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:53.511 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:53.542 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:53.589 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:53.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:53.611 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x56072281e6e0": { 2024-04-23T18:52:53.673 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:52:53.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-23T18:52:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:52:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-23T18:52:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-23T18:52:53.837 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:52:53.847 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-23T18:52:53.857 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-23T18:52:53.866 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-23T18:52:53.921 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-04-23T18:52:53.933 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-04-23T18:52:53.946 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:52:53.958 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:54.017 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:54.054 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:54.061 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:54.070 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:52:54.119 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:52:54.129 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-23T18:52:54.193 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-23T18:52:54.204 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-23T18:52:54.212 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-23T18:52:54.276 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-23T18:52:54.284 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:52:54.292 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:52:54.301 DEBUG:tasks.check_counter:Getting stats from j 2024-04-23T18:52:54.358 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-23T18:52:54.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:38 smithi019 ceph-mon[29215]: pgmap v840: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:54.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:40 smithi019 ceph-mon[29215]: pgmap v841: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:54.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:40 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:54.426 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:42 smithi019 ceph-mon[29215]: pgmap v842: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:54.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:44 smithi019 ceph-mon[29215]: pgmap v843: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:54.471 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:44 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:54.481 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:44 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:54.523 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:44 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:54.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:44 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:54.593 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:44 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:54.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:44 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:54.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:44 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:54.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:46 smithi019 ceph-mon[29215]: pgmap v844: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:54.660 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:48 smithi019 ceph-mon[29215]: pgmap v845: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:54.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:50 smithi019 ceph-mon[29215]: pgmap v846: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:54.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:52 smithi019 ceph-mon[29215]: pgmap v847: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:54.756 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:54 smithi019 ceph-mon[29215]: pgmap v848: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:54.767 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:44 smithi155 ceph-mon[33468]: pgmap v843: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:54.779 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:44 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:54.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:44 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:54.845 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:44 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:54.855 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:44 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:54.864 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:44 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:54.887 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:44 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:54.940 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:44 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:54.948 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:46 smithi155 ceph-mon[33468]: pgmap v844: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:54.993 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:48 smithi155 ceph-mon[33468]: pgmap v845: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:55.004 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:50 smithi155 ceph-mon[33468]: pgmap v846: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:55.017 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:52 smithi155 ceph-mon[33468]: pgmap v847: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:55.052 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:54 smithi155 ceph-mon[33468]: pgmap v848: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:55.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:44 smithi044 ceph-mon[32458]: pgmap v843: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:55.072 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:44 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:55.083 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:44 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:55.145 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:44 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:55.159 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:44 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:55.171 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:44 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:55.180 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:44 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:55.233 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:44 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:52:55.268 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:46 smithi044 ceph-mon[32458]: pgmap v844: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:55.280 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:48 smithi044 ceph-mon[32458]: pgmap v845: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:55.294 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:50 smithi044 ceph-mon[32458]: pgmap v846: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:55.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:52 smithi044 ceph-mon[32458]: pgmap v847: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:55.397 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:54 smithi044 ceph-mon[32458]: pgmap v848: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:52:56.198 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:56 smithi019 ceph-mon[29215]: pgmap v849: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:56.564 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:56 smithi044 ceph-mon[32458]: pgmap v849: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:56.606 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:56 smithi155 ceph-mon[33468]: pgmap v849: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:52:56.937 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-23T18:52:57.030 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:52:57.039 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-04-23T18:52:57.048 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-04-23T18:52:57.115 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-04-23T18:52:57.125 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-04-23T18:52:57.137 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-23T18:52:57.285 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-23T18:52:57.355 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001379271, 2024-04-23T18:52:57.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-04-23T18:52:57.379 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000194269, 2024-04-23T18:52:57.503 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:52:57.513 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:57.527 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:57.542 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:57.554 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:57.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:57.627 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:57.693 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:57.710 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:57.767 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:57.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:57.786 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:52:57.797 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:52:57.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:57.888 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:52:57.897 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3, 2024-04-23T18:52:57.906 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 360, 2024-04-23T18:52:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 480, 2024-04-23T18:52:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 297936, 2024-04-23T18:52:57.974 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-04-23T18:52:57.983 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-23T18:52:57.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.029704417, 2024-04-23T18:52:58.053 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.022755279, 2024-04-23T18:52:58.062 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002262888, 2024-04-23T18:52:58.153 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:52:58.163 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:58.218 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 357, 2024-04-23T18:52:58.228 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.020802046, 2024-04-23T18:52:58.238 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000058269 2024-04-23T18:52:58.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:58.307 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:58.353 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:58.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:58.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:58.449 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-23T18:52:58.459 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 297936 2024-04-23T18:52:58.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:58.481 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:52:58.532 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 485, 2024-04-23T18:52:58.542 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 472, 2024-04-23T18:52:58.593 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1667040, 2024-04-23T18:52:58.602 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 291440, 2024-04-23T18:52:58.688 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-23T18:52:58.700 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-23T18:52:58.713 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.085121782, 2024-04-23T18:52:58.729 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.031926068, 2024-04-23T18:52:58.769 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.047531482, 2024-04-23T18:52:58.803 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.010081360, 2024-04-23T18:52:58.815 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:52:58.827 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 472, 2024-04-23T18:52:58.943 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.026449530, 2024-04-23T18:52:58.956 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000056037 2024-04-23T18:52:58.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:58.979 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:52:58.989 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:52:59.045 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:52:59.055 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:52:59.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1667040, 2024-04-23T18:52:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 291440 2024-04-23T18:52:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-23T18:52:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-23T18:52:59.282 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-23T18:52:59.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:52:59.340 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-23T18:52:59.347 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:52:59.435 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-23T18:52:59.442 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:52:59.450 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:52:59.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:52:59.466 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:52:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:52:59.534 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:52:59.594 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:52:59.600 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:52:59.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:52:59.661 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:52:59.670 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:52:59.677 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:52:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:52:59.736 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-23T18:52:59.743 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:52:59.752 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-23T18:52:59.881 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:52:59.888 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:52:59.898 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:52:59.907 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:52:59.916 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:52:59.981 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:52:59.991 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:53:00.076 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-23T18:53:00.084 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:53:00.145 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:53:00.156 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:53:00.166 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-23T18:53:00.176 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-23T18:53:00.219 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-23T18:53:00.252 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:53:00.263 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:53:00.276 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:53:00.346 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:53:00.355 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148692, 2024-04-23T18:53:00.370 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 30, 2024-04-23T18:53:00.383 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:53:00.394 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-23T18:53:00.404 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-23T18:53:00.464 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-23T18:53:00.542 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:53:00.554 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-23T18:53:00.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:53:00.640 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-23T18:53:00.653 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-23T18:53:00.667 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-23T18:53:00.694 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:53:00.736 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:53:00.745 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-23T18:53:00.759 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-23T18:53:00.863 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-23T18:53:00.877 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-23T18:53:00.889 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-23T18:53:00.901 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-23T18:53:00.913 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-23T18:53:00.975 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-23T18:53:00.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-23T18:53:01.067 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:53:01.077 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-23T18:53:01.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-23T18:53:01.139 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-23T18:53:01.148 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 484, 2024-04-23T18:53:01.157 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1596011, 2024-04-23T18:53:01.192 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:01.219 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 484, 2024-04-23T18:53:01.226 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-23T18:53:01.236 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-23T18:53:01.279 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 484, 2024-04-23T18:53:01.289 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1596011, 2024-04-23T18:53:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-23T18:53:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-23T18:53:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-23T18:53:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-23T18:53:01.371 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:53:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-23T18:53:01.433 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-23T18:53:01.479 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:57 smithi019 ceph-mon[29215]: pgmap v850: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:01.488 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:57 smithi155 ceph-mon[33468]: pgmap v850: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:01.496 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:52:59 smithi155 ceph-mon[33468]: pgmap v851: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:01.504 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:57 smithi044 ceph-mon[32458]: pgmap v850: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:01.563 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:52:59 smithi044 ceph-mon[32458]: pgmap v851: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:01.614 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-23T18:53:01.623 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-23T18:53:01.630 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-23T18:53:01.704 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-23T18:53:01.713 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-23T18:53:01.720 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-23T18:53:01.728 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-23T18:53:01.736 DEBUG:tasks.check_counter:Getting stats from b 2024-04-23T18:53:01.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-23T18:53:01.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:52:59 smithi019 ceph-mon[29215]: pgmap v851: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:01.897 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:01 smithi044 ceph-mon[32458]: pgmap v852: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:02.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:01 smithi019 ceph-mon[29215]: pgmap v852: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:02.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:01 smithi155 ceph-mon[33468]: pgmap v852: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:53:03.898 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:03 smithi044 ceph-mon[32458]: pgmap v853: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:03.907 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-23T18:53:03.926 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:53:03.981 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 4806, 2024-04-23T18:53:03.990 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 6142, 2024-04-23T18:53:03.998 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 2915034, 2024-04-23T18:53:04.078 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 7432567, 2024-04-23T18:53:04.084 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 76, 2024-04-23T18:53:04.094 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 9, 2024-04-23T18:53:04.105 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.581603182, 2024-04-23T18:53:04.114 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.226298305, 2024-04-23T18:53:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.246672159, 2024-04-23T18:53:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.087160167, 2024-04-23T18:53:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:53:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6141, 2024-04-23T18:53:04.297 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 1.071354340, 2024-04-23T18:53:04.310 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000174459 2024-04-23T18:53:04.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:04.426 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:53:04.499 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 7881, 2024-04-23T18:53:04.535 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001833174, 2024-04-23T18:53:04.546 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000232 2024-04-23T18:53:04.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:04.622 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1774368, 2024-04-23T18:53:04.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 980240 2024-04-23T18:53:04.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:04.657 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:53:04.668 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 4840, 2024-04-23T18:53:04.733 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 5577, 2024-04-23T18:53:04.745 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1769324, 2024-04-23T18:53:04.755 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 12220717, 2024-04-23T18:53:04.765 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 171, 2024-04-23T18:53:04.818 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2024-04-23T18:53:04.834 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.671488034, 2024-04-23T18:53:04.844 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.211943036, 2024-04-23T18:53:04.925 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.297825886, 2024-04-23T18:53:04.968 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.153074964, 2024-04-23T18:53:05.001 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:53:05.011 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 5574, 2024-04-23T18:53:05.018 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.613494344, 2024-04-23T18:53:05.096 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000110063 2024-04-23T18:53:05.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:05.116 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:53:05.124 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6250, 2024-04-23T18:53:05.134 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001415030, 2024-04-23T18:53:05.143 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000226 2024-04-23T18:53:05.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:05.216 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1382688, 2024-04-23T18:53:05.228 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 1735104 2024-04-23T18:53:05.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:05.278 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:53:05.286 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 8470, 2024-04-23T18:53:05.344 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 7981, 2024-04-23T18:53:05.430 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 2829770, 2024-04-23T18:53:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 16171292, 2024-04-23T18:53:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 24, 2024-04-23T18:53:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 7, 2024-04-23T18:53:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.905544873, 2024-04-23T18:53:05.548 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.282336763, 2024-04-23T18:53:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.330648233, 2024-04-23T18:53:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.265507193, 2024-04-23T18:53:05.576 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:53:05.633 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 7980, 2024-04-23T18:53:05.642 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.386219624, 2024-04-23T18:53:05.649 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000048398 2024-04-23T18:53:05.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:05.709 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:53:05.716 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 10380, 2024-04-23T18:53:05.723 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.002443691, 2024-04-23T18:53:05.737 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000235 2024-04-23T18:53:05.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 887216, 2024-04-23T18:53:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 287536 2024-04-23T18:53:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:05.934 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-23T18:53:05.943 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-04-23T18:53:05.952 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-23T18:53:05.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-04-23T18:53:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-23T18:53:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-23T18:53:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6802, 2024-04-23T18:53:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 2.156175382, 2024-04-23T18:53:06.094 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000316991 2024-04-23T18:53:06.103 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:06.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:06.174 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-04-23T18:53:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-23T18:53:06.267 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-23T18:53:06.278 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 15, 2024-04-23T18:53:06.287 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000366528, 2024-04-23T18:53:06.370 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000024435 2024-04-23T18:53:06.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:06.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:06.398 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-04-23T18:53:06.406 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 1001, 2024-04-23T18:53:06.415 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 1001, 2024-04-23T18:53:06.468 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-04-23T18:53:06.477 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1001, 2024-04-23T18:53:06.487 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 12.943283228, 2024-04-23T18:53:06.518 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.012930352 2024-04-23T18:53:06.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:06.535 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-04-23T18:53:06.591 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2024-04-23T18:53:06.671 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-04-23T18:53:06.701 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-04-23T18:53:06.710 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 476, 2024-04-23T18:53:06.718 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-04-23T18:53:06.792 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-04-23T18:53:06.800 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 45, 2024-04-23T18:53:06.808 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 18, 2024-04-23T18:53:06.817 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 27, 2024-04-23T18:53:06.825 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-04-23T18:53:06.882 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 19, 2024-04-23T18:53:06.892 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 0, 2024-04-23T18:53:06.901 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 3, 2024-04-23T18:53:06.910 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 3, 2024-04-23T18:53:06.946 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 2, 2024-04-23T18:53:06.955 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 1099, 2024-04-23T18:53:06.964 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 1087, 2024-04-23T18:53:07.013 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2024-04-23T18:53:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2024-04-23T18:53:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:53:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-04-23T18:53:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-04-23T18:53:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-04-23T18:53:07.204 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-04-23T18:53:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-04-23T18:53:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-04-23T18:53:07.225 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-04-23T18:53:07.232 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-04-23T18:53:07.281 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-04-23T18:53:07.289 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:53:07.296 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-04-23T18:53:07.326 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 4, 2024-04-23T18:53:07.334 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 234, 2024-04-23T18:53:07.341 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 6, 2024-04-23T18:53:07.420 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:53:07.457 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-04-23T18:53:07.484 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:53:07.491 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:53:07.499 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:53:07.561 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:53:07.569 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:53:07.576 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-04-23T18:53:07.583 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:53:07.591 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 654, 2024-04-23T18:53:07.618 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 651, 2024-04-23T18:53:07.626 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 599, 2024-04-23T18:53:07.633 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:53:07.640 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:53:07.671 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 771, 2024-04-23T18:53:07.680 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 720, 2024-04-23T18:53:07.689 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 4, 2024-04-23T18:53:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 629 2024-04-23T18:53:07.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:07.907 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-04-23T18:53:07.978 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 24, 2024-04-23T18:53:07.984 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-04-23T18:53:07.992 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-04-23T18:53:07.999 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2024-04-23T18:53:08.011 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 49, 2024-04-23T18:53:08.021 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 2, 2024-04-23T18:53:08.031 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-04-23T18:53:08.090 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:53:08.099 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 3, 2024-04-23T18:53:08.108 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 1, 2024-04-23T18:53:08.118 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-04-23T18:53:08.173 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-04-23T18:53:08.183 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-04-23T18:53:08.193 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:53:08.257 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:53:08.316 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-04-23T18:53:08.348 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-04-23T18:53:08.357 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 12, 2024-04-23T18:53:08.397 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 164, 2024-04-23T18:53:08.406 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:53:08.417 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-04-23T18:53:08.428 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-04-23T18:53:08.437 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:53:08.446 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-04-23T18:53:08.455 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-04-23T18:53:08.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:08.525 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-04-23T18:53:08.534 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-04-23T18:53:08.604 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 2773, 2024-04-23T18:53:08.613 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 2699, 2024-04-23T18:53:08.622 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 2699, 2024-04-23T18:53:08.682 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 74, 2024-04-23T18:53:08.795 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-04-23T18:53:08.816 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 1, 2024-04-23T18:53:08.881 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 96, 2024-04-23T18:53:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 95, 2024-04-23T18:53:08.899 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 95, 2024-04-23T18:53:08.907 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 1, 2024-04-23T18:53:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 1, 2024-04-23T18:53:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-04-23T18:53:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 1, 2024-04-23T18:53:08.969 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 28395630, 2024-04-23T18:53:08.980 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 28469326, 2024-04-23T18:53:08.988 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 0, 2024-04-23T18:53:08.995 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-04-23T18:53:09.055 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2661, 2024-04-23T18:53:09.061 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 5.931593917, 2024-04-23T18:53:09.068 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.002229084 2024-04-23T18:53:09.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 0 2024-04-23T18:53:09.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:09.298 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-04-23T18:53:09.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 39, 2024-04-23T18:53:09.312 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 40, 2024-04-23T18:53:09.319 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 1, 2024-04-23T18:53:09.327 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 20, 2024-04-23T18:53:09.335 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 20, 2024-04-23T18:53:09.341 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 0, 2024-04-23T18:53:09.382 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 45, 2024-04-23T18:53:09.389 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 47, 2024-04-23T18:53:09.395 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 2, 2024-04-23T18:53:09.402 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 3, 2024-04-23T18:53:09.445 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 22, 2024-04-23T18:53:09.453 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 19, 2024-04-23T18:53:09.460 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 61300, 2024-04-23T18:53:09.546 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 231800 2024-04-23T18:53:09.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:09.622 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-04-23T18:53:09.684 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 2321, 2024-04-23T18:53:09.692 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 286, 2024-04-23T18:53:09.700 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 1001, 2024-04-23T18:53:09.707 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 57, 2024-04-23T18:53:09.714 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 572, 2024-04-23T18:53:09.729 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-04-23T18:53:09.736 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 14, 2024-04-23T18:53:09.779 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.013500328, 2024-04-23T18:53:09.786 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000964309 2024-04-23T18:53:09.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:09.802 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-04-23T18:53:09.829 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 284, 2024-04-23T18:53:09.836 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.056822072, 2024-04-23T18:53:09.844 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000200077 2024-04-23T18:53:09.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:09.924 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-04-23T18:53:09.969 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:10.034 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:10.041 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:10.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:10.055 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-04-23T18:53:10.063 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:10.071 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:10.080 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-04-23T18:53:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3, 2024-04-23T18:53:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001839608, 2024-04-23T18:53:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000613202 2024-04-23T18:53:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-04-23T18:53:10.264 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:10.329 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:10.357 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:10.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:10.427 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-04-23T18:53:10.436 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-04-23T18:53:10.444 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000462019, 2024-04-23T18:53:10.453 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000462019 2024-04-23T18:53:10.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:10.471 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-04-23T18:53:10.478 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:10.534 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:10.543 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:10.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:10.587 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-04-23T18:53:10.598 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:10.608 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-04-23T18:53:10.794 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:10.805 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:10.814 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-04-23T18:53:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 27, 2024-04-23T18:53:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.005636434, 2024-04-23T18:53:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000208756 2024-04-23T18:53:10.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:10.950 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-04-23T18:53:10.958 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 8, 2024-04-23T18:53:10.990 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 9.042066166, 2024-04-23T18:53:11.001 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 1.130258270 2024-04-23T18:53:11.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:11.089 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-04-23T18:53:11.140 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:11.174 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:11.260 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:11.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:11.277 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-04-23T18:53:11.283 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 27, 2024-04-23T18:53:11.289 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.144093221, 2024-04-23T18:53:11.297 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.005336785 2024-04-23T18:53:11.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-04-23T18:53:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-04-23T18:53:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000214973, 2024-04-23T18:53:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000214973 2024-04-23T18:53:11.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:11.461 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-04-23T18:53:11.469 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-04-23T18:53:11.503 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.002326669, 2024-04-23T18:53:11.567 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001163334 2024-04-23T18:53:11.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-04-23T18:53:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-04-23T18:53:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.002122247, 2024-04-23T18:53:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001061123 2024-04-23T18:53:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:11.787 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-04-23T18:53:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:11.840 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:11.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:11.855 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-04-23T18:53:11.886 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:11.893 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:11.900 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-04-23T18:53:12.114 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-04-23T18:53:12.169 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.014719702, 2024-04-23T18:53:12.177 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.007359851 2024-04-23T18:53:12.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:12.195 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-04-23T18:53:12.202 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 21, 2024-04-23T18:53:12.209 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.101367492, 2024-04-23T18:53:12.216 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.004827023 2024-04-23T18:53:12.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:12.281 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-04-23T18:53:12.289 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:12.298 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:12.323 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:12.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:12.341 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-04-23T18:53:12.428 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 600, 2024-04-23T18:53:12.503 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 3.532584018, 2024-04-23T18:53:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.005887640 2024-04-23T18:53:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-04-23T18:53:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:12.666 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-04-23T18:53:12.674 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:12.734 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:12.743 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:12.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:12.782 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-04-23T18:53:12.790 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:12.799 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:12.885 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:12.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:13.016 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-04-23T18:53:13.085 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 8, 2024-04-23T18:53:13.094 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.023218472, 2024-04-23T18:53:13.103 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.002902309 2024-04-23T18:53:13.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:13.120 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-04-23T18:53:13.129 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:13.138 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:13.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:13.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:13.220 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-04-23T18:53:13.229 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-04-23T18:53:13.256 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.002309807, 2024-04-23T18:53:13.265 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.002309807 2024-04-23T18:53:13.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:13.355 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:53:13.423 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:53:13.433 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-04-23T18:53:13.501 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:13.511 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:13.519 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:13.526 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:13.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:13.539 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-04-23T18:53:13.546 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 2, 2024-04-23T18:53:13.598 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 6, 2024-04-23T18:53:13.606 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 4, 2024-04-23T18:53:13.613 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 2, 2024-04-23T18:53:13.620 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-04-23T18:53:13.674 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 28, 2024-04-23T18:53:13.682 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 14, 2024-04-23T18:53:13.690 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 1587, 2024-04-23T18:53:13.720 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-04-23T18:53:13.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:13.797 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-23T18:53:13.868 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:53:13.876 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-23T18:53:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:53:13.890 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:53:13.898 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:53:13.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:53:13.915 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:53:13.980 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:53:13.987 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:53:13.997 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:53:14.004 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:53:14.073 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:53:14.081 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:53:14.088 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:53:14.171 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:53:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-23T18:53:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:53:14.323 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-23T18:53:14.331 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:53:14.338 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:53:14.345 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:53:14.352 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:53:14.359 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:53:14.366 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:53:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:53:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-23T18:53:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:53:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:53:14.498 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:53:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-23T18:53:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-23T18:53:14.585 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-23T18:53:14.639 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:53:14.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:53:14.744 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:53:14.762 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:53:14.792 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 388230, 2024-04-23T18:53:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 162, 2024-04-23T18:53:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 264, 2024-04-23T18:53:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 3, 2024-04-23T18:53:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-23T18:53:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-23T18:53:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:53:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-23T18:53:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:53:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-23T18:53:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 8032, 2024-04-23T18:53:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 167, 2024-04-23T18:53:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:53:15.024 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:53:15.030 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-23T18:53:15.103 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-23T18:53:15.112 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 171573, 2024-04-23T18:53:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 1299, 2024-04-23T18:53:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-23T18:53:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-23T18:53:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-23T18:53:15.157 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-23T18:53:15.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:15.227 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-04-23T18:53:15.236 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-04-23T18:53:15.247 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-04-23T18:53:15.279 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 4773, 2024-04-23T18:53:15.290 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 25894128, 2024-04-23T18:53:15.298 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 32, 2024-04-23T18:53:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 4773, 2024-04-23T18:53:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-04-23T18:53:15.468 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4773, 2024-04-23T18:53:15.528 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 29.127530068, 2024-04-23T18:53:15.536 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.006102562 2024-04-23T18:53:15.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:15.554 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-04-23T18:53:15.562 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-04-23T18:53:15.571 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4773, 2024-04-23T18:53:15.580 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 5773 2024-04-23T18:53:15.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:15.661 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 4773, 2024-04-23T18:53:15.668 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 1, 2024-04-23T18:53:15.673 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 4772, 2024-04-23T18:53:15.679 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-04-23T18:53:15.686 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-04-23T18:53:15.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 456, 2024-04-23T18:53:15.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 29, 2024-04-23T18:53:15.798 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 1, 2024-04-23T18:53:15.844 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 2662, 2024-04-23T18:53:15.910 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 301, 2024-04-23T18:53:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-04-23T18:53:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-04-23T18:53:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-04-23T18:53:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-04-23T18:53:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 523, 2024-04-23T18:53:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-04-23T18:53:16.032 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-04-23T18:53:16.041 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-04-23T18:53:16.048 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 0, 2024-04-23T18:53:16.056 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 58, 2024-04-23T18:53:16.123 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:53:16.129 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-04-23T18:53:16.136 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:53:16.181 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-04-23T18:53:16.239 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-04-23T18:53:16.246 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-04-23T18:53:16.300 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:53:16.306 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-04-23T18:53:16.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:53:16.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 599, 2024-04-23T18:53:16.326 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-04-23T18:53:16.332 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-04-23T18:53:16.339 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-04-23T18:53:16.378 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-04-23T18:53:16.384 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-04-23T18:53:16.389 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-04-23T18:53:16.395 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-04-23T18:53:16.428 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-04-23T18:53:16.434 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-04-23T18:53:16.441 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-04-23T18:53:16.509 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-04-23T18:53:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-04-23T18:53:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-04-23T18:53:16.663 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-04-23T18:53:16.669 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-04-23T18:53:16.676 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-04-23T18:53:16.683 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 132, 2024-04-23T18:53:16.690 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-04-23T18:53:16.698 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 47, 2024-04-23T18:53:16.704 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 12, 2024-04-23T18:53:16.745 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 14, 2024-04-23T18:53:16.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 2, 2024-04-23T18:53:16.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-04-23T18:53:16.772 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 1127, 2024-04-23T18:53:16.780 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 0, 2024-04-23T18:53:16.788 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 17 2024-04-23T18:53:16.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:16.832 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-04-23T18:53:16.906 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 1, 2024-04-23T18:53:16.953 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:53:17.000 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-04-23T18:53:17.006 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-04-23T18:53:17.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-04-23T18:53:17.034 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 457 2024-04-23T18:53:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-04-23T18:53:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-04-23T18:53:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 1, 2024-04-23T18:53:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-04-23T18:53:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 1, 2024-04-23T18:53:17.174 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 1, 2024-04-23T18:53:17.186 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 1, 2024-04-23T18:53:17.195 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-04-23T18:53:17.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:17.333 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:53:17.396 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:17.477 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-23T18:53:17.483 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:17.490 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 18112, 2024-04-23T18:53:17.498 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 5606142, 2024-04-23T18:53:17.507 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:17.515 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 18112, 2024-04-23T18:53:17.523 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:53:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:53:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 18112, 2024-04-23T18:53:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 5606142, 2024-04-23T18:53:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:17.663 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:17.672 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:17.726 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:17.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:17.813 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-04-23T18:53:17.877 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:17.884 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-23T18:53:17.891 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:17.899 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:53:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:53:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:53:17.971 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 4773, 2024-04-23T18:53:17.978 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 25894277, 2024-04-23T18:53:17.985 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 4773, 2024-04-23T18:53:17.992 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 25894277, 2024-04-23T18:53:18.024 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:18.031 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:18.038 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:18.045 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:18.220 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-04-23T18:53:18.226 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-04-23T18:53:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:18.247 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:53:18.254 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:53:18.262 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:18.307 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:53:18.315 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 4773, 2024-04-23T18:53:18.321 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 4773, 2024-04-23T18:53:18.328 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 4773, 2024-04-23T18:53:18.357 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 4773, 2024-04-23T18:53:18.364 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:18.371 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:18.435 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:18.497 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:18.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:18.604 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:53:18.610 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:18.616 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-23T18:53:18.623 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:18.629 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 1, 2024-04-23T18:53:18.637 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 149, 2024-04-23T18:53:18.695 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:18.703 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 1, 2024-04-23T18:53:18.710 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:53:18.717 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:53:18.748 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 1, 2024-04-23T18:53:18.756 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 149, 2024-04-23T18:53:18.763 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:18.839 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:18.906 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:18.940 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:19.002 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:19.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:19.016 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x56039974c500": { 2024-04-23T18:53:19.025 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 60, 2024-04-23T18:53:19.033 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-23T18:53:19.045 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:19.054 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 2773, 2024-04-23T18:53:19.125 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 24275022, 2024-04-23T18:53:19.134 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:19.143 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 2773, 2024-04-23T18:53:19.151 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:53:19.185 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:53:19.193 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2661, 2024-04-23T18:53:19.201 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 24274962, 2024-04-23T18:53:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:19.417 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:19.428 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:19.437 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:19.445 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-23T18:53:19.458 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-23T18:53:19.467 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-23T18:53:19.475 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=24 2024-04-23T18:53:19.564 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-23T18:53:19.575 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-23T18:53:19.583 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=27 2024-04-23T18:53:19.593 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=21 2024-04-23T18:53:19.643 DEBUG:tasks.check_counter:Getting stats from e 2024-04-23T18:53:19.654 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-23T18:53:19.663 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:03 smithi155 ceph-mon[33468]: pgmap v853: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:19.727 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:05 smithi155 ceph-mon[33468]: pgmap v854: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:53:19.824 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:06 smithi155 ceph-mon[33468]: osdmap e133: 12 total, 12 up, 12 in 2024-04-23T18:53:20.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:07 smithi155 ceph-mon[33468]: pgmap v856: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:20.493 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:09 smithi155 ceph-mon[33468]: pgmap v857: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:20.549 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:11 smithi155 ceph-mon[33468]: pgmap v858: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:20.558 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:14 smithi155 ceph-mon[33468]: pgmap v859: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:20.565 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:14 smithi155 ceph-mon[33468]: osdmap e134: 12 total, 12 up, 12 in 2024-04-23T18:53:20.571 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:16 smithi155 ceph-mon[33468]: pgmap v861: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-23T18:53:20.578 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:18 smithi155 ceph-mon[33468]: pgmap v862: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:53:20.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:03 smithi019 ceph-mon[29215]: pgmap v853: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:20.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:05 smithi019 ceph-mon[29215]: pgmap v854: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:53:20.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:06 smithi019 ceph-mon[29215]: osdmap e133: 12 total, 12 up, 12 in 2024-04-23T18:53:20.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:07 smithi019 ceph-mon[29215]: pgmap v856: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:20.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:09 smithi019 ceph-mon[29215]: pgmap v857: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:20.726 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:11 smithi019 ceph-mon[29215]: pgmap v858: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:20.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:14 smithi019 ceph-mon[29215]: pgmap v859: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:20.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:14 smithi019 ceph-mon[29215]: osdmap e134: 12 total, 12 up, 12 in 2024-04-23T18:53:20.826 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:16 smithi019 ceph-mon[29215]: pgmap v861: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-23T18:53:20.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:18 smithi019 ceph-mon[29215]: pgmap v862: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:53:20.945 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:05 smithi044 ceph-mon[32458]: pgmap v854: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:53:20.999 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:06 smithi044 ceph-mon[32458]: osdmap e133: 12 total, 12 up, 12 in 2024-04-23T18:53:21.006 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:07 smithi044 ceph-mon[32458]: pgmap v856: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:21.013 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:09 smithi044 ceph-mon[32458]: pgmap v857: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:21.020 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:11 smithi044 ceph-mon[32458]: pgmap v858: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:21.029 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:14 smithi044 ceph-mon[32458]: pgmap v859: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:53:21.036 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:14 smithi044 ceph-mon[32458]: osdmap e134: 12 total, 12 up, 12 in 2024-04-23T18:53:21.043 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:16 smithi044 ceph-mon[32458]: pgmap v861: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-23T18:53:21.049 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:18 smithi044 ceph-mon[32458]: pgmap v862: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:53:21.100 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:20 smithi044 ceph-mon[32458]: pgmap v863: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:53:21.108 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:20 smithi019 ceph-mon[29215]: pgmap v863: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:53:21.116 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:20 smithi155 ceph-mon[33468]: pgmap v863: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:53:21.207 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:53:22.314 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:22 smithi044 ceph-mon[32458]: pgmap v864: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:53:22.324 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:22 smithi155 ceph-mon[33468]: pgmap v864: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:53:22.512 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:22 smithi019 ceph-mon[29215]: pgmap v864: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:53:23.001 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-23T18:53:23.029 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:53:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 493, 2024-04-23T18:53:23.176 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 493, 2024-04-23T18:53:23.184 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 170329, 2024-04-23T18:53:23.193 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 156017, 2024-04-23T18:53:23.202 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 6, 2024-04-23T18:53:23.211 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-04-23T18:53:23.219 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.057315957, 2024-04-23T18:53:23.304 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.022879730, 2024-04-23T18:53:23.312 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.012568227, 2024-04-23T18:53:23.318 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.014590159, 2024-04-23T18:53:23.327 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:53:23.406 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 492, 2024-04-23T18:53:23.414 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.033659101, 2024-04-23T18:53:23.420 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000068412 2024-04-23T18:53:23.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:23.494 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:53:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:23.629 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:23.638 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:23.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:23.651 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-23T18:53:23.657 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T18:53:23.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:23.671 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:53:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1007, 2024-04-23T18:53:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 1107, 2024-04-23T18:53:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 332074, 2024-04-23T18:53:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 583348, 2024-04-23T18:53:23.822 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-04-23T18:53:23.834 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2024-04-23T18:53:23.845 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.078745232, 2024-04-23T18:53:23.930 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.022484494, 2024-04-23T18:53:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.020594127, 2024-04-23T18:53:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.029338934, 2024-04-23T18:53:24.063 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:53:24.072 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1105, 2024-04-23T18:53:24.082 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.038859001, 2024-04-23T18:53:24.090 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000035166 2024-04-23T18:53:24.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:24.107 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:53:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:24.123 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-23T18:53:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 263440 2024-04-23T18:53:24.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:24.273 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:53:24.353 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 157, 2024-04-23T18:53:24.379 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 157, 2024-04-23T18:53:24.412 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 206493, 2024-04-23T18:53:24.486 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 85460, 2024-04-23T18:53:24.494 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-04-23T18:53:24.503 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-04-23T18:53:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.024702964, 2024-04-23T18:53:24.522 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.009595967, 2024-04-23T18:53:24.531 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.015744082, 2024-04-23T18:53:24.539 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.003855742, 2024-04-23T18:53:24.547 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:53:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 156, 2024-04-23T18:53:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.010156231, 2024-04-23T18:53:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000065104 2024-04-23T18:53:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:53:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:24.703 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:24.771 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:24.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:24.879 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 204576, 2024-04-23T18:53:24.885 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 84496 2024-04-23T18:53:24.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:24.899 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-23T18:53:24.908 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-04-23T18:53:24.915 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-23T18:53:24.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:24.991 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-04-23T18:53:24.999 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-23T18:53:25.006 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-23T18:53:25.013 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2483, 2024-04-23T18:53:25.047 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.065309222, 2024-04-23T18:53:25.052 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000026302 2024-04-23T18:53:25.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:25.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:25.170 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-04-23T18:53:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-23T18:53:25.291 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-23T18:53:25.298 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:25.307 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:25.314 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:25.321 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:25.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:25.334 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-04-23T18:53:25.402 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 0, 2024-04-23T18:53:25.409 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 0, 2024-04-23T18:53:25.417 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-04-23T18:53:25.424 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:25.459 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:25.466 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:25.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:25.521 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-04-23T18:53:25.587 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2024-04-23T18:53:25.621 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-04-23T18:53:25.687 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-04-23T18:53:25.694 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 0, 2024-04-23T18:53:25.701 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-04-23T18:53:25.708 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-04-23T18:53:25.715 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 8, 2024-04-23T18:53:25.722 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 1, 2024-04-23T18:53:25.729 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 7, 2024-04-23T18:53:25.759 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-04-23T18:53:25.767 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 8, 2024-04-23T18:53:25.774 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 0, 2024-04-23T18:53:25.781 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2024-04-23T18:53:25.826 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2024-04-23T18:53:25.834 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 2, 2024-04-23T18:53:25.840 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 0, 2024-04-23T18:53:25.887 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 0, 2024-04-23T18:53:25.945 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2024-04-23T18:53:25.971 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2024-04-23T18:53:26.023 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:53:26.033 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-04-23T18:53:26.042 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-04-23T18:53:26.051 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-04-23T18:53:26.060 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-04-23T18:53:26.069 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-04-23T18:53:26.079 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-04-23T18:53:26.139 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-04-23T18:53:26.149 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-04-23T18:53:26.157 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-04-23T18:53:26.166 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:53:26.205 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-04-23T18:53:26.211 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 4, 2024-04-23T18:53:26.220 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 234, 2024-04-23T18:53:26.270 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 6, 2024-04-23T18:53:26.370 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:53:26.396 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-04-23T18:53:26.453 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:53:26.462 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:53:26.470 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:53:26.480 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:53:26.488 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:53:26.497 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-04-23T18:53:26.505 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:53:26.513 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:53:26.571 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:53:26.580 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:53:26.588 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:53:26.620 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:53:26.629 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2024-04-23T18:53:26.637 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:53:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-04-23T18:53:26.703 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2024-04-23T18:53:26.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:26.788 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-04-23T18:53:26.797 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-04-23T18:53:26.806 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-04-23T18:53:26.815 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-04-23T18:53:26.824 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2024-04-23T18:53:26.833 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-04-23T18:53:26.841 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-04-23T18:53:26.850 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-04-23T18:53:26.913 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:53:26.922 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-04-23T18:53:26.930 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-04-23T18:53:26.939 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-04-23T18:53:27.013 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-04-23T18:53:27.021 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-04-23T18:53:27.078 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:53:27.145 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:53:27.180 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-04-23T18:53:27.267 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-04-23T18:53:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:53:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:53:27.288 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:53:27.294 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-04-23T18:53:27.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-04-23T18:53:27.309 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:53:27.363 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-04-23T18:53:27.374 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-04-23T18:53:27.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:27.390 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-04-23T18:53:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-04-23T18:53:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 0, 2024-04-23T18:53:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-04-23T18:53:27.537 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-04-23T18:53:27.620 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 80, 2024-04-23T18:53:27.663 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-04-23T18:53:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-04-23T18:53:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 0, 2024-04-23T18:53:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-04-23T18:53:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-04-23T18:53:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-04-23T18:53:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 4, 2024-04-23T18:53:27.768 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-04-23T18:53:27.818 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-04-23T18:53:27.825 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 28395630, 2024-04-23T18:53:27.832 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 28469438, 2024-04-23T18:53:27.839 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 28469326, 2024-04-23T18:53:27.846 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-04-23T18:53:27.854 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:27.888 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:28.104 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 80 2024-04-23T18:53:28.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:28.175 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-04-23T18:53:28.182 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 9, 2024-04-23T18:53:28.189 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 9, 2024-04-23T18:53:28.195 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 0, 2024-04-23T18:53:28.203 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 7, 2024-04-23T18:53:28.211 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 7, 2024-04-23T18:53:28.220 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 0, 2024-04-23T18:53:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 8, 2024-04-23T18:53:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 8, 2024-04-23T18:53:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 0, 2024-04-23T18:53:28.354 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2024-04-23T18:53:28.365 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 0, 2024-04-23T18:53:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2024-04-23T18:53:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 50028, 2024-04-23T18:53:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 215416 2024-04-23T18:53:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-04-23T18:53:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 0, 2024-04-23T18:53:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-04-23T18:53:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 0, 2024-04-23T18:53:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 0, 2024-04-23T18:53:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2024-04-23T18:53:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-04-23T18:53:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:28.820 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:28.829 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:28.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:28.884 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-04-23T18:53:28.892 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:28.971 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:29.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:29.132 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-04-23T18:53:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-04-23T18:53:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-04-23T18:53:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-04-23T18:53:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-04-23T18:53:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:29.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-04-23T18:53:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-04-23T18:53:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-04-23T18:53:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-04-23T18:53:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-04-23T18:53:30.293 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:31.023 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:24 smithi019 ceph-mon[29215]: pgmap v865: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:53:31.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:26 smithi019 ceph-mon[29215]: pgmap v866: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 88 B/s wr, 13 op/s 2024-04-23T18:53:31.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:28 smithi019 ceph-mon[29215]: pgmap v867: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:31.218 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:30 smithi019 ceph-mon[29215]: pgmap v868: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:31.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:24 smithi155 ceph-mon[33468]: pgmap v865: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:53:31.416 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:26 smithi155 ceph-mon[33468]: pgmap v866: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 88 B/s wr, 13 op/s 2024-04-23T18:53:31.428 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:28 smithi155 ceph-mon[33468]: pgmap v867: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:31.436 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:30 smithi155 ceph-mon[33468]: pgmap v868: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:31.445 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:31.454 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:31.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:31.472 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-04-23T18:53:31.550 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:31.558 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:31.569 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:31.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:31.648 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-04-23T18:53:31.656 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:31.665 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:31.775 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:31.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-04-23T18:53:32.028 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:32.035 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:32.042 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:32.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:32.056 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-04-23T18:53:32.064 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:32.071 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:32.122 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:32.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:32.139 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-04-23T18:53:32.145 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:32.187 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:32.195 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:32.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:32.255 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-04-23T18:53:32.277 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:32.328 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:32.405 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:32.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:32.420 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-04-23T18:53:32.427 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:32.434 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:32.441 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:32.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:32.486 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-04-23T18:53:32.495 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:32.502 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:32.509 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-04-23T18:53:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:32.620 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:32.660 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:32.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:32.778 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-04-23T18:53:32.786 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:32.792 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:32.799 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:32.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:32.814 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-04-23T18:53:32.822 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:32.867 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:32.873 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:32.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:32.887 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-04-23T18:53:32.927 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:32.934 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:32.978 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:32.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:33.007 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-04-23T18:53:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-04-23T18:53:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:33.166 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:33.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:33.238 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-04-23T18:53:33.250 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:33.259 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:33.303 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:33.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:33.412 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-04-23T18:53:33.420 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:33.469 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:33.518 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:33.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:33.578 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-04-23T18:53:33.587 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:33.597 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:33.605 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:33.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:33.620 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:53:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:53:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-04-23T18:53:33.710 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:33.718 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:33.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:33.845 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-04-23T18:53:33.861 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 2, 2024-04-23T18:53:33.889 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 2, 2024-04-23T18:53:33.965 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 0, 2024-04-23T18:53:33.972 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 0, 2024-04-23T18:53:33.978 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-04-23T18:53:33.987 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 0, 2024-04-23T18:53:33.996 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2024-04-23T18:53:34.004 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 2867, 2024-04-23T18:53:34.033 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-04-23T18:53:34.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:34.083 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-23T18:53:34.089 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:53:34.098 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-23T18:53:34.144 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:53:34.150 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:53:34.206 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:53:34.217 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:53:34.266 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:53:34.321 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:53:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:53:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:53:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:53:34.413 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:53:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:53:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:53:34.434 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:53:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-23T18:53:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:53:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-23T18:53:34.520 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:53:34.587 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:53:34.593 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:53:34.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:53:34.735 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:53:34.742 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:53:34.771 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:53:34.851 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-23T18:53:34.858 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:53:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:53:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:53:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-23T18:53:34.885 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-23T18:53:34.894 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-23T18:53:34.902 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:53:34.961 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:53:34.968 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:53:34.976 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:53:35.019 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 197467, 2024-04-23T18:53:35.028 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 56, 2024-04-23T18:53:35.121 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:53:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-04-23T18:53:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-23T18:53:35.238 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-23T18:53:35.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:53:35.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-23T18:53:35.329 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:53:35.338 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-23T18:53:35.346 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 8032, 2024-04-23T18:53:35.355 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 167, 2024-04-23T18:53:35.364 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:53:35.370 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:53:35.432 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-23T18:53:35.441 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-23T18:53:35.452 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 52760, 2024-04-23T18:53:35.524 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 301, 2024-04-23T18:53:35.530 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-23T18:53:35.540 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-23T18:53:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-23T18:53:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-23T18:53:35.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:35.781 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-04-23T18:53:35.787 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-04-23T18:53:35.795 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-04-23T18:53:35.803 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 1498, 2024-04-23T18:53:35.810 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 26, 2024-04-23T18:53:35.816 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 4, 2024-04-23T18:53:35.823 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 1498, 2024-04-23T18:53:35.831 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-04-23T18:53:35.903 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1498, 2024-04-23T18:53:35.911 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.846986450, 2024-04-23T18:53:35.918 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000565411 2024-04-23T18:53:35.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:36.001 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-04-23T18:53:36.104 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-04-23T18:53:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1498, 2024-04-23T18:53:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 1499 2024-04-23T18:53:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 1498, 2024-04-23T18:53:36.302 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 1498, 2024-04-23T18:53:36.310 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 0, 2024-04-23T18:53:36.319 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-04-23T18:53:36.328 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-04-23T18:53:36.336 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 970, 2024-04-23T18:53:36.346 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 0, 2024-04-23T18:53:36.426 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 525, 2024-04-23T18:53:36.434 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 0, 2024-04-23T18:53:36.442 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 0, 2024-04-23T18:53:36.449 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-04-23T18:53:36.502 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-04-23T18:53:36.512 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-04-23T18:53:36.587 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-04-23T18:53:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 0, 2024-04-23T18:53:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-04-23T18:53:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-04-23T18:53:36.779 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-04-23T18:53:36.785 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 2, 2024-04-23T18:53:36.792 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 0, 2024-04-23T18:53:36.800 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:53:36.807 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-04-23T18:53:36.813 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:53:36.820 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-04-23T18:53:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-04-23T18:53:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-04-23T18:53:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:53:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-04-23T18:53:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:53:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-04-23T18:53:36.983 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-04-23T18:53:37.019 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-04-23T18:53:37.079 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-04-23T18:53:37.085 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-04-23T18:53:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-04-23T18:53:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-04-23T18:53:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-04-23T18:53:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-04-23T18:53:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-04-23T18:53:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-04-23T18:53:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-04-23T18:53:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-04-23T18:53:37.269 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-04-23T18:53:37.276 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-04-23T18:53:37.283 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-04-23T18:53:37.318 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-04-23T18:53:37.325 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 134, 2024-04-23T18:53:37.401 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-04-23T18:53:37.459 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 16, 2024-04-23T18:53:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 6, 2024-04-23T18:53:37.501 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 6, 2024-04-23T18:53:37.582 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 0, 2024-04-23T18:53:37.589 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-04-23T18:53:37.598 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 0, 2024-04-23T18:53:37.607 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 2, 2024-04-23T18:53:37.615 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 0 2024-04-23T18:53:37.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:37.633 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-04-23T18:53:37.690 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 0, 2024-04-23T18:53:37.700 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:53:37.709 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-04-23T18:53:37.716 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-04-23T18:53:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-04-23T18:53:37.879 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-04-23T18:53:37.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:38.005 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-04-23T18:53:38.014 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-04-23T18:53:38.073 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:53:38.079 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-04-23T18:53:38.087 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-04-23T18:53:38.097 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-04-23T18:53:38.105 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-04-23T18:53:38.114 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-04-23T18:53:38.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:53:38.216 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:38.224 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-23T18:53:38.233 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 1653, 2024-04-23T18:53:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 549225, 2024-04-23T18:53:38.403 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:38.488 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 1653, 2024-04-23T18:53:38.528 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:53:38.536 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:53:38.600 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 1653, 2024-04-23T18:53:38.606 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 549225, 2024-04-23T18:53:38.619 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:38.626 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:38.633 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:38.640 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:38.647 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:38.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:38.686 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-04-23T18:53:38.693 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:38.700 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-23T18:53:38.758 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:38.764 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:53:38.836 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:53:38.868 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:38.928 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:53:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 1498, 2024-04-23T18:53:39.007 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 76666, 2024-04-23T18:53:39.013 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 1498, 2024-04-23T18:53:39.020 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 76666, 2024-04-23T18:53:39.027 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:39.034 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:39.041 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:39.048 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:39.101 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:39.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:39.116 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-04-23T18:53:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:39.203 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-04-23T18:53:39.209 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:53:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:53:39.351 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:39.357 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:53:39.412 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 1498, 2024-04-23T18:53:39.418 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 1498, 2024-04-23T18:53:39.425 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 1498, 2024-04-23T18:53:39.432 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1498, 2024-04-23T18:53:39.439 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:39.445 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:39.452 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:39.459 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:53:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:39.603 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-23T18:53:39.669 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:39.731 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:53:39.776 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:53:39.785 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:39.847 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:53:39.853 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:53:39.862 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:53:39.870 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-04-23T18:53:39.880 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-04-23T18:53:39.888 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:39.955 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:39.964 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:39.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:39.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:40.034 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x55b130df6640": { 2024-04-23T18:53:40.042 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:40.110 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-23T18:53:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:40.278 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:53:40.286 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:53:40.359 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:40.367 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:53:40.376 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:53:40.385 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:53:40.394 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-04-23T18:53:40.403 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-04-23T18:53:40.413 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:40.485 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:40.492 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:40.501 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:40.512 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:40.583 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-23T18:53:40.673 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-23T18:53:40.704 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-23T18:53:40.728 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-23T18:53:40.735 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-23T18:53:40.808 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-23T18:53:40.815 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:53:40.824 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:53:40.834 DEBUG:tasks.check_counter:Getting stats from h 2024-04-23T18:53:40.841 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-23T18:53:40.851 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:24 smithi044 ceph-mon[32458]: pgmap v865: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:53:40.858 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:26 smithi044 ceph-mon[32458]: pgmap v866: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 88 B/s wr, 13 op/s 2024-04-23T18:53:40.882 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:28 smithi044 ceph-mon[32458]: pgmap v867: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:40.892 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:30 smithi044 ceph-mon[32458]: pgmap v868: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:40.897 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:32 smithi044 ceph-mon[32458]: pgmap v869: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:40.904 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:34 smithi044 ceph-mon[32458]: pgmap v870: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:40.942 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:36 smithi044 ceph-mon[32458]: pgmap v871: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:40.949 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:38 smithi044 ceph-mon[32458]: pgmap v872: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:40.998 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:40 smithi044 ceph-mon[32458]: pgmap v873: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:41.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:32 smithi019 ceph-mon[29215]: pgmap v869: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:41.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:34 smithi019 ceph-mon[29215]: pgmap v870: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:41.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:36 smithi019 ceph-mon[29215]: pgmap v871: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:41.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:38 smithi019 ceph-mon[29215]: pgmap v872: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:41.228 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:40 smithi019 ceph-mon[29215]: pgmap v873: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:41.234 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:32 smithi155 ceph-mon[33468]: pgmap v869: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:41.239 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:34 smithi155 ceph-mon[33468]: pgmap v870: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:41.249 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:36 smithi155 ceph-mon[33468]: pgmap v871: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:41.255 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:38 smithi155 ceph-mon[33468]: pgmap v872: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:41.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:40 smithi155 ceph-mon[33468]: pgmap v873: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:41.354 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:53:42.333 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:42 smithi019 ceph-mon[29215]: pgmap v874: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:42.469 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:42 smithi044 ceph-mon[32458]: pgmap v874: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:42.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:42 smithi155 ceph-mon[33468]: pgmap v874: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:43.105 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-23T18:53:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:53:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2314, 2024-04-23T18:53:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 1838, 2024-04-23T18:53:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 648409, 2024-04-23T18:53:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 644784, 2024-04-23T18:53:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 9, 2024-04-23T18:53:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-04-23T18:53:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.227098507, 2024-04-23T18:53:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.107510756, 2024-04-23T18:53:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.121826448, 2024-04-23T18:53:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.010146155, 2024-04-23T18:53:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:53:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1841, 2024-04-23T18:53:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.099486895, 2024-04-23T18:53:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000054039 2024-04-23T18:53:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:53:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3925, 2024-04-23T18:53:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001250196, 2024-04-23T18:53:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000318 2024-04-23T18:53:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-23T18:53:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T18:53:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:53:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 21, 2024-04-23T18:53:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 385, 2024-04-23T18:53:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 5198, 2024-04-23T18:53:43.687 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 913592, 2024-04-23T18:53:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 11, 2024-04-23T18:53:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2024-04-23T18:53:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.024631835, 2024-04-23T18:53:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.015964843, 2024-04-23T18:53:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.004102568, 2024-04-23T18:53:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000325595, 2024-04-23T18:53:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:53:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 382, 2024-04-23T18:53:44.021 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.041386393, 2024-04-23T18:53:44.030 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000108341 2024-04-23T18:53:44.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:44.048 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:53:44.057 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 8, 2024-04-23T18:53:44.065 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000001327, 2024-04-23T18:53:44.073 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000165 2024-04-23T18:53:44.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:44.090 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1408, 2024-04-23T18:53:44.098 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 901488 2024-04-23T18:53:44.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:44.179 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:53:44.283 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 624, 2024-04-23T18:53:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 703, 2024-04-23T18:53:44.403 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1774864, 2024-04-23T18:53:44.412 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 414610, 2024-04-23T18:53:44.482 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 15, 2024-04-23T18:53:44.515 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 6, 2024-04-23T18:53:44.522 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.126189802, 2024-04-23T18:53:44.531 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.044573884, 2024-04-23T18:53:44.539 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.071122552, 2024-04-23T18:53:44.548 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.013105738, 2024-04-23T18:53:44.557 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:53:44.565 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 698, 2024-04-23T18:53:44.574 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.038477643, 2024-04-23T18:53:44.583 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000055125 2024-04-23T18:53:44.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:44.677 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:53:44.686 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 160, 2024-04-23T18:53:44.756 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000062525, 2024-04-23T18:53:44.808 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000390 2024-04-23T18:53:44.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:44.872 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1772608, 2024-04-23T18:53:44.923 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 340800 2024-04-23T18:53:44.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:44.936 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-23T18:53:44.942 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-04-23T18:53:44.972 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-23T18:53:44.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:44.986 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-04-23T18:53:44.995 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-23T18:53:45.001 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-23T18:53:45.008 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 56, 2024-04-23T18:53:45.051 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001748537, 2024-04-23T18:53:45.058 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000031223 2024-04-23T18:53:45.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:45.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:45.241 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-04-23T18:53:45.303 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-23T18:53:45.311 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-23T18:53:45.382 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-04-23T18:53:45.406 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000008050, 2024-04-23T18:53:45.413 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000004025 2024-04-23T18:53:45.418 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:45.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:45.431 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-04-23T18:53:45.438 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 4, 2024-04-23T18:53:45.444 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 2, 2024-04-23T18:53:45.451 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-04-23T18:53:45.458 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-04-23T18:53:45.515 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000312463, 2024-04-23T18:53:45.522 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000156231 2024-04-23T18:53:45.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:45.631 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-04-23T18:53:45.683 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 2, 2024-04-23T18:53:45.760 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-04-23T18:53:45.767 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-04-23T18:53:45.798 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 11, 2024-04-23T18:53:45.805 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-04-23T18:53:45.888 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-04-23T18:53:45.895 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 17, 2024-04-23T18:53:45.901 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 1, 2024-04-23T18:53:45.919 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 16, 2024-04-23T18:53:45.936 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-04-23T18:53:45.956 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 13, 2024-04-23T18:53:45.984 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 4, 2024-04-23T18:53:46.015 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 1, 2024-04-23T18:53:46.036 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 1, 2024-04-23T18:53:46.046 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 3, 2024-04-23T18:53:46.057 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 40, 2024-04-23T18:53:46.069 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 31, 2024-04-23T18:53:46.167 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 1, 2024-04-23T18:53:46.215 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 7, 2024-04-23T18:53:46.224 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:53:46.272 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-04-23T18:53:46.279 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-04-23T18:53:46.324 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-04-23T18:53:46.335 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-04-23T18:53:46.344 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-04-23T18:53:46.353 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-04-23T18:53:46.364 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-04-23T18:53:46.370 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-04-23T18:53:46.378 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-04-23T18:53:46.387 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:53:46.448 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-04-23T18:53:46.456 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 5, 2024-04-23T18:53:46.466 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 298, 2024-04-23T18:53:46.558 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 6, 2024-04-23T18:53:46.565 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:53:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-04-23T18:53:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:53:46.748 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:53:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:53:46.767 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:53:46.776 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:53:46.808 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-04-23T18:53:46.814 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:53:46.823 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 1, 2024-04-23T18:53:46.831 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:53:46.839 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:53:46.848 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:53:46.921 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:53:46.930 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 1, 2024-04-23T18:53:46.939 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:53:47.005 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-04-23T18:53:47.012 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 1 2024-04-23T18:53:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-04-23T18:53:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-04-23T18:53:47.160 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 16, 2024-04-23T18:53:47.167 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 11, 2024-04-23T18:53:47.176 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 12, 2024-04-23T18:53:47.187 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-04-23T18:53:47.196 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-04-23T18:53:47.204 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-04-23T18:53:47.211 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:53:47.220 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-04-23T18:53:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-04-23T18:53:47.287 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-04-23T18:53:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-04-23T18:53:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-04-23T18:53:47.386 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:53:47.393 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:53:47.528 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-04-23T18:53:47.534 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-04-23T18:53:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 12, 2024-04-23T18:53:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 71, 2024-04-23T18:53:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:53:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-04-23T18:53:47.672 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-04-23T18:53:47.679 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:53:47.687 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-04-23T18:53:47.694 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-04-23T18:53:47.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:47.770 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-04-23T18:53:47.778 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-04-23T18:53:47.785 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 7, 2024-04-23T18:53:47.793 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-04-23T18:53:47.864 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-04-23T18:53:47.871 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 7, 2024-04-23T18:53:47.953 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-04-23T18:53:47.960 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-04-23T18:53:47.992 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 2, 2024-04-23T18:53:48.033 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-04-23T18:53:48.041 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-04-23T18:53:48.048 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-04-23T18:53:48.055 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-04-23T18:53:48.062 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-04-23T18:53:48.069 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-04-23T18:53:48.076 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-04-23T18:53:48.083 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 4196264, 2024-04-23T18:53:48.134 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 0, 2024-04-23T18:53:48.143 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-04-23T18:53:48.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6, 2024-04-23T18:53:48.156 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.011566698, 2024-04-23T18:53:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001927783 2024-04-23T18:53:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:48.287 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 0 2024-04-23T18:53:48.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:48.356 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-04-23T18:53:48.392 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 21, 2024-04-23T18:53:48.401 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 23, 2024-04-23T18:53:48.409 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 2, 2024-04-23T18:53:48.418 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 20, 2024-04-23T18:53:48.427 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 23, 2024-04-23T18:53:48.437 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 3, 2024-04-23T18:53:48.446 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 17, 2024-04-23T18:53:48.456 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 21, 2024-04-23T18:53:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 4, 2024-04-23T18:53:48.510 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 1, 2024-04-23T18:53:48.519 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 2, 2024-04-23T18:53:48.528 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 1, 2024-04-23T18:53:48.594 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 52912, 2024-04-23T18:53:48.600 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 223608 2024-04-23T18:53:48.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:48.682 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-04-23T18:53:48.722 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 5, 2024-04-23T18:53:48.751 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-04-23T18:53:48.760 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 4, 2024-04-23T18:53:48.769 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 30, 2024-04-23T18:53:48.777 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 71, 2024-04-23T18:53:48.787 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-04-23T18:53:48.796 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:48.804 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:48.850 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:48.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:48.869 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-04-23T18:53:48.877 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-04-23T18:53:48.885 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000180877, 2024-04-23T18:53:48.956 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000180877 2024-04-23T18:53:48.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:49.031 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-04-23T18:53:49.037 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:49.105 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:49.136 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:49.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:49.153 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-04-23T18:53:49.161 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:49.168 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:49.176 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:49.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:49.249 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-04-23T18:53:49.258 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:49.266 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:49.273 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:49.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:49.350 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-04-23T18:53:49.358 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:49.422 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:49.428 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:49.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:49.496 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-04-23T18:53:49.503 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-04-23T18:53:49.511 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000131586, 2024-04-23T18:53:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000131586 2024-04-23T18:53:49.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:49.531 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-04-23T18:53:49.537 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:49.565 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:49.572 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:49.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:49.586 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-04-23T18:53:49.593 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:49.669 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:49.676 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:49.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:49.754 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-04-23T18:53:49.830 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:49.867 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:49.874 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:49.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:49.888 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-04-23T18:53:49.895 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:49.901 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:49.908 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:49.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:49.964 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-04-23T18:53:49.974 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:44 smithi155 ceph-mon[33468]: pgmap v875: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:49.981 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:44 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:53:49.988 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:46 smithi155 ceph-mon[33468]: pgmap v876: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:50.056 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:47 smithi155 ceph-mon[33468]: pgmap v877: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:50.065 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:49 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:53:50.072 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:49 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:53:50.187 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:49 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:53:50.225 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:49 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:53:50.278 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:49 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:53:50.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:49 smithi155 ceph-mon[33468]: pgmap v878: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:50.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:44 smithi019 ceph-mon[29215]: pgmap v875: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:50.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:44 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:53:50.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:46 smithi019 ceph-mon[29215]: pgmap v876: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:50.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:47 smithi019 ceph-mon[29215]: pgmap v877: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:50.326 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:49 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:53:50.335 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:49 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:53:50.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:49 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:53:50.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:49 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:53:50.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:49 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:53:50.413 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:49 smithi019 ceph-mon[29215]: pgmap v878: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:50.498 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:50.505 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:50.620 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:50.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:50.705 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-04-23T18:53:50.730 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:50.740 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:50.750 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:50.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:50.767 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-04-23T18:53:50.775 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:50.783 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:50.791 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:50.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:50.851 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-04-23T18:53:50.859 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:50.868 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:50.925 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:50.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:50.988 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-04-23T18:53:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:51.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:51.178 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-04-23T18:53:51.185 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:51.191 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:51.198 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:51.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:51.263 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-04-23T18:53:51.270 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:51.277 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:51.284 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:51.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:51.371 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-04-23T18:53:51.379 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:51.434 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:51.448 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:51.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:51.530 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-04-23T18:53:51.537 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:51.544 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:51.551 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:51.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:51.565 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-04-23T18:53:51.572 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:51.617 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:51.625 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:51.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:51.638 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-04-23T18:53:51.645 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:51.730 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:51.736 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-04-23T18:53:51.842 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-04-23T18:53:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:51.932 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:51.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:51.998 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-04-23T18:53:52.005 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:52.014 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:52.021 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:52.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:52.092 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-04-23T18:53:52.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:52.159 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:52.183 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:52.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:52.232 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-04-23T18:53:52.241 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:52.251 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:52.260 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:52.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:52.277 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-04-23T18:53:52.323 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:52.332 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:52.340 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:52.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:52.356 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-04-23T18:53:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:52.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:52.517 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:53:52.565 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:53:52.576 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-04-23T18:53:52.585 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:52.594 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:52.604 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:52.613 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:52.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:52.628 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-04-23T18:53:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2024-04-23T18:53:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 3, 2024-04-23T18:53:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 2, 2024-04-23T18:53:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 1, 2024-04-23T18:53:52.797 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-04-23T18:53:52.807 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 1, 2024-04-23T18:53:52.855 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2024-04-23T18:53:52.863 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 1652, 2024-04-23T18:53:52.877 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-04-23T18:53:52.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:52.937 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-23T18:53:52.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:53:52.953 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-23T18:53:52.961 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:53:52.971 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:53:52.980 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:53:53.016 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:53:53.024 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:53:53.031 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:53:53.038 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:53:53.045 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:53:53.099 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:53:53.107 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:53:53.179 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:53:53.186 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:53:53.209 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:53:53.258 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-23T18:53:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:53:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-23T18:53:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:53:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:53:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:53:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:53:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:53:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:53:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:53:53.358 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-23T18:53:53.365 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:53:53.435 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:53:53.443 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:53:53.510 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-23T18:53:53.516 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-23T18:53:53.573 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-23T18:53:53.600 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:53:53.608 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:53:53.615 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:53:53.622 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:53:53.629 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 238999, 2024-04-23T18:53:53.635 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 76, 2024-04-23T18:53:53.641 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:53:53.696 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-04-23T18:53:53.703 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-23T18:53:53.709 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-23T18:53:53.717 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:53:53.724 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-23T18:53:53.809 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:53:53.818 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-23T18:53:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 8032, 2024-04-23T18:53:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 167, 2024-04-23T18:53:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:53:53.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:53:53.957 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-23T18:53:53.964 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-23T18:53:53.972 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 88117, 2024-04-23T18:53:53.980 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 357, 2024-04-23T18:53:53.990 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-23T18:53:53.997 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-23T18:53:54.068 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-23T18:53:54.077 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-23T18:53:54.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:54.094 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-04-23T18:53:54.103 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-04-23T18:53:54.156 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-04-23T18:53:54.180 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 43, 2024-04-23T18:53:54.241 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 12352, 2024-04-23T18:53:54.248 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 33, 2024-04-23T18:53:54.259 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 43, 2024-04-23T18:53:54.284 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-04-23T18:53:54.293 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 43, 2024-04-23T18:53:54.304 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.233996281, 2024-04-23T18:53:54.315 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.005441773 2024-04-23T18:53:54.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:54.329 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-04-23T18:53:54.337 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-04-23T18:53:54.407 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 43, 2024-04-23T18:53:54.419 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 65 2024-04-23T18:53:54.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:54.438 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 43, 2024-04-23T18:53:54.446 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 0, 2024-04-23T18:53:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 43, 2024-04-23T18:53:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-04-23T18:53:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-04-23T18:53:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 0, 2024-04-23T18:53:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 10, 2024-04-23T18:53:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 0, 2024-04-23T18:53:54.638 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 6, 2024-04-23T18:53:54.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 8, 2024-04-23T18:53:54.653 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-04-23T18:53:54.661 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-04-23T18:53:54.670 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-04-23T18:53:54.679 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-04-23T18:53:54.704 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 6, 2024-04-23T18:53:54.714 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-04-23T18:53:54.723 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-04-23T18:53:54.732 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-04-23T18:53:54.742 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 0, 2024-04-23T18:53:54.803 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 20, 2024-04-23T18:53:54.811 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:53:54.871 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-04-23T18:53:54.879 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:53:54.910 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-04-23T18:53:54.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-04-23T18:53:54.945 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-04-23T18:53:54.954 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:53:54.962 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-04-23T18:53:54.970 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:53:54.979 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-04-23T18:53:54.988 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-04-23T18:53:54.996 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-04-23T18:53:55.046 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-04-23T18:53:55.052 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-04-23T18:53:55.062 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-04-23T18:53:55.069 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-04-23T18:53:55.126 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-04-23T18:53:55.131 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-04-23T18:53:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-04-23T18:53:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-04-23T18:53:55.270 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-04-23T18:53:55.315 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-04-23T18:53:55.322 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-04-23T18:53:55.329 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-04-23T18:53:55.337 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-04-23T18:53:55.343 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-04-23T18:53:55.351 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 134, 2024-04-23T18:53:55.358 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-04-23T18:53:55.365 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 48, 2024-04-23T18:53:55.418 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 8, 2024-04-23T18:53:55.426 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 12, 2024-04-23T18:53:55.433 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 4, 2024-04-23T18:53:55.440 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-04-23T18:53:55.484 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 14, 2024-04-23T18:53:55.491 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 0, 2024-04-23T18:53:55.557 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 1 2024-04-23T18:53:55.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:55.584 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-04-23T18:53:55.627 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 1, 2024-04-23T18:53:55.634 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:53:55.641 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-04-23T18:53:55.647 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-04-23T18:53:55.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:55.660 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-04-23T18:53:55.667 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 82 2024-04-23T18:53:55.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:55.721 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-04-23T18:53:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-04-23T18:53:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:53:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-04-23T18:53:55.815 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-04-23T18:53:55.822 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-04-23T18:53:55.870 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-04-23T18:53:55.877 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-04-23T18:53:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:55.995 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:53:56.003 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:56.009 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-23T18:53:56.018 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:56.030 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 2955, 2024-04-23T18:53:56.037 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 2128396, 2024-04-23T18:53:56.045 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:56.111 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 2955, 2024-04-23T18:53:56.121 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:53:56.128 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:53:56.138 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2955, 2024-04-23T18:53:56.147 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 2128396, 2024-04-23T18:53:56.214 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:56.224 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:56.296 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:56.306 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:56.342 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-04-23T18:53:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-23T18:53:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:53:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:53:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:56.500 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:53:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 43, 2024-04-23T18:53:56.519 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 12352, 2024-04-23T18:53:56.528 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 43, 2024-04-23T18:53:56.586 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 12352, 2024-04-23T18:53:56.603 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:56.769 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:56.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:56.789 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-04-23T18:53:56.798 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:56.807 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-04-23T18:53:56.815 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:56.824 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:53:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:53:56.889 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:56.897 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:53:56.906 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 43, 2024-04-23T18:53:56.914 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 43, 2024-04-23T18:53:57.028 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 43, 2024-04-23T18:53:57.035 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 43, 2024-04-23T18:53:57.096 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:57.103 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:53:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-23T18:53:57.218 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:57.279 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:53:57.286 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:53:57.297 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:57.304 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:53:57.312 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:53:57.353 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:53:57.395 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-04-23T18:53:57.454 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-04-23T18:53:57.461 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:57.469 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:57.502 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:57.553 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:57.560 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:57.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:53:57.574 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x55ad1b85a500": { 2024-04-23T18:53:57.581 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:53:57.588 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-23T18:53:57.642 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:53:57.649 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 7, 2024-04-23T18:53:57.656 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1960, 2024-04-23T18:53:57.662 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:53:57.669 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 7, 2024-04-23T18:53:57.744 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:53:57.750 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:53:57.811 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 6, 2024-04-23T18:53:57.822 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1960, 2024-04-23T18:53:57.859 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:53:57.900 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:53:57.907 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:53:57.913 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:53:57.921 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:57.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:53:57.935 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-23T18:53:57.943 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-23T18:53:58.006 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-23T18:53:58.013 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-23T18:53:58.021 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=16 2024-04-23T18:53:58.029 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-23T18:53:58.036 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:53:58.088 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:53:58.095 DEBUG:tasks.check_counter:Getting stats from k 2024-04-23T18:53:58.200 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-23T18:53:58.209 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:44 smithi044 ceph-mon[32458]: pgmap v875: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:58.286 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:44 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:53:58.327 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:46 smithi044 ceph-mon[32458]: pgmap v876: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:58.336 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:47 smithi044 ceph-mon[32458]: pgmap v877: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:58.346 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:49 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:53:58.354 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:49 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:53:58.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:49 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:53:58.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:49 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:53:58.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:49 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:53:58.403 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:49 smithi044 ceph-mon[32458]: pgmap v878: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:58.412 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:51 smithi044 ceph-mon[32458]: pgmap v879: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:58.423 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:53 smithi044 ceph-mon[32458]: pgmap v880: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:58.432 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:55 smithi044 ceph-mon[32458]: pgmap v881: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:58.440 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:57 smithi044 ceph-mon[32458]: pgmap v882: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:58.517 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:51 smithi155 ceph-mon[33468]: pgmap v879: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:58.525 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:53 smithi155 ceph-mon[33468]: pgmap v880: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:58.623 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:55 smithi155 ceph-mon[33468]: pgmap v881: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:58.634 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:57 smithi155 ceph-mon[33468]: pgmap v882: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:58.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:51 smithi019 ceph-mon[29215]: pgmap v879: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:58.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:53 smithi019 ceph-mon[29215]: pgmap v880: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:53:58.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:55 smithi019 ceph-mon[29215]: pgmap v881: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:53:58.729 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:57 smithi019 ceph-mon[29215]: pgmap v882: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:58.776 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:53:59.910 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:53:59 smithi044 ceph-mon[32458]: pgmap v883: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:00.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:53:59 smithi019 ceph-mon[29215]: pgmap v883: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:00.074 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:53:59 smithi155 ceph-mon[33468]: pgmap v883: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:00.475 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-23T18:54:00.483 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:54:00.490 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2352, 2024-04-23T18:54:00.497 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2336, 2024-04-23T18:54:00.589 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 2352064, 2024-04-23T18:54:00.597 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 881976, 2024-04-23T18:54:00.606 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-04-23T18:54:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-04-23T18:54:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.305763261, 2024-04-23T18:54:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.118742689, 2024-04-23T18:54:00.769 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.099747830, 2024-04-23T18:54:00.776 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.068720766, 2024-04-23T18:54:00.783 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:00.791 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2335, 2024-04-23T18:54:00.801 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.126417469, 2024-04-23T18:54:00.808 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000054140 2024-04-23T18:54:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1748608, 2024-04-23T18:54:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 310848 2024-04-23T18:54:01.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:01.137 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:54:01.183 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 8, 2024-04-23T18:54:01.237 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 376, 2024-04-23T18:54:01.244 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 3916, 2024-04-23T18:54:01.250 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 819914, 2024-04-23T18:54:01.262 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 6, 2024-04-23T18:54:01.269 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2024-04-23T18:54:01.277 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.022406863, 2024-04-23T18:54:01.348 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.016124030, 2024-04-23T18:54:01.356 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.002123139, 2024-04-23T18:54:01.419 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000045723, 2024-04-23T18:54:01.427 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:01.487 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 374, 2024-04-23T18:54:01.497 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.022851034, 2024-04-23T18:54:01.505 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000061099 2024-04-23T18:54:01.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:01.726 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:01.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:01.742 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-23T18:54:01.751 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 819216 2024-04-23T18:54:01.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:01.814 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:54:01.825 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 3535, 2024-04-23T18:54:01.833 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 3537, 2024-04-23T18:54:01.906 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 900502, 2024-04-23T18:54:01.914 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 1124137, 2024-04-23T18:54:02.004 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-04-23T18:54:02.010 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-04-23T18:54:02.020 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.224947159, 2024-04-23T18:54:02.029 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.055076243, 2024-04-23T18:54:02.110 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.060241521, 2024-04-23T18:54:02.119 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.094488495, 2024-04-23T18:54:02.194 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3535, 2024-04-23T18:54:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.093131086, 2024-04-23T18:54:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000026345 2024-04-23T18:54:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:02.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:02.387 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:54:02.465 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:54:02.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:02.480 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-23T18:54:02.488 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-04-23T18:54:02.564 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-23T18:54:02.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:02.677 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-04-23T18:54:02.707 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-23T18:54:02.717 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-23T18:54:02.724 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 8919, 2024-04-23T18:54:02.731 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.226134652, 2024-04-23T18:54:02.739 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000025354 2024-04-23T18:54:02.776 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:54:02.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:02.793 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-04-23T18:54:02.856 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-23T18:54:02.864 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-23T18:54:02.953 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:02.960 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:02.967 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:02.973 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:54:03.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:03.074 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-04-23T18:54:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 0, 2024-04-23T18:54:03.196 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 0, 2024-04-23T18:54:03.203 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-04-23T18:54:03.210 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:03.217 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:03.223 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:03.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:03.275 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-04-23T18:54:03.337 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2024-04-23T18:54:03.345 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-04-23T18:54:03.352 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-04-23T18:54:03.435 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 0, 2024-04-23T18:54:03.444 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-04-23T18:54:03.455 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-04-23T18:54:03.462 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 0, 2024-04-23T18:54:03.599 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 0, 2024-04-23T18:54:03.607 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 0, 2024-04-23T18:54:03.702 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-04-23T18:54:03.753 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 0, 2024-04-23T18:54:03.759 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 0, 2024-04-23T18:54:03.766 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2024-04-23T18:54:03.773 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2024-04-23T18:54:03.779 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 1, 2024-04-23T18:54:03.817 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 0, 2024-04-23T18:54:03.824 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 0, 2024-04-23T18:54:03.831 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2024-04-23T18:54:03.893 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2024-04-23T18:54:03.900 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:54:03.980 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-04-23T18:54:03.989 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-04-23T18:54:03.998 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-04-23T18:54:04.007 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-04-23T18:54:04.088 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-04-23T18:54:04.097 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-04-23T18:54:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-04-23T18:54:04.261 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-04-23T18:54:04.270 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-04-23T18:54:04.278 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:54:04.288 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-04-23T18:54:04.298 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 0, 2024-04-23T18:54:04.328 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 0, 2024-04-23T18:54:04.337 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2024-04-23T18:54:04.405 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:54:04.414 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-04-23T18:54:04.423 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:54:04.483 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:54:04.494 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:54:04.503 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:54:04.512 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:54:04.521 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-04-23T18:54:04.531 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:54:04.643 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:54:04.678 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:54:04.686 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:54:04.696 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:54:04.705 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:54:04.714 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2024-04-23T18:54:04.753 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:54:04.762 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-04-23T18:54:04.830 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2024-04-23T18:54:04.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:04.846 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-04-23T18:54:04.924 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-04-23T18:54:04.932 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-04-23T18:54:04.942 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-04-23T18:54:04.951 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2024-04-23T18:54:05.013 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-04-23T18:54:05.022 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-04-23T18:54:05.110 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-04-23T18:54:05.139 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:54:05.148 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-04-23T18:54:05.155 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-04-23T18:54:05.162 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-04-23T18:54:05.169 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-04-23T18:54:05.200 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-04-23T18:54:05.207 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:54:05.274 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:54:05.282 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-04-23T18:54:05.289 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-04-23T18:54:05.356 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:54:05.363 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:54:05.370 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:54:05.377 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-04-23T18:54:05.421 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-04-23T18:54:05.429 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:54:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-04-23T18:54:05.589 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-04-23T18:54:05.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:05.603 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-04-23T18:54:05.609 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-04-23T18:54:05.617 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 0, 2024-04-23T18:54:05.650 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-04-23T18:54:05.658 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-04-23T18:54:05.721 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 5, 2024-04-23T18:54:05.728 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-04-23T18:54:05.737 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-04-23T18:54:05.804 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 0, 2024-04-23T18:54:05.811 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-04-23T18:54:05.818 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-04-23T18:54:05.825 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-04-23T18:54:05.831 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-04-23T18:54:05.838 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-04-23T18:54:05.951 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-04-23T18:54:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-04-23T18:54:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 4195815, 2024-04-23T18:54:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 4195424, 2024-04-23T18:54:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-04-23T18:54:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:06.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:06.122 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 5 2024-04-23T18:54:06.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:06.179 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-04-23T18:54:06.185 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 3, 2024-04-23T18:54:06.194 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 3, 2024-04-23T18:54:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 0, 2024-04-23T18:54:06.279 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 1, 2024-04-23T18:54:06.286 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 1, 2024-04-23T18:54:06.424 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 0, 2024-04-23T18:54:06.494 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 0, 2024-04-23T18:54:06.515 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 0, 2024-04-23T18:54:06.542 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 0, 2024-04-23T18:54:06.560 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2024-04-23T18:54:06.568 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 0, 2024-04-23T18:54:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2024-04-23T18:54:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 50008, 2024-04-23T18:54:06.642 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 215416 2024-04-23T18:54:06.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:06.658 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-04-23T18:54:06.665 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 0, 2024-04-23T18:54:06.715 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-04-23T18:54:06.723 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 0, 2024-04-23T18:54:06.732 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 0, 2024-04-23T18:54:06.806 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2024-04-23T18:54:06.815 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-04-23T18:54:06.901 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:06.971 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:06.980 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:06.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:06.997 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-04-23T18:54:07.006 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-04-23T18:54:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:07.197 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:07.205 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:07.222 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-04-23T18:54:07.323 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:07.329 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:07.401 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:07.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:07.455 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-04-23T18:54:07.464 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:07.472 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:07.479 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:07.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:07.518 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-04-23T18:54:07.524 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:07.590 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:07.598 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:07.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:07.668 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-04-23T18:54:07.674 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:07.679 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:07.909 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-04-23T18:54:07.939 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:07.945 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:07.952 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:07.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:07.966 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-04-23T18:54:08.005 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:08.012 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:08.060 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:08.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:08.075 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-04-23T18:54:08.135 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:08.142 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:08.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:08.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:08.257 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-04-23T18:54:08.264 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:08.359 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:08.389 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:08.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:08.405 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-04-23T18:54:08.414 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:08.423 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:01 smithi155 ceph-mon[33468]: pgmap v884: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:08.455 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:03 smithi155 ceph-mon[33468]: pgmap v885: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:08.462 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:05 smithi155 ceph-mon[33468]: pgmap v886: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:54:08.516 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:06 smithi155 ceph-mon[33468]: osdmap e135: 12 total, 12 up, 12 in 2024-04-23T18:54:08.523 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:07 smithi155 ceph-mon[33468]: pgmap v888: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:08.530 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:01 smithi019 ceph-mon[29215]: pgmap v884: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:08.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:03 smithi019 ceph-mon[29215]: pgmap v885: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:08.632 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:05 smithi019 ceph-mon[29215]: pgmap v886: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:54:08.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:06 smithi019 ceph-mon[29215]: osdmap e135: 12 total, 12 up, 12 in 2024-04-23T18:54:08.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:07 smithi019 ceph-mon[29215]: pgmap v888: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:08.729 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:08.736 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:08.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:08.888 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-04-23T18:54:08.898 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:08.907 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:08.955 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-04-23T18:54:08.964 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:08.990 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:08.999 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:09.053 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-04-23T18:54:09.062 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:09.071 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:09.079 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-04-23T18:54:09.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:09.293 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:09.302 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:09.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:09.319 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-04-23T18:54:09.329 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:09.360 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:09.370 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:09.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:09.435 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-04-23T18:54:09.444 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:09.520 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:09.529 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:09.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:09.546 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-04-23T18:54:09.614 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:09.621 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:09.691 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:09.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:09.736 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-04-23T18:54:09.744 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:09.752 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:09.761 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:09.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:09.778 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-04-23T18:54:09.837 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:09.846 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:09.854 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:09.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:09.908 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-04-23T18:54:09.915 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:09.922 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:10.003 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:10.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-04-23T18:54:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-04-23T18:54:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-04-23T18:54:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:10.371 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:10.378 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:10.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:10.453 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-04-23T18:54:10.459 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:10.583 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-04-23T18:54:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:10.782 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-04-23T18:54:10.793 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:10.802 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:10.869 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:10.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:10.884 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-04-23T18:54:10.893 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:10.974 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:10.982 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:11.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:11.113 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:54:11.119 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:54:11.128 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-04-23T18:54:11.137 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:11.145 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:11.181 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:11.187 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:54:11.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:11.264 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-04-23T18:54:11.271 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2024-04-23T18:54:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 2, 2024-04-23T18:54:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 1, 2024-04-23T18:54:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 1, 2024-04-23T18:54:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-04-23T18:54:11.476 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 0, 2024-04-23T18:54:11.482 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2024-04-23T18:54:11.566 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 2900, 2024-04-23T18:54:11.643 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-04-23T18:54:11.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:11.657 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-23T18:54:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:54:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-23T18:54:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:54:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:54:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:54:11.785 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:54:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:54:11.863 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:54:11.870 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:54:11.877 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:54:11.885 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:54:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:54:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:54:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:54:12.096 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:54:12.103 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-23T18:54:12.110 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:54:12.118 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-23T18:54:12.126 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:54:12.179 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:54:12.186 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:54:12.193 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:54:12.254 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:54:12.262 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:54:12.336 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:54:12.342 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-23T18:54:12.349 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:54:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:54:12.402 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:54:12.409 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-23T18:54:12.482 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-23T18:54:12.562 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-23T18:54:12.569 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:54:12.576 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:54:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:54:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:54:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 182087, 2024-04-23T18:54:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 48, 2024-04-23T18:54:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:54:12.698 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-04-23T18:54:12.707 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-23T18:54:12.713 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-23T18:54:12.811 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:54:12.817 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-23T18:54:12.826 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:54:12.869 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-23T18:54:12.880 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 8032, 2024-04-23T18:54:12.889 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 167, 2024-04-23T18:54:12.955 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:54:12.965 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:54:12.974 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-23T18:54:12.982 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-23T18:54:12.991 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 8512, 2024-04-23T18:54:13.021 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 13, 2024-04-23T18:54:13.101 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-23T18:54:13.110 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-23T18:54:13.168 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-23T18:54:13.176 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-23T18:54:13.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:13.254 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-04-23T18:54:13.263 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-04-23T18:54:13.271 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-04-23T18:54:13.280 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 5333, 2024-04-23T18:54:13.289 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 21, 2024-04-23T18:54:13.365 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 3, 2024-04-23T18:54:13.432 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 5333, 2024-04-23T18:54:13.440 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-04-23T18:54:13.448 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 5333, 2024-04-23T18:54:13.456 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 2.841954336, 2024-04-23T18:54:13.464 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000532899 2024-04-23T18:54:13.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:13.577 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-04-23T18:54:13.585 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-04-23T18:54:13.643 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 5333, 2024-04-23T18:54:13.652 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 5334 2024-04-23T18:54:13.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:13.733 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 5333, 2024-04-23T18:54:13.742 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 5333, 2024-04-23T18:54:13.751 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 0, 2024-04-23T18:54:13.760 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-04-23T18:54:13.768 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-04-23T18:54:13.832 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 3529, 2024-04-23T18:54:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 0, 2024-04-23T18:54:13.893 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 1802, 2024-04-23T18:54:13.901 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 0, 2024-04-23T18:54:13.909 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 0, 2024-04-23T18:54:13.917 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-04-23T18:54:13.951 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-04-23T18:54:13.998 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-04-23T18:54:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-04-23T18:54:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 0, 2024-04-23T18:54:14.067 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-04-23T18:54:14.143 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-04-23T18:54:14.151 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-04-23T18:54:14.158 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 1, 2024-04-23T18:54:14.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 0, 2024-04-23T18:54:14.172 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:54:14.180 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-04-23T18:54:14.282 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:54:14.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-04-23T18:54:14.333 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-04-23T18:54:14.340 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-04-23T18:54:14.347 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:54:14.354 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-04-23T18:54:14.385 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:54:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-04-23T18:54:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-04-23T18:54:14.497 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-04-23T18:54:14.505 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-04-23T18:54:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-04-23T18:54:14.625 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-04-23T18:54:14.632 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-04-23T18:54:14.638 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-04-23T18:54:14.645 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-04-23T18:54:14.652 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-04-23T18:54:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-04-23T18:54:14.789 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-04-23T18:54:14.796 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-04-23T18:54:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-04-23T18:54:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-04-23T18:54:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-04-23T18:54:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-04-23T18:54:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 134, 2024-04-23T18:54:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-04-23T18:54:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 48, 2024-04-23T18:54:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 4, 2024-04-23T18:54:15.053 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 5, 2024-04-23T18:54:15.060 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 1, 2024-04-23T18:54:15.067 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-04-23T18:54:15.075 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 0, 2024-04-23T18:54:15.082 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 2, 2024-04-23T18:54:15.089 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 0 2024-04-23T18:54:15.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:15.181 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-04-23T18:54:15.258 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 0, 2024-04-23T18:54:15.268 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:54:15.276 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-04-23T18:54:15.284 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-04-23T18:54:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-04-23T18:54:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-04-23T18:54:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:15.449 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-04-23T18:54:15.512 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-04-23T18:54:15.519 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:54:15.527 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-04-23T18:54:15.536 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-04-23T18:54:15.545 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-04-23T18:54:15.556 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-04-23T18:54:15.615 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-04-23T18:54:15.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:15.681 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:54:15.689 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:54:15.698 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-23T18:54:15.707 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:54:15.746 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 5891, 2024-04-23T18:54:15.755 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 2694848, 2024-04-23T18:54:15.764 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:15.831 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 5891, 2024-04-23T18:54:15.840 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:54:15.901 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:54:15.907 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 5891, 2024-04-23T18:54:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 2694848, 2024-04-23T18:54:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:54:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:16.039 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:16.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:54:16.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:16.134 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-04-23T18:54:16.143 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:54:16.153 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-23T18:54:16.187 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:54:16.243 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:54:16.250 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:54:16.297 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:16.303 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:54:16.311 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 5333, 2024-04-23T18:54:16.412 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 1516, 2024-04-23T18:54:16.419 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 5333, 2024-04-23T18:54:16.427 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1516, 2024-04-23T18:54:16.433 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:54:16.440 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:54:16.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:16.619 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-04-23T18:54:16.626 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:54:16.659 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-04-23T18:54:16.710 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:54:16.736 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:54:16.744 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:54:16.751 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:16.818 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:54:16.826 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 5333, 2024-04-23T18:54:16.833 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 5333, 2024-04-23T18:54:16.840 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 5333, 2024-04-23T18:54:16.847 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 5333, 2024-04-23T18:54:16.853 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:54:16.973 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:54:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:54:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:54:17.154 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-23T18:54:17.209 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:54:17.216 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:54:17.223 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:54:17.344 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:17.352 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:54:17.359 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:54:17.365 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:54:17.371 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-04-23T18:54:17.378 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-04-23T18:54:17.448 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:54:17.500 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:17.506 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:17.516 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:54:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-23T18:54:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x55910f23a6e0": { 2024-04-23T18:54:17.659 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-23T18:54:17.725 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-23T18:54:17.736 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-23T18:54:17.745 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-23T18:54:17.817 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-23T18:54:17.830 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:17.838 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-23T18:54:17.848 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-23T18:54:17.856 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-23T18:54:17.865 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-04-23T18:54:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-04-23T18:54:17.977 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-23T18:54:17.989 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-23T18:54:17.998 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-23T18:54:18.007 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-23T18:54:18.016 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:54:18.051 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-23T18:54:18.059 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-23T18:54:18.106 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-23T18:54:18.137 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-23T18:54:18.149 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-23T18:54:18.223 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-23T18:54:18.230 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-23T18:54:18.238 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:54:18.245 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:54:18.252 DEBUG:tasks.check_counter:Getting stats from c 2024-04-23T18:54:18.260 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-23T18:54:18.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:01 smithi044 ceph-mon[32458]: pgmap v884: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:18.387 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:03 smithi044 ceph-mon[32458]: pgmap v885: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:18.396 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:05 smithi044 ceph-mon[32458]: pgmap v886: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:54:18.433 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:06 smithi044 ceph-mon[32458]: osdmap e135: 12 total, 12 up, 12 in 2024-04-23T18:54:18.440 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:07 smithi044 ceph-mon[32458]: pgmap v888: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:18.446 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:09 smithi044 ceph-mon[32458]: pgmap v889: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:18.484 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:11 smithi044 ceph-mon[32458]: pgmap v890: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:18.526 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:14 smithi044 ceph-mon[32458]: pgmap v891: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:18.575 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:14 smithi044 ceph-mon[32458]: osdmap e136: 12 total, 12 up, 12 in 2024-04-23T18:54:18.582 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:16 smithi044 ceph-mon[32458]: pgmap v893: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-23T18:54:18.589 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:18 smithi044 ceph-mon[32458]: pgmap v894: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:54:18.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:09 smithi019 ceph-mon[29215]: pgmap v889: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:18.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:11 smithi019 ceph-mon[29215]: pgmap v890: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:18.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:14 smithi019 ceph-mon[29215]: pgmap v891: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:18.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:14 smithi019 ceph-mon[29215]: osdmap e136: 12 total, 12 up, 12 in 2024-04-23T18:54:18.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:16 smithi019 ceph-mon[29215]: pgmap v893: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-23T18:54:18.685 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:09 smithi155 ceph-mon[33468]: pgmap v889: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:18.747 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:11 smithi155 ceph-mon[33468]: pgmap v890: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:18.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:14 smithi155 ceph-mon[33468]: pgmap v891: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:54:18.798 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:14 smithi155 ceph-mon[33468]: osdmap e136: 12 total, 12 up, 12 in 2024-04-23T18:54:18.806 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:16 smithi155 ceph-mon[33468]: pgmap v893: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-04-23T18:54:18.814 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:18 smithi155 ceph-mon[33468]: pgmap v894: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:54:18.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:18 smithi019 ceph-mon[29215]: pgmap v894: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:54:19.076 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:54:20.242 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:20 smithi155 ceph-mon[33468]: pgmap v895: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:54:20.317 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:20 smithi044 ceph-mon[32458]: pgmap v895: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:54:20.512 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:20 smithi019 ceph-mon[29215]: pgmap v895: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:54:20.900 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-23T18:54:20.908 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:54:20.916 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 45, 2024-04-23T18:54:20.922 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 47, 2024-04-23T18:54:20.928 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 13789, 2024-04-23T18:54:20.937 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 65417, 2024-04-23T18:54:20.943 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 17, 2024-04-23T18:54:21.047 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-04-23T18:54:21.151 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.008562699, 2024-04-23T18:54:21.163 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.000938129, 2024-04-23T18:54:21.173 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.005205853, 2024-04-23T18:54:21.183 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000579549, 2024-04-23T18:54:21.193 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:21.244 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 46, 2024-04-23T18:54:21.389 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.018041808, 2024-04-23T18:54:21.452 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000392213 2024-04-23T18:54:21.463 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:21.476 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:21.486 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 25, 2024-04-23T18:54:21.498 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000002884, 2024-04-23T18:54:21.507 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000115 2024-04-23T18:54:21.517 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:21.526 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 6016, 2024-04-23T18:54:21.533 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 4272 2024-04-23T18:54:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:21.665 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:54:21.679 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 18, 2024-04-23T18:54:21.685 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 390, 2024-04-23T18:54:21.695 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 4539, 2024-04-23T18:54:21.702 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 832055, 2024-04-23T18:54:21.752 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 14, 2024-04-23T18:54:21.814 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-04-23T18:54:21.822 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.024904536, 2024-04-23T18:54:21.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.015098560, 2024-04-23T18:54:21.881 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.005223904, 2024-04-23T18:54:21.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000374674, 2024-04-23T18:54:21.893 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:21.900 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 387, 2024-04-23T18:54:21.907 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.041037897, 2024-04-23T18:54:21.915 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000106041 2024-04-23T18:54:21.922 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:22.013 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:22.151 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 8, 2024-04-23T18:54:22.159 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000001148, 2024-04-23T18:54:22.166 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000143 2024-04-23T18:54:22.173 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:22.179 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-04-23T18:54:22.210 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 826384 2024-04-23T18:54:22.272 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:22.313 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:54:22.320 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2975, 2024-04-23T18:54:22.326 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2543, 2024-04-23T18:54:22.333 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 2423924, 2024-04-23T18:54:22.339 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1018151, 2024-04-23T18:54:22.346 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 10, 2024-04-23T18:54:22.354 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-04-23T18:54:22.364 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.303440089, 2024-04-23T18:54:22.370 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.120008056, 2024-04-23T18:54:22.430 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.177608143, 2024-04-23T18:54:22.491 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.025381380, 2024-04-23T18:54:22.500 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:22.510 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2542, 2024-04-23T18:54:22.520 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.126690466, 2024-04-23T18:54:22.529 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000049838 2024-04-23T18:54:22.566 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:22.615 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:22.653 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4162, 2024-04-23T18:54:22.662 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001451529, 2024-04-23T18:54:22.672 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000348 2024-04-23T18:54:22.680 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:22.690 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1770832, 2024-04-23T18:54:22.699 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 350496 2024-04-23T18:54:22.709 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:22.717 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-04-23T18:54:22.726 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-04-23T18:54:22.788 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-04-23T18:54:22.881 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:22.890 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-04-23T18:54:22.900 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-04-23T18:54:22.909 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-04-23T18:54:22.916 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 47, 2024-04-23T18:54:22.966 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001742256, 2024-04-23T18:54:23.008 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000037069 2024-04-23T18:54:23.063 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:23.071 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:23.084 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-04-23T18:54:23.093 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-04-23T18:54:23.104 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-04-23T18:54:23.114 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-04-23T18:54:23.121 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000006900, 2024-04-23T18:54:23.130 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000003450 2024-04-23T18:54:23.138 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:23.224 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:23.344 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-04-23T18:54:23.350 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 10, 2024-04-23T18:54:23.357 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 6, 2024-04-23T18:54:23.364 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-04-23T18:54:23.370 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6, 2024-04-23T18:54:23.471 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.002690226, 2024-04-23T18:54:23.608 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000448371 2024-04-23T18:54:23.664 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:23.670 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-04-23T18:54:23.677 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 4, 2024-04-23T18:54:23.684 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-04-23T18:54:23.690 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-04-23T18:54:23.697 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 11, 2024-04-23T18:54:23.704 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-04-23T18:54:23.710 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-04-23T18:54:23.717 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 17, 2024-04-23T18:54:23.724 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 1, 2024-04-23T18:54:23.860 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 16, 2024-04-23T18:54:23.868 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-04-23T18:54:23.874 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 13, 2024-04-23T18:54:23.880 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 5, 2024-04-23T18:54:23.888 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 1, 2024-04-23T18:54:23.966 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 1, 2024-04-23T18:54:24.063 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 3, 2024-04-23T18:54:24.130 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 52, 2024-04-23T18:54:24.138 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 40, 2024-04-23T18:54:24.149 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 1, 2024-04-23T18:54:24.161 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 8, 2024-04-23T18:54:24.168 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:54:24.176 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-04-23T18:54:24.186 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-04-23T18:54:24.193 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-04-23T18:54:24.201 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-04-23T18:54:24.262 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-04-23T18:54:24.364 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-04-23T18:54:24.376 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-04-23T18:54:24.386 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-04-23T18:54:24.396 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-04-23T18:54:24.404 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:54:24.457 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-04-23T18:54:24.529 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 5, 2024-04-23T18:54:24.598 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 298, 2024-04-23T18:54:24.605 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-04-23T18:54:24.613 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:54:24.620 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-04-23T18:54:24.627 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:54:24.633 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:54:24.642 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:54:24.648 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:54:24.655 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:54:24.779 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-04-23T18:54:24.825 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:54:24.833 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 3, 2024-04-23T18:54:24.839 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:54:24.846 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:54:24.853 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:54:24.907 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:54:24.967 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 3, 2024-04-23T18:54:25.021 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:54:25.029 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-04-23T18:54:25.036 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-04-23T18:54:25.043 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:25.052 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-04-23T18:54:25.059 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-04-23T18:54:25.066 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 20, 2024-04-23T18:54:25.072 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 13, 2024-04-23T18:54:25.079 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 13, 2024-04-23T18:54:25.187 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-04-23T18:54:25.249 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-04-23T18:54:25.256 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-04-23T18:54:25.262 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:54:25.269 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-04-23T18:54:25.287 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-04-23T18:54:25.333 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-04-23T18:54:25.374 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-04-23T18:54:25.432 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-04-23T18:54:25.442 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:54:25.450 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:54:25.458 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-04-23T18:54:25.466 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-04-23T18:54:25.475 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 12, 2024-04-23T18:54:25.485 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 71, 2024-04-23T18:54:25.495 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:54:25.504 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-04-23T18:54:25.570 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-04-23T18:54:25.670 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:54:25.681 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-04-23T18:54:25.690 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-04-23T18:54:25.700 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:25.707 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-04-23T18:54:25.757 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-04-23T18:54:25.799 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 7, 2024-04-23T18:54:25.863 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-04-23T18:54:25.872 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-04-23T18:54:25.881 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 7, 2024-04-23T18:54:25.889 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-04-23T18:54:25.898 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-04-23T18:54:25.907 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 2, 2024-04-23T18:54:25.915 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-04-23T18:54:25.924 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-04-23T18:54:25.934 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 2, 2024-04-23T18:54:26.004 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 2, 2024-04-23T18:54:26.085 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-04-23T18:54:26.092 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-04-23T18:54:26.100 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-04-23T18:54:26.106 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 4196347, 2024-04-23T18:54:26.113 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-04-23T18:54:26.119 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-04-23T18:54:26.151 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6, 2024-04-23T18:54:26.158 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.009486863, 2024-04-23T18:54:26.211 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001581143 2024-04-23T18:54:26.218 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:26.225 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-04-23T18:54:26.233 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:26.242 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-04-23T18:54:26.253 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 21, 2024-04-23T18:54:26.270 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 23, 2024-04-23T18:54:26.277 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 2, 2024-04-23T18:54:26.337 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 20, 2024-04-23T18:54:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 23, 2024-04-23T18:54:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 3, 2024-04-23T18:54:26.397 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 17, 2024-04-23T18:54:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 22, 2024-04-23T18:54:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 5, 2024-04-23T18:54:26.483 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 1, 2024-04-23T18:54:26.540 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 3, 2024-04-23T18:54:26.599 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 2, 2024-04-23T18:54:26.605 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 55608, 2024-04-23T18:54:26.611 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 223608 2024-04-23T18:54:26.621 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:26.632 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-04-23T18:54:26.642 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 14, 2024-04-23T18:54:26.652 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-04-23T18:54:26.661 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 10, 2024-04-23T18:54:26.669 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 27, 2024-04-23T18:54:26.729 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 72, 2024-04-23T18:54:26.807 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-04-23T18:54:26.819 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:26.829 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:26.839 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:26.848 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:26.899 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-04-23T18:54:26.961 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4, 2024-04-23T18:54:27.019 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.002171802, 2024-04-23T18:54:27.029 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000542950 2024-04-23T18:54:27.039 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:27.046 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-04-23T18:54:27.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:27.068 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:27.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:27.085 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:27.093 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-04-23T18:54:27.153 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:27.225 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:27.235 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:27.242 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:27.250 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-04-23T18:54:27.257 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-04-23T18:54:27.300 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000518424, 2024-04-23T18:54:27.368 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000259212 2024-04-23T18:54:27.419 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:27.426 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-04-23T18:54:27.433 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:27.440 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:27.446 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:27.453 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:27.460 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-04-23T18:54:27.468 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:27.475 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:27.520 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:27.587 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:27.594 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-04-23T18:54:27.602 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:27.607 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:27.613 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:27.648 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:27.751 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-04-23T18:54:27.807 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:27.813 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:27.820 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:27.827 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:27.834 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-04-23T18:54:27.840 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:27.846 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:27.852 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:27.859 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:27.911 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-04-23T18:54:27.982 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:27.995 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:28.003 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:28.011 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:28.020 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-04-23T18:54:28.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:22 smithi019 ceph-mon[29215]: pgmap v896: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:54:28.102 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:24 smithi019 ceph-mon[29215]: pgmap v897: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:54:28.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:26 smithi019 ceph-mon[29215]: pgmap v898: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 88 B/s wr, 13 op/s 2024-04-23T18:54:28.162 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:22 smithi044 ceph-mon[32458]: pgmap v896: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:54:28.169 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:24 smithi044 ceph-mon[32458]: pgmap v897: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:54:28.179 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:26 smithi044 ceph-mon[32458]: pgmap v898: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 88 B/s wr, 13 op/s 2024-04-23T18:54:28.189 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:28.198 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:28.206 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:28.215 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:28.224 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-04-23T18:54:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:28.353 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:28.365 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:28.374 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:28.381 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-04-23T18:54:28.388 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:28.423 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:28.496 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:28.563 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:28.569 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-04-23T18:54:28.576 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:28.586 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:28.593 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:28.599 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:28.605 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-04-23T18:54:28.610 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:28.619 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:28.686 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:28.758 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:28.764 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-04-23T18:54:28.771 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:28.778 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:28.785 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:28.815 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:28.882 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-04-23T18:54:28.935 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:28.942 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:28.950 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:28.956 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:28.963 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-04-23T18:54:28.971 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:28.978 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:28.985 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:28.992 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:29.077 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-04-23T18:54:29.141 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:29.148 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:29.154 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:29.161 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:29.170 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-04-23T18:54:29.198 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:29.248 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:29.302 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-04-23T18:54:29.323 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:29.333 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:29.341 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:29.350 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:29.361 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-04-23T18:54:29.369 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:29.436 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:29.511 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:29.523 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:29.533 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-04-23T18:54:29.543 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:29.554 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:29.590 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:29.663 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:29.738 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-04-23T18:54:29.746 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:29.754 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:29.764 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:29.774 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:29.781 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-04-23T18:54:29.790 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:29.800 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:29.809 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:29.877 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:29.953 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-04-23T18:54:29.964 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:29.973 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:30.006 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:30.017 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:30.048 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-04-23T18:54:30.112 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:30.137 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:30.145 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:30.152 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:30.159 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-04-23T18:54:30.168 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:30.175 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:30.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:30.188 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:30.195 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:54:30.260 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:54:30.313 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-04-23T18:54:30.366 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:30.374 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:30.381 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:30.388 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:30.423 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:30.470 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-04-23T18:54:30.531 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-04-23T18:54:30.540 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2024-04-23T18:54:30.549 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2024-04-23T18:54:30.554 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2024-04-23T18:54:30.560 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-04-23T18:54:30.568 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 4, 2024-04-23T18:54:30.575 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 1, 2024-04-23T18:54:30.582 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1640, 2024-04-23T18:54:30.588 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-04-23T18:54:30.652 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:30.722 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-04-23T18:54:30.734 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:54:30.742 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-04-23T18:54:30.750 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:54:30.759 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:54:30.766 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:54:30.838 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:54:30.893 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:54:30.902 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:54:30.910 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:54:30.918 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:54:30.928 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:54:30.938 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:54:30.948 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:54:30.956 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:54:30.966 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:54:31.027 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-04-23T18:54:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:54:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-04-23T18:54:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:54:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:54:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:54:31.148 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:54:31.208 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:54:31.215 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:54:31.284 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:54:31.293 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-04-23T18:54:31.299 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:54:31.306 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:54:31.312 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:54:31.319 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-04-23T18:54:31.326 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-04-23T18:54:31.333 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-04-23T18:54:31.385 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:54:31.491 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:54:31.499 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:54:31.505 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:54:31.512 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 246119, 2024-04-23T18:54:31.519 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 75, 2024-04-23T18:54:31.560 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:54:31.611 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-04-23T18:54:31.662 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-04-23T18:54:31.670 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-04-23T18:54:31.677 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:54:31.683 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-04-23T18:54:31.690 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:54:31.698 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-04-23T18:54:31.706 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8032, 2024-04-23T18:54:31.712 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 167, 2024-04-23T18:54:31.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:54:31.819 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:54:31.942 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-04-23T18:54:31.951 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-04-23T18:54:31.958 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 88149, 2024-04-23T18:54:31.964 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 361, 2024-04-23T18:54:31.971 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-04-23T18:54:32.006 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-04-23T18:54:32.077 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-04-23T18:54:32.134 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-04-23T18:54:32.145 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:32.152 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-04-23T18:54:32.161 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-04-23T18:54:32.171 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-04-23T18:54:32.184 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 45, 2024-04-23T18:54:32.191 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 12580, 2024-04-23T18:54:32.201 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 33, 2024-04-23T18:54:32.211 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 45, 2024-04-23T18:54:32.285 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-04-23T18:54:32.376 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 45, 2024-04-23T18:54:32.387 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.226995267, 2024-04-23T18:54:32.395 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.005044339 2024-04-23T18:54:32.405 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:32.413 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-04-23T18:54:32.448 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-04-23T18:54:32.495 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 45, 2024-04-23T18:54:32.556 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 69 2024-04-23T18:54:32.568 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:32.577 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 45, 2024-04-23T18:54:32.586 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 0, 2024-04-23T18:54:32.596 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 45, 2024-04-23T18:54:32.605 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-04-23T18:54:32.614 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-04-23T18:54:32.623 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2024-04-23T18:54:32.633 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 10, 2024-04-23T18:54:32.667 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 0, 2024-04-23T18:54:32.774 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 6, 2024-04-23T18:54:32.782 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 8, 2024-04-23T18:54:32.789 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-04-23T18:54:32.795 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-04-23T18:54:32.802 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-04-23T18:54:32.835 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-04-23T18:54:32.881 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 6, 2024-04-23T18:54:32.887 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-04-23T18:54:32.938 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-04-23T18:54:32.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-04-23T18:54:32.952 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 0, 2024-04-23T18:54:32.959 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 20, 2024-04-23T18:54:32.965 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:54:32.972 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-04-23T18:54:32.978 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:54:32.985 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-04-23T18:54:33.052 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-04-23T18:54:33.124 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-04-23T18:54:33.131 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:54:33.139 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-04-23T18:54:33.148 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:54:33.158 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-04-23T18:54:33.196 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-04-23T18:54:33.236 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-04-23T18:54:33.284 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-04-23T18:54:33.293 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-04-23T18:54:33.302 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-04-23T18:54:33.311 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-04-23T18:54:33.320 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-04-23T18:54:33.329 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-04-23T18:54:33.338 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-04-23T18:54:33.347 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-04-23T18:54:33.356 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-04-23T18:54:33.401 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-04-23T18:54:33.483 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-04-23T18:54:33.490 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-04-23T18:54:33.496 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-04-23T18:54:33.505 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-04-23T18:54:33.514 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 136, 2024-04-23T18:54:33.582 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-04-23T18:54:33.636 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 50, 2024-04-23T18:54:33.644 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 8, 2024-04-23T18:54:33.684 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 20, 2024-04-23T18:54:33.693 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 12, 2024-04-23T18:54:33.702 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-04-23T18:54:33.710 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 17, 2024-04-23T18:54:33.719 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 0, 2024-04-23T18:54:33.736 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 2 2024-04-23T18:54:33.744 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-04-23T18:54:33.818 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-04-23T18:54:33.873 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:54:33.882 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-04-23T18:54:33.891 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-04-23T18:54:33.900 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:33.909 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-04-23T18:54:33.945 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 82 2024-04-23T18:54:33.985 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:34.039 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-04-23T18:54:34.047 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-04-23T18:54:34.056 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:54:34.063 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-04-23T18:54:34.072 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-04-23T18:54:34.079 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-04-23T18:54:34.085 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-04-23T18:54:34.092 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-04-23T18:54:34.099 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:34.160 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:54:34.225 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:54:34.233 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-04-23T18:54:34.239 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:54:34.245 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 3034, 2024-04-23T18:54:34.253 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2134316, 2024-04-23T18:54:34.299 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:34.349 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 3034, 2024-04-23T18:54:34.409 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:54:34.417 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:54:34.425 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3034, 2024-04-23T18:54:34.433 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2134316, 2024-04-23T18:54:34.441 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:54:34.449 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:34.456 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:34.463 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:34.472 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:34.559 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:34.631 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-04-23T18:54:34.638 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:54:34.645 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-04-23T18:54:34.651 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:54:34.657 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:54:34.689 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:54:34.742 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:34.789 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:54:34.796 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 45, 2024-04-23T18:54:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 12580, 2024-04-23T18:54:34.809 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 45, 2024-04-23T18:54:34.815 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 12580, 2024-04-23T18:54:34.822 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:54:34.829 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:34.836 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:34.843 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:34.851 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:34.922 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:34.928 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-04-23T18:54:34.936 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:54:34.943 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-04-23T18:54:34.950 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:54:34.956 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:54:35.059 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:54:35.112 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:35.119 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:54:35.127 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 45, 2024-04-23T18:54:35.134 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 45, 2024-04-23T18:54:35.143 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 45, 2024-04-23T18:54:35.151 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 45, 2024-04-23T18:54:35.160 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:54:35.168 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:35.177 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:35.226 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:35.294 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:35.305 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:35.312 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:54:35.320 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:54:35.329 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-04-23T18:54:35.370 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:54:35.427 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:54:35.435 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:54:35.490 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:35.498 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:54:35.512 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:54:35.521 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:54:35.530 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-04-23T18:54:35.540 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-04-23T18:54:35.548 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:54:35.557 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:35.617 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:35.663 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:35.669 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:35.678 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:35.687 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x557a8cd0c500": { 2024-04-23T18:54:35.696 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:54:35.728 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-04-23T18:54:35.797 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:54:35.805 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 7, 2024-04-23T18:54:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2043, 2024-04-23T18:54:35.874 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:35.883 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 7, 2024-04-23T18:54:35.892 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:54:35.901 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:54:35.909 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 6, 2024-04-23T18:54:35.917 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2043, 2024-04-23T18:54:35.925 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:54:36.001 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:36.074 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:36.080 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:36.091 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:36.102 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:36.111 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-23T18:54:36.137 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-23T18:54:36.177 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-23T18:54:36.185 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-23T18:54:36.232 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=20 2024-04-23T18:54:36.241 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-23T18:54:36.248 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:54:36.256 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:54:36.262 DEBUG:tasks.check_counter:Getting stats from f 2024-04-23T18:54:36.269 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-23T18:54:36.277 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:22 smithi155 ceph-mon[33468]: pgmap v896: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:54:36.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:24 smithi155 ceph-mon[33468]: pgmap v897: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:54:36.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:26 smithi155 ceph-mon[33468]: pgmap v898: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 88 B/s wr, 13 op/s 2024-04-23T18:54:36.380 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:28 smithi155 ceph-mon[33468]: pgmap v899: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:36.398 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:30 smithi155 ceph-mon[33468]: pgmap v900: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:36.405 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:32 smithi155 ceph-mon[33468]: pgmap v901: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:36.411 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:34 smithi155 ceph-mon[33468]: pgmap v902: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:36.419 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:36 smithi155 ceph-mon[33468]: pgmap v903: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:36.487 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:28 smithi019 ceph-mon[29215]: pgmap v899: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:36.529 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:30 smithi019 ceph-mon[29215]: pgmap v900: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:36.584 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:32 smithi019 ceph-mon[29215]: pgmap v901: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:36.592 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:34 smithi019 ceph-mon[29215]: pgmap v902: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:36.599 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:28 smithi044 ceph-mon[32458]: pgmap v899: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:36.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:30 smithi044 ceph-mon[32458]: pgmap v900: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:36.613 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:32 smithi044 ceph-mon[32458]: pgmap v901: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:36.621 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:34 smithi044 ceph-mon[32458]: pgmap v902: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:36.628 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:36 smithi019 ceph-mon[29215]: pgmap v903: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:36.636 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:36 smithi044 ceph-mon[32458]: pgmap v903: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:36.671 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:54:38.179 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:38 smithi155 ceph-mon[33468]: pgmap v904: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:38.458 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-23T18:54:38.466 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:54:38.475 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 49, 2024-04-23T18:54:38.484 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 58, 2024-04-23T18:54:38.493 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 15194, 2024-04-23T18:54:38.559 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 66815, 2024-04-23T18:54:38.668 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 21, 2024-04-23T18:54:38.677 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-04-23T18:54:38.802 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.010638303, 2024-04-23T18:54:38.812 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.001046098, 2024-04-23T18:54:38.821 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.006607690, 2024-04-23T18:54:38.830 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000610381, 2024-04-23T18:54:38.838 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:38.847 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 52, 2024-04-23T18:54:38.853 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.010031214, 2024-04-23T18:54:38.862 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000192907 2024-04-23T18:54:38.870 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:38.879 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:38.981 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 50, 2024-04-23T18:54:38.989 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000009749, 2024-04-23T18:54:38.997 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000194 2024-04-23T18:54:39.005 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:39.013 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 9696, 2024-04-23T18:54:39.100 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 9424 2024-04-23T18:54:39.135 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:39.143 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:54:39.222 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2361, 2024-04-23T18:54:39.229 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1877, 2024-04-23T18:54:39.237 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 649987, 2024-04-23T18:54:39.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 537475, 2024-04-23T18:54:39.250 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 16, 2024-04-23T18:54:39.259 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-04-23T18:54:39.267 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.199036237, 2024-04-23T18:54:39.276 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.079059140, 2024-04-23T18:54:39.330 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.121290493, 2024-04-23T18:54:39.363 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.012697958, 2024-04-23T18:54:39.370 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:39.378 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1879, 2024-04-23T18:54:39.386 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.090818852, 2024-04-23T18:54:39.427 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000048333 2024-04-23T18:54:39.507 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:39.564 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:39.573 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4028, 2024-04-23T18:54:39.704 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001100080, 2024-04-23T18:54:39.710 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000273 2024-04-23T18:54:39.717 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:39.724 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-23T18:54:39.731 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 132864 2024-04-23T18:54:39.738 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:39.746 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:54:39.753 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 612, 2024-04-23T18:54:39.788 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 937, 2024-04-23T18:54:39.846 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1765101, 2024-04-23T18:54:39.853 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1117068, 2024-04-23T18:54:39.860 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 10, 2024-04-23T18:54:39.866 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-04-23T18:54:39.914 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.118372257, 2024-04-23T18:54:40.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.050640574, 2024-04-23T18:54:40.080 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.055037193, 2024-04-23T18:54:40.087 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.014132527, 2024-04-23T18:54:40.167 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:40.173 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 936, 2024-04-23T18:54:40.181 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.071464240, 2024-04-23T18:54:40.189 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000076350 2024-04-23T18:54:40.196 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:40.202 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:40.212 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:40.219 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:40.255 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:40.380 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:40.386 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1760144, 2024-04-23T18:54:40.393 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 1108352 2024-04-23T18:54:40.400 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:40.435 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-04-23T18:54:40.550 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-04-23T18:54:40.594 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-04-23T18:54:40.603 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:40.681 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-04-23T18:54:40.690 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-04-23T18:54:40.699 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-04-23T18:54:40.708 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 53, 2024-04-23T18:54:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.002018672, 2024-04-23T18:54:40.726 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000038088 2024-04-23T18:54:40.735 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:40.743 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:40.779 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-04-23T18:54:40.814 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-04-23T18:54:40.825 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-04-23T18:54:40.835 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-04-23T18:54:40.842 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000012504, 2024-04-23T18:54:40.878 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000006252 2024-04-23T18:54:40.951 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:41.005 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:41.014 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-04-23T18:54:41.142 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 9, 2024-04-23T18:54:41.149 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 6, 2024-04-23T18:54:41.157 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-04-23T18:54:41.166 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6, 2024-04-23T18:54:41.174 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.003067454, 2024-04-23T18:54:41.183 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000511242 2024-04-23T18:54:41.191 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:41.199 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-04-23T18:54:41.251 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 3, 2024-04-23T18:54:41.346 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-04-23T18:54:41.355 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-04-23T18:54:41.363 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 11, 2024-04-23T18:54:41.372 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-04-23T18:54:41.416 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-04-23T18:54:41.505 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 17, 2024-04-23T18:54:41.572 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 1, 2024-04-23T18:54:41.581 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 16, 2024-04-23T18:54:41.676 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-04-23T18:54:41.683 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 13, 2024-04-23T18:54:41.689 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 5, 2024-04-23T18:54:41.696 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-04-23T18:54:41.702 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-04-23T18:54:41.711 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 3, 2024-04-23T18:54:41.717 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 49, 2024-04-23T18:54:41.723 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 39, 2024-04-23T18:54:41.730 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 1, 2024-04-23T18:54:41.738 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 7, 2024-04-23T18:54:41.784 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:54:41.814 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-04-23T18:54:41.822 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-04-23T18:54:41.861 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-04-23T18:54:41.929 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-04-23T18:54:41.999 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-04-23T18:54:42.006 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-04-23T18:54:42.014 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-04-23T18:54:42.131 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-04-23T18:54:42.138 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-04-23T18:54:42.144 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:54:42.152 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-04-23T18:54:42.159 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 5, 2024-04-23T18:54:42.166 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 298, 2024-04-23T18:54:42.173 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-04-23T18:54:42.221 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:54:42.282 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-04-23T18:54:42.292 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:54:42.298 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:54:42.307 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:54:42.362 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:54:42.446 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:54:42.525 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-04-23T18:54:42.533 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:54:42.540 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 5, 2024-04-23T18:54:42.612 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:54:42.619 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:54:42.628 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:54:42.638 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:54:42.647 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 5, 2024-04-23T18:54:42.654 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:54:42.665 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-04-23T18:54:42.762 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-04-23T18:54:42.822 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:42.829 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-04-23T18:54:42.837 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-04-23T18:54:42.848 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 16, 2024-04-23T18:54:42.889 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 11, 2024-04-23T18:54:42.971 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 12, 2024-04-23T18:54:42.980 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-04-23T18:54:43.035 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-04-23T18:54:43.108 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-04-23T18:54:43.117 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:54:43.126 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-04-23T18:54:43.135 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-04-23T18:54:43.154 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-04-23T18:54:43.163 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-04-23T18:54:43.171 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-04-23T18:54:43.182 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:54:43.220 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:54:43.249 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-04-23T18:54:43.258 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-04-23T18:54:43.268 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 12, 2024-04-23T18:54:43.278 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 71, 2024-04-23T18:54:43.312 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:54:43.403 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-04-23T18:54:43.467 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-04-23T18:54:43.476 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:54:43.589 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-04-23T18:54:43.596 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-04-23T18:54:43.604 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:43.613 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-04-23T18:54:43.622 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-04-23T18:54:43.630 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 8, 2024-04-23T18:54:43.639 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-04-23T18:54:43.648 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-04-23T18:54:43.657 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 8, 2024-04-23T18:54:43.712 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-04-23T18:54:43.721 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-04-23T18:54:43.730 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 2, 2024-04-23T18:54:43.739 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-04-23T18:54:43.795 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-04-23T18:54:43.878 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 2, 2024-04-23T18:54:43.944 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 2, 2024-04-23T18:54:43.952 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-04-23T18:54:44.039 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-04-23T18:54:44.047 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-04-23T18:54:44.053 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 4196488, 2024-04-23T18:54:44.062 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-04-23T18:54:44.069 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-04-23T18:54:44.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 7, 2024-04-23T18:54:44.084 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.014491154, 2024-04-23T18:54:44.091 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002070164 2024-04-23T18:54:44.146 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:44.183 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-04-23T18:54:44.189 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:44.196 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-04-23T18:54:44.204 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 21, 2024-04-23T18:54:44.279 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 23, 2024-04-23T18:54:44.374 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 2, 2024-04-23T18:54:44.441 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 20, 2024-04-23T18:54:44.449 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 22, 2024-04-23T18:54:44.522 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 2, 2024-04-23T18:54:44.529 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 17, 2024-04-23T18:54:44.535 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 22, 2024-04-23T18:54:44.542 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 5, 2024-04-23T18:54:44.549 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-04-23T18:54:44.556 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 5, 2024-04-23T18:54:44.564 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 5, 2024-04-23T18:54:44.570 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 54308, 2024-04-23T18:54:44.612 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 223608 2024-04-23T18:54:44.675 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:44.686 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-04-23T18:54:44.696 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 13, 2024-04-23T18:54:44.703 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-04-23T18:54:44.752 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 9, 2024-04-23T18:54:44.804 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 12, 2024-04-23T18:54:44.810 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 72, 2024-04-23T18:54:44.872 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-04-23T18:54:44.934 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:44.942 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:44.950 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:44.957 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:44.963 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-04-23T18:54:44.970 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-04-23T18:54:44.978 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001791631, 2024-04-23T18:54:44.986 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000895815 2024-04-23T18:54:45.063 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:45.093 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-04-23T18:54:45.100 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:45.108 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:45.117 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:45.151 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:45.202 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-04-23T18:54:45.263 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:45.272 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:45.361 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:45.370 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:45.380 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-04-23T18:54:45.391 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4, 2024-04-23T18:54:45.400 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001275823, 2024-04-23T18:54:45.408 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000318955 2024-04-23T18:54:45.417 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:45.425 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-04-23T18:54:45.487 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:45.519 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:45.529 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:45.538 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:45.548 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-04-23T18:54:45.577 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:45.678 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:45.687 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:45.760 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:45.841 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-04-23T18:54:45.847 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:45.854 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:45.859 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:45.866 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:45.875 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-04-23T18:54:45.882 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:45.889 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:45.928 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:45.961 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:45.967 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-04-23T18:54:45.974 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:45.981 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:46.017 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:46.078 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:46.118 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-04-23T18:54:46.125 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:46.182 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:46.189 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:46.196 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:46.203 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-04-23T18:54:46.211 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:38 smithi044 ceph-mon[32458]: pgmap v904: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:46.218 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:40 smithi044 ceph-mon[32458]: pgmap v905: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:46.228 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:42 smithi044 ceph-mon[32458]: pgmap v906: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:46.235 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:44 smithi044 ceph-mon[32458]: pgmap v907: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:46.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:38 smithi019 ceph-mon[29215]: pgmap v904: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:46.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:40 smithi019 ceph-mon[29215]: pgmap v905: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:46.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:42 smithi019 ceph-mon[29215]: pgmap v906: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:46.351 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:44 smithi019 ceph-mon[29215]: pgmap v907: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:46.358 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:46.393 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:46.444 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:46.484 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:46.493 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-04-23T18:54:46.566 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:46.573 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:46.579 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:46.587 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:46.594 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-04-23T18:54:46.601 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:46.608 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:46.616 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:46.677 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:46.712 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-04-23T18:54:46.719 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:46.730 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:46.737 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:46.745 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:46.836 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-04-23T18:54:46.888 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:46.896 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:46.957 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:46.963 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:46.972 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-04-23T18:54:46.982 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:46.990 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:46.999 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:47.007 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:47.017 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-04-23T18:54:47.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:47.080 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:47.089 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:47.097 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:47.105 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-04-23T18:54:47.170 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:47.269 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:47.310 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:47.319 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:47.392 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-04-23T18:54:47.399 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:47.406 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:47.413 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:47.420 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:47.430 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-04-23T18:54:47.437 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:47.444 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:47.494 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:47.571 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:47.577 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-04-23T18:54:47.585 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:47.591 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:47.628 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:47.686 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:47.761 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-04-23T18:54:47.769 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:47.829 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:47.837 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:47.844 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:47.851 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-04-23T18:54:47.858 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:47.865 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:47.872 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:47.881 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:47.954 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-04-23T18:54:48.008 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:48.016 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:48.025 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:48.034 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:48.084 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-04-23T18:54:48.130 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:48.210 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:48.217 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:48.269 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:48.277 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-04-23T18:54:48.284 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:48.293 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:48.300 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:48.306 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:48.315 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-04-23T18:54:48.324 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:48.391 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:48.423 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:48.431 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:48.438 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-04-23T18:54:48.446 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:48.490 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:48.515 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:48.572 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:48.579 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:54:48.659 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:54:48.665 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-04-23T18:54:48.672 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:48.679 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:48.686 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:48.693 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:48.699 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:48.706 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-04-23T18:54:48.713 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 0, 2024-04-23T18:54:48.732 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2024-04-23T18:54:48.761 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 3, 2024-04-23T18:54:48.768 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 0, 2024-04-23T18:54:48.775 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-04-23T18:54:48.805 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 5, 2024-04-23T18:54:48.900 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 1, 2024-04-23T18:54:48.907 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 88, 2024-04-23T18:54:48.965 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-04-23T18:54:49.045 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:49.052 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-04-23T18:54:49.059 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:54:49.065 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-04-23T18:54:49.072 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:54:49.080 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:54:49.088 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:54:49.095 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:54:49.159 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:54:49.191 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:54:49.201 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:54:49.208 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:54:49.216 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:54:49.251 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:54:49.323 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:54:49.412 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:54:49.419 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:54:49.511 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-04-23T18:54:49.517 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:54:49.524 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-04-23T18:54:49.531 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:54:49.538 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:54:49.546 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:54:49.554 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:54:49.561 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:54:49.602 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:54:49.690 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:54:49.698 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-04-23T18:54:49.704 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:54:49.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:54:49.751 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:54:49.843 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-04-23T18:54:49.900 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-04-23T18:54:49.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-04-23T18:54:49.981 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:54:49.989 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:54:49.997 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:54:50.005 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:54:50.013 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 245440, 2024-04-23T18:54:50.020 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 70, 2024-04-23T18:54:50.028 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:54:50.037 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-04-23T18:54:50.077 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-04-23T18:54:50.151 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-04-23T18:54:50.159 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:54:50.165 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-04-23T18:54:50.172 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:54:50.209 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-04-23T18:54:50.271 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8032, 2024-04-23T18:54:50.331 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 167, 2024-04-23T18:54:50.340 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:54:50.402 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:54:50.424 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-04-23T18:54:50.430 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-04-23T18:54:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 87797, 2024-04-23T18:54:50.448 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 361, 2024-04-23T18:54:50.457 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-04-23T18:54:50.465 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-04-23T18:54:50.502 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-04-23T18:54:50.563 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-04-23T18:54:50.572 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:50.582 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-04-23T18:54:50.591 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-04-23T18:54:50.626 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-04-23T18:54:50.710 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 46, 2024-04-23T18:54:50.776 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 12757, 2024-04-23T18:54:50.785 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 29, 2024-04-23T18:54:50.793 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 46, 2024-04-23T18:54:50.865 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-04-23T18:54:50.872 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 46, 2024-04-23T18:54:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.221995373, 2024-04-23T18:54:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004825986 2024-04-23T18:54:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-04-23T18:54:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-04-23T18:54:50.951 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 46, 2024-04-23T18:54:50.997 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 71 2024-04-23T18:54:51.003 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:51.010 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 46, 2024-04-23T18:54:51.016 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 0, 2024-04-23T18:54:51.050 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 46, 2024-04-23T18:54:51.118 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-04-23T18:54:51.162 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-04-23T18:54:51.170 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2024-04-23T18:54:51.253 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 10, 2024-04-23T18:54:51.261 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 0, 2024-04-23T18:54:51.267 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 7, 2024-04-23T18:54:51.283 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 7, 2024-04-23T18:54:51.290 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-04-23T18:54:51.296 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-04-23T18:54:51.304 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-04-23T18:54:51.311 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-04-23T18:54:51.352 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 6, 2024-04-23T18:54:51.404 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-04-23T18:54:51.412 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-04-23T18:54:51.420 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-04-23T18:54:51.428 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 0, 2024-04-23T18:54:51.477 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 20, 2024-04-23T18:54:51.563 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:54:51.626 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-04-23T18:54:51.634 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:54:51.685 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-04-23T18:54:51.691 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-04-23T18:54:51.699 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-04-23T18:54:51.706 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:54:51.713 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-04-23T18:54:51.720 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:54:51.728 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-04-23T18:54:51.735 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-04-23T18:54:51.773 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-04-23T18:54:51.896 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-04-23T18:54:51.905 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-04-23T18:54:51.914 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-04-23T18:54:51.924 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-04-23T18:54:51.958 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-04-23T18:54:52.025 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-04-23T18:54:52.073 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-04-23T18:54:52.081 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-04-23T18:54:52.163 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-04-23T18:54:52.169 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-04-23T18:54:52.178 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-04-23T18:54:52.188 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-04-23T18:54:52.196 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-04-23T18:54:52.206 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-04-23T18:54:52.215 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 136, 2024-04-23T18:54:52.223 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-04-23T18:54:52.259 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 50, 2024-04-23T18:54:52.354 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 9, 2024-04-23T18:54:52.365 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 24, 2024-04-23T18:54:52.373 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 15, 2024-04-23T18:54:52.381 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-04-23T18:54:52.435 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 19, 2024-04-23T18:54:52.506 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 0, 2024-04-23T18:54:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 2 2024-04-23T18:54:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:52.672 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-04-23T18:54:52.679 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-04-23T18:54:52.686 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:54:52.693 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-04-23T18:54:52.700 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-04-23T18:54:52.707 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:52.714 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-04-23T18:54:52.721 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 82 2024-04-23T18:54:52.759 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:52.826 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-04-23T18:54:52.834 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-04-23T18:54:52.841 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:54:52.848 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-04-23T18:54:52.883 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-04-23T18:54:52.935 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-04-23T18:54:52.941 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-04-23T18:54:52.994 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-04-23T18:54:53.058 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:53.065 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:54:53.074 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:54:53.081 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-04-23T18:54:53.088 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:54:53.095 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 3018, 2024-04-23T18:54:53.103 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2124300, 2024-04-23T18:54:53.109 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:53.157 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 3018, 2024-04-23T18:54:53.234 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:54:53.239 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:54:53.247 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3018, 2024-04-23T18:54:53.254 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2124300, 2024-04-23T18:54:53.300 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:54:53.392 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:53.449 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:53.459 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:53.569 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:53.579 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:53.590 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-04-23T18:54:53.599 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:54:53.607 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-04-23T18:54:53.618 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:54:53.628 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:54:53.637 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:54:53.681 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:53.729 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:54:53.739 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 46, 2024-04-23T18:54:53.748 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 12757, 2024-04-23T18:54:53.757 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 46, 2024-04-23T18:54:53.791 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 12757, 2024-04-23T18:54:53.869 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:54:53.878 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:53.932 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:53.995 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:54.004 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:54.019 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-04-23T18:54:54.026 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:54:54.035 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-04-23T18:54:54.043 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:54:54.050 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:54:54.193 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:54:54.271 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:54.278 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:54:54.286 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 46, 2024-04-23T18:54:54.294 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 46, 2024-04-23T18:54:54.302 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 46, 2024-04-23T18:54:54.498 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 46, 2024-04-23T18:54:54.575 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:54:54.584 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:54.593 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:54.644 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:54.651 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:54.658 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:54.665 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:54:54.672 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:54:54.679 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-04-23T18:54:54.686 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:54:54.734 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:54:54.804 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:54:54.812 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:54.820 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:54:54.827 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:54:54.891 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:54:54.943 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-04-23T18:54:54.998 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-04-23T18:54:55.005 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:54:55.082 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:55.088 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:55.096 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:55.103 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:55.110 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:55.117 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55c43c0a6500": { 2024-04-23T18:54:55.124 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:54:55.131 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-04-23T18:54:55.176 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:54:55.237 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 8, 2024-04-23T18:54:55.244 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2184, 2024-04-23T18:54:55.252 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:54:55.260 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 8, 2024-04-23T18:54:55.267 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:54:55.333 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:54:55.398 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 7, 2024-04-23T18:54:55.406 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2184, 2024-04-23T18:54:55.444 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:54:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:55.458 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:55.465 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:55.472 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:55.479 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:54:55.486 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-23T18:54:55.494 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-23T18:54:55.547 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-23T18:54:55.596 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-23T18:54:55.602 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=16 2024-04-23T18:54:55.609 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-23T18:54:55.616 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:54:55.622 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:54:55.704 DEBUG:tasks.check_counter:Getting stats from i 2024-04-23T18:54:55.713 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-23T18:54:55.761 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:40 smithi155 ceph-mon[33468]: pgmap v905: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:55.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:42 smithi155 ceph-mon[33468]: pgmap v906: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:55.838 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:44 smithi155 ceph-mon[33468]: pgmap v907: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:55.850 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:46 smithi155 ceph-mon[33468]: pgmap v908: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:55.859 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:48 smithi155 ceph-mon[33468]: pgmap v909: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:55.868 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:49 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:54:55.877 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:49 smithi155 ceph-mon[33468]: pgmap v910: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:55.886 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:50 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:54:55.895 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:50 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:54:55.945 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:50 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:54:56.012 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:51 smithi155 ceph-mon[33468]: pgmap v911: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:56.021 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:53 smithi155 ceph-mon[33468]: pgmap v912: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:56.032 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:46 smithi044 ceph-mon[32458]: pgmap v908: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:56.041 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:48 smithi044 ceph-mon[32458]: pgmap v909: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:56.105 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:49 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:54:56.181 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:49 smithi044 ceph-mon[32458]: pgmap v910: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:56.267 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:50 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:54:56.277 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:50 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:54:56.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:50 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:54:56.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:51 smithi044 ceph-mon[32458]: pgmap v911: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:56.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:53 smithi044 ceph-mon[32458]: pgmap v912: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:56.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:46 smithi019 ceph-mon[29215]: pgmap v908: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:56.409 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:48 smithi019 ceph-mon[29215]: pgmap v909: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:56.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:49 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:54:56.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:49 smithi019 ceph-mon[29215]: pgmap v910: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:56.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:50 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:54:56.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:50 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:54:56.553 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:50 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:54:56.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:51 smithi019 ceph-mon[29215]: pgmap v911: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:56.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:53 smithi019 ceph-mon[29215]: pgmap v912: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:56.577 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:55 smithi019 ceph-mon[29215]: pgmap v913: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:56.635 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:55 smithi044 ceph-mon[32458]: pgmap v913: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:56.704 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:55 smithi155 ceph-mon[33468]: pgmap v913: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:54:56.764 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:54:57.769 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:57 smithi155 ceph-mon[33468]: pgmap v914: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:58.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:57 smithi019 ceph-mon[29215]: pgmap v914: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:58.047 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-23T18:54:58.055 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:54:58.063 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2982, 2024-04-23T18:54:58.070 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2455, 2024-04-23T18:54:58.077 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 2438051, 2024-04-23T18:54:58.083 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 954236, 2024-04-23T18:54:58.090 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 10, 2024-04-23T18:54:58.153 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-04-23T18:54:58.242 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.302242422, 2024-04-23T18:54:58.248 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.118988005, 2024-04-23T18:54:58.255 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.173778691, 2024-04-23T18:54:58.265 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.026747781, 2024-04-23T18:54:58.333 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:58.364 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2454, 2024-04-23T18:54:58.440 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.120421896, 2024-04-23T18:54:58.447 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000049071 2024-04-23T18:54:58.458 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:58.466 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:58.564 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4060, 2024-04-23T18:54:58.573 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001242943, 2024-04-23T18:54:58.583 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000306 2024-04-23T18:54:58.593 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:58.602 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1776368, 2024-04-23T18:54:58.611 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 322080 2024-04-23T18:54:58.650 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:58.704 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:54:58.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 45, 2024-04-23T18:54:58.723 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 470, 2024-04-23T18:54:58.732 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 11270, 2024-04-23T18:54:58.773 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 933903, 2024-04-23T18:54:58.853 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 21, 2024-04-23T18:54:58.901 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-04-23T18:54:58.910 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.032348006, 2024-04-23T18:54:58.920 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.018126859, 2024-04-23T18:54:59.000 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.009700343, 2024-04-23T18:54:59.009 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000276634, 2024-04-23T18:54:59.018 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:59.027 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 468, 2024-04-23T18:54:59.035 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.033658234, 2024-04-23T18:54:59.044 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000071919 2024-04-23T18:54:59.054 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:59.083 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:59.138 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 101, 2024-04-23T18:54:59.148 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000024198, 2024-04-23T18:54:59.157 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000239 2024-04-23T18:54:59.165 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:59.240 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 7616, 2024-04-23T18:54:59.328 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 857568 2024-04-23T18:54:59.398 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:59.405 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:54:59.414 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 15, 2024-04-23T18:54:59.492 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 17, 2024-04-23T18:54:59.500 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 4117, 2024-04-23T18:54:59.506 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 12463, 2024-04-23T18:54:59.513 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 11, 2024-04-23T18:54:59.520 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-04-23T18:54:59.528 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.006425054, 2024-04-23T18:54:59.535 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.000266716, 2024-04-23T18:54:59.588 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.005299479, 2024-04-23T18:54:59.644 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000364316, 2024-04-23T18:54:59.651 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:54:59.659 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 15, 2024-04-23T18:54:59.667 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.044880265, 2024-04-23T18:54:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002992017 2024-04-23T18:54:59.792 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:59.799 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:54:59.871 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:54:59.878 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:54:59.960 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:54:59.967 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:59.975 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:54:59.982 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:54:59.990 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:54:59.997 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-04-23T18:55:00.004 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-04-23T18:55:00.049 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-04-23T18:55:00.103 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:00.110 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-04-23T18:55:00.117 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-04-23T18:55:00.127 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-04-23T18:55:00.168 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 51, 2024-04-23T18:55:00.223 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001446101, 2024-04-23T18:55:00.268 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000028354 2024-04-23T18:55:00.275 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:00.282 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:00.385 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-04-23T18:55:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-04-23T18:55:00.419 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-04-23T18:55:00.424 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-04-23T18:55:00.430 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000013804, 2024-04-23T18:55:00.438 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000006902 2024-04-23T18:55:00.446 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:00.483 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:00.531 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-04-23T18:55:00.540 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-04-23T18:55:00.550 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-04-23T18:55:00.559 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-04-23T18:55:00.593 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:00.690 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:00.700 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:00.763 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:00.772 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-04-23T18:55:00.851 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-04-23T18:55:00.861 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-04-23T18:55:00.871 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-04-23T18:55:00.880 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 11, 2024-04-23T18:55:00.890 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-04-23T18:55:00.899 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-04-23T18:55:00.908 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 17, 2024-04-23T18:55:00.941 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 1, 2024-04-23T18:55:01.023 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 16, 2024-04-23T18:55:01.031 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-04-23T18:55:01.039 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 13, 2024-04-23T18:55:01.047 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 4, 2024-04-23T18:55:01.080 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-04-23T18:55:01.164 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-04-23T18:55:01.222 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 3, 2024-04-23T18:55:01.231 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 38, 2024-04-23T18:55:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 30, 2024-04-23T18:55:01.342 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-04-23T18:55:01.348 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 8, 2024-04-23T18:55:01.357 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:55:01.366 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-04-23T18:55:01.375 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-04-23T18:55:01.385 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-04-23T18:55:01.394 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-04-23T18:55:01.424 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-04-23T18:55:01.490 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-04-23T18:55:01.497 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-04-23T18:55:01.504 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-04-23T18:55:01.512 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-04-23T18:55:01.541 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:55:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-04-23T18:55:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 5, 2024-04-23T18:55:01.667 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 298, 2024-04-23T18:55:01.673 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-04-23T18:55:01.771 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:55:01.779 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-04-23T18:55:01.786 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:55:01.794 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:55:01.802 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:55:01.810 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:55:01.817 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:55:01.862 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-04-23T18:55:01.910 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:55:01.918 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:55:01.926 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:55:01.935 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:55:01.980 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:55:02.037 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:55:02.108 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-04-23T18:55:02.116 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:55:02.124 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-04-23T18:55:02.194 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-04-23T18:55:02.202 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:02.209 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-04-23T18:55:02.217 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-04-23T18:55:02.224 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 21, 2024-04-23T18:55:02.231 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 14, 2024-04-23T18:55:02.238 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 12, 2024-04-23T18:55:02.288 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-04-23T18:55:02.351 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-04-23T18:55:02.358 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-04-23T18:55:02.365 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:55:02.372 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-04-23T18:55:02.414 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-04-23T18:55:02.475 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-04-23T18:55:02.544 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-04-23T18:55:02.553 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-04-23T18:55:02.560 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:55:02.659 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:55:02.670 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-04-23T18:55:02.680 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-04-23T18:55:02.689 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 12, 2024-04-23T18:55:02.695 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 71, 2024-04-23T18:55:02.701 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:55:02.708 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-04-23T18:55:02.745 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-04-23T18:55:02.801 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:55:02.813 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-04-23T18:55:02.821 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-04-23T18:55:02.830 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:02.874 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-04-23T18:55:02.965 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-04-23T18:55:03.025 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 5, 2024-04-23T18:55:03.033 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-04-23T18:55:03.043 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-04-23T18:55:03.109 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 5, 2024-04-23T18:55:03.118 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-04-23T18:55:03.126 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-04-23T18:55:03.135 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 2, 2024-04-23T18:55:03.143 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-04-23T18:55:03.152 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-04-23T18:55:03.161 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 2, 2024-04-23T18:55:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 2, 2024-04-23T18:55:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-04-23T18:55:03.286 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-04-23T18:55:03.295 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-04-23T18:55:03.303 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 4195815, 2024-04-23T18:55:03.355 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-04-23T18:55:03.437 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-04-23T18:55:03.505 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4, 2024-04-23T18:55:03.514 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.007544780, 2024-04-23T18:55:03.523 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001886195 2024-04-23T18:55:03.582 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:03.592 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-04-23T18:55:03.600 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:03.606 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-04-23T18:55:03.614 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 21, 2024-04-23T18:55:03.623 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 23, 2024-04-23T18:55:03.632 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 2, 2024-04-23T18:55:03.678 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 20, 2024-04-23T18:55:03.778 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 23, 2024-04-23T18:55:03.785 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 3, 2024-04-23T18:55:03.794 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 17, 2024-04-23T18:55:03.804 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 21, 2024-04-23T18:55:03.838 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 4, 2024-04-23T18:55:03.906 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-04-23T18:55:03.969 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-04-23T18:55:03.976 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-04-23T18:55:03.985 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 53492, 2024-04-23T18:55:04.035 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 223608 2024-04-23T18:55:04.041 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:04.048 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-04-23T18:55:04.055 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-04-23T18:55:04.068 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-04-23T18:55:04.078 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-04-23T18:55:04.088 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 23, 2024-04-23T18:55:04.148 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 71, 2024-04-23T18:55:04.226 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-04-23T18:55:04.234 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:04.241 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:04.249 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:04.257 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:04.324 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-04-23T18:55:04.332 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:04.393 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:04.401 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:04.512 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:04.520 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-04-23T18:55:04.528 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:04.535 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:04.542 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:04.549 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:04.556 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-04-23T18:55:04.564 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-04-23T18:55:04.733 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:04.780 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:04.787 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:04.821 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:04.844 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-04-23T18:55:04.913 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:04.921 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:04.927 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:04.935 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:04.942 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-04-23T18:55:04.949 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:04.955 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:05.003 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:05.076 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:05.083 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-04-23T18:55:05.092 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:05.101 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:05.110 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:05.159 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:05.223 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-04-23T18:55:05.232 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:05.241 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:05.337 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:05.344 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:05.352 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-04-23T18:55:05.361 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:05.370 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:05.378 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:05.386 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:05.431 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-04-23T18:55:05.507 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:05.516 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:05.525 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:05.534 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:05.542 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-04-23T18:55:05.596 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:05.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:54:59 smithi019 ceph-mon[29215]: pgmap v915: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:05.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:01 smithi019 ceph-mon[29215]: pgmap v916: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:05.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:03 smithi019 ceph-mon[29215]: pgmap v917: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:05.762 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:57 smithi044 ceph-mon[32458]: pgmap v914: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:05.769 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:54:59 smithi044 ceph-mon[32458]: pgmap v915: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:05.778 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:01 smithi044 ceph-mon[32458]: pgmap v916: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:05.786 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:03 smithi044 ceph-mon[32458]: pgmap v917: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:05.795 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:05.804 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:05.812 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:05.856 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-04-23T18:55:05.910 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:05.920 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:05.929 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:05.936 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:05.979 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-04-23T18:55:06.013 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:06.022 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:06.091 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:06.100 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:06.149 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-04-23T18:55:06.158 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:06.167 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:06.173 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:06.180 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-04-23T18:55:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:06.236 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:06.292 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:06.300 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:06.307 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-04-23T18:55:06.314 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:06.354 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:06.390 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:06.397 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:06.468 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-04-23T18:55:06.476 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:06.526 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:06.534 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:06.542 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:06.551 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-04-23T18:55:06.558 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:06.566 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:06.573 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:06.623 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:06.680 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-04-23T18:55:06.686 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:06.693 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:06.700 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:06.737 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:06.784 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-04-23T18:55:06.792 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:06.848 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:06.856 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:06.896 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:06.903 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-04-23T18:55:06.909 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:06.916 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:06.922 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:06.933 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:06.940 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-04-23T18:55:07.002 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:07.096 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:07.103 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:07.109 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:07.115 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-04-23T18:55:07.122 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:07.195 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:07.261 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:07.268 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:07.275 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-04-23T18:55:07.358 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:07.364 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:07.372 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:07.382 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:07.391 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-04-23T18:55:07.400 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:07.409 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:07.456 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:07.542 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:07.553 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-04-23T18:55:07.564 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:07.573 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:07.612 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:07.662 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:07.735 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-04-23T18:55:07.744 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:07.753 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:07.840 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:07.847 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:07.854 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-04-23T18:55:07.861 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:07.870 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:07.879 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:07.888 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:07.936 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:55:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:55:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-04-23T18:55:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:08.095 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:08.163 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:08.222 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:08.231 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-04-23T18:55:08.237 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-04-23T18:55:08.301 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 2, 2024-04-23T18:55:08.308 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 1, 2024-04-23T18:55:08.316 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2024-04-23T18:55:08.323 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-04-23T18:55:08.331 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-04-23T18:55:08.340 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-04-23T18:55:08.348 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1676, 2024-04-23T18:55:08.389 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-04-23T18:55:08.463 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:08.469 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-04-23T18:55:08.476 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:55:08.483 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-04-23T18:55:08.546 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:55:08.604 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:55:08.650 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:55:08.658 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:55:08.665 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:55:08.753 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:55:08.760 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:55:08.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:55:08.779 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:55:08.786 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:55:08.792 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:55:08.805 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:55:08.838 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:55:08.957 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-04-23T18:55:08.965 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:55:08.974 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-04-23T18:55:08.981 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:55:09.039 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:55:09.104 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:55:09.173 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:55:09.180 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:55:09.188 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:55:09.299 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:55:09.307 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-04-23T18:55:09.314 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:55:09.324 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:55:09.333 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:55:09.340 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-04-23T18:55:09.347 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-04-23T18:55:09.393 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-04-23T18:55:09.479 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:55:09.488 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:55:09.498 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:55:09.506 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:55:09.539 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 238039, 2024-04-23T18:55:09.595 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 73, 2024-04-23T18:55:09.653 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:55:09.662 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-04-23T18:55:09.672 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-04-23T18:55:09.760 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-04-23T18:55:09.767 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:55:09.777 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-04-23T18:55:09.788 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:55:09.797 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-04-23T18:55:09.806 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8032, 2024-04-23T18:55:09.815 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 167, 2024-04-23T18:55:09.860 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:55:09.868 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:55:09.877 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-04-23T18:55:09.886 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-04-23T18:55:09.894 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 87717, 2024-04-23T18:55:09.922 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 351, 2024-04-23T18:55:10.013 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-04-23T18:55:10.067 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-04-23T18:55:10.076 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-04-23T18:55:10.087 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-04-23T18:55:10.168 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:10.175 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-04-23T18:55:10.183 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-04-23T18:55:10.192 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-04-23T18:55:10.202 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 40, 2024-04-23T18:55:10.210 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 11813, 2024-04-23T18:55:10.219 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 32, 2024-04-23T18:55:10.255 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 40, 2024-04-23T18:55:10.346 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-04-23T18:55:10.354 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 40, 2024-04-23T18:55:10.362 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.226995286, 2024-04-23T18:55:10.370 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.005674882 2024-04-23T18:55:10.405 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:10.506 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-04-23T18:55:10.566 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-04-23T18:55:10.574 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 40, 2024-04-23T18:55:10.582 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 62 2024-04-23T18:55:10.700 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:10.707 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 40, 2024-04-23T18:55:10.714 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 0, 2024-04-23T18:55:10.725 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 40, 2024-04-23T18:55:10.731 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-04-23T18:55:10.737 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-04-23T18:55:10.744 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2024-04-23T18:55:10.793 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 10, 2024-04-23T18:55:10.891 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 0, 2024-04-23T18:55:10.899 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 4, 2024-04-23T18:55:10.907 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 7, 2024-04-23T18:55:10.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-04-23T18:55:10.947 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-04-23T18:55:10.995 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-04-23T18:55:11.063 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-04-23T18:55:11.070 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 6, 2024-04-23T18:55:11.077 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-04-23T18:55:11.167 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-04-23T18:55:11.173 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-04-23T18:55:11.181 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 0, 2024-04-23T18:55:11.187 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 20, 2024-04-23T18:55:11.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:55:11.201 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-04-23T18:55:11.208 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:55:11.216 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-04-23T18:55:11.224 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-04-23T18:55:11.255 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-04-23T18:55:11.299 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:55:11.306 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-04-23T18:55:11.338 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:55:11.410 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-04-23T18:55:11.476 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-04-23T18:55:11.484 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-04-23T18:55:11.490 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-04-23T18:55:11.559 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-04-23T18:55:11.565 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-04-23T18:55:11.571 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-04-23T18:55:11.580 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-04-23T18:55:11.591 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-04-23T18:55:11.601 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-04-23T18:55:11.610 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-04-23T18:55:11.646 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-04-23T18:55:11.716 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-04-23T18:55:11.725 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-04-23T18:55:11.734 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-04-23T18:55:11.746 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-04-23T18:55:11.778 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-04-23T18:55:11.836 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 136, 2024-04-23T18:55:11.905 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-04-23T18:55:11.914 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 50, 2024-04-23T18:55:11.923 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 9, 2024-04-23T18:55:11.983 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 22, 2024-04-23T18:55:11.990 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 13, 2024-04-23T18:55:11.999 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-04-23T18:55:12.009 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 14, 2024-04-23T18:55:12.016 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 0, 2024-04-23T18:55:12.025 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 1 2024-04-23T18:55:12.034 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:12.084 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-04-23T18:55:12.182 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-04-23T18:55:12.189 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:55:12.198 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-04-23T18:55:12.207 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-04-23T18:55:12.237 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-04-23T18:55:12.358 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 82 2024-04-23T18:55:12.367 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:12.376 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-04-23T18:55:12.467 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-04-23T18:55:12.473 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:55:12.482 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-04-23T18:55:12.490 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-04-23T18:55:12.499 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-04-23T18:55:12.510 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-04-23T18:55:12.518 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-04-23T18:55:12.587 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:12.670 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:55:12.678 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:55:12.687 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-04-23T18:55:12.697 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:55:12.729 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 3038, 2024-04-23T18:55:12.786 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2145383, 2024-04-23T18:55:12.855 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:55:12.864 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 3038, 2024-04-23T18:55:12.873 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:55:12.948 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:55:12.954 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3038, 2024-04-23T18:55:12.966 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2145383, 2024-04-23T18:55:12.974 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:55:12.982 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:12.990 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:12.997 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:13.037 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:13.090 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:13.097 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-04-23T18:55:13.105 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:55:13.112 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-04-23T18:55:13.277 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:55:13.313 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:55:13.346 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:55:13.380 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:55:13.413 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:55:13.446 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 40, 2024-04-23T18:55:13.480 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 11813, 2024-04-23T18:55:13.513 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 40, 2024-04-23T18:55:13.547 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 11813, 2024-04-23T18:55:13.580 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:55:13.613 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:13.647 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:13.680 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:13.713 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:13.747 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:13.780 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-04-23T18:55:13.830 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:55:13.906 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-04-23T18:55:13.939 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:55:13.972 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:55:14.005 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:55:14.039 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:55:14.072 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:55:14.105 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 40, 2024-04-23T18:55:14.139 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 40, 2024-04-23T18:55:14.172 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 40, 2024-04-23T18:55:14.205 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 40, 2024-04-23T18:55:14.239 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:55:14.272 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:14.306 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:14.339 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:14.372 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:14.431 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:14.464 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:55:14.512 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:55:14.547 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-04-23T18:55:14.581 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:55:14.614 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:55:14.647 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:55:14.681 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:55:14.715 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:55:14.748 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:55:14.781 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:55:14.815 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-04-23T18:55:14.848 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-04-23T18:55:14.881 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:55:14.915 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:14.948 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:14.998 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:15.032 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:15.065 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:15.124 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x5595759d0500": { 2024-04-23T18:55:15.157 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:55:15.190 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-04-23T18:55:15.223 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:55:15.257 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 5, 2024-04-23T18:55:15.290 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1511, 2024-04-23T18:55:15.323 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:55:15.357 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 5, 2024-04-23T18:55:15.390 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:55:15.424 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:55:15.457 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 4, 2024-04-23T18:55:15.490 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1511, 2024-04-23T18:55:15.524 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:55:15.582 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:15.615 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:15.649 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:15.682 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:15.745 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:15.795 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-23T18:55:15.833 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-23T18:55:15.866 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-23T18:55:15.899 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-23T18:55:15.932 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=21 2024-04-23T18:55:15.966 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-23T18:55:15.999 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:55:16.032 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:55:16.066 DEBUG:tasks.check_counter:Getting stats from l 2024-04-23T18:55:16.099 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 46706838-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-23T18:55:16.134 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:54:59 smithi155 ceph-mon[33468]: pgmap v915: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:16.182 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:01 smithi155 ceph-mon[33468]: pgmap v916: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:16.216 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:03 smithi155 ceph-mon[33468]: pgmap v917: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:16.249 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:05 smithi155 ceph-mon[33468]: pgmap v918: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:55:16.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:07 smithi155 ceph-mon[33468]: pgmap v919: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:55:16.316 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:09 smithi155 ceph-mon[33468]: pgmap v920: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:55:16.365 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:11 smithi155 ceph-mon[33468]: pgmap v921: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:55:16.399 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:11 smithi155 ceph-mon[33468]: osdmap e137: 12 total, 12 up, 12 in 2024-04-23T18:55:16.441 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:13 smithi155 ceph-mon[33468]: pgmap v923: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:55:16.474 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:15 smithi155 ceph-mon[33468]: pgmap v924: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:16.510 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:05 smithi019 ceph-mon[29215]: pgmap v918: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:55:16.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:07 smithi019 ceph-mon[29215]: pgmap v919: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:55:16.574 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:09 smithi019 ceph-mon[29215]: pgmap v920: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:55:16.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:11 smithi019 ceph-mon[29215]: pgmap v921: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:55:16.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:11 smithi019 ceph-mon[29215]: osdmap e137: 12 total, 12 up, 12 in 2024-04-23T18:55:16.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:13 smithi019 ceph-mon[29215]: pgmap v923: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:55:16.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:15 smithi019 ceph-mon[29215]: pgmap v924: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:16.758 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:05 smithi044 ceph-mon[32458]: pgmap v918: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:55:16.791 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:07 smithi044 ceph-mon[32458]: pgmap v919: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:55:16.825 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:09 smithi044 ceph-mon[32458]: pgmap v920: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:55:16.858 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:11 smithi044 ceph-mon[32458]: pgmap v921: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:55:16.892 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:11 smithi044 ceph-mon[32458]: osdmap e137: 12 total, 12 up, 12 in 2024-04-23T18:55:16.925 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:13 smithi044 ceph-mon[32458]: pgmap v923: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T18:55:16.998 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:15 smithi044 ceph-mon[32458]: pgmap v924: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:17.035 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:55:17.825 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:17 smithi155 ceph-mon[33468]: pgmap v925: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:18.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:17 smithi019 ceph-mon[29215]: pgmap v925: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:18.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:17 smithi044 ceph-mon[32458]: pgmap v925: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:18.760 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-23T18:55:18.800 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:55:18.833 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 582, 2024-04-23T18:55:18.867 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 566, 2024-04-23T18:55:18.900 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1757810, 2024-04-23T18:55:18.950 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 325316, 2024-04-23T18:55:18.983 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2024-04-23T18:55:19.017 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 1, 2024-04-23T18:55:19.050 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.099740405, 2024-04-23T18:55:19.083 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.035679105, 2024-04-23T18:55:19.117 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.054658429, 2024-04-23T18:55:19.150 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.013614362, 2024-04-23T18:55:19.246 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:55:19.283 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 565, 2024-04-23T18:55:19.317 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.029693204, 2024-04-23T18:55:19.350 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000052554 2024-04-23T18:55:19.383 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:19.417 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:55:19.450 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:19.484 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:19.517 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:19.567 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:19.601 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1756976, 2024-04-23T18:55:19.634 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 324992 2024-04-23T18:55:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:19.701 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:55:19.734 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1814, 2024-04-23T18:55:19.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2200, 2024-04-23T18:55:19.801 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 446094, 2024-04-23T18:55:19.850 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1423661, 2024-04-23T18:55:19.884 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2024-04-23T18:55:19.918 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-04-23T18:55:19.951 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.217682190, 2024-04-23T18:55:19.984 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.068148085, 2024-04-23T18:55:20.019 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.048772359, 2024-04-23T18:55:20.051 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.078552160, 2024-04-23T18:55:20.084 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:55:20.134 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2197, 2024-04-23T18:55:20.168 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.107685085, 2024-04-23T18:55:20.201 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000049014 2024-04-23T18:55:20.234 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:20.268 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:55:20.301 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:20.335 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:20.368 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:20.401 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:20.451 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:55:20.485 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 847440 2024-04-23T18:55:20.518 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:20.551 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:55:20.585 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 3757, 2024-04-23T18:55:20.663 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 3757, 2024-04-23T18:55:20.701 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1126875, 2024-04-23T18:55:20.751 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1193960, 2024-04-23T18:55:20.785 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-04-23T18:55:20.818 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-04-23T18:55:20.852 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.423323057, 2024-04-23T18:55:20.885 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.132382020, 2024-04-23T18:55:20.918 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.111413887, 2024-04-23T18:55:20.952 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.140601683, 2024-04-23T18:55:21.043 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:55:21.102 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3756, 2024-04-23T18:55:21.177 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.164418657, 2024-04-23T18:55:21.243 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000043774 2024-04-23T18:55:21.310 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:21.368 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:55:21.427 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:21.494 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:21.643 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:21.710 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-23T18:55:21.794 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T18:55:21.844 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:21.894 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-04-23T18:55:21.944 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-04-23T18:55:21.994 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-04-23T18:55:22.044 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:22.094 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-04-23T18:55:22.144 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-04-23T18:55:22.194 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-04-23T18:55:22.298 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 9123, 2024-04-23T18:55:22.344 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.302016366, 2024-04-23T18:55:22.394 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000033104 2024-04-23T18:55:22.444 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:22.494 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:22.569 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-04-23T18:55:22.619 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-04-23T18:55:22.669 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-04-23T18:55:22.736 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:22.786 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:22.836 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:22.886 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:22.936 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:22.986 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-04-23T18:55:23.036 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-04-23T18:55:23.086 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-04-23T18:55:23.137 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-04-23T18:55:23.211 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:23.261 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:23.311 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:23.361 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:23.436 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-04-23T18:55:23.486 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-04-23T18:55:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-04-23T18:55:23.586 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-04-23T18:55:23.636 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-04-23T18:55:23.687 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-04-23T18:55:23.736 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-04-23T18:55:23.786 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 0, 2024-04-23T18:55:23.837 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 0, 2024-04-23T18:55:23.887 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 0, 2024-04-23T18:55:23.937 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-04-23T18:55:23.987 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 0, 2024-04-23T18:55:24.037 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-04-23T18:55:24.087 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-04-23T18:55:24.137 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-04-23T18:55:24.187 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 1, 2024-04-23T18:55:24.279 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-04-23T18:55:24.329 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-04-23T18:55:24.379 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-04-23T18:55:24.429 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-04-23T18:55:24.479 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:55:24.529 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-04-23T18:55:24.579 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-04-23T18:55:24.637 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-04-23T18:55:24.679 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-04-23T18:55:24.729 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-04-23T18:55:24.779 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-04-23T18:55:24.829 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-04-23T18:55:24.879 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-04-23T18:55:24.929 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-04-23T18:55:25.053 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:55:25.112 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-04-23T18:55:25.221 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 0, 2024-04-23T18:55:25.271 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 0, 2024-04-23T18:55:25.321 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-04-23T18:55:25.371 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:55:25.421 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-04-23T18:55:25.488 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:55:25.537 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:55:25.587 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:55:25.638 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:55:25.688 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:55:25.738 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-04-23T18:55:25.787 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:55:25.837 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:55:25.888 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:55:25.938 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:55:26.007 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:55:26.088 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:55:26.138 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-04-23T18:55:26.188 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:55:26.238 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-04-23T18:55:26.288 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-04-23T18:55:26.338 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-04-23T18:55:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-04-23T18:55:26.489 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-04-23T18:55:26.539 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-04-23T18:55:26.588 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-04-23T18:55:26.639 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-04-23T18:55:26.689 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-04-23T18:55:26.739 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-04-23T18:55:26.789 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:55:26.839 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-04-23T18:55:26.914 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-04-23T18:55:26.964 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-04-23T18:55:27.014 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-04-23T18:55:27.064 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-04-23T18:55:27.114 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:55:27.164 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:55:27.218 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-04-23T18:55:27.264 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-04-23T18:55:27.314 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:55:27.364 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:55:27.414 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:55:27.464 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-04-23T18:55:27.514 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-04-23T18:55:27.564 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:55:27.614 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-04-23T18:55:27.664 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-04-23T18:55:27.739 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:27.789 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-04-23T18:55:27.839 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-04-23T18:55:27.889 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-04-23T18:55:27.939 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-04-23T18:55:27.989 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-04-23T18:55:28.039 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 7, 2024-04-23T18:55:28.089 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-04-23T18:55:28.139 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-04-23T18:55:28.198 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-04-23T18:55:28.248 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-04-23T18:55:28.298 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-04-23T18:55:28.348 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 2, 2024-04-23T18:55:28.398 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 2, 2024-04-23T18:55:28.448 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-04-23T18:55:28.498 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-04-23T18:55:28.573 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-04-23T18:55:28.623 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 4196264, 2024-04-23T18:55:28.733 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 4196123, 2024-04-23T18:55:28.831 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-04-23T18:55:28.881 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:28.931 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:28.981 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:29.031 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:29.082 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 7 2024-04-23T18:55:29.131 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:29.181 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-04-23T18:55:29.231 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 3, 2024-04-23T18:55:29.282 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 3, 2024-04-23T18:55:29.331 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 0, 2024-04-23T18:55:29.382 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 1, 2024-04-23T18:55:29.432 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 1, 2024-04-23T18:55:29.490 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 0, 2024-04-23T18:55:29.540 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 0, 2024-04-23T18:55:29.590 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 0, 2024-04-23T18:55:29.683 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 0, 2024-04-23T18:55:29.732 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-04-23T18:55:29.782 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-04-23T18:55:29.832 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-04-23T18:55:29.882 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 49916, 2024-04-23T18:55:29.932 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 215416 2024-04-23T18:55:29.982 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:30.032 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-04-23T18:55:30.082 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-04-23T18:55:30.132 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-04-23T18:55:30.182 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-04-23T18:55:30.232 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-04-23T18:55:30.282 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-04-23T18:55:30.349 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-04-23T18:55:30.399 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:30.449 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:30.499 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:30.582 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:30.632 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-04-23T18:55:30.682 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:30.732 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:30.783 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:30.832 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:30.882 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-04-23T18:55:30.933 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:30.982 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:31.033 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-04-23T18:55:31.208 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:31.258 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:31.307 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:31.357 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:31.408 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-04-23T18:55:31.484 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:31.533 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:31.583 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:31.633 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:31.683 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-04-23T18:55:31.775 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:31.825 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:31.875 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:31.925 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:31.975 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-04-23T18:55:32.025 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:32.175 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:32.225 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:32.325 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-04-23T18:55:32.375 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:32.425 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:32.542 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:32.608 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:32.658 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-04-23T18:55:32.708 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:32.758 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:32.808 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:32.858 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:32.909 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-04-23T18:55:32.984 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:33.033 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:33.100 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:33.150 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:33.200 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-04-23T18:55:33.250 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:33.300 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:33.350 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:33.400 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:33.495 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-04-23T18:55:33.542 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:33.594 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:33.642 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:33.692 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:33.742 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-04-23T18:55:33.792 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:33.842 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:33.892 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:33.959 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:34.009 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-04-23T18:55:34.059 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:34.109 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:34.159 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:34.209 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:34.259 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-04-23T18:55:34.309 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:34.407 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:34.451 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:34.501 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:34.551 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-04-23T18:55:34.601 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:34.651 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:34.701 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:34.751 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:34.809 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-04-23T18:55:34.859 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:34.909 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:34.959 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:35.010 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:35.060 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-04-23T18:55:35.110 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:35.160 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:35.210 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:35.294 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:35.355 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-04-23T18:55:35.401 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:35.452 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:35.501 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:35.552 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:35.601 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-04-23T18:55:35.668 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:35.718 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:35.768 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:35.818 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-04-23T18:55:35.918 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:35.968 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:36.027 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:36.077 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:36.127 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-04-23T18:55:36.231 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:36.277 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:36.327 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:36.377 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:36.427 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-04-23T18:55:36.477 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:36.544 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:36.594 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:36.644 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:36.729 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-04-23T18:55:36.768 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:36.801 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:36.835 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:36.868 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:36.910 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-04-23T18:55:36.943 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:36.977 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:37.033 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:37.068 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:37.102 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-04-23T18:55:37.135 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:37.168 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:37.202 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:37.269 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:37.302 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-04-23T18:55:37.335 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:37.369 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:37.402 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:37.435 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:37.469 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-04-23T18:55:37.502 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:37.585 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:37.619 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:37.690 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-04-23T18:55:37.727 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:37.761 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:37.794 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:37.861 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:37.894 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:55:37.927 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:55:37.961 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-04-23T18:55:37.994 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:38.027 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:38.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:38.094 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:38.128 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:38.161 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-04-23T18:55:38.194 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-04-23T18:55:38.228 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2024-04-23T18:55:38.261 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2024-04-23T18:55:38.332 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2024-04-23T18:55:38.369 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-04-23T18:55:38.403 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-04-23T18:55:38.453 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-04-23T18:55:38.495 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 4153, 2024-04-23T18:55:38.528 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-04-23T18:55:38.561 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:38.603 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-04-23T18:55:38.636 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:55:38.670 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-04-23T18:55:38.703 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:55:38.736 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:55:38.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:55:38.803 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:55:38.836 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:55:38.870 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:55:38.903 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:55:38.978 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:55:39.011 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:55:39.095 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:55:39.128 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:55:39.162 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:55:39.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:55:39.228 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-04-23T18:55:39.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:55:39.304 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-04-23T18:55:39.337 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:55:39.370 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:55:39.403 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:55:39.504 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:55:39.537 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:55:39.570 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:55:39.603 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:55:39.637 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-04-23T18:55:39.715 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:55:39.779 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:55:39.836 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:55:39.871 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-04-23T18:55:39.937 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-04-23T18:55:39.970 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-04-23T18:55:40.004 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:55:40.045 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:55:40.079 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:55:40.112 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:55:40.145 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 189383, 2024-04-23T18:55:40.179 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 50, 2024-04-23T18:55:40.212 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:55:40.246 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-04-23T18:55:40.279 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-04-23T18:55:40.321 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-04-23T18:55:40.354 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:55:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-04-23T18:55:40.446 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:55:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-04-23T18:55:40.513 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8152, 2024-04-23T18:55:40.546 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 170, 2024-04-23T18:55:40.579 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:55:40.621 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:55:40.655 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-04-23T18:55:40.763 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-04-23T18:55:40.796 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 8512, 2024-04-23T18:55:40.829 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 13, 2024-04-23T18:55:40.863 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-04-23T18:55:40.896 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-04-23T18:55:40.930 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-04-23T18:55:40.963 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-04-23T18:55:40.996 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:41.046 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-04-23T18:55:41.107 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-04-23T18:55:41.155 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-04-23T18:55:41.195 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 5566, 2024-04-23T18:55:41.230 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 21, 2024-04-23T18:55:41.263 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 3, 2024-04-23T18:55:41.297 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 5566, 2024-04-23T18:55:41.339 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-04-23T18:55:41.380 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5566, 2024-04-23T18:55:41.422 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3.182933648, 2024-04-23T18:55:41.464 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000571852 2024-04-23T18:55:41.497 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:41.539 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-04-23T18:55:41.572 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-04-23T18:55:41.614 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5566, 2024-04-23T18:55:41.655 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 5567 2024-04-23T18:55:41.714 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:41.757 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 5566, 2024-04-23T18:55:41.813 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 5566, 2024-04-23T18:55:41.847 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-04-23T18:55:41.889 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-04-23T18:55:41.931 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-04-23T18:55:41.972 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 3683, 2024-04-23T18:55:42.014 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-04-23T18:55:42.056 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1881, 2024-04-23T18:55:42.097 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-04-23T18:55:42.131 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-04-23T18:55:42.164 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-04-23T18:55:42.206 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-04-23T18:55:42.248 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-04-23T18:55:42.290 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-04-23T18:55:42.331 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-04-23T18:55:42.390 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-04-23T18:55:42.431 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-04-23T18:55:42.473 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-04-23T18:55:42.527 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 1, 2024-04-23T18:55:42.564 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-04-23T18:55:42.606 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:55:42.648 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-04-23T18:55:42.690 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:55:42.731 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-04-23T18:55:42.773 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-04-23T18:55:42.815 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-04-23T18:55:42.857 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:55:42.898 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-04-23T18:55:42.940 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:55:42.982 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-04-23T18:55:43.023 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-04-23T18:55:43.082 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-04-23T18:55:43.123 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-04-23T18:55:43.173 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-04-23T18:55:43.215 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-04-23T18:55:43.264 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-04-23T18:55:43.298 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-04-23T18:55:43.332 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-04-23T18:55:43.365 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-04-23T18:55:43.407 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-04-23T18:55:43.440 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-04-23T18:55:43.473 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-04-23T18:55:43.507 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-04-23T18:55:43.540 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-04-23T18:55:43.574 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-04-23T18:55:43.607 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-04-23T18:55:43.640 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 138, 2024-04-23T18:55:43.691 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-04-23T18:55:43.724 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 52, 2024-04-23T18:55:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 5, 2024-04-23T18:55:43.791 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 8, 2024-04-23T18:55:43.841 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 3, 2024-04-23T18:55:43.903 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-04-23T18:55:43.941 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-04-23T18:55:43.974 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-04-23T18:55:44.007 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-04-23T18:55:44.041 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:44.074 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-04-23T18:55:44.107 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-04-23T18:55:44.141 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:55:44.207 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-04-23T18:55:44.241 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-04-23T18:55:44.274 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:44.324 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-04-23T18:55:44.357 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-04-23T18:55:44.391 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:44.424 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-04-23T18:55:44.458 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-04-23T18:55:44.491 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:55:44.543 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-04-23T18:55:44.583 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-04-23T18:55:44.616 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-04-23T18:55:44.658 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-04-23T18:55:44.691 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-04-23T18:55:44.725 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:44.758 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:55:44.791 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:55:44.825 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-04-23T18:55:44.858 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:55:44.900 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 6149, 2024-04-23T18:55:44.933 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2744647, 2024-04-23T18:55:44.966 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:55:45.000 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 6149, 2024-04-23T18:55:45.033 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:55:45.066 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:55:45.100 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 6149, 2024-04-23T18:55:45.154 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2744647, 2024-04-23T18:55:45.192 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:55:45.225 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:45.258 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:45.292 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:45.325 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:45.367 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:45.400 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-04-23T18:55:45.433 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:55:45.484 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-04-23T18:55:45.517 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:55:45.550 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:55:45.592 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:55:45.625 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:55:45.659 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:55:45.692 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 5566, 2024-04-23T18:55:45.725 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 1965, 2024-04-23T18:55:45.782 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 5566, 2024-04-23T18:55:45.817 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1965, 2024-04-23T18:55:45.850 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:55:45.892 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:45.926 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:45.959 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:45.992 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:46.026 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:46.084 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-04-23T18:55:46.126 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:55:46.159 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-04-23T18:55:46.192 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:55:46.226 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:55:46.259 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:55:46.293 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:55:46.326 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:55:46.359 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 5566, 2024-04-23T18:55:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 5566, 2024-04-23T18:55:46.459 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 5566, 2024-04-23T18:55:46.493 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 5566, 2024-04-23T18:55:46.526 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:55:46.559 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:46.601 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:46.635 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:46.693 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:46.726 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:46.759 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:55:46.793 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:55:46.826 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-04-23T18:55:46.868 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:55:46.901 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:55:46.935 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:55:46.968 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:55:47.010 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:55:47.078 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:55:47.118 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:55:47.151 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-04-23T18:55:47.185 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-04-23T18:55:47.251 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:55:47.286 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:47.335 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:47.369 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:47.402 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:47.435 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-23T18:55:47.468 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x558b7ce2c6e0": { 2024-04-23T18:55:47.502 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-04-23T18:55:47.535 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-04-23T18:55:47.569 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-04-23T18:55:47.602 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-04-23T18:55:47.710 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-04-23T18:55:47.744 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-04-23T18:55:47.810 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-04-23T18:55:47.844 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-04-23T18:55:47.877 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-04-23T18:55:47.910 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-04-23T18:55:47.952 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-04-23T18:55:48.002 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-04-23T18:55:48.036 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-04-23T18:55:48.086 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-04-23T18:55:48.119 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-04-23T18:55:48.152 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:48.186 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-23T18:55:48.219 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-23T18:55:48.253 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:19 smithi155 ceph-mon[33468]: osdmap e138: 12 total, 12 up, 12 in 2024-04-23T18:55:48.289 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-23T18:55:48.319 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-23T18:55:48.352 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-23T18:55:48.401 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-23T18:55:48.471 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-23T18:55:48.511 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:55:48.544 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:55:48.578 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-23T18:55:48.638 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-23T18:55:48.661 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:55:48.694 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:55:48.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:19 smithi019 ceph-mon[29215]: osdmap e138: 12 total, 12 up, 12 in 2024-04-23T18:55:48.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:20 smithi019 ceph-mon[29215]: pgmap v927: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:55:48.811 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:22 smithi019 ceph-mon[29215]: pgmap v928: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 309 B/s wr, 10 op/s 2024-04-23T18:55:48.844 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:24 smithi019 ceph-mon[29215]: pgmap v929: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-23T18:55:48.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:26 smithi019 ceph-mon[29215]: pgmap v930: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:48.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:28 smithi019 ceph-mon[29215]: pgmap v931: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:49.020 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:30 smithi019 ceph-mon[29215]: pgmap v932: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 193 B/s wr, 9 op/s 2024-04-23T18:55:49.053 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:32 smithi019 ceph-mon[29215]: pgmap v933: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-23T18:55:49.086 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:34 smithi019 ceph-mon[29215]: pgmap v934: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:49.138 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:36 smithi019 ceph-mon[29215]: pgmap v935: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:49.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:38 smithi019 ceph-mon[29215]: pgmap v936: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:49.245 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:40 smithi019 ceph-mon[29215]: pgmap v937: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:49.278 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:42 smithi019 ceph-mon[29215]: pgmap v938: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:49.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:44 smithi019 ceph-mon[29215]: pgmap v939: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:49.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:45 smithi019 ceph-mon[29215]: pgmap v940: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:49.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:47 smithi019 ceph-mon[29215]: pgmap v941: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:49.454 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:19 smithi044 ceph-mon[32458]: osdmap e138: 12 total, 12 up, 12 in 2024-04-23T18:55:49.486 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:20 smithi044 ceph-mon[32458]: pgmap v927: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:55:49.520 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:22 smithi044 ceph-mon[32458]: pgmap v928: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 309 B/s wr, 10 op/s 2024-04-23T18:55:49.553 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:24 smithi044 ceph-mon[32458]: pgmap v929: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-23T18:55:49.587 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:26 smithi044 ceph-mon[32458]: pgmap v930: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:49.637 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:28 smithi044 ceph-mon[32458]: pgmap v931: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:49.670 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:30 smithi044 ceph-mon[32458]: pgmap v932: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 193 B/s wr, 9 op/s 2024-04-23T18:55:49.703 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:32 smithi044 ceph-mon[32458]: pgmap v933: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-23T18:55:49.753 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:34 smithi044 ceph-mon[32458]: pgmap v934: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:49.787 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:36 smithi044 ceph-mon[32458]: pgmap v935: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:49.820 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:38 smithi044 ceph-mon[32458]: pgmap v936: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:49.878 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:40 smithi044 ceph-mon[32458]: pgmap v937: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:49.912 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:42 smithi044 ceph-mon[32458]: pgmap v938: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:49.962 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:44 smithi044 ceph-mon[32458]: pgmap v939: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:50.004 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:45 smithi044 ceph-mon[32458]: pgmap v940: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:50.037 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:47 smithi044 ceph-mon[32458]: pgmap v941: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:50.072 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:49 smithi044 ceph-mon[32458]: pgmap v942: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:50.104 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:49 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:20 smithi155 ceph-mon[33468]: pgmap v927: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T18:55:50.254 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:22 smithi155 ceph-mon[33468]: pgmap v928: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 309 B/s wr, 10 op/s 2024-04-23T18:55:50.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:24 smithi155 ceph-mon[33468]: pgmap v929: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-23T18:55:50.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:26 smithi155 ceph-mon[33468]: pgmap v930: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:50.354 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:28 smithi155 ceph-mon[33468]: pgmap v931: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T18:55:50.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:30 smithi155 ceph-mon[33468]: pgmap v932: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 193 B/s wr, 9 op/s 2024-04-23T18:55:50.421 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:32 smithi155 ceph-mon[33468]: pgmap v933: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-23T18:55:50.463 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:34 smithi155 ceph-mon[33468]: pgmap v934: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:50.504 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:36 smithi155 ceph-mon[33468]: pgmap v935: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:50.546 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:38 smithi155 ceph-mon[33468]: pgmap v936: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:50.613 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:40 smithi155 ceph-mon[33468]: pgmap v937: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:50.663 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:42 smithi155 ceph-mon[33468]: pgmap v938: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:50.705 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:44 smithi155 ceph-mon[33468]: pgmap v939: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:50.746 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:45 smithi155 ceph-mon[33468]: pgmap v940: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:50.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:47 smithi155 ceph-mon[33468]: pgmap v941: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:50.821 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:49 smithi155 ceph-mon[33468]: pgmap v942: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:50.855 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:49 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:55:50.897 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:55:50.930 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:55:50.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:49 smithi019 ceph-mon[29215]: pgmap v942: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:51.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:49 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:55:51.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:50 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:55:51.163 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:50 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:55:51.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:50 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:55:51.247 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:50 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:55:51.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:50 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:55:51.333 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:50 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:55:51.424 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:50 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:55:51.488 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:50 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:55:51.530 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:50 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' 2024-04-23T18:55:51.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:55:51.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.726+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1344759545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818107280 msgr2=0x7fe81810ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:51.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.726+0000 7fe81fb8e640 1 --2- 172.21.15.19:0/1344759545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818107280 0x7fe81810ee50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe810009ca0 tx=0x7fe81002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:55:51.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.726+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1344759545 shutdown_connections 2024-04-23T18:55:51.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.726+0000 7fe81fb8e640 1 --2- 172.21.15.19:0/1344759545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe818102bf0 0x7fe818106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:51.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.726+0000 7fe81fb8e640 1 --2- 172.21.15.19:0/1344759545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe818107280 0x7fe81810ee50 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.726+0000 7fe81fb8e640 1 --2- 172.21.15.19:0/1344759545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe818102240 0x7fe818102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:51.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.726+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1344759545 >> 172.21.15.19:0/1344759545 conn(0x7fe8180790e0 msgr2=0x7fe8180794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:52.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.726+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1344759545 shutdown_connections 2024-04-23T18:55:52.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.726+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1344759545 wait complete. 2024-04-23T18:55:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81fb8e640 1 Processor -- start 2024-04-23T18:55:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81fb8e640 1 -- start start 2024-04-23T18:55:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81fb8e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe818102240 0x7fe818198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:52.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81fb8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe818107280 0x7fe818198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:52.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81fb8e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81819cfe0 0x7fe81819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:52.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81fb8e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe818112200 con 0x7fe81819cfe0 2024-04-23T18:55:52.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81fb8e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe818112080 con 0x7fe818107280 2024-04-23T18:55:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81fb8e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe818112380 con 0x7fe818102240 2024-04-23T18:55:52.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81e12b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81819cfe0 0x7fe81819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:52.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81e12b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81819cfe0 0x7fe81819d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51660/0 (socket says 172.21.15.19:51660) 2024-04-23T18:55:52.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81e12b640 1 -- 172.21.15.19:0/1543005801 learned_addr learned my addr 172.21.15.19:0/1543005801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:55:52.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81d92a640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe818102240 0x7fe818198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.727+0000 7fe81d129640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe818107280 0x7fe818198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe81e12b640 1 -- 172.21.15.19:0/1543005801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe818102240 msgr2=0x7fe818198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:52.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe81e12b640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe818102240 0x7fe818198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe81e12b640 1 -- 172.21.15.19:0/1543005801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe818107280 msgr2=0x7fe818198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:52.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe81e12b640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe818107280 0x7fe818198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe81e12b640 1 -- 172.21.15.19:0/1543005801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81819dad0 con 0x7fe81819cfe0 2024-04-23T18:55:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe81d129640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe818107280 0x7fe818198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:53.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe81e12b640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81819cfe0 0x7fe81819d490 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe8100098d0 tx=0x7fe8100317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:53.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe80affd640 1 -- 172.21.15.19:0/1543005801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810003b70 con 0x7fe81819cfe0 2024-04-23T18:55:53.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1543005801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81819dd60 con 0x7fe81819cfe0 2024-04-23T18:55:53.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe80affd640 1 -- 172.21.15.19:0/1543005801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe810003d10 con 0x7fe81819cfe0 2024-04-23T18:55:53.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe80affd640 1 -- 172.21.15.19:0/1543005801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81002fa20 con 0x7fe81819cfe0 2024-04-23T18:55:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.728+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1543005801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8181a5580 con 0x7fe81819cfe0 2024-04-23T18:55:53.374 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:52 smithi155 ceph-mon[33468]: pgmap v943: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:53.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:52 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1543005801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:55:53.440 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:52 smithi044 ceph-mon[32458]: pgmap v943: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:53.473 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:52 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1543005801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:55:53.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.729+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1543005801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe818199a70 con 0x7fe81819cfe0 2024-04-23T18:55:53.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.730+0000 7fe80affd640 1 -- 172.21.15.19:0/1543005801 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe81002fc20 con 0x7fe81819cfe0 2024-04-23T18:55:53.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.731+0000 7fe80affd640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8040a3d20 0x7fe8040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:53.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.731+0000 7fe81d92a640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8040a3d20 0x7fe8040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:53.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.731+0000 7fe80affd640 1 -- 172.21.15.19:0/1543005801 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8513+0+0 (secure 0 0 0) 0x7fe8100f1bb0 con 0x7fe81819cfe0 2024-04-23T18:55:53.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.731+0000 7fe81d92a640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8040a3d20 0x7fe8040a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009bd0 tx=0x7fe80c005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:53.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.736+0000 7fe80affd640 1 -- 172.21.15.19:0/1543005801 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8100be360 con 0x7fe81819cfe0 2024-04-23T18:55:53.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.926+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1543005801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe818102bf0 con 0x7fe81819cfe0 2024-04-23T18:55:53.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.928+0000 7fe80affd640 1 -- 172.21.15.19:0/1543005801 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+23112 (secure 0 0 0) 0x7fe8100dce20 con 0x7fe81819cfe0 2024-04-23T18:55:53.920 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:55:54.023 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":138,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:55:18.054911+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":53,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/304874449":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1108384152":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6801/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6800/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/983971095":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2556512605":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":40,"length":1},{"begin":53,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:55:54.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1543005801 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8040a3d20 msgr2=0x7fe8040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:54.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8040a3d20 0x7fe8040a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009bd0 tx=0x7fe80c005f90 comp rx=0 tx=0).stop 2024-04-23T18:55:54.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1543005801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81819cfe0 msgr2=0x7fe81819d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:54.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81819cfe0 0x7fe81819d490 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe8100098d0 tx=0x7fe8100317b0 comp rx=0 tx=0).stop 2024-04-23T18:55:54.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1543005801 shutdown_connections 2024-04-23T18:55:54.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fe8040a3d20 0x7fe8040a61e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe818107280 0x7fe818198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:54.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81819cfe0 0x7fe81819d490 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:54.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 --2- 172.21.15.19:0/1543005801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe818102240 0x7fe818198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1543005801 >> 172.21.15.19:0/1543005801 conn(0x7fe8180790e0 msgr2=0x7fe8181002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:54.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1543005801 shutdown_connections 2024-04-23T18:55:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:51.930+0000 7fe81fb8e640 1 -- 172.21.15.19:0/1543005801 wait complete. 2024-04-23T18:55:54.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:07.365995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:23:53.296504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:23:54.573484+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 53, 'snap_epoch': 137, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:55:54.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:55:54.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:52 smithi019 ceph-mon[29215]: pgmap v943: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:54.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:52 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1543005801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:55:54.715 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:54 smithi019 ceph-mon[29215]: pgmap v944: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:54.749 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:54 smithi155 ceph-mon[33468]: pgmap v944: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:54.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:54 smithi044 ceph-mon[32458]: pgmap v944: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:55.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 -- 172.21.15.19:0/1959150199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9cc101670 msgr2=0x7fa9cc10e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:55.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 --2- 172.21.15.19:0/1959150199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9cc101670 0x7fa9cc10e3c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009900 tx=0x7fa9bc02a410 comp rx=0 tx=0).stop 2024-04-23T18:55:55.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 -- 172.21.15.19:0/1959150199 shutdown_connections 2024-04-23T18:55:55.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 --2- 172.21.15.19:0/1959150199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9cc10e900 0x7fa9cc110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:55.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 --2- 172.21.15.19:0/1959150199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9cc101670 0x7fa9cc10e3c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:55.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 --2- 172.21.15.19:0/1959150199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9cc100d50 0x7fa9cc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:55.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 -- 172.21.15.19:0/1959150199 >> 172.21.15.19:0/1959150199 conn(0x7fa9cc077f50 msgr2=0x7fa9cc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:55.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 -- 172.21.15.19:0/1959150199 shutdown_connections 2024-04-23T18:55:55.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 -- 172.21.15.19:0/1959150199 wait complete. 2024-04-23T18:55:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 Processor -- start 2024-04-23T18:55:55.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.013+0000 7fa9d45e1640 1 -- start start 2024-04-23T18:55:55.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d45e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9cc100d50 0x7fa9cc1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:55.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d45e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9cc101670 0x7fa9cc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:55.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d45e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9cc10e900 0x7fa9cc19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d45e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1143c0 con 0x7fa9cc100d50 2024-04-23T18:55:55.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d45e1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc114240 con 0x7fa9cc101670 2024-04-23T18:55:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d45e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc114540 con 0x7fa9cc10e900 2024-04-23T18:55:55.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d237d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9cc100d50 0x7fa9cc1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d237d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9cc100d50 0x7fa9cc1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51666/0 (socket says 172.21.15.19:51666) 2024-04-23T18:55:55.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d237d640 1 -- 172.21.15.19:0/4158003851 learned_addr learned my addr 172.21.15.19:0/4158003851 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:55:55.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d1b7c640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9cc101670 0x7fa9cc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d2b7e640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9cc10e900 0x7fa9cc19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d237d640 1 -- 172.21.15.19:0/4158003851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9cc10e900 msgr2=0x7fa9cc19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d237d640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9cc10e900 0x7fa9cc19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d237d640 1 -- 172.21.15.19:0/4158003851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9cc101670 msgr2=0x7fa9cc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:56.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d237d640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9cc101670 0x7fa9cc1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:56.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d237d640 1 -- 172.21.15.19:0/4158003851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9cc19afb0 con 0x7fa9cc100d50 2024-04-23T18:55:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9d237d640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9cc100d50 0x7fa9cc1a0670 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800d950 tx=0x7fa9c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.014+0000 7fa9c37fe640 1 -- 172.21.15.19:0/4158003851 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8017070 con 0x7fa9cc100d50 2024-04-23T18:55:56.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.015+0000 7fa9d45e1640 1 -- 172.21.15.19:0/4158003851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9cc19b2a0 con 0x7fa9cc100d50 2024-04-23T18:55:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.015+0000 7fa9c37fe640 1 -- 172.21.15.19:0/4158003851 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c8009c30 con 0x7fa9cc100d50 2024-04-23T18:55:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.015+0000 7fa9c37fe640 1 -- 172.21.15.19:0/4158003851 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c800f620 con 0x7fa9cc100d50 2024-04-23T18:55:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.015+0000 7fa9d45e1640 1 -- 172.21.15.19:0/4158003851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9cc1a74c0 con 0x7fa9cc100d50 2024-04-23T18:55:56.409 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:56 smithi155 ceph-mon[33468]: pgmap v945: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:56.471 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:56 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/4158003851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:55:56.508 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:56 smithi044 ceph-mon[32458]: pgmap v945: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:56.542 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:56 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/4158003851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:55:56.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.016+0000 7fa9d45e1640 1 -- 172.21.15.19:0/4158003851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9cc0698e0 con 0x7fa9cc100d50 2024-04-23T18:55:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.018+0000 7fa9c37fe640 1 -- 172.21.15.19:0/4158003851 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa9c8010460 con 0x7fa9cc100d50 2024-04-23T18:55:56.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.018+0000 7fa9c37fe640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa9b80a3d20 0x7fa9b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.018+0000 7fa9d1b7c640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa9b80a3d20 0x7fa9b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:56.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.018+0000 7fa9c37fe640 1 -- 172.21.15.19:0/4158003851 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8513+0+0 (secure 0 0 0) 0x7fa9c80d0370 con 0x7fa9cc100d50 2024-04-23T18:55:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.019+0000 7fa9d1b7c640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa9b80a3d20 0x7fa9b80a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc031440 tx=0x7fa9bc005b50 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:56.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.022+0000 7fa9c37fe640 1 -- 172.21.15.19:0/4158003851 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9c8009de0 con 0x7fa9cc100d50 2024-04-23T18:55:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.209+0000 7fa9d45e1640 1 -- 172.21.15.19:0/4158003851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa9cc109a90 con 0x7fa9cc100d50 2024-04-23T18:55:56.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.209+0000 7fa9c37fe640 1 -- 172.21.15.19:0/4158003851 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) ==== 93+0+10 (secure 0 0 0) 0x7fa9c809c3c0 con 0x7fa9cc100d50 2024-04-23T18:55:56.917 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-23T18:55:56.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 -- 172.21.15.19:0/4158003851 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa9b80a3d20 msgr2=0x7fa9b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa9b80a3d20 0x7fa9b80a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc031440 tx=0x7fa9bc005b50 comp rx=0 tx=0).stop 2024-04-23T18:55:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 -- 172.21.15.19:0/4158003851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9cc100d50 msgr2=0x7fa9cc1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:57.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9cc100d50 0x7fa9cc1a0670 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800d950 tx=0x7fa9c800de10 comp rx=0 tx=0).stop 2024-04-23T18:55:57.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 -- 172.21.15.19:0/4158003851 shutdown_connections 2024-04-23T18:55:57.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fa9b80a3d20 0x7fa9b80a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9cc101670 0x7fa9cc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9cc10e900 0x7fa9cc19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 --2- 172.21.15.19:0/4158003851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9cc100d50 0x7fa9cc1a0670 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 -- 172.21.15.19:0/4158003851 >> 172.21.15.19:0/4158003851 conn(0x7fa9cc077f50 msgr2=0x7fa9cc1100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:57.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 -- 172.21.15.19:0/4158003851 shutdown_connections 2024-04-23T18:55:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:55.211+0000 7fa9d45e1640 1 -- 172.21.15.19:0/4158003851 wait complete. 2024-04-23T18:55:57.419 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:55:57.459 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:56 smithi019 ceph-mon[29215]: pgmap v945: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:55:57.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:56 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/4158003851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.647+0000 7f0c06f29640 1 -- 172.21.15.19:0/1529870137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c00101a40 msgr2=0x7f0c0010dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.647+0000 7f0c06f29640 1 --2- 172.21.15.19:0/1529870137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c00101a40 0x7f0c0010dca0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009ca0 tx=0x7f0bf802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.647+0000 7f0c06f29640 1 -- 172.21.15.19:0/1529870137 shutdown_connections 2024-04-23T18:55:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.647+0000 7f0c06f29640 1 --2- 172.21.15.19:0/1529870137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c001010a0 0x7f0c00101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.647+0000 7f0c06f29640 1 --2- 172.21.15.19:0/1529870137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c00101a40 0x7f0c0010dca0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.647+0000 7f0c06f29640 1 --2- 172.21.15.19:0/1529870137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c00105dd0 0x7f0c001061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.647+0000 7f0c06f29640 1 -- 172.21.15.19:0/1529870137 >> 172.21.15.19:0/1529870137 conn(0x7f0c00077c40 msgr2=0x7f0c00078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:57.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.647+0000 7f0c06f29640 1 -- 172.21.15.19:0/1529870137 shutdown_connections 2024-04-23T18:55:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.647+0000 7f0c06f29640 1 -- 172.21.15.19:0/1529870137 wait complete. 2024-04-23T18:55:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c06f29640 1 Processor -- start 2024-04-23T18:55:58.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c06f29640 1 -- start start 2024-04-23T18:55:58.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c06f29640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c001010a0 0x7f0c00198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:58.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c06f29640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c00101a40 0x7f0c00198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:58.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c06f29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c00105dd0 0x7f0c0019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c06f29640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00111160 con 0x7f0c001010a0 2024-04-23T18:55:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c06f29640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00110fe0 con 0x7f0c00105dd0 2024-04-23T18:55:58.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c06f29640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001112e0 con 0x7f0c00101a40 2024-04-23T18:55:58.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c04cc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c001010a0 0x7f0c00198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c04cc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c001010a0 0x7f0c00198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51686/0 (socket says 172.21.15.19:51686) 2024-04-23T18:55:58.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c04cc5640 1 -- 172.21.15.19:0/538616154 learned_addr learned my addr 172.21.15.19:0/538616154 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:55:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0c054c6640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c00105dd0 0x7f0c0019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:58.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.648+0000 7f0bfffff640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c00101a40 0x7f0c00198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:58.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0c04cc5640 1 -- 172.21.15.19:0/538616154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c00101a40 msgr2=0x7f0c00198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0c04cc5640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c00101a40 0x7f0c00198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:58.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0c04cc5640 1 -- 172.21.15.19:0/538616154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c00105dd0 msgr2=0x7f0c0019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0c04cc5640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c00105dd0 0x7f0c0019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:58.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0c04cc5640 1 -- 172.21.15.19:0/538616154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0019d6c0 con 0x7f0c001010a0 2024-04-23T18:55:58.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0c04cc5640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c001010a0 0x7f0c00198720 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400d950 tx=0x7f0bf400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:58.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0bfdffb640 1 -- 172.21.15.19:0/538616154 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4017030 con 0x7f0c001010a0 2024-04-23T18:55:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0bfdffb640 1 -- 172.21.15.19:0/538616154 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf4004020 con 0x7f0c001010a0 2024-04-23T18:55:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0bfdffb640 1 -- 172.21.15.19:0/538616154 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40047a0 con 0x7f0c001010a0 2024-04-23T18:55:58.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0c06f29640 1 -- 172.21.15.19:0/538616154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0019d9b0 con 0x7f0c001010a0 2024-04-23T18:55:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.649+0000 7f0c06f29640 1 -- 172.21.15.19:0/538616154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c001a5240 con 0x7f0c001010a0 2024-04-23T18:55:58.927 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:58 smithi155 ceph-mon[33468]: pgmap v946: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:58.960 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:55:58 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/538616154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:55:59.002 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:58 smithi044 ceph-mon[32458]: pgmap v946: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:59.064 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:55:58 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/538616154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:55:59.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.650+0000 7f0c06f29640 1 -- 172.21.15.19:0/538616154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c00102540 con 0x7f0c001010a0 2024-04-23T18:55:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.651+0000 7f0bfdffb640 1 -- 172.21.15.19:0/538616154 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0bf4009a30 con 0x7f0c001010a0 2024-04-23T18:55:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.652+0000 7f0bfdffb640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.652+0000 7f0bfffff640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:59.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.652+0000 7f0bfdffb640 1 -- 172.21.15.19:0/538616154 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8513+0+0 (secure 0 0 0) 0x7f0bf40d0310 con 0x7f0c001010a0 2024-04-23T18:55:59.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.652+0000 7f0bfffff640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0095f0 tx=0x7f0bec005f90 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:59.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.657+0000 7f0bfdffb640 1 -- 172.21.15.19:0/538616154 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bf409c9b0 con 0x7f0c001010a0 2024-04-23T18:55:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.843+0000 7f0c06f29640 1 -- 172.21.15.19:0/538616154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0c0010a370 con 0x7f0c001010a0 2024-04-23T18:55:59.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.844+0000 7f0bfdffb640 1 -- 172.21.15.19:0/538616154 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) ==== 104+0+11 (secure 0 0 0) 0x7f0bf4096e20 con 0x7f0c001010a0 2024-04-23T18:55:59.445 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-23T18:55:59.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 -- 172.21.15.19:0/538616154 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0bf00a3d20 msgr2=0x7f0bf00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:59.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0095f0 tx=0x7f0bec005f90 comp rx=0 tx=0).stop 2024-04-23T18:55:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 -- 172.21.15.19:0/538616154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c001010a0 msgr2=0x7f0c00198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c001010a0 0x7f0c00198720 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400d950 tx=0x7f0bf400de10 comp rx=0 tx=0).stop 2024-04-23T18:55:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 -- 172.21.15.19:0/538616154 shutdown_connections 2024-04-23T18:55:59.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:59.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c00105dd0 0x7f0c0019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:59.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c00101a40 0x7f0c00198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:59.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 --2- 172.21.15.19:0/538616154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c001010a0 0x7f0c00198720 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:59.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 -- 172.21.15.19:0/538616154 >> 172.21.15.19:0/538616154 conn(0x7f0c00077c40 msgr2=0x7f0c0006a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:59.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 -- 172.21.15.19:0/538616154 shutdown_connections 2024-04-23T18:55:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:55:57.846+0000 7f0c06f29640 1 -- 172.21.15.19:0/538616154 wait complete. 2024-04-23T18:55:59.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:56:00.020 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:58 smithi019 ceph-mon[29215]: pgmap v946: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:00.065 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:55:58 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/538616154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:56:00.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.224+0000 7f04875cd640 1 -- 172.21.15.19:0/4086820777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0480106160 msgr2=0x7f048010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:00.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.224+0000 7f04875cd640 1 --2- 172.21.15.19:0/4086820777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0480106160 0x7f048010dd30 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0478009ca0 tx=0x7f047802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:56:00.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.224+0000 7f04875cd640 1 -- 172.21.15.19:0/4086820777 shutdown_connections 2024-04-23T18:56:00.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.224+0000 7f04875cd640 1 --2- 172.21.15.19:0/4086820777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0480106160 0x7f048010dd30 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:00.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.224+0000 7f04875cd640 1 --2- 172.21.15.19:0/4086820777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0480101a40 0x7f0480105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:00.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.224+0000 7f04875cd640 1 --2- 172.21.15.19:0/4086820777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0480101090 0x7f0480101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.224+0000 7f04875cd640 1 -- 172.21.15.19:0/4086820777 >> 172.21.15.19:0/4086820777 conn(0x7f0480077f50 msgr2=0x7f0480078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.224+0000 7f04875cd640 1 -- 172.21.15.19:0/4086820777 shutdown_connections 2024-04-23T18:56:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.224+0000 7f04875cd640 1 -- 172.21.15.19:0/4086820777 wait complete. 2024-04-23T18:56:00.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.224+0000 7f04875cd640 1 Processor -- start 2024-04-23T18:56:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f04875cd640 1 -- start start 2024-04-23T18:56:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f04875cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0480101090 0x7f0480199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f04875cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0480106160 0x7f0480199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f04875cd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048019a060 0x7f04801a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:00.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f04875cd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04801110d0 con 0x7f0480106160 2024-04-23T18:56:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f04875cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0480110f50 con 0x7f0480101090 2024-04-23T18:56:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f04875cd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0480111250 con 0x7f048019a060 2024-04-23T18:56:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0484b68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0480106160 0x7f0480199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:00.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0484b68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0480106160 0x7f0480199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56698/0 (socket says 172.21.15.19:56698) 2024-04-23T18:56:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0484b68640 1 -- 172.21.15.19:0/1961937769 learned_addr learned my addr 172.21.15.19:0/1961937769 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:56:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0485369640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0480101090 0x7f0480199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:01.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0485b6a640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048019a060 0x7f04801a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0484b68640 1 -- 172.21.15.19:0/1961937769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048019a060 msgr2=0x7f04801a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0484b68640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048019a060 0x7f04801a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0484b68640 1 -- 172.21.15.19:0/1961937769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0480101090 msgr2=0x7f0480199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0484b68640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0480101090 0x7f0480199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0484b68640 1 -- 172.21.15.19:0/1961937769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04801a4de0 con 0x7f0480106160 2024-04-23T18:56:01.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0485b6a640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048019a060 0x7f04801a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:56:01.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.225+0000 7f0484b68640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0480106160 0x7f0480199ad0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f047000d740 tx=0x7f047000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:01.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.226+0000 7f04767fc640 1 -- 172.21.15.19:0/1961937769 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470017070 con 0x7f0480106160 2024-04-23T18:56:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.226+0000 7f04875cd640 1 -- 172.21.15.19:0/1961937769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04801a50d0 con 0x7f0480106160 2024-04-23T18:56:01.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.226+0000 7f04767fc640 1 -- 172.21.15.19:0/1961937769 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047000de50 con 0x7f0480106160 2024-04-23T18:56:01.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.226+0000 7f04767fc640 1 -- 172.21.15.19:0/1961937769 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470004ce0 con 0x7f0480106160 2024-04-23T18:56:01.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.226+0000 7f04875cd640 1 -- 172.21.15.19:0/1961937769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04801a5620 con 0x7f0480106160 2024-04-23T18:56:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.227+0000 7f04875cd640 1 -- 172.21.15.19:0/1961937769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0480068d70 con 0x7f0480106160 2024-04-23T18:56:01.630 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:00 smithi155 ceph-mon[33468]: pgmap v947: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:01.675 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:01 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1961937769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:56:01.712 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:00 smithi044 ceph-mon[32458]: pgmap v947: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:01.746 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:01 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1961937769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:56:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.228+0000 7f04767fc640 1 -- 172.21.15.19:0/1961937769 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0470004e80 con 0x7f0480106160 2024-04-23T18:56:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.229+0000 7f04767fc640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f046c0a3d20 0x7f046c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.229+0000 7f0485369640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f046c0a3d20 0x7f046c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:01.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.229+0000 7f04767fc640 1 -- 172.21.15.19:0/1961937769 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8513+0+0 (secure 0 0 0) 0x7f04700d05d0 con 0x7f0480106160 2024-04-23T18:56:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.229+0000 7f0485369640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f046c0a3d20 0x7f046c0a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f047c005d00 tx=0x7f047c00b410 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.232+0000 7f04767fc640 1 -- 172.21.15.19:0/1961937769 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f047009cc10 con 0x7f0480106160 2024-04-23T18:56:02.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.432+0000 7f04875cd640 1 -- 172.21.15.19:0/1961937769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0480109390 con 0x7f0480106160 2024-04-23T18:56:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.432+0000 7f04767fc640 1 -- 172.21.15.19:0/1961937769 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) ==== 100+0+11 (secure 0 0 0) 0x7f047009c510 con 0x7f0480106160 2024-04-23T18:56:02.096 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-23T18:56:02.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.434+0000 7f04875cd640 1 -- 172.21.15.19:0/1961937769 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f046c0a3d20 msgr2=0x7f046c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.434+0000 7f04875cd640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f046c0a3d20 0x7f046c0a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f047c005d00 tx=0x7f047c00b410 comp rx=0 tx=0).stop 2024-04-23T18:56:02.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.435+0000 7f04875cd640 1 -- 172.21.15.19:0/1961937769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0480106160 msgr2=0x7f0480199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.435+0000 7f04875cd640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0480106160 0x7f0480199ad0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f047000d740 tx=0x7f047000dc00 comp rx=0 tx=0).stop 2024-04-23T18:56:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.435+0000 7f04875cd640 1 -- 172.21.15.19:0/1961937769 shutdown_connections 2024-04-23T18:56:02.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.435+0000 7f04875cd640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f046c0a3d20 0x7f046c0a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:02.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.435+0000 7f04875cd640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048019a060 0x7f04801a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:02.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.435+0000 7f04875cd640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0480106160 0x7f0480199ad0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:02.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.435+0000 7f04875cd640 1 --2- 172.21.15.19:0/1961937769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0480101090 0x7f0480199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:02.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.435+0000 7f04875cd640 1 -- 172.21.15.19:0/1961937769 >> 172.21.15.19:0/1961937769 conn(0x7f0480077f50 msgr2=0x7f04800ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:02.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.435+0000 7f04875cd640 1 -- 172.21.15.19:0/1961937769 shutdown_connections 2024-04-23T18:56:02.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:00.435+0000 7f04875cd640 1 -- 172.21.15.19:0/1961937769 wait complete. 2024-04-23T18:56:02.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:00 smithi019 ceph-mon[29215]: pgmap v947: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:02.639 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:02 smithi155 ceph-mon[33468]: pgmap v948: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:56:02.672 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:02 smithi044 ceph-mon[32458]: pgmap v948: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:56:02.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T18:56:02.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:01 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1961937769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:56:02.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:02 smithi019 ceph-mon[29215]: pgmap v948: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:56:02.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.923+0000 7fc46f013640 1 -- 172.21.15.19:0/1944922725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46810c620 msgr2=0x7fc46810ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.923+0000 7fc46f013640 1 --2- 172.21.15.19:0/1944922725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46810c620 0x7fc46810ea50 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc460009ca0 tx=0x7fc46002a950 comp rx=0 tx=0).stop 2024-04-23T18:56:03.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.923+0000 7fc46f013640 1 -- 172.21.15.19:0/1944922725 shutdown_connections 2024-04-23T18:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.923+0000 7fc46f013640 1 --2- 172.21.15.19:0/1944922725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc468103b70 0x7fc46810c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:03.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.923+0000 7fc46f013640 1 --2- 172.21.15.19:0/1944922725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46810c620 0x7fc46810ea50 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:03.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.923+0000 7fc46f013640 1 --2- 172.21.15.19:0/1944922725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468103250 0x7fc468103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.923+0000 7fc46f013640 1 -- 172.21.15.19:0/1944922725 >> 172.21.15.19:0/1944922725 conn(0x7fc468077f80 msgr2=0x7fc468078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.923+0000 7fc46f013640 1 -- 172.21.15.19:0/1944922725 shutdown_connections 2024-04-23T18:56:03.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.923+0000 7fc46f013640 1 -- 172.21.15.19:0/1944922725 wait complete. 2024-04-23T18:56:03.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46f013640 1 Processor -- start 2024-04-23T18:56:03.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46f013640 1 -- start start 2024-04-23T18:56:03.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46f013640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc468103250 0x7fc4681a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46f013640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468103b70 0x7fc4681a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:03.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46f013640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46810c620 0x7fc46819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:03.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46f013640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc468112240 con 0x7fc46810c620 2024-04-23T18:56:03.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46f013640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681120c0 con 0x7fc468103250 2024-04-23T18:56:03.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46f013640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681123c0 con 0x7fc468103b70 2024-04-23T18:56:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46d5b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46810c620 0x7fc46819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46d5b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46810c620 0x7fc46819a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56708/0 (socket says 172.21.15.19:56708) 2024-04-23T18:56:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc46d5b0640 1 -- 172.21.15.19:0/1625336674 learned_addr learned my addr 172.21.15.19:0/1625336674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:56:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.924+0000 7fc467fff640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468103b70 0x7fc4681a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:03.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc46cdaf640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc468103250 0x7fc4681a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc46d5b0640 1 -- 172.21.15.19:0/1625336674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468103b70 msgr2=0x7fc4681a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc46d5b0640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468103b70 0x7fc4681a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc46d5b0640 1 -- 172.21.15.19:0/1625336674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc468103250 msgr2=0x7fc4681a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc46d5b0640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc468103250 0x7fc4681a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:04.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc46d5b0640 1 -- 172.21.15.19:0/1625336674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46819af70 con 0x7fc46810c620 2024-04-23T18:56:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc46d5b0640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46810c620 0x7fc46819a7a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fc46002ae70 tx=0x7fc46002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:04.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc465ffb640 1 -- 172.21.15.19:0/1625336674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460034040 con 0x7fc46810c620 2024-04-23T18:56:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc465ffb640 1 -- 172.21.15.19:0/1625336674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4600308f0 con 0x7fc46810c620 2024-04-23T18:56:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc46f013640 1 -- 172.21.15.19:0/1625336674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46819b200 con 0x7fc46810c620 2024-04-23T18:56:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc465ffb640 1 -- 172.21.15.19:0/1625336674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4600046f0 con 0x7fc46810c620 2024-04-23T18:56:04.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.925+0000 7fc46f013640 1 -- 172.21.15.19:0/1625336674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4681a7480 con 0x7fc46810c620 2024-04-23T18:56:04.348 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:04 smithi155 ceph-mon[33468]: pgmap v949: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:04.381 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:04 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1625336674' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T18:56:04.434 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:04 smithi044 ceph-mon[32458]: pgmap v949: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:04.473 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:04 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1625336674' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T18:56:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.927+0000 7fc46f013640 1 -- 172.21.15.19:0/1625336674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46806adb0 con 0x7fc46810c620 2024-04-23T18:56:04.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.928+0000 7fc465ffb640 1 -- 172.21.15.19:0/1625336674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc460037020 con 0x7fc46810c620 2024-04-23T18:56:04.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.929+0000 7fc465ffb640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc4580a3da0 0x7fc4580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.929+0000 7fc46cdaf640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc4580a3da0 0x7fc4580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.929+0000 7fc465ffb640 1 -- 172.21.15.19:0/1625336674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8513+0+0 (secure 0 0 0) 0x7fc4600f14f0 con 0x7fc46810c620 2024-04-23T18:56:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.929+0000 7fc46cdaf640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc4580a3da0 0x7fc4580a6260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005fd0 tx=0x7fc45c00b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:02.933+0000 7fc465ffb640 1 -- 172.21.15.19:0/1625336674 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4600bdca0 con 0x7fc46810c620 2024-04-23T18:56:04.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.132+0000 7fc46f013640 1 -- 172.21.15.19:0/1625336674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc4681a7650 con 0x7fc46810c620 2024-04-23T18:56:04.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.133+0000 7fc465ffb640 1 -- 172.21.15.19:0/1625336674 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v138) ==== 81+0+944 (secure 0 0 0) 0x7fc4600bd5a0 con 0x7fc46810c620 2024-04-23T18:56:04.898 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/304874449 2024-04-24T18:26:08.116888+0000 2024-04-23T18:56:04.931 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1108384152 2024-04-24T18:26:08.116888+0000 2024-04-23T18:56:04.965 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/1993809272 2024-04-24T18:26:08.116888+0000 2024-04-23T18:56:04.998 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/1993809272 2024-04-24T18:26:08.116888+0000 2024-04-23T18:56:05.031 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/983971095 2024-04-24T18:26:08.116888+0000 2024-04-23T18:56:05.065 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1473447325 2024-04-24T18:16:35.264133+0000 2024-04-23T18:56:05.136 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2431053142 2024-04-24T18:16:35.264133+0000 2024-04-23T18:56:05.173 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3826407819 2024-04-24T18:16:06.817907+0000 2024-04-23T18:56:05.207 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/4193386941 2024-04-24T18:16:06.817907+0000 2024-04-23T18:56:05.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1551393169 2024-04-24T18:16:35.264133+0000 2024-04-23T18:56:05.273 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2431053142 2024-04-24T18:16:35.264133+0000 2024-04-23T18:56:05.307 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/4087353417 2024-04-24T18:16:06.817907+0000 2024-04-23T18:56:05.357 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/1085063871 2024-04-24T18:16:06.817907+0000 2024-04-23T18:56:05.390 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/1085063871 2024-04-24T18:16:06.817907+0000 2024-04-23T18:56:05.463 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2556512605 2024-04-24T18:26:08.116888+0000 2024-04-23T18:56:05.535 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2920477219 2024-04-24T18:16:35.264133+0000 2024-04-23T18:56:05.575 INFO:teuthology.orchestra.run.smithi019.stderr:listed 16 entries 2024-04-23T18:56:05.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 -- 172.21.15.19:0/1625336674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc4580a3da0 msgr2=0x7fc4580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc4580a3da0 0x7fc4580a6260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005fd0 tx=0x7fc45c00b390 comp rx=0 tx=0).stop 2024-04-23T18:56:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 -- 172.21.15.19:0/1625336674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46810c620 msgr2=0x7fc46819a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46810c620 0x7fc46819a7a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fc46002ae70 tx=0x7fc46002f5d0 comp rx=0 tx=0).stop 2024-04-23T18:56:05.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 -- 172.21.15.19:0/1625336674 shutdown_connections 2024-04-23T18:56:05.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc4580a3da0 0x7fc4580a6260 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:05.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46810c620 0x7fc46819a7a0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:05.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468103b70 0x7fc4681a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 --2- 172.21.15.19:0/1625336674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc468103250 0x7fc4681a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 -- 172.21.15.19:0/1625336674 >> 172.21.15.19:0/1625336674 conn(0x7fc468077f80 msgr2=0x7fc468108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:05.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 -- 172.21.15.19:0/1625336674 shutdown_connections 2024-04-23T18:56:06.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:03.135+0000 7fc46f013640 1 -- 172.21.15.19:0/1625336674 wait complete. 2024-04-23T18:56:06.041 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-23T18:56:06.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:56:06.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-23T18:56:06.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:04 smithi019 ceph-mon[29215]: pgmap v949: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:06.274 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:04 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1625336674' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T18:56:06.326 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:06 smithi155 ceph-mon[33468]: pgmap v950: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:56:06.349 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:06 smithi019 ceph-mon[29215]: pgmap v950: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:56:06.383 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:56:06.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:56:06.449 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T18:56:06.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:06 smithi044 ceph-mon[32458]: pgmap v950: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:56:06.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:56:06.641 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-23T18:56:06.692 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T18:56:06.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:56:06.766 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T18:56:06.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:56:06.849 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-23T18:56:06.942 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-23T18:56:06.966 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-23T18:56:07.018 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:56:07.068 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16518 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/82ddca01e6295a4ddbbfcf42af033c9fdce303cde89ef5391bc9a6cb7412fd0a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PZB2S73FSW4TRTBDY6H5H6U435:/var/lib/containers/storage/overlay/l/I6BURPU5ZVGEEON6NZKFOI7ADO,upperdir=/var/lib/containers/storage/overlay/82ddca01e6295a4ddbbfcf42af033c9fdce303cde89ef5391bc9a6cb7412fd0a/diff,workdir=/var/lib/containers/storage/overlay/82ddca01e6295a4ddbbfcf42af033c9fdce303cde89ef5391bc9a6cb7412fd0a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8842a9e0589f6b5003efee13008997839fbf6f93e871227df8b89f481e4d24c1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PZB2S73FSW4TRTBDY6H5H6U435:/var/lib/containers/storage/overlay/l/I6BURPU5ZVGEEON6NZKFOI7ADO,upperdir=/var/lib/containers/storage/overlay/8842a9e0589f6b5003efee13008997839fbf6f93e871227df8b89f481e4d24c1/diff,workdir=/var/lib/containers/storage/overlay/8842a9e0589f6b5003efee13008997839fbf6f93e871227df8b89f481e4d24c1/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/6b3b96f379cf30d5065d658947bd0c041df759dc3a026d3c4a49c5778fd53d92/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PZB2S73FSW4TRTBDY6H5H6U435:/var/lib/containers/storage/overlay/l/I6BURPU5ZVGEEON6NZKFOI7ADO,upperdir=/var/lib/containers/storage/overlay/6b3b96f379cf30d5065d658947bd0c041df759dc3a026d3c4a49c5778fd53d92/diff,workdir=/var/lib/containers/storage/overlay/6b3b96f379cf30d5065d658947bd0c041df759dc3a026d3c4a49c5778fd53d92/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fcacb2a3349cb5bc8d9c9605f31df3bb272a0de19c099c17fd57b7b8f31b17ca/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PZB2S73FSW4TRTBDY6H5H6U435:/var/lib/containers/storage/overlay/l/I6BURPU5ZVGEEON6NZKFOI7ADO,upperdir=/var/lib/containers/storage/overlay/fcacb2a3349cb5bc8d9c9605f31df3bb272a0de19c099c17fd57b7b8f31b17ca/diff,workdir=/var/lib/containers/storage/overlay/fcacb2a3349cb5bc8d9c9605f31df3bb272a0de19c099c17fd57b7b8f31b17ca/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/383325296a2c2b9e0ab1408e6d326256b725318f1e407594bd116ffdada0c761/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PZB2S73FSW4TRTBDY6H5H6U435:/var/lib/containers/storage/overlay/l/I6BURPU5ZVGEEON6NZKFOI7ADO,upperdir=/var/lib/containers/storage/overlay/383325296a2c2b9e0ab1408e6d326256b725318f1e407594bd116ffdada0c761/diff,workdir=/var/lib/containers/storage/overlay/383325296a2c2b9e0ab1408e6d326256b725318f1e407594bd116ffdada0c761/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/af2e2744710b6cc68f4ae2922eb56348e173b639201ca1c5f422d792125a33ff/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PZB2S73FSW4TRTBDY6H5H6U435:/var/lib/containers/storage/overlay/l/I6BURPU5ZVGEEON6NZKFOI7ADO,upperdir=/var/lib/containers/storage/overlay/af2e2744710b6cc68f4ae2922eb56348e173b639201ca1c5f422d792125a33ff/diff,workdir=/var/lib/containers/storage/overlay/af2e2744710b6cc68f4ae2922eb56348e173b639201ca1c5f422d792125a33ff/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f5a92b648bb6496244af604186a82a38ee71ccd653afbdc535b08f67b35b568d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PZB2S73FSW4TRTBDY6H5H6U435:/var/lib/containers/storage/overlay/l/I6BURPU5ZVGEEON6NZKFOI7ADO,upperdir=/var/lib/containers/storage/overlay/f5a92b648bb6496244af604186a82a38ee71ccd653afbdc535b08f67b35b568d/diff,workdir=/var/lib/containers/storage/overlay/f5a92b648bb6496244af604186a82a38ee71ccd653afbdc535b08f67b35b568d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/55ef79e4dc18af2ddfff6750fed2b9a4b896a6d075f53d59f410b0bad61782b0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PZB2S73FSW4TRTBDY6H5H6U435:/var/lib/containers/storage/overlay/l/I6BURPU5ZVGEEON6NZKFOI7ADO,upperdir=/var/lib/containers/storage/overlay/55ef79e4dc18af2ddfff6750fed2b9a4b896a6d075f53d59f410b0bad61782b0/diff,workdir=/var/lib/containers/storage/overlay/55ef79e4dc18af2ddfff6750fed2b9a4b896a6d075f53d59f410b0bad61782b0/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6d299bb8c8048b126d678205014b7c5c9e5d99856299c76b4c6c826c17fd6b80/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PZB2S73FSW4TRTBDY6H5H6U435:/var/lib/containers/storage/overlay/l/I6BURPU5ZVGEEON6NZKFOI7ADO,upperdir=/var/lib/containers/storage/overlay/6d299bb8c8048b126d678205014b7c5c9e5d99856299c76b4c6c826c17fd6b80/diff,workdir=/var/lib/containers/storage/overlay/6d299bb8c8048b126d678205014b7c5c9e5d99856299c76b4c6c826c17fd6b80/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/926d2db06da07ecfb5618b2bd0afcaf4a6bc2b5995c9b68d3766ff3d7b34172d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PZB2S73FSW4TRTBDY6H5H6U435:/var/lib/containers/storage/overlay/l/I6BURPU5ZVGEEON6NZKFOI7ADO,upperdir=/var/lib/containers/storage/overlay/926d2db06da07ecfb5618b2bd0afcaf4a6bc2b5995c9b68d3766ff3d7b34172d/diff,workdir=/var/lib/containers/storage/overlay/926d2db06da07ecfb5618b2bd0afcaf4a6bc2b5995c9b68d3766ff3d7b34172d/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-23T18:56:07.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:56:07.124 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-23T18:56:07.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:56:07.233 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-23T18:56:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-23T18:56:07.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:56:07.377 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-23T18:56:07.421 INFO:tasks.cephadm:Teardown begin 2024-04-23T18:56:07.442 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T18:56:07.475 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:56:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.650+0000 7f80c7a8a640 1 -- 172.21.15.19:0/898593110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c0102bf0 msgr2=0x7f80c0106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:07.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.650+0000 7f80c7a8a640 1 --2- 172.21.15.19:0/898593110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c0102bf0 0x7f80c0106ba0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009900 tx=0x7f80b002a410 comp rx=0 tx=0).stop 2024-04-23T18:56:07.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.651+0000 7f80c7a8a640 1 -- 172.21.15.19:0/898593110 shutdown_connections 2024-04-23T18:56:07.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.651+0000 7f80c7a8a640 1 --2- 172.21.15.19:0/898593110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c0107280 0x7f80c010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:07.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.651+0000 7f80c7a8a640 1 --2- 172.21.15.19:0/898593110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c0102bf0 0x7f80c0106ba0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:07.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.651+0000 7f80c7a8a640 1 --2- 172.21.15.19:0/898593110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80c0102240 0x7f80c0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:07.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.651+0000 7f80c7a8a640 1 -- 172.21.15.19:0/898593110 >> 172.21.15.19:0/898593110 conn(0x7f80c00790e0 msgr2=0x7f80c00794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:07.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.651+0000 7f80c7a8a640 1 -- 172.21.15.19:0/898593110 shutdown_connections 2024-04-23T18:56:07.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.651+0000 7f80c7a8a640 1 -- 172.21.15.19:0/898593110 wait complete. 2024-04-23T18:56:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.651+0000 7f80c7a8a640 1 Processor -- start 2024-04-23T18:56:08.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.651+0000 7f80c7a8a640 1 -- start start 2024-04-23T18:56:08.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c7a8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80c0102240 0x7f80c01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:08.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c7a8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c0107280 0x7f80c0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:08.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c7a8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c019cfa0 0x7f80c019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:08.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c7a8a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80c01121a0 con 0x7f80c019cfa0 2024-04-23T18:56:08.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c7a8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0112020 con 0x7f80c0102240 2024-04-23T18:56:08.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c7a8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0112320 con 0x7f80c0107280 2024-04-23T18:56:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c6027640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c019cfa0 0x7f80c019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c6027640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c019cfa0 0x7f80c019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56734/0 (socket says 172.21.15.19:56734) 2024-04-23T18:56:08.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c5826640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80c0102240 0x7f80c01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c6027640 1 -- 172.21.15.19:0/149694861 learned_addr learned my addr 172.21.15.19:0/149694861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:56:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c5025640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c0107280 0x7f80c0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:08.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c6027640 1 -- 172.21.15.19:0/149694861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c0107280 msgr2=0x7f80c0198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c6027640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c0107280 0x7f80c0198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:08.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c6027640 1 -- 172.21.15.19:0/149694861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80c0102240 msgr2=0x7f80c01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c6027640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80c0102240 0x7f80c01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c6027640 1 -- 172.21.15.19:0/149694861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c019da90 con 0x7f80c019cfa0 2024-04-23T18:56:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c5025640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c0107280 0x7f80c0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:56:08.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80c6027640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c019cfa0 0x7f80c019d450 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f80b800cce0 tx=0x7f80b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80b6ffd640 1 -- 172.21.15.19:0/149694861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b800f450 con 0x7f80c019cfa0 2024-04-23T18:56:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.652+0000 7f80b6ffd640 1 -- 172.21.15.19:0/149694861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b800faa0 con 0x7f80c019cfa0 2024-04-23T18:56:08.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.653+0000 7f80c7a8a640 1 -- 172.21.15.19:0/149694861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c019dd80 con 0x7f80c019cfa0 2024-04-23T18:56:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.653+0000 7f80b6ffd640 1 -- 172.21.15.19:0/149694861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8018b70 con 0x7f80c019cfa0 2024-04-23T18:56:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.653+0000 7f80c7a8a640 1 -- 172.21.15.19:0/149694861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c01a5520 con 0x7f80c019cfa0 2024-04-23T18:56:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.654+0000 7f80c7a8a640 1 -- 172.21.15.19:0/149694861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c00691c0 con 0x7f80c019cfa0 2024-04-23T18:56:09.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.655+0000 7f80b6ffd640 1 -- 172.21.15.19:0/149694861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f80b8018d10 con 0x7f80c019cfa0 2024-04-23T18:56:09.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.655+0000 7f80b6ffd640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f80ac0a3d20 0x7f80ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.656+0000 7f80b6ffd640 1 -- 172.21.15.19:0/149694861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8513+0+0 (secure 0 0 0) 0x7f80b80d0c70 con 0x7f80c019cfa0 2024-04-23T18:56:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.656+0000 7f80c5826640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f80ac0a3d20 0x7f80ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:09.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.656+0000 7f80c5826640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f80ac0a3d20 0x7f80ac0a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00bbe0 tx=0x7f80bc00b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:09.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.660+0000 7f80b6ffd640 1 -- 172.21.15.19:0/149694861 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80b8014030 con 0x7f80c019cfa0 2024-04-23T18:56:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.886+0000 7f80c7a8a640 1 -- 172.21.15.19:0/149694861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80c0102bf0 con 0x7f80c019cfa0 2024-04-23T18:56:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.887+0000 7f80b6ffd640 1 -- 172.21.15.19:0/149694861 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11914 (secure 0 0 0) 0x7f80b809d2b0 con 0x7f80c019cfa0 2024-04-23T18:56:09.385 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:56:09.418 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:45:17.914370+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26843":{"gid":26843,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/332625709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":332625709},{"type":"v1","addr":"172.21.15.44:6835","nonce":332625709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:56:09.452 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:08 smithi044 ceph-mon[32458]: pgmap v951: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:56:09.485 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:08 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/149694861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:56:09.560 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:08 smithi155 ceph-mon[33468]: pgmap v951: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:56:09.593 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:08 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/149694861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:56:09.627 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 31 2024-04-23T18:56:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 -- 172.21.15.19:0/149694861 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f80ac0a3d20 msgr2=0x7f80ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f80ac0a3d20 0x7f80ac0a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00bbe0 tx=0x7f80bc00b390 comp rx=0 tx=0).stop 2024-04-23T18:56:09.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 -- 172.21.15.19:0/149694861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c019cfa0 msgr2=0x7f80c019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:09.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c019cfa0 0x7f80c019d450 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f80b800cce0 tx=0x7f80b800a620 comp rx=0 tx=0).stop 2024-04-23T18:56:09.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 -- 172.21.15.19:0/149694861 shutdown_connections 2024-04-23T18:56:09.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f80ac0a3d20 0x7f80ac0a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:09.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c019cfa0 0x7f80c019d450 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:09.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c0107280 0x7f80c0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 --2- 172.21.15.19:0/149694861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80c0102240 0x7f80c01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 -- 172.21.15.19:0/149694861 >> 172.21.15.19:0/149694861 conn(0x7f80c00790e0 msgr2=0x7f80c0100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:10.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 -- 172.21.15.19:0/149694861 shutdown_connections 2024-04-23T18:56:10.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:07.889+0000 7f80c7a8a640 1 -- 172.21.15.19:0/149694861 wait complete. 2024-04-23T18:56:10.119 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:56:10.177 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:08 smithi019 ceph-mon[29215]: pgmap v951: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:56:10.232 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:08 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/149694861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:56:10.326 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:10 smithi155 ceph-mon[33468]: pgmap v952: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:56:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.386+0000 7f0d8795d640 1 -- 172.21.15.19:0/2825773328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d80102bf0 msgr2=0x7f0d80106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.386+0000 7f0d8795d640 1 --2- 172.21.15.19:0/2825773328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d80102bf0 0x7f0d80106ba0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009900 tx=0x7f0d7002a410 comp rx=0 tx=0).stop 2024-04-23T18:56:10.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.386+0000 7f0d8795d640 1 -- 172.21.15.19:0/2825773328 shutdown_connections 2024-04-23T18:56:10.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.386+0000 7f0d8795d640 1 --2- 172.21.15.19:0/2825773328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80107280 0x7f0d8010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.386+0000 7f0d8795d640 1 --2- 172.21.15.19:0/2825773328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d80102bf0 0x7f0d80106ba0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.386+0000 7f0d8795d640 1 --2- 172.21.15.19:0/2825773328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d80102240 0x7f0d80102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.386+0000 7f0d8795d640 1 -- 172.21.15.19:0/2825773328 >> 172.21.15.19:0/2825773328 conn(0x7f0d800790e0 msgr2=0x7f0d800794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:10.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.386+0000 7f0d8795d640 1 -- 172.21.15.19:0/2825773328 shutdown_connections 2024-04-23T18:56:10.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.386+0000 7f0d8795d640 1 -- 172.21.15.19:0/2825773328 wait complete. 2024-04-23T18:56:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d8795d640 1 Processor -- start 2024-04-23T18:56:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d8795d640 1 -- start start 2024-04-23T18:56:10.812 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:10 smithi044 ceph-mon[32458]: pgmap v952: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:56:10.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d8795d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102240 0x7f0d80198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:10.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d8795d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d80107280 0x7f0d80198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d8795d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8019d020 0x7f0d8019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:10.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d8795d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801121a0 con 0x7f0d80107280 2024-04-23T18:56:11.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d8795d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d80112020 con 0x7f0d8019d020 2024-04-23T18:56:11.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d8795d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d80112320 con 0x7f0d80102240 2024-04-23T18:56:11.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d84ef8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d80107280 0x7f0d80198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:11.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d84ef8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d80107280 0x7f0d80198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36734/0 (socket says 172.21.15.19:36734) 2024-04-23T18:56:11.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d84ef8640 1 -- 172.21.15.19:0/1378497176 learned_addr learned my addr 172.21.15.19:0/1378497176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:56:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.387+0000 7f0d856f9640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102240 0x7f0d80198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d85efa640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8019d020 0x7f0d8019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:11.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d84ef8640 1 -- 172.21.15.19:0/1378497176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102240 msgr2=0x7f0d80198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d84ef8640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102240 0x7f0d80198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:11.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d84ef8640 1 -- 172.21.15.19:0/1378497176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8019d020 msgr2=0x7f0d8019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:11.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d84ef8640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8019d020 0x7f0d8019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:11.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d84ef8640 1 -- 172.21.15.19:0/1378497176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8019db10 con 0x7f0d80107280 2024-04-23T18:56:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d84ef8640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d80107280 0x7f0d80198ca0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0d7002a930 tx=0x7f0d70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:11.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d767fc640 1 -- 172.21.15.19:0/1378497176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7003f030 con 0x7f0d80107280 2024-04-23T18:56:11.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d767fc640 1 -- 172.21.15.19:0/1378497176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d70003aa0 con 0x7f0d80107280 2024-04-23T18:56:11.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d8795d640 1 -- 172.21.15.19:0/1378497176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8019dda0 con 0x7f0d80107280 2024-04-23T18:56:11.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d767fc640 1 -- 172.21.15.19:0/1378497176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70035070 con 0x7f0d80107280 2024-04-23T18:56:11.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.388+0000 7f0d8795d640 1 -- 172.21.15.19:0/1378497176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d801a5560 con 0x7f0d80107280 2024-04-23T18:56:11.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.389+0000 7f0d8795d640 1 -- 172.21.15.19:0/1378497176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d800691c0 con 0x7f0d80107280 2024-04-23T18:56:11.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.391+0000 7f0d767fc640 1 -- 172.21.15.19:0/1378497176 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d70042030 con 0x7f0d80107280 2024-04-23T18:56:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.391+0000 7f0d767fc640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d6c0a3d20 0x7f0d6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:11.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.391+0000 7f0d856f9640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d6c0a3d20 0x7f0d6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.391+0000 7f0d767fc640 1 -- 172.21.15.19:0/1378497176 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8513+0+0 (secure 0 0 0) 0x7f0d700f1240 con 0x7f0d80107280 2024-04-23T18:56:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.392+0000 7f0d856f9640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d6c0a3d20 0x7f0d6c0a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00bbe0 tx=0x7f0d7c00b390 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:11.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.395+0000 7f0d767fc640 1 -- 172.21.15.19:0/1378497176 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d700bd290 con 0x7f0d80107280 2024-04-23T18:56:12.053 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:10 smithi019 ceph-mon[29215]: pgmap v952: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T18:56:12.087 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:11 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1378497176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:56:12.120 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:11 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1378497176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:56:12.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.620+0000 7f0d8795d640 1 -- 172.21.15.19:0/1378497176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d80199ab0 con 0x7f0d80107280 2024-04-23T18:56:12.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.621+0000 7f0d767fc640 1 -- 172.21.15.19:0/1378497176 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11914 (secure 0 0 0) 0x7f0d80199ab0 con 0x7f0d80107280 2024-04-23T18:56:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:56:12.278 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2967585130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2967585130},{"type":"v1","addr":"172.21.15.19:6835","nonce":2967585130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/781445125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":781445125},{"type":"v1","addr":"172.21.15.19:6837","nonce":781445125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:23:55.611471+0000","modified":"2024-04-23T18:45:17.914370+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24508,"mds_2":24491,"mds_3":24487,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/716138605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":716138605},{"type":"v1","addr":"172.21.15.19:6839","nonce":716138605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2593314342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2593314342},{"type":"v1","addr":"172.21.15.19:6841","nonce":2593314342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/3128313210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3128313210},{"type":"v1","addr":"172.21.15.155:6833","nonce":3128313210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6833/3545031610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3545031610},{"type":"v1","addr":"172.21.15.44:6833","nonce":3545031610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/698922157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":698922157},{"type":"v1","addr":"172.21.15.155:6835","nonce":698922157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6837/980782961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":980782961},{"type":"v1","addr":"172.21.15.155:6837","nonce":980782961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/728759484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":728759484},{"type":"v1","addr":"172.21.15.44:6837","nonce":728759484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/3934638415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3934638415},{"type":"v1","addr":"172.21.15.155:6839","nonce":3934638415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/2221834529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2221834529},{"type":"v1","addr":"172.21.15.44:6839","nonce":2221834529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26843":{"gid":26843,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/332625709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":332625709},{"type":"v1","addr":"172.21.15.44:6835","nonce":332625709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24491,24521,24469,24487,24508]},"id":1}]} 2024-04-23T18:56:12.313 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 31 2024-04-23T18:56:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 -- 172.21.15.19:0/1378497176 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d6c0a3d20 msgr2=0x7f0d6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:12.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d6c0a3d20 0x7f0d6c0a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00bbe0 tx=0x7f0d7c00b390 comp rx=0 tx=0).stop 2024-04-23T18:56:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 -- 172.21.15.19:0/1378497176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d80107280 msgr2=0x7f0d80198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:12.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d80107280 0x7f0d80198ca0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0d7002a930 tx=0x7f0d70004270 comp rx=0 tx=0).stop 2024-04-23T18:56:12.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 -- 172.21.15.19:0/1378497176 shutdown_connections 2024-04-23T18:56:12.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f0d6c0a3d20 0x7f0d6c0a61e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:12.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8019d020 0x7f0d8019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:12.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d80107280 0x7f0d80198ca0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:12.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 --2- 172.21.15.19:0/1378497176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102240 0x7f0d80198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:12.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 -- 172.21.15.19:0/1378497176 >> 172.21.15.19:0/1378497176 conn(0x7f0d800790e0 msgr2=0x7f0d80100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 -- 172.21.15.19:0/1378497176 shutdown_connections 2024-04-23T18:56:12.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:10.623+0000 7f0d8795d640 1 -- 172.21.15.19:0/1378497176 wait complete. 2024-04-23T18:56:12.855 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:56:12.888 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:11 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1378497176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:56:12.921 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:12 smithi019 ceph-mon[29215]: pgmap v953: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:56:12.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:12 smithi019 ceph-mon[29215]: osdmap e139: 12 total, 12 up, 12 in 2024-04-23T18:56:12.988 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:12 smithi155 ceph-mon[33468]: pgmap v953: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:56:13.021 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:12 smithi155 ceph-mon[33468]: osdmap e139: 12 total, 12 up, 12 in 2024-04-23T18:56:13.070 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:12 smithi044 ceph-mon[32458]: pgmap v953: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T18:56:13.104 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:12 smithi044 ceph-mon[32458]: osdmap e139: 12 total, 12 up, 12 in 2024-04-23T18:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.107+0000 7f8b3704e640 1 -- 172.21.15.19:0/2672199080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3010c520 msgr2=0x7f8b3010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:13.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.107+0000 7f8b3704e640 1 --2- 172.21.15.19:0/2672199080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3010c520 0x7f8b3010e9c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009ca0 tx=0x7f8b2802a9e0 comp rx=0 tx=0).stop 2024-04-23T18:56:13.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.107+0000 7f8b3704e640 1 -- 172.21.15.19:0/2672199080 shutdown_connections 2024-04-23T18:56:13.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.107+0000 7f8b3704e640 1 --2- 172.21.15.19:0/2672199080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b30103b00 0x7f8b3010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:13.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.107+0000 7f8b3704e640 1 --2- 172.21.15.19:0/2672199080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3010c520 0x7f8b3010e9c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:13.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.107+0000 7f8b3704e640 1 --2- 172.21.15.19:0/2672199080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b301031e0 0x7f8b301035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:13.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.107+0000 7f8b3704e640 1 -- 172.21.15.19:0/2672199080 >> 172.21.15.19:0/2672199080 conn(0x7f8b30077f50 msgr2=0x7f8b30078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:13.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.107+0000 7f8b3704e640 1 -- 172.21.15.19:0/2672199080 shutdown_connections 2024-04-23T18:56:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.107+0000 7f8b3704e640 1 -- 172.21.15.19:0/2672199080 wait complete. 2024-04-23T18:56:13.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b3704e640 1 Processor -- start 2024-04-23T18:56:13.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b3704e640 1 -- start start 2024-04-23T18:56:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b3704e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b301031e0 0x7f8b30100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b3704e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b30103b00 0x7f8b30101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:13.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b3704e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3010c520 0x7f8b301018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b3704e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301121b0 con 0x7f8b3010c520 2024-04-23T18:56:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b3704e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30112030 con 0x7f8b30103b00 2024-04-23T18:56:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b3704e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30112330 con 0x7f8b301031e0 2024-04-23T18:56:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b355eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3010c520 0x7f8b301018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:13.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b355eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3010c520 0x7f8b301018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36752/0 (socket says 172.21.15.19:36752) 2024-04-23T18:56:13.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b355eb640 1 -- 172.21.15.19:0/599232641 learned_addr learned my addr 172.21.15.19:0/599232641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:56:13.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b2ffff640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b30103b00 0x7f8b30101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b34dea640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b301031e0 0x7f8b30100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b355eb640 1 -- 172.21.15.19:0/599232641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b301031e0 msgr2=0x7f8b30100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b355eb640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b301031e0 0x7f8b30100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:14.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b355eb640 1 -- 172.21.15.19:0/599232641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b30103b00 msgr2=0x7f8b30101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:14.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b355eb640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b30103b00 0x7f8b30101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:14.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b355eb640 1 -- 172.21.15.19:0/599232641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b301a6b90 con 0x7f8b3010c520 2024-04-23T18:56:14.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b2ffff640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b30103b00 0x7f8b30101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:56:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.108+0000 7f8b34dea640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b301031e0 0x7f8b30100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:56:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.109+0000 7f8b355eb640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3010c520 0x7f8b301018b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8b28006010 tx=0x7f8b2802fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:14.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.109+0000 7f8b2dffb640 1 -- 172.21.15.19:0/599232641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28033040 con 0x7f8b3010c520 2024-04-23T18:56:14.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.109+0000 7f8b2dffb640 1 -- 172.21.15.19:0/599232641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b2802e7a0 con 0x7f8b3010c520 2024-04-23T18:56:14.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.109+0000 7f8b3704e640 1 -- 172.21.15.19:0/599232641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b301a6e20 con 0x7f8b3010c520 2024-04-23T18:56:14.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.109+0000 7f8b2dffb640 1 -- 172.21.15.19:0/599232641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b280437b0 con 0x7f8b3010c520 2024-04-23T18:56:14.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.109+0000 7f8b3704e640 1 -- 172.21.15.19:0/599232641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b301a7280 con 0x7f8b3010c520 2024-04-23T18:56:14.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.110+0000 7f8b3704e640 1 -- 172.21.15.19:0/599232641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b30068d70 con 0x7f8b3010c520 2024-04-23T18:56:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.111+0000 7f8b2dffb640 1 -- 172.21.15.19:0/599232641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b28036070 con 0x7f8b3010c520 2024-04-23T18:56:14.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.112+0000 7f8b2dffb640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b180a3d50 0x7f8b180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:14.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.112+0000 7f8b34dea640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b180a3d50 0x7f8b180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.112+0000 7f8b2dffb640 1 -- 172.21.15.19:0/599232641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 8557+0+0 (secure 0 0 0) 0x7f8b280f1e30 con 0x7f8b3010c520 2024-04-23T18:56:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.112+0000 7f8b34dea640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b180a3d50 0x7f8b180a6210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009b40 tx=0x7f8b20005e50 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.116+0000 7f8b2dffb640 1 -- 172.21.15.19:0/599232641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b2803e070 con 0x7f8b3010c520 2024-04-23T18:56:14.756 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:14 smithi155 ceph-mon[33468]: pgmap v955: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:56:14.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:14 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/599232641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:56:14.822 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:14 smithi044 ceph-mon[32458]: pgmap v955: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:56:14.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:14 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/599232641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:56:14.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.303+0000 7f8b3704e640 1 -- 172.21.15.19:0/599232641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b30102310 con 0x7f8b3010c520 2024-04-23T18:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.305+0000 7f8b2dffb640 1 -- 172.21.15.19:0/599232641 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+23180 (secure 0 0 0) 0x7f8b280098b0 con 0x7f8b3010c520 2024-04-23T18:56:14.982 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-23T18:56:15.014 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":139,"fsid":"46706838-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:15:32.046592+0000","modified":"2024-04-23T18:56:11.077546+0000","last_up_change":"2024-04-23T18:23:01.624434+0000","last_in_change":"2024-04-23T18:22:40.556076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:07.365995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:23:53.296504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:23:54.573484+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":55,"snap_epoch":139,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78ea8328-c293-4ef2-af0d-81bfe0dccc4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6803","nonce":4279899748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6805","nonce":4279899748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6809","nonce":4279899748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":4279899748},{"type":"v1","addr":"172.21.15.19:6807","nonce":4279899748}]},"public_addr":"172.21.15.19:6803/4279899748","cluster_addr":"172.21.15.19:6805/4279899748","heartbeat_back_addr":"172.21.15.19:6809/4279899748","heartbeat_front_addr":"172.21.15.19:6807/4279899748","state":["exists","up"]},{"osd":1,"uuid":"3e9e3631-24c8-42d1-84c6-48cf0cda9283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6801","nonce":2943619157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6803","nonce":2943619157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6807","nonce":2943619157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2943619157},{"type":"v1","addr":"172.21.15.44:6805","nonce":2943619157}]},"public_addr":"172.21.15.44:6801/2943619157","cluster_addr":"172.21.15.44:6803/2943619157","heartbeat_back_addr":"172.21.15.44:6807/2943619157","heartbeat_front_addr":"172.21.15.44:6805/2943619157","state":["exists","up"]},{"osd":2,"uuid":"680b8f94-cb10-444d-a1a4-22c12f9cbe1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6801","nonce":3680785490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6803","nonce":3680785490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6807","nonce":3680785490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3680785490},{"type":"v1","addr":"172.21.15.155:6805","nonce":3680785490}]},"public_addr":"172.21.15.155:6801/3680785490","cluster_addr":"172.21.15.155:6803/3680785490","heartbeat_back_addr":"172.21.15.155:6807/3680785490","heartbeat_front_addr":"172.21.15.155:6805/3680785490","state":["exists","up"]},{"osd":3,"uuid":"31e55972-9f96-4714-b681-fa4af600b35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6811","nonce":3983142737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6813","nonce":3983142737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6817","nonce":3983142737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3983142737},{"type":"v1","addr":"172.21.15.19:6815","nonce":3983142737}]},"public_addr":"172.21.15.19:6811/3983142737","cluster_addr":"172.21.15.19:6813/3983142737","heartbeat_back_addr":"172.21.15.19:6817/3983142737","heartbeat_front_addr":"172.21.15.19:6815/3983142737","state":["exists","up"]},{"osd":4,"uuid":"a48cc342-4cea-4410-8b19-0f049a183667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6809","nonce":3405107831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6811","nonce":3405107831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6815","nonce":3405107831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3405107831},{"type":"v1","addr":"172.21.15.44:6813","nonce":3405107831}]},"public_addr":"172.21.15.44:6809/3405107831","cluster_addr":"172.21.15.44:6811/3405107831","heartbeat_back_addr":"172.21.15.44:6815/3405107831","heartbeat_front_addr":"172.21.15.44:6813/3405107831","state":["exists","up"]},{"osd":5,"uuid":"777efb4b-82b9-4ce6-8187-80697bffc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6809","nonce":2897505924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6811","nonce":2897505924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6815","nonce":2897505924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2897505924},{"type":"v1","addr":"172.21.15.155:6813","nonce":2897505924}]},"public_addr":"172.21.15.155:6809/2897505924","cluster_addr":"172.21.15.155:6811/2897505924","heartbeat_back_addr":"172.21.15.155:6815/2897505924","heartbeat_front_addr":"172.21.15.155:6813/2897505924","state":["exists","up"]},{"osd":6,"uuid":"a0be6429-6e18-42e7-b385-34826780f118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6819","nonce":2995627219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6821","nonce":2995627219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6825","nonce":2995627219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2995627219},{"type":"v1","addr":"172.21.15.19:6823","nonce":2995627219}]},"public_addr":"172.21.15.19:6819/2995627219","cluster_addr":"172.21.15.19:6821/2995627219","heartbeat_back_addr":"172.21.15.19:6825/2995627219","heartbeat_front_addr":"172.21.15.19:6823/2995627219","state":["exists","up"]},{"osd":7,"uuid":"71f568c1-298c-4d3a-a191-02ffd057f620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6817","nonce":2128528085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6819","nonce":2128528085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6823","nonce":2128528085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2128528085},{"type":"v1","addr":"172.21.15.44:6821","nonce":2128528085}]},"public_addr":"172.21.15.44:6817/2128528085","cluster_addr":"172.21.15.44:6819/2128528085","heartbeat_back_addr":"172.21.15.44:6823/2128528085","heartbeat_front_addr":"172.21.15.44:6821/2128528085","state":["exists","up"]},{"osd":8,"uuid":"49dae33b-4d26-4145-a4eb-3ffd2dc50e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6817","nonce":3480056050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6819","nonce":3480056050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6823","nonce":3480056050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3480056050},{"type":"v1","addr":"172.21.15.155:6821","nonce":3480056050}]},"public_addr":"172.21.15.155:6817/3480056050","cluster_addr":"172.21.15.155:6819/3480056050","heartbeat_back_addr":"172.21.15.155:6823/3480056050","heartbeat_front_addr":"172.21.15.155:6821/3480056050","state":["exists","up"]},{"osd":9,"uuid":"7715f388-45b3-47c4-82a2-d67a2de63b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6827","nonce":2566199237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6829","nonce":2566199237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6833","nonce":2566199237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2566199237},{"type":"v1","addr":"172.21.15.19:6831","nonce":2566199237}]},"public_addr":"172.21.15.19:6827/2566199237","cluster_addr":"172.21.15.19:6829/2566199237","heartbeat_back_addr":"172.21.15.19:6833/2566199237","heartbeat_front_addr":"172.21.15.19:6831/2566199237","state":["exists","up"]},{"osd":10,"uuid":"393911c2-6938-4f7a-93d6-6c187f236b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6825","nonce":2033313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6827","nonce":2033313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6831","nonce":2033313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2033313090},{"type":"v1","addr":"172.21.15.44:6829","nonce":2033313090}]},"public_addr":"172.21.15.44:6825/2033313090","cluster_addr":"172.21.15.44:6827/2033313090","heartbeat_back_addr":"172.21.15.44:6831/2033313090","heartbeat_front_addr":"172.21.15.44:6829/2033313090","state":["exists","up"]},{"osd":11,"uuid":"35d577a0-4fd5-4e4d-a402-71434ca33c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6825","nonce":3982643864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6827","nonce":3982643864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6831","nonce":3982643864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3982643864},{"type":"v1","addr":"172.21.15.155:6829","nonce":3982643864}]},"public_addr":"172.21.15.155:6825/3982643864","cluster_addr":"172.21.15.155:6827/3982643864","heartbeat_back_addr":"172.21.15.155:6831/3982643864","heartbeat_front_addr":"172.21.15.155:6829/3982643864","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:15.788040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:40.134628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:04.175629+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:30.059781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:55.665668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:21.182671+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:48.464441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:14.124778+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:39.545089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:07.065199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:32.927482+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:59.209422+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/304874449":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1108384152":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6801/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:6800/1993809272":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/983971095":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/1473447325":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6800/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/3826407819":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/4193386941":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/1551393169":"2024-04-24T18:16:35.264133+0000","172.21.15.19:6801/2431053142":"2024-04-24T18:16:35.264133+0000","172.21.15.19:0/4087353417":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6800/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:6801/1085063871":"2024-04-24T18:16:06.817907+0000","172.21.15.19:0/2556512605":"2024-04-24T18:26:08.116888+0000","172.21.15.19:0/2920477219":"2024-04-24T18:16:35.264133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":42,"length":1},{"begin":55,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":42,"length":1},{"begin":55,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:56:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 -- 172.21.15.19:0/599232641 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b180a3d50 msgr2=0x7f8b180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:15.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b180a3d50 0x7f8b180a6210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009b40 tx=0x7f8b20005e50 comp rx=0 tx=0).stop 2024-04-23T18:56:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 -- 172.21.15.19:0/599232641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3010c520 msgr2=0x7f8b301018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3010c520 0x7f8b301018b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8b28006010 tx=0x7f8b2802fc00 comp rx=0 tx=0).stop 2024-04-23T18:56:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 -- 172.21.15.19:0/599232641 shutdown_connections 2024-04-23T18:56:15.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f8b180a3d50 0x7f8b180a6210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:15.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b30103b00 0x7f8b30101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3010c520 0x7f8b301018b0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 --2- 172.21.15.19:0/599232641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b301031e0 0x7f8b30100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 -- 172.21.15.19:0/599232641 >> 172.21.15.19:0/599232641 conn(0x7f8b30077f50 msgr2=0x7f8b300fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 -- 172.21.15.19:0/599232641 shutdown_connections 2024-04-23T18:56:15.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:13.307+0000 7f8b3704e640 1 -- 172.21.15.19:0/599232641 wait complete. 2024-04-23T18:56:15.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T18:56:15.526 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:14 smithi019 ceph-mon[29215]: pgmap v955: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:56:15.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:14 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/599232641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:56:15.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.732+0000 7fc6d6144640 1 -- 172.21.15.19:0/4170203052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6d0103230 msgr2=0x7fc6d0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:15.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.732+0000 7fc6d6144640 1 --2- 172.21.15.19:0/4170203052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6d0103230 0x7fc6d0103610 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009900 tx=0x7fc6c002a480 comp rx=0 tx=0).stop 2024-04-23T18:56:15.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.732+0000 7fc6d6144640 1 -- 172.21.15.19:0/4170203052 shutdown_connections 2024-04-23T18:56:15.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.732+0000 7fc6d6144640 1 --2- 172.21.15.19:0/4170203052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d010c600 0x7fc6d010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:15.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.732+0000 7fc6d6144640 1 --2- 172.21.15.19:0/4170203052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6d0103b50 0x7fc6d010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:15.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.732+0000 7fc6d6144640 1 --2- 172.21.15.19:0/4170203052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6d0103230 0x7fc6d0103610 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.732+0000 7fc6d6144640 1 -- 172.21.15.19:0/4170203052 >> 172.21.15.19:0/4170203052 conn(0x7fc6d0077f80 msgr2=0x7fc6d0078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:16.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.732+0000 7fc6d6144640 1 -- 172.21.15.19:0/4170203052 shutdown_connections 2024-04-23T18:56:16.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.733+0000 7fc6d6144640 1 -- 172.21.15.19:0/4170203052 wait complete. 2024-04-23T18:56:16.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.733+0000 7fc6d6144640 1 Processor -- start 2024-04-23T18:56:16.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.733+0000 7fc6d6144640 1 -- start start 2024-04-23T18:56:16.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6d6144640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6d0103230 0x7fc6d0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:16.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6d6144640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d0103b50 0x7fc6d01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:16.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6d6144640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6d010c600 0x7fc6d0101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:16.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6d6144640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0112240 con 0x7fc6d010c600 2024-04-23T18:56:16.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6d6144640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d01120c0 con 0x7fc6d0103b50 2024-04-23T18:56:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6d6144640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d01123c0 con 0x7fc6d0103230 2024-04-23T18:56:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6cffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6d010c600 0x7fc6d0101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6cffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6d010c600 0x7fc6d0101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36772/0 (socket says 172.21.15.19:36772) 2024-04-23T18:56:16.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6cffff640 1 -- 172.21.15.19:0/2987345332 learned_addr learned my addr 172.21.15.19:0/2987345332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:56:16.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6ceffd640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d0103b50 0x7fc6d01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6cf7fe640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6d0103230 0x7fc6d0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6cffff640 1 -- 172.21.15.19:0/2987345332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6d0103230 msgr2=0x7fc6d0100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6cffff640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6d0103230 0x7fc6d0100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:16.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6cffff640 1 -- 172.21.15.19:0/2987345332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d0103b50 msgr2=0x7fc6d01013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:16.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6cffff640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d0103b50 0x7fc6d01013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:16.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6cffff640 1 -- 172.21.15.19:0/2987345332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d01a6d10 con 0x7fc6d010c600 2024-04-23T18:56:16.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6cffff640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6d010c600 0x7fc6d0101900 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009fd0 tx=0x7fc6c800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6ccff9640 1 -- 172.21.15.19:0/2987345332 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8017070 con 0x7fc6d010c600 2024-04-23T18:56:16.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6ccff9640 1 -- 172.21.15.19:0/2987345332 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c8003e20 con 0x7fc6d010c600 2024-04-23T18:56:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.734+0000 7fc6d6144640 1 -- 172.21.15.19:0/2987345332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d01a7000 con 0x7fc6d010c600 2024-04-23T18:56:17.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.735+0000 7fc6ccff9640 1 -- 172.21.15.19:0/2987345332 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8010460 con 0x7fc6d010c600 2024-04-23T18:56:17.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.735+0000 7fc6d6144640 1 -- 172.21.15.19:0/2987345332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d01a74d0 con 0x7fc6d010c600 2024-04-23T18:56:17.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.735+0000 7fc6d6144640 1 -- 172.21.15.19:0/2987345332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d006add0 con 0x7fc6d010c600 2024-04-23T18:56:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.737+0000 7fc6ccff9640 1 -- 172.21.15.19:0/2987345332 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6c8003970 con 0x7fc6d010c600 2024-04-23T18:56:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.737+0000 7fc6ccff9640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc6b80a3d20 0x7fc6b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.738+0000 7fc6cf7fe640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc6b80a3d20 0x7fc6b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.738+0000 7fc6ccff9640 1 -- 172.21.15.19:0/2987345332 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 8557+0+0 (secure 0 0 0) 0x7fc6c80d0530 con 0x7fc6d010c600 2024-04-23T18:56:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.738+0000 7fc6cf7fe640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc6b80a3d20 0x7fc6b80a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0031440 tx=0x7fc6c00095f0 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:17.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.741+0000 7fc6ccff9640 1 -- 172.21.15.19:0/2987345332 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6c809cbb0 con 0x7fc6d010c600 2024-04-23T18:56:17.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.968+0000 7fc6d6144640 1 -- 172.21.15.19:0/2987345332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc6d0107720 con 0x7fc6d010c600 2024-04-23T18:56:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.970+0000 7fc6ccff9640 1 -- 172.21.15.19:0/2987345332 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) ==== 161+0+0 (secure 0 0 0) 0x7fc6c809c4b0 con 0x7fc6d010c600 2024-04-23T18:56:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T18:56:17.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 -- 172.21.15.19:0/2987345332 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc6b80a3d20 msgr2=0x7fc6b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:17.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc6b80a3d20 0x7fc6b80a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0031440 tx=0x7fc6c00095f0 comp rx=0 tx=0).stop 2024-04-23T18:56:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 -- 172.21.15.19:0/2987345332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6d010c600 msgr2=0x7fc6d0101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6d010c600 0x7fc6d0101900 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009fd0 tx=0x7fc6c800b260 comp rx=0 tx=0).stop 2024-04-23T18:56:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 -- 172.21.15.19:0/2987345332 shutdown_connections 2024-04-23T18:56:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fc6b80a3d20 0x7fc6b80a61e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:17.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d0103b50 0x7fc6d01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:17.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6d010c600 0x7fc6d0101900 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:17.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 --2- 172.21.15.19:0/2987345332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6d0103230 0x7fc6d0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 -- 172.21.15.19:0/2987345332 >> 172.21.15.19:0/2987345332 conn(0x7fc6d0077f80 msgr2=0x7fc6d00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 -- 172.21.15.19:0/2987345332 shutdown_connections 2024-04-23T18:56:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:15.973+0000 7fc6d6144640 1 -- 172.21.15.19:0/2987345332 wait complete. 2024-04-23T18:56:17.975 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T18:56:18.009 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mon[33468]: pgmap v956: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 10 op/s 2024-04-23T18:56:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2987345332' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T18:56:18.075 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mon[33468]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T18:56:18.126 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mon[33468]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:56:18.199 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/2987345332' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T18:56:18.233 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mon[33468]: osdmap e140: 12 total, 12 up, 12 in 2024-04-23T18:56:18.266 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mon[33468]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-23T18:56:18.300 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b[62807]: ignoring --setuser ceph since I am not root 2024-04-23T18:56:18.333 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b[62807]: ignoring --setgroup ceph since I am not root 2024-04-23T18:56:18.366 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mds[62811]: -- 172.21.15.44:0/3165703859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b943b55a40 con 0x55b943b28800 2024-04-23T18:56:18.400 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b[62807]: starting mds.b at 2024-04-23T18:56:18.434 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a[70760]: ignoring --setuser ceph since I am not root 2024-04-23T18:56:18.493 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a[70760]: ignoring --setgroup ceph since I am not root 2024-04-23T18:56:18.525 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-mds[70766]: -- 172.21.15.19:0/199125617 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560dc506ba40 con 0x560dc503e800 2024-04-23T18:56:18.558 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a[70760]: starting mds.a at 2024-04-23T18:56:18.617 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c[63613]: ignoring --setuser ceph since I am not root 2024-04-23T18:56:18.650 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c[63613]: ignoring --setgroup ceph since I am not root 2024-04-23T18:56:18.683 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mds[63617]: -- 172.21.15.155:0/2218951501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55692d21ba40 con 0x55692d1ee800 2024-04-23T18:56:18.717 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c[63613]: starting mds.c at 2024-04-23T18:56:18.751 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69558]: ignoring --setuser ceph since I am not root 2024-04-23T18:56:18.792 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69558]: ignoring --setgroup ceph since I am not root 2024-04-23T18:56:18.857 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mds[69562]: -- 172.21.15.44:0/1274956314 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55887f785a40 con 0x55887f758800 2024-04-23T18:56:18.892 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69558]: starting mds.e at 2024-04-23T18:56:18.926 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:15 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a[29211]: 2024-04-23T18:56:15.969+0000 7fe97eba8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:56:18.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-mon[29215]: pgmap v956: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 10 op/s 2024-04-23T18:56:18.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2987345332' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T18:56:19.025 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-mon[29215]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T18:56:19.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-mon[29215]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:56:19.100 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/2987345332' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T18:56:19.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-mon[29215]: osdmap e140: 12 total, 12 up, 12 in 2024-04-23T18:56:19.176 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-mon[29215]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-23T18:56:19.226 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f[63308]: ignoring --setuser ceph since I am not root 2024-04-23T18:56:19.259 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f[63308]: ignoring --setgroup ceph since I am not root 2024-04-23T18:56:19.292 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mds[63312]: -- 172.21.15.155:0/1913863399 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562474101a40 con 0x5624740d4c00 2024-04-23T18:56:19.329 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f[63308]: starting mds.f at 2024-04-23T18:56:19.362 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h[62501]: ignoring --setuser ceph since I am not root 2024-04-23T18:56:19.395 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h[62501]: ignoring --setgroup ceph since I am not root 2024-04-23T18:56:19.442 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mds[62505]: -- 172.21.15.44:0/2782434004 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea947b5a40 con 0x55ea94789000 2024-04-23T18:56:19.513 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h[62501]: starting mds.h at 2024-04-23T18:56:19.671 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g[70359]: ignoring --setuser ceph since I am not root 2024-04-23T18:56:19.746 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g[70359]: ignoring --setgroup ceph since I am not root 2024-04-23T18:56:19.779 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-mds[70363]: -- 172.21.15.19:0/3830774122 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5571e4011a40 con 0x5571e3fe4800 2024-04-23T18:56:19.812 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:56:16 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g[70359]: starting mds.g at 2024-04-23T18:56:19.856 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l[63000]: ignoring --setuser ceph since I am not root 2024-04-23T18:56:19.896 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l[63000]: ignoring --setgroup ceph since I am not root 2024-04-23T18:56:19.929 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mds[63004]: -- 172.21.15.155:0/2332200088 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5555a0e95a40 con 0x5555a0e68800 2024-04-23T18:56:19.962 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l[63000]: starting mds.l at 2024-04-23T18:56:20.038 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k[61882]: ignoring --setuser ceph since I am not root 2024-04-23T18:56:20.071 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k[61882]: ignoring --setgroup ceph since I am not root 2024-04-23T18:56:20.104 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mds[61886]: -- 172.21.15.44:0/3588921830 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed890f1a40 con 0x55ed890c4800 2024-04-23T18:56:20.154 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k[61882]: starting mds.k at 2024-04-23T18:56:20.188 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-i[63914]: ignoring --setuser ceph since I am not root 2024-04-23T18:56:20.221 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-i[63914]: ignoring --setgroup ceph since I am not root 2024-04-23T18:56:20.254 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-mds[63918]: -- 172.21.15.155:0/664542486 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e2266fa40 con 0x561e22642800 2024-04-23T18:56:20.288 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:56:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-i[63914]: starting mds.i at 2024-04-23T18:56:20.325 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: pgmap v958: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-23T18:56:20.363 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6832/2871286470,v1:172.21.15.155:6834/2871286470] up:boot 2024-04-23T18:56:20.403 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6833/1445021829,v1:172.21.15.155:6835/1445021829] up:boot 2024-04-23T18:56:20.459 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6833/2189735838,v1:172.21.15.44:6835/2189735838] up:boot 2024-04-23T18:56:20.504 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6832/860498879,v1:172.21.15.44:6834/860498879] up:boot 2024-04-23T18:56:20.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.19:6838/807723773,v1:172.21.15.19:6840/807723773] up:boot 2024-04-23T18:56:20.571 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.19:6839/1770450267,v1:172.21.15.19:6841/1770450267] up:boot 2024-04-23T18:56:20.605 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6837/1347089856,v1:172.21.15.155:6839/1347089856] up:boot 2024-04-23T18:56:20.646 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.155:6836/1482333101,v1:172.21.15.155:6838/1482333101] up:boot 2024-04-23T18:56:20.680 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6836/3805858248,v1:172.21.15.44:6838/3805858248] up:boot 2024-04-23T18:56:20.734 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: mds.? [v2:172.21.15.44:6837/3891284197,v1:172.21.15.44:6839/3891284197] up:boot 2024-04-23T18:56:20.771 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-23T18:56:20.805 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:56:20.838 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:56:20.871 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:56:20.905 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:56:20.954 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:56:20.988 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:56:21.022 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:56:21.091 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:56:21.143 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:56:21.180 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:17 smithi155 ceph-mon[33468]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:56:21.240 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:19 smithi155 ceph-mon[33468]: osdmap e141: 12 total, 12 up, 12 in 2024-04-23T18:56:21.281 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: pgmap v958: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-23T18:56:21.330 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6832/2871286470,v1:172.21.15.155:6834/2871286470] up:boot 2024-04-23T18:56:21.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6833/1445021829,v1:172.21.15.155:6835/1445021829] up:boot 2024-04-23T18:56:21.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6833/2189735838,v1:172.21.15.44:6835/2189735838] up:boot 2024-04-23T18:56:21.446 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6832/860498879,v1:172.21.15.44:6834/860498879] up:boot 2024-04-23T18:56:21.480 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.19:6838/807723773,v1:172.21.15.19:6840/807723773] up:boot 2024-04-23T18:56:21.514 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.19:6839/1770450267,v1:172.21.15.19:6841/1770450267] up:boot 2024-04-23T18:56:21.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6837/1347089856,v1:172.21.15.155:6839/1347089856] up:boot 2024-04-23T18:56:21.580 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.155:6836/1482333101,v1:172.21.15.155:6838/1482333101] up:boot 2024-04-23T18:56:21.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6836/3805858248,v1:172.21.15.44:6838/3805858248] up:boot 2024-04-23T18:56:21.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: mds.? [v2:172.21.15.44:6837/3891284197,v1:172.21.15.44:6839/3891284197] up:boot 2024-04-23T18:56:21.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-23T18:56:21.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:56:21.787 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:56:21.838 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:56:21.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:56:21.906 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:56:21.947 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:56:21.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:56:22.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:56:22.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:56:22.095 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:17 smithi019 ceph-mon[29215]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:56:22.131 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:19 smithi019 ceph-mon[29215]: osdmap e141: 12 total, 12 up, 12 in 2024-04-23T18:56:22.166 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mon[32458]: pgmap v956: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 10 op/s 2024-04-23T18:56:22.198 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2987345332' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T18:56:22.231 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mon[32458]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T18:56:22.266 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mon[32458]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:56:22.300 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/2987345332' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T18:56:22.339 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mon[32458]: osdmap e140: 12 total, 12 up, 12 in 2024-04-23T18:56:22.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:16 smithi044 ceph-mon[32458]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-23T18:56:22.460 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: pgmap v958: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-23T18:56:22.513 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6832/2871286470,v1:172.21.15.155:6834/2871286470] up:boot 2024-04-23T18:56:22.548 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6833/1445021829,v1:172.21.15.155:6835/1445021829] up:boot 2024-04-23T18:56:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6833/2189735838,v1:172.21.15.44:6835/2189735838] up:boot 2024-04-23T18:56:22.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6832/860498879,v1:172.21.15.44:6834/860498879] up:boot 2024-04-23T18:56:22.673 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.19:6838/807723773,v1:172.21.15.19:6840/807723773] up:boot 2024-04-23T18:56:22.706 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.19:6839/1770450267,v1:172.21.15.19:6841/1770450267] up:boot 2024-04-23T18:56:22.740 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6837/1347089856,v1:172.21.15.155:6839/1347089856] up:boot 2024-04-23T18:56:22.792 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.155:6836/1482333101,v1:172.21.15.155:6838/1482333101] up:boot 2024-04-23T18:56:22.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6836/3805858248,v1:172.21.15.44:6838/3805858248] up:boot 2024-04-23T18:56:22.865 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: mds.? [v2:172.21.15.44:6837/3891284197,v1:172.21.15.44:6839/3891284197] up:boot 2024-04-23T18:56:22.898 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-23T18:56:22.931 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:56:22.965 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:56:23.005 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:56:23.040 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:56:23.073 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:56:23.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:56:23.159 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:56:23.273 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:56:23.307 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:56:23.348 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:17 smithi044 ceph-mon[32458]: from='mgr.25004 172.21.15.19:0/3096649862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:56:23.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:19 smithi044 ceph-mon[32458]: osdmap e141: 12 total, 12 up, 12 in 2024-04-23T18:56:23.415 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:20 smithi044 ceph-mon[32458]: pgmap v960: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T18:56:23.448 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:21 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1532990408' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:56:23.485 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:20 smithi155 ceph-mon[33468]: pgmap v960: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T18:56:23.524 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:21 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1532990408' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:56:23.557 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:22 smithi155 ceph-mon[33468]: pgmap v961: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 127 B/s wr, 8 op/s 2024-04-23T18:56:23.590 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:22 smithi155 ceph-mon[33468]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T18:56:23.640 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:22 smithi155 ceph-mon[33468]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T18:56:23.674 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:22 smithi155 ceph-mon[33468]: Cluster is now healthy 2024-04-23T18:56:23.707 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:22 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1532990408' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:56:23.740 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:22 smithi155 ceph-mon[33468]: fsmap 12 up:standby 2024-04-23T18:56:23.776 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:22 smithi044 ceph-mon[32458]: pgmap v961: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 127 B/s wr, 8 op/s 2024-04-23T18:56:23.825 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:22 smithi044 ceph-mon[32458]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T18:56:23.907 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:22 smithi044 ceph-mon[32458]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T18:56:23.941 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:22 smithi044 ceph-mon[32458]: Cluster is now healthy 2024-04-23T18:56:23.974 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:22 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1532990408' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:56:24.007 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:22 smithi044 ceph-mon[32458]: fsmap 12 up:standby 2024-04-23T18:56:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.975+0000 7feedc19b640 1 -- 172.21.15.19:0/2698626684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feed4107270 msgr2=0x7feed410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.975+0000 7feedc19b640 1 --2- 172.21.15.19:0/2698626684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feed4107270 0x7feed410ee40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7feecc009ca0 tx=0x7feecc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:56:24.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.975+0000 7feedc19b640 1 -- 172.21.15.19:0/2698626684 shutdown_connections 2024-04-23T18:56:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.975+0000 7feedc19b640 1 --2- 172.21.15.19:0/2698626684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feed4107270 0x7feed410ee40 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:24.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.975+0000 7feedc19b640 1 --2- 172.21.15.19:0/2698626684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed406b1f0 0x7feed4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:24.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.975+0000 7feedc19b640 1 --2- 172.21.15.19:0/2698626684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed406a840 0x7feed406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:24.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.975+0000 7feedc19b640 1 -- 172.21.15.19:0/2698626684 >> 172.21.15.19:0/2698626684 conn(0x7feed4077eb0 msgr2=0x7feed40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feedc19b640 1 -- 172.21.15.19:0/2698626684 shutdown_connections 2024-04-23T18:56:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feedc19b640 1 -- 172.21.15.19:0/2698626684 wait complete. 2024-04-23T18:56:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feedc19b640 1 Processor -- start 2024-04-23T18:56:24.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feedc19b640 1 -- start start 2024-04-23T18:56:24.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feedc19b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feed406a840 0x7feed41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:24.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feedc19b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed4107270 0x7feed4198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feedc19b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed419cf90 0x7feed419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feedc19b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feed41121d0 con 0x7feed406a840 2024-04-23T18:56:24.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feedc19b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feed4112050 con 0x7feed4107270 2024-04-23T18:56:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feedc19b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feed4112350 con 0x7feed419cf90 2024-04-23T18:56:24.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.976+0000 7feed9f37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feed406a840 0x7feed41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feed9f37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feed406a840 0x7feed41986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38156/0 (socket says 172.21.15.19:38156) 2024-04-23T18:56:24.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feed9f37640 1 -- 172.21.15.19:0/1532990408 learned_addr learned my addr 172.21.15.19:0/1532990408 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:56:24.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feeda738640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed419cf90 0x7feed419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feed9736640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed4107270 0x7feed4198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:24.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feed9f37640 1 -- 172.21.15.19:0/1532990408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed419cf90 msgr2=0x7feed419d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feed9f37640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed419cf90 0x7feed419d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feed9f37640 1 -- 172.21.15.19:0/1532990408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed4107270 msgr2=0x7feed4198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feed9f37640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed4107270 0x7feed4198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feed9f37640 1 -- 172.21.15.19:0/1532990408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed419da80 con 0x7feed406a840 2024-04-23T18:56:25.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feeda738640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed419cf90 0x7feed419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:56:25.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feed9f37640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feed406a840 0x7feed41986d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7feed000c890 tx=0x7feed000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:25.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feecaffd640 1 -- 172.21.15.19:0/1532990408 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0004220 con 0x7feed406a840 2024-04-23T18:56:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feedc19b640 1 -- 172.21.15.19:0/1532990408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed419dd70 con 0x7feed406a840 2024-04-23T18:56:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feecaffd640 1 -- 172.21.15.19:0/1532990408 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed00043c0 con 0x7feed406a840 2024-04-23T18:56:25.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feecaffd640 1 -- 172.21.15.19:0/1532990408 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0004a00 con 0x7feed406a840 2024-04-23T18:56:25.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.977+0000 7feedc19b640 1 -- 172.21.15.19:0/1532990408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41a55b0 con 0x7feed406a840 2024-04-23T18:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.978+0000 7feedc19b640 1 -- 172.21.15.19:0/1532990408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed40691c0 con 0x7feed406a840 2024-04-23T18:56:25.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.979+0000 7feecaffd640 1 -- 172.21.15.19:0/1532990408 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feed00079b0 con 0x7feed406a840 2024-04-23T18:56:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.980+0000 7feecaffd640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feec00a3d20 0x7feec00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:25.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.980+0000 7feed9736640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feec00a3d20 0x7feec00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:25.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.980+0000 7feecaffd640 1 -- 172.21.15.19:0/1532990408 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 8943+0+0 (secure 0 0 0) 0x7feed00d0c00 con 0x7feed406a840 2024-04-23T18:56:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.981+0000 7feed9736640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feec00a3d20 0x7feec00a61e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feec4005fd0 tx=0x7feec400b510 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:25.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:19.984+0000 7feecaffd640 1 -- 172.21.15.19:0/1532990408 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feed0014030 con 0x7feed406a840 2024-04-23T18:56:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:20.222+0000 7feedc19b640 1 -- 172.21.15.19:0/1532990408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7feed410a4f0 con 0x7feed406a840 2024-04-23T18:56:25.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:20 smithi019 ceph-mon[29215]: pgmap v960: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T18:56:25.801 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:24 smithi155 ceph-mon[33468]: pgmap v962: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 127 B/s wr, 2 op/s 2024-04-23T18:56:25.851 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:24 smithi044 ceph-mon[32458]: pgmap v962: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 127 B/s wr, 2 op/s 2024-04-23T18:56:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.082+0000 7feecaffd640 1 -- 172.21.15.19:0/1532990408 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) ==== 104+0+0 (secure 0 0 0) 0x7feed009d0a0 con 0x7feed406a840 2024-04-23T18:56:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.085+0000 7feedc19b640 1 -- 172.21.15.19:0/1532990408 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feec00a3d20 msgr2=0x7feec00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.085+0000 7feedc19b640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feec00a3d20 0x7feec00a61e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feec4005fd0 tx=0x7feec400b510 comp rx=0 tx=0).stop 2024-04-23T18:56:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.085+0000 7feedc19b640 1 -- 172.21.15.19:0/1532990408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feed406a840 msgr2=0x7feed41986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:26.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.085+0000 7feedc19b640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feed406a840 0x7feed41986d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7feed000c890 tx=0x7feed000cd50 comp rx=0 tx=0).stop 2024-04-23T18:56:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.086+0000 7feedc19b640 1 -- 172.21.15.19:0/1532990408 shutdown_connections 2024-04-23T18:56:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.086+0000 7feedc19b640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7feec00a3d20 0x7feec00a61e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:26.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.086+0000 7feedc19b640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed4107270 0x7feed4198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:26.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.086+0000 7feedc19b640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed419cf90 0x7feed419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.086+0000 7feedc19b640 1 --2- 172.21.15.19:0/1532990408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feed406a840 0x7feed41986d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.086+0000 7feedc19b640 1 -- 172.21.15.19:0/1532990408 >> 172.21.15.19:0/1532990408 conn(0x7feed4077eb0 msgr2=0x7feed4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.086+0000 7feedc19b640 1 -- 172.21.15.19:0/1532990408 shutdown_connections 2024-04-23T18:56:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:21.086+0000 7feedc19b640 1 -- 172.21.15.19:0/1532990408 wait complete. 2024-04-23T18:56:26.435 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T18:56:26.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:21 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1532990408' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:56:26.518 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:22 smithi019 ceph-mon[29215]: pgmap v961: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 127 B/s wr, 8 op/s 2024-04-23T18:56:26.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:22 smithi019 ceph-mon[29215]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T18:56:26.584 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:22 smithi019 ceph-mon[29215]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T18:56:26.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:22 smithi019 ceph-mon[29215]: Cluster is now healthy 2024-04-23T18:56:26.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:22 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1532990408' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:56:26.785 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:22 smithi019 ceph-mon[29215]: fsmap 12 up:standby 2024-04-23T18:56:26.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:24 smithi019 ceph-mon[29215]: pgmap v962: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 127 B/s wr, 2 op/s 2024-04-23T18:56:26.852 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:26 smithi019 ceph-mon[29215]: pgmap v963: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 118 B/s wr, 2 op/s 2024-04-23T18:56:26.885 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:26 smithi155 ceph-mon[33468]: pgmap v963: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 118 B/s wr, 2 op/s 2024-04-23T18:56:26.918 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:26 smithi044 ceph-mon[32458]: pgmap v963: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 118 B/s wr, 2 op/s 2024-04-23T18:56:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.974+0000 7fdddfee1640 1 -- 172.21.15.19:0/805484023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddd81018e0 msgr2=0x7fddd8105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:27.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.974+0000 7fdddfee1640 1 --2- 172.21.15.19:0/805484023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddd81018e0 0x7fddd8105da0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009900 tx=0x7fddc802a510 comp rx=0 tx=0).stop 2024-04-23T18:56:27.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.974+0000 7fdddfee1640 1 -- 172.21.15.19:0/805484023 shutdown_connections 2024-04-23T18:56:27.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.974+0000 7fdddfee1640 1 --2- 172.21.15.19:0/805484023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd81062e0 0x7fddd8106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.974+0000 7fdddfee1640 1 --2- 172.21.15.19:0/805484023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddd81018e0 0x7fddd8105da0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:27.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.974+0000 7fdddfee1640 1 --2- 172.21.15.19:0/805484023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd8100fc0 0x7fddd81013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:27.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.974+0000 7fdddfee1640 1 -- 172.21.15.19:0/805484023 >> 172.21.15.19:0/805484023 conn(0x7fddd8077f50 msgr2=0x7fddd8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:27.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.974+0000 7fdddfee1640 1 -- 172.21.15.19:0/805484023 shutdown_connections 2024-04-23T18:56:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.974+0000 7fdddfee1640 1 -- 172.21.15.19:0/805484023 wait complete. 2024-04-23T18:56:27.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddfee1640 1 Processor -- start 2024-04-23T18:56:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddfee1640 1 -- start start 2024-04-23T18:56:27.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddfee1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd8100fc0 0x7fddd819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:27.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddfee1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddd81018e0 0x7fddd819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddfee1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd81062e0 0x7fddd81a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:27.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddfee1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8118800 con 0x7fddd81018e0 2024-04-23T18:56:27.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddfee1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8118680 con 0x7fddd8100fc0 2024-04-23T18:56:27.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddfee1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8118980 con 0x7fddd81062e0 2024-04-23T18:56:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddd47c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddd81018e0 0x7fddd819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:27.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddd47c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddd81018e0 0x7fddd819f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38172/0 (socket says 172.21.15.19:38172) 2024-04-23T18:56:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddd47c640 1 -- 172.21.15.19:0/1930582156 learned_addr learned my addr 172.21.15.19:0/1930582156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:56:27.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fddddc7d640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd8100fc0 0x7fddd819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:27.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fddde47e640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd81062e0 0x7fddd81a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:27.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddd47c640 1 -- 172.21.15.19:0/1930582156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd81062e0 msgr2=0x7fddd81a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:27.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddd47c640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd81062e0 0x7fddd81a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:27.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddd47c640 1 -- 172.21.15.19:0/1930582156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd8100fc0 msgr2=0x7fddd819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:27.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddd47c640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd8100fc0 0x7fddd819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:27.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fdddd47c640 1 -- 172.21.15.19:0/1930582156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd81a3e40 con 0x7fddd81018e0 2024-04-23T18:56:28.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.975+0000 7fddde47e640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd81062e0 0x7fddd81a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:56:28.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.976+0000 7fdddd47c640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddd81018e0 0x7fddd819f390 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fddc8005940 tx=0x7fddc80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.976+0000 7fddceffd640 1 -- 172.21.15.19:0/1930582156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc803f030 con 0x7fddd81018e0 2024-04-23T18:56:28.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.976+0000 7fddceffd640 1 -- 172.21.15.19:0/1930582156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc8003cb0 con 0x7fddd81018e0 2024-04-23T18:56:28.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.976+0000 7fdddfee1640 1 -- 172.21.15.19:0/1930582156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd81a40d0 con 0x7fddd81018e0 2024-04-23T18:56:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.976+0000 7fddceffd640 1 -- 172.21.15.19:0/1930582156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8035070 con 0x7fddd81018e0 2024-04-23T18:56:28.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.976+0000 7fdddfee1640 1 -- 172.21.15.19:0/1930582156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd81ab9c0 con 0x7fddd81018e0 2024-04-23T18:56:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.977+0000 7fdddfee1640 1 -- 172.21.15.19:0/1930582156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd8068d70 con 0x7fddd81018e0 2024-04-23T18:56:28.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.978+0000 7fddceffd640 1 -- 172.21.15.19:0/1930582156 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fddc8042030 con 0x7fddd81018e0 2024-04-23T18:56:28.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.979+0000 7fddceffd640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fddc40a3d20 0x7fddc40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.979+0000 7fddceffd640 1 -- 172.21.15.19:0/1930582156 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 8943+0+0 (secure 0 0 0) 0x7fddc80f1590 con 0x7fddd81018e0 2024-04-23T18:56:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.979+0000 7fddddc7d640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fddc40a3d20 0x7fddc40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:28.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.979+0000 7fddddc7d640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fddc40a3d20 0x7fddc40a61e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005fd0 tx=0x7fddd4009870 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:28.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:26.982+0000 7fddceffd640 1 -- 172.21.15.19:0/1930582156 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fddc80bda30 con 0x7fddd81018e0 2024-04-23T18:56:28.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:27.171+0000 7fdddfee1640 1 -- 172.21.15.19:0/1930582156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fddd81abb90 con 0x7fddd81018e0 2024-04-23T18:56:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.098+0000 7fddceffd640 1 -- 172.21.15.19:0/1930582156 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) ==== 181+0+0 (secure 0 0 0) 0x7fddc80bd330 con 0x7fddd81018e0 2024-04-23T18:56:28.603 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-23T18:56:28.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.101+0000 7fdddfee1640 1 -- 172.21.15.19:0/1930582156 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fddc40a3d20 msgr2=0x7fddc40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:28.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.101+0000 7fdddfee1640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fddc40a3d20 0x7fddc40a61e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005fd0 tx=0x7fddd4009870 comp rx=0 tx=0).stop 2024-04-23T18:56:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.101+0000 7fdddfee1640 1 -- 172.21.15.19:0/1930582156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddd81018e0 msgr2=0x7fddd819f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.101+0000 7fdddfee1640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddd81018e0 0x7fddd819f390 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fddc8005940 tx=0x7fddc80042f0 comp rx=0 tx=0).stop 2024-04-23T18:56:28.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.102+0000 7fdddfee1640 1 -- 172.21.15.19:0/1930582156 shutdown_connections 2024-04-23T18:56:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.102+0000 7fdddfee1640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7fddc40a3d20 0x7fddc40a61e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:28.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.102+0000 7fdddfee1640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd81062e0 0x7fddd81a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:28.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.102+0000 7fdddfee1640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddd81018e0 0x7fddd819f390 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.102+0000 7fdddfee1640 1 --2- 172.21.15.19:0/1930582156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd8100fc0 0x7fddd819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:29.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.102+0000 7fdddfee1640 1 -- 172.21.15.19:0/1930582156 >> 172.21.15.19:0/1930582156 conn(0x7fddd8077f50 msgr2=0x7fddd8103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:29.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.103+0000 7fdddfee1640 1 -- 172.21.15.19:0/1930582156 shutdown_connections 2024-04-23T18:56:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:28.103+0000 7fdddfee1640 1 -- 172.21.15.19:0/1930582156 wait complete. 2024-04-23T18:56:29.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T18:56:29.171 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:28 smithi155 ceph-mon[33468]: pgmap v964: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-04-23T18:56:29.203 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:28 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1930582156' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:56:29.237 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:28 smithi044 ceph-mon[32458]: pgmap v964: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-04-23T18:56:29.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:28 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1930582156' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:56:29.303 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:28 smithi019 ceph-mon[29215]: pgmap v964: 97 pgs: 97 active+clean; 4.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-04-23T18:56:29.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:28 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1930582156' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:56:29.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:29 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/1930582156' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:56:29.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:29 smithi019 ceph-mon[29215]: osdmap e142: 12 total, 12 up, 12 in 2024-04-23T18:56:29.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 -- 172.21.15.19:0/2183321016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be01031d0 msgr2=0x7f3be01035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 --2- 172.21.15.19:0/2183321016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be01031d0 0x7f3be01035b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009900 tx=0x7f3bd002a510 comp rx=0 tx=0).stop 2024-04-23T18:56:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 -- 172.21.15.19:0/2183321016 shutdown_connections 2024-04-23T18:56:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 --2- 172.21.15.19:0/2183321016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be010c510 0x7f3be010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 --2- 172.21.15.19:0/2183321016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3be0103af0 0x7f3be010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 --2- 172.21.15.19:0/2183321016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be01031d0 0x7f3be01035b0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:29.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 -- 172.21.15.19:0/2183321016 >> 172.21.15.19:0/2183321016 conn(0x7f3be0077f50 msgr2=0x7f3be0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 -- 172.21.15.19:0/2183321016 shutdown_connections 2024-04-23T18:56:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 -- 172.21.15.19:0/2183321016 wait complete. 2024-04-23T18:56:29.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 Processor -- start 2024-04-23T18:56:29.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.517+0000 7f3be6e8a640 1 -- start start 2024-04-23T18:56:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be6e8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be01031d0 0x7f3be0100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:29.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be6e8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3be0103af0 0x7f3be0101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be6e8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be010c510 0x7f3be01018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:30.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be6e8a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0112180 con 0x7f3be010c510 2024-04-23T18:56:30.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be6e8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0112000 con 0x7f3be01031d0 2024-04-23T18:56:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be6e8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0112300 con 0x7f3be0103af0 2024-04-23T18:56:30.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be5427640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be010c510 0x7f3be01018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be5427640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be010c510 0x7f3be01018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38200/0 (socket says 172.21.15.19:38200) 2024-04-23T18:56:30.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be5427640 1 -- 172.21.15.19:0/941046129 learned_addr learned my addr 172.21.15.19:0/941046129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-23T18:56:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be4c26640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be01031d0 0x7f3be0100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3bdffff640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3be0103af0 0x7f3be0101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:30.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be5427640 1 -- 172.21.15.19:0/941046129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3be0103af0 msgr2=0x7f3be0101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:30.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be5427640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3be0103af0 0x7f3be0101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be5427640 1 -- 172.21.15.19:0/941046129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be01031d0 msgr2=0x7f3be0100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be5427640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be01031d0 0x7f3be0100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:30.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be5427640 1 -- 172.21.15.19:0/941046129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be01a6ba0 con 0x7f3be010c510 2024-04-23T18:56:30.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.518+0000 7f3be5427640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be010c510 0x7f3be01018a0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800cad0 tx=0x7f3bd800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.519+0000 7f3bddffb640 1 -- 172.21.15.19:0/941046129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800ae20 con 0x7f3be010c510 2024-04-23T18:56:30.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.519+0000 7f3bddffb640 1 -- 172.21.15.19:0/941046129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd8005020 con 0x7f3be010c510 2024-04-23T18:56:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.519+0000 7f3be6e8a640 1 -- 172.21.15.19:0/941046129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be01a6e90 con 0x7f3be010c510 2024-04-23T18:56:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.519+0000 7f3bddffb640 1 -- 172.21.15.19:0/941046129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800f6b0 con 0x7f3be010c510 2024-04-23T18:56:30.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.519+0000 7f3be6e8a640 1 -- 172.21.15.19:0/941046129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be01a7360 con 0x7f3be010c510 2024-04-23T18:56:30.914 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:29 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/1930582156' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:56:30.946 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:29 smithi155 ceph-mon[33468]: osdmap e142: 12 total, 12 up, 12 in 2024-04-23T18:56:30.979 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:30 smithi155 ceph-mon[33468]: pgmap v966: 65 pgs: 65 active+clean; 577 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:56:31.021 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:30 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/941046129' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:56:31.088 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:29 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/1930582156' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:56:31.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:29 smithi044 ceph-mon[32458]: osdmap e142: 12 total, 12 up, 12 in 2024-04-23T18:56:31.180 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:30 smithi044 ceph-mon[32458]: pgmap v966: 65 pgs: 65 active+clean; 577 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:56:31.213 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:30 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/941046129' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:56:31.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.520+0000 7f3be6e8a640 1 -- 172.21.15.19:0/941046129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be0068d70 con 0x7f3be010c510 2024-04-23T18:56:31.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.521+0000 7f3bddffb640 1 -- 172.21.15.19:0/941046129 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3bd80051d0 con 0x7f3be010c510 2024-04-23T18:56:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.522+0000 7f3bddffb640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3bc80a3d50 0x7f3bc80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.522+0000 7f3be4c26640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3bc80a3d50 0x7f3bc80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.522+0000 7f3bddffb640 1 -- 172.21.15.19:0/941046129 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 8345+0+0 (secure 0 0 0) 0x7f3bd80d0790 con 0x7f3be010c510 2024-04-23T18:56:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.522+0000 7f3be4c26640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3bc80a3d50 0x7f3bc80a6210 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0031440 tx=0x7f3bd0005e80 comp rx=0 tx=0).ready entity=mgr.25004 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.526+0000 7f3bddffb640 1 -- 172.21.15.19:0/941046129 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3bd8014030 con 0x7f3be010c510 2024-04-23T18:56:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:29.715+0000 7f3be6e8a640 1 -- 172.21.15.19:0/941046129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3be0107100 con 0x7f3be010c510 2024-04-23T18:56:31.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.105+0000 7f3bddffb640 1 -- 172.21.15.19:0/941046129 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) ==== 169+0+0 (secure 0 0 0) 0x7f3bd809ce80 con 0x7f3be010c510 2024-04-23T18:56:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-23T18:56:31.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 -- 172.21.15.19:0/941046129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3bc80a3d50 msgr2=0x7f3bc80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:31.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3bc80a3d50 0x7f3bc80a6210 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0031440 tx=0x7f3bd0005e80 comp rx=0 tx=0).stop 2024-04-23T18:56:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 -- 172.21.15.19:0/941046129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be010c510 msgr2=0x7f3be01018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:31.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be010c510 0x7f3be01018a0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800cad0 tx=0x7f3bd800cf90 comp rx=0 tx=0).stop 2024-04-23T18:56:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 -- 172.21.15.19:0/941046129 shutdown_connections 2024-04-23T18:56:31.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.19:6800/1177101685,v1:172.21.15.19:6801/1177101685] conn(0x7f3bc80a3d50 0x7f3bc80a6210 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be010c510 0x7f3be01018a0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:31.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3be0103af0 0x7f3be0101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 --2- 172.21.15.19:0/941046129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be01031d0 0x7f3be0100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:32.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 -- 172.21.15.19:0/941046129 >> 172.21.15.19:0/941046129 conn(0x7f3be0077f50 msgr2=0x7f3be00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:32.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 -- 172.21.15.19:0/941046129 shutdown_connections 2024-04-23T18:56:32.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-23T18:56:30.109+0000 7f3be6e8a640 1 -- 172.21.15.19:0/941046129 wait complete. 2024-04-23T18:56:32.131 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:56:32.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:30 smithi019 ceph-mon[29215]: pgmap v966: 65 pgs: 65 active+clean; 577 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:56:32.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:30 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/941046129' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:56:32.305 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:31 smithi019 ceph-mon[29215]: from='client.? 172.21.15.19:0/941046129' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:56:32.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:31 smithi019 ceph-mon[29215]: osdmap e143: 12 total, 12 up, 12 in 2024-04-23T18:56:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:31 smithi019 ceph-mon[29215]: pgmap v968: 1 pgs: 1 active+clean; 577 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:56:32.422 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:31 smithi155 ceph-mon[33468]: from='client.? 172.21.15.19:0/941046129' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:56:32.455 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:31 smithi155 ceph-mon[33468]: osdmap e143: 12 total, 12 up, 12 in 2024-04-23T18:56:32.489 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:31 smithi155 ceph-mon[33468]: pgmap v968: 1 pgs: 1 active+clean; 577 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:56:32.554 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:31 smithi044 ceph-mon[32458]: from='client.? 172.21.15.19:0/941046129' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:56:32.589 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:31 smithi044 ceph-mon[32458]: osdmap e143: 12 total, 12 up, 12 in 2024-04-23T18:56:32.622 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:31 smithi044 ceph-mon[32458]: pgmap v968: 1 pgs: 1 active+clean; 577 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:56:32.674 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:56:32.738 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:56:32.796 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-23T18:56:32.822 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:56:32.875 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:56:32.916 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:56:33.009 INFO:tasks.cephadm:Stopping all daemons... 2024-04-23T18:56:33.043 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-23T18:56:33.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mon.a 2024-04-23T18:56:33.453 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:33 smithi019 systemd[1]: Stopping Ceph mon.a for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:56:33.763 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a[29211]: 2024-04-23T18:56:33.455+0000 7fe9843b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:56:33.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a[29211]: 2024-04-23T18:56:33.455+0000 7fe9843b3700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-23T18:56:33.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:33 smithi019 podman[100726]: 2024-04-23 18:56:33.690568287 +0000 UTC m=+0.367907158 container died c42d35d9ae5c8e74c58c49a5fa587e4e57c1562907de4a1a8f7b28bb7525f015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0) 2024-04-23T18:56:34.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:34 smithi019 podman[100726]: 2024-04-23 18:56:34.039530404 +0000 UTC m=+0.716869261 container cleanup c42d35d9ae5c8e74c58c49a5fa587e4e57c1562907de4a1a8f7b28bb7525f015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T18:56:34.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 23 18:56:34 smithi019 bash[100726]: ceph-46706838-019d-11ef-bc93-c7b262605968-mon-a 2024-04-23T18:56:34.517 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mon.a.service' 2024-04-23T18:56:34.610 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:56:34.633 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-23T18:56:34.664 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-23T18:56:34.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mon.b 2024-04-23T18:56:35.066 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:34 smithi044 systemd[1]: Stopping Ceph mon.b for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:56:35.566 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:35 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mon-b[32454]: 2024-04-23T18:56:35.176+0000 7f9b37dd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:56:35.598 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:35 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mon-b[32454]: 2024-04-23T18:56:35.176+0000 7f9b37dd0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-23T18:56:35.697 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:35 smithi044 podman[75240]: 2024-04-23 18:56:35.391277404 +0000 UTC m=+0.450847033 container died 4edd044bd4d648fa83b8d50525e6550410a7457155b763ab3fed4499a2e117bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:56:36.041 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:35 smithi044 podman[75240]: 2024-04-23 18:56:35.785456798 +0000 UTC m=+0.845026426 container cleanup 4edd044bd4d648fa83b8d50525e6550410a7457155b763ab3fed4499a2e117bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:56:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 23 18:56:35 smithi044 bash[75240]: ceph-46706838-019d-11ef-bc93-c7b262605968-mon-b 2024-04-23T18:56:36.177 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mon.b.service' 2024-04-23T18:56:36.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:56:36.273 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-23T18:56:36.306 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-23T18:56:36.340 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mon.c 2024-04-23T18:56:36.749 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:36 smithi155 systemd[1]: Stopping Ceph mon.c for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:56:37.076 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mon-c[33463]: 2024-04-23T18:56:36.751+0000 7f88835a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:56:37.140 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mon-c[33463]: 2024-04-23T18:56:36.751+0000 7f88835a8700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-23T18:56:37.173 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:36 smithi155 podman[74870]: 2024-04-23 18:56:36.946492529 +0000 UTC m=+0.320255899 container died 88528f6d2fdd42a3f10680ccfd54ef31e3d4d9e4ab4140087243bfcd5bc33f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:56:37.826 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:37 smithi155 podman[74870]: 2024-04-23 18:56:37.47062996 +0000 UTC m=+0.844393338 container cleanup 88528f6d2fdd42a3f10680ccfd54ef31e3d4d9e4ab4140087243bfcd5bc33f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-23T18:56:37.890 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:37 smithi155 bash[74870]: ceph-46706838-019d-11ef-bc93-c7b262605968-mon-c 2024-04-23T18:56:37.923 INFO:journalctl@ceph.mon.c.smithi155.stdout:Apr 23 18:56:37 smithi155 podman[74885]: 2024-04-23 18:56:37.672268883 +0000 UTC m=+0.720502290 container remove 88528f6d2fdd42a3f10680ccfd54ef31e3d4d9e4ab4140087243bfcd5bc33f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:56:37.957 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mon.c.service' 2024-04-23T18:56:38.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:56:38.117 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-23T18:56:38.185 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-23T18:56:38.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.x 2024-04-23T18:56:38.625 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:56:38 smithi019 systemd[1]: Stopping Ceph mgr.x for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:56:38.950 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:56:38 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:56:38.627+0000 7f6b9f08f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:56:38.998 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:56:38 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x[29447]: 2024-04-23T18:56:38.627+0000 7f6b9f08f700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-23T18:56:39.031 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:56:38 smithi019 podman[100936]: 2024-04-23 18:56:38.659103208 +0000 UTC m=+0.167749042 container died 6f128d5987ef75bda4e61740cb0bdf0d336b5002288d42c8a4eaf135526aeef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-23T18:56:39.263 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:56:39 smithi019 podman[100936]: 2024-04-23 18:56:39.062753652 +0000 UTC m=+0.571399492 container cleanup 6f128d5987ef75bda4e61740cb0bdf0d336b5002288d42c8a4eaf135526aeef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-23T18:56:39.340 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 23 18:56:39 smithi019 bash[100936]: ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-x 2024-04-23T18:56:39.444 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-23T18:56:39.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:56:39.557 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-23T18:56:39.590 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-23T18:56:39.623 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.y 2024-04-23T18:56:40.027 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:56:39 smithi044 systemd[1]: Stopping Ceph mgr.y for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:56:40.316 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:56:40 smithi044 podman[75363]: 2024-04-23 18:56:40.030738181 +0000 UTC m=+0.165717839 container died 853cb65869c8bbb93346c39e6a81a6f80486e5d09a3356223446f51290ffb2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-23T18:56:40.757 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:56:40 smithi044 podman[75363]: 2024-04-23 18:56:40.550045233 +0000 UTC m=+0.685024902 container cleanup 853cb65869c8bbb93346c39e6a81a6f80486e5d09a3356223446f51290ffb2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:56:40.824 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:56:40 smithi044 bash[75363]: ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y 2024-04-23T18:56:40.979 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-23T18:56:41.037 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:56:40 smithi044 podman[75377]: 2024-04-23 18:56:40.760813689 +0000 UTC m=+0.725588581 container remove 853cb65869c8bbb93346c39e6a81a6f80486e5d09a3356223446f51290ffb2cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:56:41.065 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:56:40 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-23T18:56:41.132 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:56:40 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-23T18:56:41.165 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:56:40 smithi044 systemd[1]: Stopped Ceph mgr.y for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:56:41.199 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 23 18:56:40 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.y.service: Consumed 26.562s CPU time. 2024-04-23T18:56:41.234 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:56:41.322 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-23T18:56:41.357 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-23T18:56:41.391 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.z 2024-04-23T18:56:41.826 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:56:41 smithi155 systemd[1]: Stopping Ceph mgr.z for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:56:42.133 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:56:41 smithi155 podman[75005]: 2024-04-23 18:56:41.845992978 +0000 UTC m=+0.225068687 container died ae6fd8769900ddb6f7943ec8553c398e5a3c5f3ea8e5e4e13ac82cdc27d5d1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-04-23T18:56:42.576 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:56:42 smithi155 podman[75005]: 2024-04-23 18:56:42.234725695 +0000 UTC m=+0.613801414 container cleanup ae6fd8769900ddb6f7943ec8553c398e5a3c5f3ea8e5e4e13ac82cdc27d5d1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-23T18:56:42.599 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:56:42 smithi155 bash[75005]: ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z 2024-04-23T18:56:42.800 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-23T18:56:42.879 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:56:42 smithi155 podman[75019]: 2024-04-23 18:56:42.586371769 +0000 UTC m=+0.735908191 container remove ae6fd8769900ddb6f7943ec8553c398e5a3c5f3ea8e5e4e13ac82cdc27d5d1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:56:42.969 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:56:42 smithi155 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-23T18:56:43.023 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:56:42 smithi155 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-04-23T18:56:43.057 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:56:42 smithi155 systemd[1]: Stopped Ceph mgr.z for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:56:43.116 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Apr 23 18:56:42 smithi155 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mgr.z.service: Consumed 25.274s CPU time. 2024-04-23T18:56:43.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:56:43.183 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-23T18:56:43.216 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-23T18:56:43.249 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.0 2024-04-23T18:56:43.613 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:43 smithi019 systemd[1]: Stopping Ceph osd.0 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:56:44.013 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:43 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0[43670]: 2024-04-23T18:56:43.616+0000 7f41dacc8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:56:44.083 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:43 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0[43670]: 2024-04-23T18:56:43.616+0000 7f41dacc8700 -1 osd.0 143 *** Got signal Terminated *** 2024-04-23T18:56:44.116 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:43 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0[43670]: 2024-04-23T18:56:43.616+0000 7f41dacc8700 -1 osd.0 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:56:49.432 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:49 smithi019 podman[101057]: 2024-04-23 18:56:49.143602428 +0000 UTC m=+5.662044522 container died 273beec7d06d6347cc103b57ecbe9a59082d33feafb97cb4cb41dcffd30b8d42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T18:56:49.764 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:49 smithi019 podman[101057]: 2024-04-23 18:56:49.559682799 +0000 UTC m=+6.078124891 container cleanup 273beec7d06d6347cc103b57ecbe9a59082d33feafb97cb4cb41dcffd30b8d42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-23T18:56:49.807 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:49 smithi019 bash[101057]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0 2024-04-23T18:56:50.258 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:49 smithi019 podman[101077]: 2024-04-23 18:56:49.786318893 +0000 UTC m=+0.638594232 container remove 273beec7d06d6347cc103b57ecbe9a59082d33feafb97cb4cb41dcffd30b8d42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-23T18:56:50.513 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:50 smithi019 podman[101179]: 2024-04-23 18:56:50.165050233 +0000 UTC m=+0.018599855 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:56:50.540 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:50 smithi019 podman[101179]: 2024-04-23 18:56:50.27069132 +0000 UTC m=+0.124240926 container create 1dbc0b654ffc230b8422a79b79953e2ba5e368950aa442508f6c8216e9a634bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-23T18:56:50.881 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:50 smithi019 podman[101179]: 2024-04-23 18:56:50.55054807 +0000 UTC m=+0.404097679 container init 1dbc0b654ffc230b8422a79b79953e2ba5e368950aa442508f6c8216e9a634bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-23T18:56:50.913 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:50 smithi019 podman[101179]: 2024-04-23 18:56:50.554139576 +0000 UTC m=+0.407689191 container start 1dbc0b654ffc230b8422a79b79953e2ba5e368950aa442508f6c8216e9a634bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-23T18:56:50.949 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:50 smithi019 podman[101179]: 2024-04-23 18:56:50.640248855 +0000 UTC m=+0.493798462 container attach 1dbc0b654ffc230b8422a79b79953e2ba5e368950aa442508f6c8216e9a634bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-23T18:56:51.264 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 23 18:56:50 smithi019 podman[101179]: 2024-04-23 18:56:50.884867507 +0000 UTC m=+0.738417120 container died 1dbc0b654ffc230b8422a79b79953e2ba5e368950aa442508f6c8216e9a634bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:56:51.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.0.service' 2024-04-23T18:56:51.771 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:56:51.807 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-23T18:56:51.840 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-23T18:56:51.874 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.1 2024-04-23T18:56:52.228 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:51 smithi044 systemd[1]: Stopping Ceph osd.1 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:56:52.566 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1[37100]: 2024-04-23T18:56:52.231+0000 7f3abf62c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:56:52.610 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1[37100]: 2024-04-23T18:56:52.231+0000 7f3abf62c700 -1 osd.1 143 *** Got signal Terminated *** 2024-04-23T18:56:52.649 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1[37100]: 2024-04-23T18:56:52.231+0000 7f3abf62c700 -1 osd.1 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:56:58.066 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:57 smithi044 podman[75498]: 2024-04-23 18:56:57.752091427 +0000 UTC m=+5.642427472 container died 5661ffd87dbeae16e11d1c0266d168982b35588fbfbbe62533cb7c467570038d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-23T18:56:58.464 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:58 smithi044 podman[75498]: 2024-04-23 18:56:58.274126643 +0000 UTC m=+6.164462699 container cleanup 5661ffd87dbeae16e11d1c0266d168982b35588fbfbbe62533cb7c467570038d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-23T18:56:58.572 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:58 smithi044 bash[75498]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1 2024-04-23T18:56:58.816 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:58 smithi044 podman[75515]: 2024-04-23 18:56:58.467971165 +0000 UTC m=+0.715638554 container remove 5661ffd87dbeae16e11d1c0266d168982b35588fbfbbe62533cb7c467570038d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T18:56:59.226 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:58 smithi044 podman[75605]: 2024-04-23 18:56:58.814240976 +0000 UTC m=+0.018478676 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:56:59.282 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:58 smithi044 podman[75605]: 2024-04-23 18:56:58.927315556 +0000 UTC m=+0.131553281 container create 6fcd753281b3c683070077a54c9ae2d2a9055f4650e60c2ce26114f970532e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:56:59.315 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:59 smithi044 podman[75605]: 2024-04-23 18:56:59.138183935 +0000 UTC m=+0.342421618 container init 6fcd753281b3c683070077a54c9ae2d2a9055f4650e60c2ce26114f970532e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-23T18:56:59.348 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:59 smithi044 podman[75605]: 2024-04-23 18:56:59.141303268 +0000 UTC m=+0.345540954 container start 6fcd753281b3c683070077a54c9ae2d2a9055f4650e60c2ce26114f970532e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-23T18:56:59.389 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:59 smithi044 podman[75605]: 2024-04-23 18:56:59.230000729 +0000 UTC m=+0.434238410 container attach 6fcd753281b3c683070077a54c9ae2d2a9055f4650e60c2ce26114f970532e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-23T18:56:59.566 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 23 18:56:59 smithi044 podman[75605]: 2024-04-23 18:56:59.4487338 +0000 UTC m=+0.652971482 container died 6fcd753281b3c683070077a54c9ae2d2a9055f4650e60c2ce26114f970532e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-23T18:57:00.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.1.service' 2024-04-23T18:57:00.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:57:00.324 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-23T18:57:00.357 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-23T18:57:00.390 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.2 2024-04-23T18:57:00.826 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:00 smithi155 systemd[1]: Stopping Ceph osd.2 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:57:01.326 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2[38659]: 2024-04-23T18:57:00.830+0000 7fa284353700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:57:01.399 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2[38659]: 2024-04-23T18:57:00.830+0000 7fa284353700 -1 osd.2 143 *** Got signal Terminated *** 2024-04-23T18:57:01.451 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2[38659]: 2024-04-23T18:57:00.830+0000 7fa284353700 -1 osd.2 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:57:06.520 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:06 smithi155 podman[75125]: 2024-04-23 18:57:06.212228924 +0000 UTC m=+5.596528830 container died 3e41951cc3fd0f5f343d8525a035572891682fbe8da6d31ba9623c86ad0f0ae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:57:06.826 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:06 smithi155 podman[75125]: 2024-04-23 18:57:06.614934553 +0000 UTC m=+5.999234501 container cleanup 3e41951cc3fd0f5f343d8525a035572891682fbe8da6d31ba9623c86ad0f0ae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-23T18:57:06.848 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:06 smithi155 bash[75125]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2 2024-04-23T18:57:07.173 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:06 smithi155 podman[75143]: 2024-04-23 18:57:06.858036134 +0000 UTC m=+0.648218077 container remove 3e41951cc3fd0f5f343d8525a035572891682fbe8da6d31ba9623c86ad0f0ae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-23T18:57:07.433 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:07 smithi155 podman[75237]: 2024-04-23 18:57:07.07963119 +0000 UTC m=+0.018043854 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:57:07.516 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:07 smithi155 podman[75237]: 2024-04-23 18:57:07.200695127 +0000 UTC m=+0.139107775 container create 4d8f5d373a36ef771de65434c66264143551bfba93a41124c010537a8d46aebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-23T18:57:07.581 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:07 smithi155 podman[75237]: 2024-04-23 18:57:07.437206897 +0000 UTC m=+0.375619546 container init 4d8f5d373a36ef771de65434c66264143551bfba93a41124c010537a8d46aebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:57:07.727 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:07 smithi155 podman[75237]: 2024-04-23 18:57:07.440801516 +0000 UTC m=+0.379214166 container start 4d8f5d373a36ef771de65434c66264143551bfba93a41124c010537a8d46aebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:57:07.756 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:07 smithi155 podman[75237]: 2024-04-23 18:57:07.545851457 +0000 UTC m=+0.484264123 container attach 4d8f5d373a36ef771de65434c66264143551bfba93a41124c010537a8d46aebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-23T18:57:08.037 INFO:journalctl@ceph.osd.2.smithi155.stdout:Apr 23 18:57:07 smithi155 podman[75237]: 2024-04-23 18:57:07.730899783 +0000 UTC m=+0.669312437 container died 4d8f5d373a36ef771de65434c66264143551bfba93a41124c010537a8d46aebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:57:08.349 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.2.service' 2024-04-23T18:57:08.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:57:08.431 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-23T18:57:08.465 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-23T18:57:08.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.3 2024-04-23T18:57:08.869 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:08 smithi019 systemd[1]: Stopping Ceph osd.3 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:57:09.264 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3[50358]: 2024-04-23T18:57:08.872+0000 7f3fe27f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:57:09.298 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3[50358]: 2024-04-23T18:57:08.872+0000 7f3fe27f8700 -1 osd.3 143 *** Got signal Terminated *** 2024-04-23T18:57:09.331 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3[50358]: 2024-04-23T18:57:08.872+0000 7f3fe27f8700 -1 osd.3 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:57:10.014 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3[50358]: 2024-04-23T18:57:09.658+0000 7f3fdade4700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.213419+0000 front 2024-04-23T18:56:48.213373+0000 (oldest deadline 2024-04-23T18:57:09.313614+0000) 2024-04-23T18:57:10.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:09 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:09.657+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:10.326 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:09 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:09.959+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:10.764 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:10 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3[50358]: 2024-04-23T18:57:10.700+0000 7f3fdade4700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.213419+0000 front 2024-04-23T18:56:48.213373+0000 (oldest deadline 2024-04-23T18:57:09.313614+0000) 2024-04-23T18:57:10.798 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:10 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:10.467+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:11.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:10 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:10.630+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:11.326 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:11 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:11.004+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:11.764 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:11 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3[50358]: 2024-04-23T18:57:11.658+0000 7f3fdade4700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.213419+0000 front 2024-04-23T18:56:48.213373+0000 (oldest deadline 2024-04-23T18:57:09.313614+0000) 2024-04-23T18:57:11.813 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:11 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:11.440+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:12.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:11 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:11.587+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:12.326 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:11 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:11.958+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:12.764 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3[50358]: 2024-04-23T18:57:12.630+0000 7f3fdade4700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.213419+0000 front 2024-04-23T18:56:48.213373+0000 (oldest deadline 2024-04-23T18:57:09.313614+0000) 2024-04-23T18:57:12.798 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:12 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:12.397+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:13.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:12 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:12.574+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:13.233 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:12 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:12.980+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:13.576 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:13.235+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:13.764 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:13 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3[50358]: 2024-04-23T18:57:13.660+0000 7f3fdade4700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.213419+0000 front 2024-04-23T18:56:48.213373+0000 (oldest deadline 2024-04-23T18:57:09.313614+0000) 2024-04-23T18:57:13.798 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:13 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:13.615+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:13.831 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:13 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:13.424+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:14.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:13.621+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:14.326 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:13.966+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:14.357 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:14 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:14.186+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:14.514 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:14 smithi019 podman[101285]: 2024-04-23 18:57:14.242813776 +0000 UTC m=+5.505137223 container died 2d0e7df7afe7c51d8eb475a2d14615f00ef2230f2ed75d18a93b0c2b15a24801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-23T18:57:14.549 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:14 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:14.395+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:14.996 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:14 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:14.626+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:15.060 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:14 smithi019 podman[101285]: 2024-04-23 18:57:14.715026604 +0000 UTC m=+5.977350035 container cleanup 2d0e7df7afe7c51d8eb475a2d14615f00ef2230f2ed75d18a93b0c2b15a24801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-23T18:57:15.099 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:14 smithi019 bash[101285]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3 2024-04-23T18:57:15.132 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:14 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:14.597+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:15.221 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:14 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:14.929+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:15.264 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:15 smithi019 podman[101305]: 2024-04-23 18:57:15.000049143 +0000 UTC m=+0.755976689 container remove 2d0e7df7afe7c51d8eb475a2d14615f00ef2230f2ed75d18a93b0c2b15a24801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-23T18:57:15.576 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:15 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:15.223+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:15.624 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:15 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:15.441+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:15.682 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:15 smithi019 podman[101392]: 2024-04-23 18:57:15.252287297 +0000 UTC m=+0.018588599 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:57:15.773 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:15 smithi019 podman[101392]: 2024-04-23 18:57:15.376062289 +0000 UTC m=+0.142363571 container create 3d70a25a109b41a83c84b01ae67b83e2c20514ed0e3d20c133bcbd7085886adf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-23T18:57:15.923 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:15 smithi019 podman[101392]: 2024-04-23 18:57:15.620338193 +0000 UTC m=+0.386639479 container init 3d70a25a109b41a83c84b01ae67b83e2c20514ed0e3d20c133bcbd7085886adf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-23T18:57:15.949 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:15 smithi019 podman[101392]: 2024-04-23 18:57:15.623872707 +0000 UTC m=+0.390173990 container start 3d70a25a109b41a83c84b01ae67b83e2c20514ed0e3d20c133bcbd7085886adf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-23T18:57:15.982 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:15 smithi019 podman[101392]: 2024-04-23 18:57:15.728712389 +0000 UTC m=+0.495013686 container attach 3d70a25a109b41a83c84b01ae67b83e2c20514ed0e3d20c133bcbd7085886adf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-23T18:57:16.041 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:15 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:15.671+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:16.074 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:15 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:15.606+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:16.196 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:15 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:15.944+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:16.226 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 23 18:57:15 smithi019 podman[101392]: 2024-04-23 18:57:15.926957449 +0000 UTC m=+0.693258738 container died 3d70a25a109b41a83c84b01ae67b83e2c20514ed0e3d20c133bcbd7085886adf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-23T18:57:16.514 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:16 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:16.427+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:16.576 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.3.service' 2024-04-23T18:57:16.608 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:16.198+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:16.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:57:16.707 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-23T18:57:16.741 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-23T18:57:16.774 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.4 2024-04-23T18:57:16.876 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:16.605+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:17.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:16 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:16.701+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:17.187 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:16 smithi044 systemd[1]: Stopping Ceph osd.4 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:57:17.225 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:16.914+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:17.566 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4[43914]: 2024-04-23T18:57:17.188+0000 7f791021b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:57:17.591 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4[43914]: 2024-04-23T18:57:17.188+0000 7f791021b700 -1 osd.4 143 *** Got signal Terminated *** 2024-04-23T18:57:17.685 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4[43914]: 2024-04-23T18:57:17.188+0000 7f791021b700 -1 osd.4 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:57:17.725 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:17 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:17.439+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:17.758 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:17 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:17.220+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:18.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:17 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:17.714+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:18.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:17.601+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:18.181 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:17 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:17.894+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:18.578 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:18 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:18.183+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:18.737 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:18 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:18.425+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:19.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:18 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:18.740+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:19.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:18 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:18.581+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:19.189 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:18 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:18.880+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:19.216 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:18 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:18.880+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:19.566 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:19 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4[43914]: 2024-04-23T18:57:19.104+0000 7f790c80f700 -1 osd.4 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.852982+0000 front 2024-04-23T18:56:53.853009+0000 (oldest deadline 2024-04-23T18:57:18.553137+0000) 2024-04-23T18:57:19.642 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:19 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:19.192+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:19.701 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:19 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:19.387+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:19.854 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:19 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:19.601+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:19.900 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:19 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:19.601+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:20.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:19 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:19.705+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:20.042 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:19 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:19.705+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:20.180 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:19 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:19.921+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:20.208 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:19 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:19.921+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:20.515 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:20.182+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:20.576 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:20 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4[43914]: 2024-04-23T18:57:20.151+0000 7f790c80f700 -1 osd.4 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.852982+0000 front 2024-04-23T18:56:53.853009+0000 (oldest deadline 2024-04-23T18:57:18.553137+0000) 2024-04-23T18:57:20.678 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:20 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:20.401+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:20.816 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:20 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:20.564+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:21.016 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:20 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:20.564+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:21.051 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:20 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:20.680+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:21.142 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:20 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:20.680+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:21.175 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:20.517+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:21.209 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:20.910+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:21.325 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:20.910+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:21.512 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:21.169+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:21.566 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:21 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4[43914]: 2024-04-23T18:57:21.172+0000 7f790c80f700 -1 osd.4 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.852982+0000 front 2024-04-23T18:56:53.853009+0000 (oldest deadline 2024-04-23T18:57:18.553137+0000) 2024-04-23T18:57:21.592 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:21 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:21.165+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:21.700 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:21 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:21.357+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:21.826 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:21.514+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:22.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:21 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:21.704+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:22.058 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:21 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:21.704+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:22.092 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:21 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:21.587+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:22.125 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:21 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:21.587+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:22.327 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:22.152+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:22.392 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:21.914+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:22.426 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:21.914+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:22.554 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:22 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:22.201+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:22.576 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:22 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4[43914]: 2024-04-23T18:57:22.157+0000 7f790c80f700 -1 osd.4 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.852982+0000 front 2024-04-23T18:56:53.853009+0000 (oldest deadline 2024-04-23T18:57:18.553137+0000) 2024-04-23T18:57:22.651 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:22 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:22.371+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:22.816 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:22 smithi044 podman[75711]: 2024-04-23 18:57:22.557533061 +0000 UTC m=+5.502523465 container died a14f9bf61dfcdfb09b5fb9a6f485f50b4b0578443ea547a93f0f374abf0c3152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T18:57:22.892 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:22 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:22.629+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:22.926 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:22 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:22.629+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:22.959 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:22.515+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:23.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:22 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:22.654+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:23.045 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:22 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:22.654+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:23.151 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:22 smithi044 podman[75711]: 2024-04-23 18:57:22.945060787 +0000 UTC m=+5.890051184 container cleanup a14f9bf61dfcdfb09b5fb9a6f485f50b4b0578443ea547a93f0f374abf0c3152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:57:23.176 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:22 smithi044 bash[75711]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4 2024-04-23T18:57:23.326 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:23.142+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:23.351 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:23.142+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:55.026601+0000 front 2024-04-23T18:56:55.026567+0000 (oldest deadline 2024-04-23T18:57:22.627196+0000) 2024-04-23T18:57:23.384 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:22.908+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:23.417 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:22.908+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:23.496 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:23 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:23.153+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:23.518 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:23 smithi044 podman[75739]: 2024-04-23 18:57:23.170818843 +0000 UTC m=+0.611334558 container remove a14f9bf61dfcdfb09b5fb9a6f485f50b4b0578443ea547a93f0f374abf0c3152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:57:23.670 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:23 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:23.417+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:23.693 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:23 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:23.417+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:23.752 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:23 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:23.658+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:23.776 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:23 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:23.658+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:23.878 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:23 smithi044 podman[75828]: 2024-04-23 18:57:23.403337013 +0000 UTC m=+0.018271633 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:57:23.909 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:23 smithi044 podman[75828]: 2024-04-23 18:57:23.514238594 +0000 UTC m=+0.129173183 container create 3b8908f27bb1dcd0618bdc52dd0525a65b0db38d1cc2d867c77ee8f4266e9a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:57:23.943 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:23.493+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:24.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:23 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:23.673+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:24.036 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:23 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:23.673+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:24.068 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:23 smithi044 podman[75828]: 2024-04-23 18:57:23.883637648 +0000 UTC m=+0.498572248 container init 3b8908f27bb1dcd0618bdc52dd0525a65b0db38d1cc2d867c77ee8f4266e9a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-23T18:57:24.101 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:23 smithi044 podman[75828]: 2024-04-23 18:57:23.886781474 +0000 UTC m=+0.501716072 container start 3b8908f27bb1dcd0618bdc52dd0525a65b0db38d1cc2d867c77ee8f4266e9a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-23T18:57:24.135 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:23 smithi044 podman[75828]: 2024-04-23 18:57:23.958406256 +0000 UTC m=+0.573340847 container attach 3b8908f27bb1dcd0618bdc52dd0525a65b0db38d1cc2d867c77ee8f4266e9a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-23T18:57:24.168 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:23.878+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:24.201 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:23.878+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:24.499 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:24 smithi044 podman[75828]: 2024-04-23 18:57:24.195288666 +0000 UTC m=+0.810223270 container died 3b8908f27bb1dcd0618bdc52dd0525a65b0db38d1cc2d867c77ee8f4266e9a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:57:24.560 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:24 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:24.164+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:24.610 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:24.150+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:24.643 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:24.150+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:55.026601+0000 front 2024-04-23T18:56:55.026567+0000 (oldest deadline 2024-04-23T18:57:22.627196+0000) 2024-04-23T18:57:24.764 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:24 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:24.660+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:24.785 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:24 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:24.660+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:24.818 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:24 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:24.424+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:24.888 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:24 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:24.424+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:24.927 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:24.535+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:24.960 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:24 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:24.649+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:24.993 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:24 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:24.649+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:25.045 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.4.service' 2024-04-23T18:57:25.078 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:24 smithi044 podman[75854]: 2024-04-23 18:57:24.795725473 +0000 UTC m=+0.591760247 container remove 3b8908f27bb1dcd0618bdc52dd0525a65b0db38d1cc2d867c77ee8f4266e9a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:57:25.110 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:24 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-23T18:57:25.144 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:24 smithi044 systemd[1]: Stopped Ceph osd.4 for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:57:25.177 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 18:57:24 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.4.service: Consumed 20.341s CPU time. 2024-04-23T18:57:25.211 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:24.854+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:25.243 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:24.854+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:25.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:57:25.335 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-23T18:57:25.369 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-23T18:57:25.402 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.5 2024-04-23T18:57:25.462 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:25.144+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:25.485 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:25.144+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:55.026601+0000 front 2024-04-23T18:56:55.026567+0000 (oldest deadline 2024-04-23T18:57:22.627196+0000) 2024-04-23T18:57:25.566 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:25 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:25.119+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:25.625 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:25 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:25.377+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:25.665 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:25 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:25.377+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:25.724 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:25 smithi155 systemd[1]: Stopping Ceph osd.5 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:57:25.753 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:25.542+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:26.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:25 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:25.628+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:26.077 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:25 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:25.628+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:26.137 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:25.726+0000 7f55a8273700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:57:26.169 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:25.726+0000 7f55a8273700 -1 osd.5 143 *** Got signal Terminated *** 2024-04-23T18:57:26.202 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:25.726+0000 7f55a8273700 -1 osd.5 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:57:26.278 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:25 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:25.627+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:26.311 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:25 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:25.627+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:26.346 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:25.865+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:26.377 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:25.865+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:26.566 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:26 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:26.117+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:26.586 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:26.151+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:26.619 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:26.151+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:55.026601+0000 front 2024-04-23T18:56:55.026567+0000 (oldest deadline 2024-04-23T18:57:22.627196+0000) 2024-04-23T18:57:26.703 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:26 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:26.382+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:26.736 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:26 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:26.382+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:26.902 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:26.580+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:27.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:26 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:26.641+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:27.044 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:26 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:26.641+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:27.078 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:26 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:26.641+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:27.112 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:26 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:26.604+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:27.144 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:26 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:26.604+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:27.178 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:26 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:26.604+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:27.326 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:27 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:27.155+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:27.353 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:27 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:27.155+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:55.026601+0000 front 2024-04-23T18:56:55.026567+0000 (oldest deadline 2024-04-23T18:57:22.627196+0000) 2024-04-23T18:57:27.386 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:26.905+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:27.419 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:26.905+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:27.566 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:27 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:27.128+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:27.659 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:27 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:27.393+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:27.703 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:27 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:27.393+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:27.826 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:27 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:27.575+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:28.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:27 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:27.662+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:28.036 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:27 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:27.662+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:28.070 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:27 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:27.662+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:28.104 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:27 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:27.585+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:28.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:27 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:27.585+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:28.186 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:27 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:27.585+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:28.221 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:27 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:27.873+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:28.253 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:27 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:27.873+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:28.303 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:27 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:27.873+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:28.528 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:28 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:28.133+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:28.553 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:28 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:28.133+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:55.026601+0000 front 2024-04-23T18:56:55.026567+0000 (oldest deadline 2024-04-23T18:57:22.627196+0000) 2024-04-23T18:57:28.587 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:28 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:28.103+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:28.764 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:28.392+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:28.820 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:28.392+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:28.854 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:28.640+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:28.887 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:28.640+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:28.920 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:28 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:28.640+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:29.014 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:28 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:28.531+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:29.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:28 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:28.615+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:29.087 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:28 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:28.615+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:29.121 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:28 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:28.615+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:29.179 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:28 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:28.912+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:29.204 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:28 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:28.912+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:29.237 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:28 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:28.912+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:29.480 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:29 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:29.182+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:29.504 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:29 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:29.182+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:55.026601+0000 front 2024-04-23T18:56:55.026567+0000 (oldest deadline 2024-04-23T18:57:22.627196+0000) 2024-04-23T18:57:29.566 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:29 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:29.100+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:29.676 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:29 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:29.357+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:29.704 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:29 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:29.357+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:29.826 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:29 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:29.483+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:30.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:29 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:29.680+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:30.038 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:29 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:29.680+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:30.071 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:29 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:29.680+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:30.105 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:29 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:29.643+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:30.184 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:29 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:29.643+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:30.221 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:29 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:29.643+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:30.255 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:29 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:29.897+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:30.288 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:29 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:29.897+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:30.321 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:29 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:29.897+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:30.530 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:30 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:30.209+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:48.224622+0000 front 2024-04-23T18:56:48.224847+0000 (oldest deadline 2024-04-23T18:57:12.324483+0000) 2024-04-23T18:57:30.554 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:30 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:30.209+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:55.026601+0000 front 2024-04-23T18:56:55.026567+0000 (oldest deadline 2024-04-23T18:57:22.627196+0000) 2024-04-23T18:57:30.613 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:30 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5[45183]: 2024-04-23T18:57:30.209+0000 7f55a085f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.328194+0000 front 2024-04-23T18:57:04.328220+0000 (oldest deadline 2024-04-23T18:57:29.628438+0000) 2024-04-23T18:57:30.647 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:30 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:30.093+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:30.680 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:30.364+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:30.713 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:30.364+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:30.746 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:30.364+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:30.827 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:30 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:30.532+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:31.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:30.651+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:31.038 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:30.651+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:31.071 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:30.651+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:31.105 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:30 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:30.671+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:31.204 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:30 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:30.671+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:31.238 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:30 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:30.671+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:31.273 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:30 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:30.857+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:31.305 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:30 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:30.857+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:31.341 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:30 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:30.857+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:31.496 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:31 smithi155 podman[75356]: 2024-04-23 18:57:31.220842194 +0000 UTC m=+5.592259376 container died 61b743ed3fe2baff80e43a9bfaa8a794bb56486f0814b418586c8d6444d6afec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-23T18:57:31.566 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:31 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:31.114+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:31.597 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:31 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:31.114+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:31.631 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:31 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:31.322+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:31.663 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:31 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:31.322+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:31.697 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:31 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:31.322+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:31.827 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:31 smithi155 podman[75356]: 2024-04-23 18:57:31.628244056 +0000 UTC m=+5.999661227 container cleanup 61b743ed3fe2baff80e43a9bfaa8a794bb56486f0814b418586c8d6444d6afec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-23T18:57:31.855 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:31 smithi155 bash[75356]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5 2024-04-23T18:57:31.889 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:31 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:31.498+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:31.922 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:31 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:31.498+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:32.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:31 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:31.622+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:32.039 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:31 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:31.622+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:32.099 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:31 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:31.622+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:32.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:31 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:31.717+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:32.172 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:31 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:31.717+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:32.205 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:31 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:31.717+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:32.304 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:31 smithi155 podman[75374]: 2024-04-23 18:57:31.871112947 +0000 UTC m=+0.649521377 container remove 61b743ed3fe2baff80e43a9bfaa8a794bb56486f0814b418586c8d6444d6afec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-23T18:57:32.331 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:31 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:31.873+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:32.380 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:31 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:31.873+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:32.414 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:31 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:31.873+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:32.567 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:32 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:32.157+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:32.622 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:32 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:32.157+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:32.656 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:32 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:32.298+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:32.689 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:32 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:32.298+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:32.722 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:32 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:32.298+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:32.781 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:32 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:32.469+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:32.814 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:32 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:32.469+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:32.848 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:32 smithi155 podman[75464]: 2024-04-23 18:57:32.21019367 +0000 UTC m=+0.018134285 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:57:32.881 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:32 smithi155 podman[75464]: 2024-04-23 18:57:32.340597511 +0000 UTC m=+0.148538158 container create 2635ece8c3b2a08f76e36a9aeab70159f87620b490b2c471289ba4988f79f7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:57:32.946 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:32 smithi155 podman[75464]: 2024-04-23 18:57:32.616495935 +0000 UTC m=+0.424436541 container init 2635ece8c3b2a08f76e36a9aeab70159f87620b490b2c471289ba4988f79f7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-23T18:57:32.981 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:32 smithi155 podman[75464]: 2024-04-23 18:57:32.619754406 +0000 UTC m=+0.427695014 container start 2635ece8c3b2a08f76e36a9aeab70159f87620b490b2c471289ba4988f79f7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-23T18:57:33.014 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:32 smithi155 podman[75464]: 2024-04-23 18:57:32.700302597 +0000 UTC m=+0.508243214 container attach 2635ece8c3b2a08f76e36a9aeab70159f87620b490b2c471289ba4988f79f7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-23T18:57:33.048 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:32 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:32.619+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:33.081 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:32 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:32.619+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:33.139 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:32 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:32.619+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:33.173 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:32 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:32.679+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:33.206 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:32 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:32.679+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:33.239 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:32 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:32.679+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:33.273 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:32 smithi155 podman[75464]: 2024-04-23 18:57:32.914234924 +0000 UTC m=+0.722175532 container died 2635ece8c3b2a08f76e36a9aeab70159f87620b490b2c471289ba4988f79f7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-23T18:57:33.306 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:32 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:32.888+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:33.340 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:32 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:32.888+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:33.373 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:32 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:32.888+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:33.509 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:33 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:33.422+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:33.548 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:33 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:33.422+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:33.582 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:33 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:33.182+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:33.615 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:33 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:33.182+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:33.696 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.5.service' 2024-04-23T18:57:33.732 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:33.323+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:33.765 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:33.323+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:33.798 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:33.323+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:33.849 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:33 smithi155 podman[75490]: 2024-04-23 18:57:33.512766601 +0000 UTC m=+0.593406479 container remove 2635ece8c3b2a08f76e36a9aeab70159f87620b490b2c471289ba4988f79f7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-23T18:57:33.895 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:33 smithi155 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-23T18:57:33.931 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:33 smithi155 systemd[1]: Stopped Ceph osd.5 for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:57:33.965 INFO:journalctl@ceph.osd.5.smithi155.stdout:Apr 23 18:57:33 smithi155 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.5.service: Consumed 24.426s CPU time. 2024-04-23T18:57:34.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:57:34.032 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-23T18:57:34.065 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-23T18:57:34.098 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.6 2024-04-23T18:57:34.135 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:33 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:33.640+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:34.165 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:33 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:33.640+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:34.198 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:33 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:33.640+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:34.232 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:33.647+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:34.265 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:33.647+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:34.322 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:33.647+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:34.374 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:33 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:33.933+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:34.407 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:33 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:33.933+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:34.457 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:33 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:33.933+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:34.566 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:34 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:34.214+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:34.607 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:34 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:34.214+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:34.641 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:34.295+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:34.674 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:34.295+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:34.707 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:34.295+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:34.741 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:34 smithi019 systemd[1]: Stopping Ceph osd.6 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:57:34.774 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:34.625+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:34.807 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:34.625+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:34.841 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:34.625+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:34.875 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:34 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:34.415+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:34.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:34 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:34.415+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:35.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:34.701+0000 7fa794fd1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:57:35.041 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:34.701+0000 7fa794fd1700 -1 osd.6 143 *** Got signal Terminated *** 2024-04-23T18:57:35.097 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:34 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:34.701+0000 7fa794fd1700 -1 osd.6 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:57:35.133 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:34 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:34.687+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:35.191 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:34 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:34.687+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:35.224 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:34 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:34.687+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:35.259 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:34 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:34.687+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:35.327 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:34 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:34.971+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:35.349 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:34 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:34.971+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:35.383 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:34 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:34.971+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:35.566 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:35 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:35.249+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:35.591 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:35 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:35.249+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:35.642 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:35 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:35.307+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:35.674 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:35 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:35.307+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:35.708 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:35 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:35.307+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:35.741 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:35 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:35.307+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:35.827 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:35 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:35.454+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:35.850 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:35 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:35.454+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:36.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:35 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:35.613+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:36.100 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:35 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:35.613+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:36.150 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:35 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:35.613+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:36.183 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:35 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:35.613+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.709159+0000 front 2024-04-23T18:57:11.709143+0000 (oldest deadline 2024-04-23T18:57:35.209251+0000) 2024-04-23T18:57:36.217 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:35 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:35.670+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:36.250 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:35 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:35.670+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:36.301 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:35 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:35.670+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:36.350 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:35 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:35.670+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:36.387 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:36.005+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:36.416 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:36.005+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:36.450 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:36.005+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:36.483 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:36.005+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:36.566 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:36 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:36.226+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:36.592 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:36 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:36.226+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:36.625 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:36.470+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:36.675 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:36.470+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:36.709 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:36.330+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:36.769 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:36.330+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:36.854 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:36.330+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:36.892 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:36.330+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:37.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:36.643+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:37.033 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:36.643+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:37.067 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:36.643+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:37.100 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:36.643+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.709159+0000 front 2024-04-23T18:57:11.709143+0000 (oldest deadline 2024-04-23T18:57:35.209251+0000) 2024-04-23T18:57:37.134 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:36 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:36.675+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:37.167 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:36 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:36.675+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:37.200 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:36 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:36.675+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:37.234 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:36 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:36.675+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:37.326 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:36.972+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:37.350 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:36.972+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:37.384 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:36.972+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:37.417 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:36 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:36.972+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:37.567 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:37 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:37.233+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:37.592 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:37 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:37.234+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:37.660 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:37 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:37.379+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:37.725 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:37 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:37.379+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:37.759 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:37 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:37.379+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:37.792 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:37 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:37.379+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:37.827 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:37 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:37.511+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:37.859 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:37 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:37.511+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:37.892 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:37 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:37.511+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:38.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:37 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:37.650+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:38.035 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:37 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:37.650+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:38.067 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:37 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:37.650+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:38.101 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:37 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:37.650+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.709159+0000 front 2024-04-23T18:57:11.709143+0000 (oldest deadline 2024-04-23T18:57:35.209251+0000) 2024-04-23T18:57:38.135 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:37 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:37.708+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:38.168 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:37 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:37.708+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:38.201 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:37 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:37.708+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:38.234 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:37 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:37.708+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:38.327 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:37 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:37.992+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:38.351 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:37 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:37.992+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:38.410 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:37 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:37.992+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:38.480 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:37 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:37.992+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:38.566 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:38 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:38.249+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:38.593 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:38 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:38.249+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:38.626 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:38 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:38.249+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:38.669 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:38 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:38.346+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:38.710 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:38 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:38.346+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:38.743 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:38 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:38.346+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:38.776 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:38 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:38.346+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:38.827 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:38 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:38.535+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:38.851 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:38 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:38.535+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:38.885 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:38 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:38.535+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:39.014 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:38 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:38.610+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:39.035 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:38 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:38.610+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:39.068 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:38 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:38.610+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:39.101 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:38 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:38.610+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.709159+0000 front 2024-04-23T18:57:11.709143+0000 (oldest deadline 2024-04-23T18:57:35.209251+0000) 2024-04-23T18:57:39.177 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:38 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:38.753+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:39.210 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:38 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:38.753+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:39.290 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:38 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:38.753+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:39.326 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:38 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:38.753+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:39.361 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:39 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:39.030+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:39.393 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:39 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:39.030+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:39.430 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:39 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:39.030+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:39.468 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:39 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:39.030+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:39.567 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:39 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:39.282+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:39.594 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:39 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:39.282+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:39.627 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:39 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:39.282+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:39.764 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:39 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:39.582+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.104391+0000 front 2024-04-23T18:56:46.104697+0000 (oldest deadline 2024-04-23T18:57:13.105225+0000) 2024-04-23T18:57:39.785 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:39 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:39.582+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.105987+0000 front 2024-04-23T18:56:53.106049+0000 (oldest deadline 2024-04-23T18:57:19.005893+0000) 2024-04-23T18:57:39.819 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:39 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:39.582+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:03.707099+0000 front 2024-04-23T18:57:03.707087+0000 (oldest deadline 2024-04-23T18:57:26.606971+0000) 2024-04-23T18:57:39.852 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:39 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6[57372]: 2024-04-23T18:57:39.582+0000 7fa78d5bd700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.709159+0000 front 2024-04-23T18:57:11.709143+0000 (oldest deadline 2024-04-23T18:57:35.209251+0000) 2024-04-23T18:57:39.886 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:39 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:39.380+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:39.952 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:39 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:39.380+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:39.986 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:39 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:39.380+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:40.019 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:39 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:39.380+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:40.073 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:39 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:39.778+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:40.111 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:39 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:39.778+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:40.144 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:39 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:39.778+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:40.178 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:39 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:39.778+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:40.211 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:39 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:39.500+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:40.244 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:39 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:39.500+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:40.278 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:39 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:39.500+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:40.327 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:40 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:40.078+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:40.353 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:40 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:40.078+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:40.386 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:40 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:40.078+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:40.419 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:40 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:40.078+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:40.514 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:40 smithi019 podman[101509]: 2024-04-23 18:57:40.241167247 +0000 UTC m=+5.667702755 container died 20f692cc5c7b64299edba308a9317444186b180d285ddd43a02efe28275b50de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:57:40.553 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:40 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:40.345+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:40.611 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:40 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:40.345+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:40.645 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:40 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:40.345+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:40.678 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:40 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:40.345+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:40.772 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:40 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:40.319+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:40.810 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:40 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:40.319+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:40.845 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:40 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:40.319+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:40.879 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:40 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:40.537+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:40.911 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:40 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:40.537+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:40.957 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:40 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:40.537+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:40.996 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:40 smithi019 podman[101509]: 2024-04-23 18:57:40.727623516 +0000 UTC m=+6.154159066 container cleanup 20f692cc5c7b64299edba308a9317444186b180d285ddd43a02efe28275b50de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-23T18:57:41.028 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:40 smithi019 bash[101509]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6 2024-04-23T18:57:41.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:40 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:40.775+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:41.095 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:40 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:40.775+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:41.128 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:40 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:40.775+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:41.162 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:40 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:40.775+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:41.264 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:40 smithi019 podman[101526]: 2024-04-23 18:57:40.995842862 +0000 UTC m=+0.745037524 container remove 20f692cc5c7b64299edba308a9317444186b180d285ddd43a02efe28275b50de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-23T18:57:41.498 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:41 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:41.087+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:41.562 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:41 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:41.087+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:41.595 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:41 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:41.087+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:41.629 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:41 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:41.087+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:41.663 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:41 smithi019 podman[101616]: 2024-04-23 18:57:41.262400013 +0000 UTC m=+0.018287218 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:57:41.734 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:41 smithi019 podman[101616]: 2024-04-23 18:57:41.388619462 +0000 UTC m=+0.144506648 container create 627686545d874cf45d1b17911605f6141403f9098c22759c862ab5761c414a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-23T18:57:41.771 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:41 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:41.318+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:41.804 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:41 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:41.318+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:41.837 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:41 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:41.318+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:41.889 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:41 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:41.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:41.921 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:41 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:41.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:41.954 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:41 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:41.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:41.987 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:41 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:41.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:42.021 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:41 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:41.502+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:42.054 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:41 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:41.502+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:42.087 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:41 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:41.502+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:42.121 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:41 smithi019 podman[101616]: 2024-04-23 18:57:41.65769891 +0000 UTC m=+0.413586098 container init 627686545d874cf45d1b17911605f6141403f9098c22759c862ab5761c414a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-23T18:57:42.171 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:41 smithi019 podman[101616]: 2024-04-23 18:57:41.661170723 +0000 UTC m=+0.417057917 container start 627686545d874cf45d1b17911605f6141403f9098c22759c862ab5761c414a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-23T18:57:42.204 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:41 smithi019 podman[101616]: 2024-04-23 18:57:41.757740926 +0000 UTC m=+0.513628119 container attach 627686545d874cf45d1b17911605f6141403f9098c22759c862ab5761c414a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416) 2024-04-23T18:57:42.246 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:41 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:41.731+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:42.279 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:41 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:41.731+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:42.312 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:41 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:41.731+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:42.369 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:41 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:41.731+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:42.405 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:42 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:42.041+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:42.438 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:42 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:42.041+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:42.471 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:42 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:42.041+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:42.505 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:42 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:42.041+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:42.538 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:41 smithi019 podman[101616]: 2024-04-23 18:57:41.965677836 +0000 UTC m=+0.721565022 container died 627686545d874cf45d1b17911605f6141403f9098c22759c862ab5761c414a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-23T18:57:42.734 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:42 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:42.328+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:42.796 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:42 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:42.328+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:42.829 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:42 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:42.328+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:42.864 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:42 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:42.534+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:42.896 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:42 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:42.534+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:42.946 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:42 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:42.534+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:42.980 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:42 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:42.279+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:43.013 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:42 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:42.279+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:43.046 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:42 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:42.279+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:43.080 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:42 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:42.279+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:43.114 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:42 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:42.738+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:43.159 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:42 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:42.738+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:43.196 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:42 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:42.738+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:43.230 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:42 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:42.738+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:43.264 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.6.service' 2024-04-23T18:57:43.298 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:42 smithi019 podman[101643]: 2024-04-23 18:57:42.879043244 +0000 UTC m=+0.907171371 container remove 627686545d874cf45d1b17911605f6141403f9098c22759c862ab5761c414a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-23T18:57:43.330 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:42 smithi019 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.6.service: Deactivated successfully. 2024-04-23T18:57:43.363 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:42 smithi019 systemd[1]: Stopped Ceph osd.6 for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:57:43.397 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 23 18:57:42 smithi019 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.6.service: Consumed 20.025s CPU time. 2024-04-23T18:57:43.431 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:43.001+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:43.475 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:43.001+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:43.530 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:43.001+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:43.563 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:43.001+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:43.598 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:43 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:43.306+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:43.630 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:43 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:43.306+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:43.693 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:43 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:43.306+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:43.722 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:43 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:43.306+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.366959+0000 front 2024-04-23T18:57:20.366980+0000 (oldest deadline 2024-04-23T18:57:43.267093+0000) 2024-04-23T18:57:43.756 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:43 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:43.266+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:43.827 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:43 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:43.266+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:43.864 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:43 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:43.266+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:43.906 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:43 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:43.266+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:43.949 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:57:43.997 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-23T18:57:44.031 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-23T18:57:44.064 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.7 2024-04-23T18:57:44.099 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:43.535+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:44.131 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:43.535+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:44.189 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:43.535+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:44.223 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:43 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:43.694+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:44.256 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:43 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:43.694+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:44.289 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:43 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:43.694+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:44.323 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:43 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:43.694+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:44.360 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:43 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:43.694+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:44.390 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:43.984+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:44.423 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:43.984+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:44.471 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:43.984+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:44.509 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:43 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:43.984+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:44.549 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:44 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:44.272+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:44.581 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:44 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:44.272+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:44.632 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:44 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:44.272+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:44.665 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:44 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:44.272+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:44.716 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:44.309+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:44.748 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:44.309+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:44.807 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:44.309+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:44.858 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:44.309+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.366959+0000 front 2024-04-23T18:57:20.366980+0000 (oldest deadline 2024-04-23T18:57:43.267093+0000) 2024-04-23T18:57:44.898 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:44 smithi044 systemd[1]: Stopping Ceph osd.7 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:57:44.932 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:44 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:44.528+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:44.965 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:44 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:44.528+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:44.998 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:44 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:44.528+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:45.032 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:44 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:44.528+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:45.067 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:44.698+0000 7f0e35976700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:57:45.098 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:44.698+0000 7f0e35976700 -1 osd.7 143 *** Got signal Terminated *** 2024-04-23T18:57:45.156 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:44.698+0000 7f0e35976700 -1 osd.7 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:57:45.190 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:44.654+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:45.224 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:44.654+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:45.257 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:44.654+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:45.318 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:44.654+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:45.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:44 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:44.654+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:45.391 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:44 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:44.946+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:45.440 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:44 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:44.946+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:45.474 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:44 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:44.946+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:45.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:44 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:44.946+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:45.582 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:44 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:44.946+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:45.616 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:45 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:45.235+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:45.649 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:45 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:45.235+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:45.682 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:45 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:45.235+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:45.724 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:45 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:45.235+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:45.757 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:45 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:45.235+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:45.791 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:45.343+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:45.858 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:45.343+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:45.891 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:45.343+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:45.924 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:45.343+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.366959+0000 front 2024-04-23T18:57:20.366980+0000 (oldest deadline 2024-04-23T18:57:43.267093+0000) 2024-04-23T18:57:45.958 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:45 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:45.495+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:45.996 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:45 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:45.495+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:46.033 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:45 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:45.495+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:46.083 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:45 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:45.495+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:46.117 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:45.687+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:46.149 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:45.687+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:46.192 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:45.687+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:46.224 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:45.687+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:46.258 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:45.687+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:46.327 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:45 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:45.912+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:46.349 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:45 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:45.912+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:46.383 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:45 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:45.912+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:46.416 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:45 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:45.912+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:46.450 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:45 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:45.912+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:46.521 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:46 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:46.189+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:46.558 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:46 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:46.189+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:46.591 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:46 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:46.189+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:46.625 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:46 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:46.189+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:46.671 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:46 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:46.189+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:46.725 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:46 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:46.370+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:46.758 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:46 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:46.370+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:46.792 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:46 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:46.370+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:46.825 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:46 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:46.370+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.366959+0000 front 2024-04-23T18:57:20.366980+0000 (oldest deadline 2024-04-23T18:57:43.267093+0000) 2024-04-23T18:57:46.876 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:46 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:46.470+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:46.908 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:46 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:46.470+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:46.942 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:46 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:46.470+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:46.975 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:46 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:46.470+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:47.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:46 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:46.659+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:47.092 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:46 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:46.659+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:47.125 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:46 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:46.659+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:47.159 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:46 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:46.659+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:47.236 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:46 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:46.659+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:47.327 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:46 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:46.872+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:47.350 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:46 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:46.872+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:47.384 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:46 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:46.872+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:47.450 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:46 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:46.872+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:47.492 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:46 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:46.872+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:47.526 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:47 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:47.205+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:47.559 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:47 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:47.205+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:47.592 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:47 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:47.205+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:47.668 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:47 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:47.205+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:47.700 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:47 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:47.205+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:47.751 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:47 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:47.325+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:47.784 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:47 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:47.325+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:47.817 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:47 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:47.325+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:47.851 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:47 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:47.325+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.366959+0000 front 2024-04-23T18:57:20.366980+0000 (oldest deadline 2024-04-23T18:57:43.267093+0000) 2024-04-23T18:57:47.885 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:47 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:47.476+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:47.917 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:47 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:47.476+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:48.008 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:47 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:47.476+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:48.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:47 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:47.476+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:48.077 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:47 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:47.705+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:48.142 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:47 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:47.705+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:48.179 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:47 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:47.705+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:48.218 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:47 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:47.705+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:48.251 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:47 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:47.705+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:48.327 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:47 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:47.850+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:48.351 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:47 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:47.850+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:48.399 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:47 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:47.850+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:48.434 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:47 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:47.850+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:48.468 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:47 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:47.850+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:48.514 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:48 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:48.173+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:48.534 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:48 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:48.173+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:48.568 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:48 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:48.173+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:48.601 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:48 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:48.173+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:48.635 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:48 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:48.173+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:48.742 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:48 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:48.355+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:48.768 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:48 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:48.355+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:48.818 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:48 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:48.355+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:48.852 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:48 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:48.355+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.366959+0000 front 2024-04-23T18:57:20.366980+0000 (oldest deadline 2024-04-23T18:57:43.267093+0000) 2024-04-23T18:57:48.888 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:48 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:48.483+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:48.927 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:48 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:48.483+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:48.960 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:48 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:48.483+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:48.994 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:48 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:48.483+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:49.067 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:48 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:48.745+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:49.093 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:48 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:48.745+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:49.127 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:48 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:48.745+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:49.160 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:48 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:48.745+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:49.194 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:48 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:48.745+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:49.327 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:48 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:48.875+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:49.352 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:48 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:48.875+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:49.385 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:48 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:48.875+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:49.419 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:48 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:48.875+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:49.487 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:48 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:48.875+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:49.553 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:49 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:49.161+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:49.585 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:49 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:49.161+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:49.619 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:49 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:49.161+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:49.653 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:49 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:49.161+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:49.694 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:49 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:49.161+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:49.789 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:49.363+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.663546+0000 front 2024-04-23T18:56:54.663491+0000 (oldest deadline 2024-04-23T18:57:20.563693+0000) 2024-04-23T18:57:49.811 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:49.363+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.764964+0000 front 2024-04-23T18:57:05.765044+0000 (oldest deadline 2024-04-23T18:57:31.065096+0000) 2024-04-23T18:57:49.844 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:49.363+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.566104+0000 front 2024-04-23T18:57:11.566193+0000 (oldest deadline 2024-04-23T18:57:37.466204+0000) 2024-04-23T18:57:49.891 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7[50513]: 2024-04-23T18:57:49.363+0000 7f0e2df62700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.366959+0000 front 2024-04-23T18:57:20.366980+0000 (oldest deadline 2024-04-23T18:57:43.267093+0000) 2024-04-23T18:57:49.928 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:49.455+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:49.961 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:49.455+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:49.994 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:49.455+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:50.028 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:49.455+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:50.066 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:49.792+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:50.094 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:49.792+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:50.128 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:49.792+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:50.202 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:49.792+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:50.254 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:49.792+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:50.327 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:49.839+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:50.353 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:49.839+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:50.386 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:49.839+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:50.419 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:49.839+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:50.453 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:49 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:49.839+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:50.514 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:50 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:50.194+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:50.536 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:50 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:50.194+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:50.605 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:50 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:50.194+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:50.645 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:50 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:50.194+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:50.678 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:50 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:50.194+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:50.712 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:50 smithi044 podman[75933]: 2024-04-23 18:57:50.262574379 +0000 UTC m=+5.672081421 container died f0686006a4ccd273d83cf568bc61af4c497ef2a939e31635d59987722e95e270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-23T18:57:50.802 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:50 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:50.484+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:50.862 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:50 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:50.484+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:50.895 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:50 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:50.484+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:50.945 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:50 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:50.484+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:51.026 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:50 smithi044 podman[75933]: 2024-04-23 18:57:50.740304752 +0000 UTC m=+6.149811801 container cleanup f0686006a4ccd273d83cf568bc61af4c497ef2a939e31635d59987722e95e270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-23T18:57:51.062 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:50 smithi044 bash[75933]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7 2024-04-23T18:57:51.103 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:50 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:50.745+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:51.137 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:50 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:50.745+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:51.170 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:50 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:50.745+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:51.203 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:50 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:50.745+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:51.253 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:50 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:50.745+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:51.287 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:50 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:50.805+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:51.320 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:50 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:50.805+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:51.382 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:50 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:50.805+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:51.420 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:50 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:50.805+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:51.454 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:50 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:50.805+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:51.488 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:51 smithi044 podman[75960]: 2024-04-23 18:57:51.000076624 +0000 UTC m=+0.737275845 container remove f0686006a4ccd273d83cf568bc61af4c497ef2a939e31635d59987722e95e270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-23T18:57:51.521 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:51 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:51.223+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:51.554 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:51 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:51.223+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:51.604 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:51 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:51.223+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:51.637 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:51 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:51.223+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:51.707 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:51 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:51.223+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:51.746 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:51 smithi044 podman[76055]: 2024-04-23 18:57:51.267645701 +0000 UTC m=+0.018593213 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:57:51.779 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:51 smithi044 podman[76055]: 2024-04-23 18:57:51.426460755 +0000 UTC m=+0.177408268 container create bd9cb02c46537660b9a07ef9e59a07b2b39021b4dab810511233f46630acb279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-23T18:57:51.812 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:51 smithi044 podman[76055]: 2024-04-23 18:57:51.653953535 +0000 UTC m=+0.404901048 container init bd9cb02c46537660b9a07ef9e59a07b2b39021b4dab810511233f46630acb279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-23T18:57:51.895 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:51.458+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:51.929 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:51.458+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:51.962 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:51.458+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:51.996 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:51.458+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:52.030 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:51 smithi044 podman[76055]: 2024-04-23 18:57:51.657273216 +0000 UTC m=+0.408220729 container start bd9cb02c46537660b9a07ef9e59a07b2b39021b4dab810511233f46630acb279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T18:57:52.090 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:51 smithi044 podman[76055]: 2024-04-23 18:57:51.728778048 +0000 UTC m=+0.479725561 container attach bd9cb02c46537660b9a07ef9e59a07b2b39021b4dab810511233f46630acb279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-04-23T18:57:52.129 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:51 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:51.778+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:52.162 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:51 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:51.778+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:52.196 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:51 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:51.778+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:52.229 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:51 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:51.778+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:52.288 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:51 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:51.778+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:52.322 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:51.773+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:52.354 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:51.773+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:52.415 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:51.773+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:52.454 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:51.773+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:52.513 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:51 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:51.773+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:52.546 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:51 smithi044 podman[76055]: 2024-04-23 18:57:51.964822569 +0000 UTC m=+0.715770087 container died bd9cb02c46537660b9a07ef9e59a07b2b39021b4dab810511233f46630acb279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T18:57:52.597 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:52 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:52.227+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:52.630 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:52 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:52.227+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:52.663 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:52 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:52.227+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:52.696 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:52 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:52.227+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:52.730 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:52 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:52.227+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:52.795 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:52 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:52.227+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:57:52.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.7.service' 2024-04-23T18:57:52.863 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:52 smithi044 podman[76082]: 2024-04-23 18:57:52.549280995 +0000 UTC m=+0.574939095 container remove bd9cb02c46537660b9a07ef9e59a07b2b39021b4dab810511233f46630acb279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-23T18:57:52.896 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:52 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.7.service: Deactivated successfully. 2024-04-23T18:57:52.955 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:52 smithi044 systemd[1]: Stopped Ceph osd.7 for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:57:52.988 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 23 18:57:52 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.7.service: Consumed 21.821s CPU time. 2024-04-23T18:57:53.022 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:52.742+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:53.055 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:52.742+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:53.113 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:52.742+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:53.147 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:52.742+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:53.180 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:52.742+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:53.213 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:52 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:52.742+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:57:53.262 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:52.427+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:53.297 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:52.427+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:53.363 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:52.427+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:53.397 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:52.427+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:53.430 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:52.787+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:53.481 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:52.787+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:53.522 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:52.787+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:53.555 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:52.787+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:53.589 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:52 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:52.787+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:53.648 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:53 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:53.207+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:53.680 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:53 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:53.207+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:53.714 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:53 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:53.207+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:53.747 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:53 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:53.207+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:53.814 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:53 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:53.207+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:53.847 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:53 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:53.207+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:57:53.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:57:53.914 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-23T18:57:53.960 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-23T18:57:53.997 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.8 2024-04-23T18:57:54.032 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:53.399+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:54.064 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:53.399+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:54.097 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:53.399+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:54.148 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:53.399+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:54.182 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:53 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:53.720+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:54.214 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:53 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:53.720+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:54.348 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:53 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:53.720+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:54.381 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:53 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:53.720+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:54.414 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:53 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:53.720+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:54.447 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:53 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:53.720+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:57:54.481 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:53.815+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:54.579 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:53.815+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:54.606 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:53.815+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:54.639 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:53.815+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:54.673 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:53 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:53.815+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:54.707 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:54.215+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:54.740 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:54.215+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:54.773 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:54.215+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:54.806 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:54.215+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:54.840 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:54.215+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:54.895 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:54 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:54.215+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:57:54.965 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:54 smithi155 systemd[1]: Stopping Ceph osd.8 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:57:54.998 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:54.379+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:55.031 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:54.379+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:55.065 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:54.379+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:55.098 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:54.379+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:55.132 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:54.498+0000 7f92e9b1d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:57:55.165 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:54.498+0000 7f92e9b1d700 -1 osd.8 143 *** Got signal Terminated *** 2024-04-23T18:57:55.237 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:54.498+0000 7f92e9b1d700 -1 osd.8 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:57:55.274 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:54.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:55.307 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:54.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:55.340 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:54.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:55.385 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:54.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:55.423 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:54.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:55.457 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:54 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:54.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:57:55.490 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:54.809+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:55.524 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:54.809+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:55.574 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:54.809+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:55.615 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:54.809+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:55.649 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:54.809+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:55.682 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:54 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:54.809+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:57:55.716 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:55 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:55.260+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:55.749 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:55 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:55.260+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:55.782 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:55 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:55.260+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:55.815 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:55 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:55.260+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:55.871 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:55 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:55.260+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:55.907 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:55 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:55.260+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:57:55.941 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:55.397+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:55.974 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:55.397+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:56.007 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:55.397+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:56.041 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:55.397+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:56.074 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:55.397+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:28.875781+0000 front 2024-04-23T18:57:28.875827+0000 (oldest deadline 2024-04-23T18:57:54.775950+0000) 2024-04-23T18:57:56.116 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:55 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:55.725+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:56.166 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:55 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:55.725+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:56.216 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:55 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:55.725+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:56.249 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:55 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:55.725+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:56.282 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:55 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:55.725+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:56.316 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:55 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:55.725+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:57:56.350 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:55.840+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:56.382 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:55.840+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:56.416 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:55.840+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:56.449 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:55.840+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:56.520 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:55.840+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:56.558 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:55 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:55.840+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:57:56.764 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:56 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:56.303+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:56.833 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:56 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:56.303+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:56.898 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:56 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:56.303+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:56.933 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:56 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:56.303+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:56.966 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:56 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:56.303+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:57.016 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:56 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:56.303+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:57:57.050 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:56.396+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:57.126 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:56.396+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:57.166 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:56.396+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:57.200 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:56.396+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:57.233 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:56.396+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:28.875781+0000 front 2024-04-23T18:57:28.875827+0000 (oldest deadline 2024-04-23T18:57:54.775950+0000) 2024-04-23T18:57:57.268 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:56.772+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:57.300 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:56.772+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:57.350 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:56.772+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:57.383 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:56.772+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:57.466 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:56.772+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:57.500 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:56 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:56.772+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:57:57.533 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:56.803+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:57.567 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:56.803+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:57.600 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:56.803+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:57.633 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:56.803+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:57.683 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:56.803+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:57.717 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:56 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:56.803+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:57:57.764 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:57 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:57.348+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:57.807 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:57 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:57.348+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:57.842 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:57 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:57.348+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:57.895 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:57 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:57.349+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:57.950 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:57 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:57.349+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:57.984 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:57 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:57.349+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:57:58.017 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:57.375+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:58.051 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:57.375+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:58.084 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:57.375+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:58.174 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:57.375+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:58.209 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:57.375+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:28.875781+0000 front 2024-04-23T18:57:28.875827+0000 (oldest deadline 2024-04-23T18:57:54.775950+0000) 2024-04-23T18:57:58.243 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:57.724+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:58.276 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:57.724+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:58.310 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:57.724+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:58.368 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:57.724+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:58.401 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:57.724+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:58.434 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:57.724+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:57:58.469 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:57.831+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:58.510 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:57.831+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:58.609 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:57.831+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:58.651 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:57.831+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:58.684 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:57.831+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:58.718 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:57 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:57.831+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:57:58.764 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:58 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:58.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:58.793 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:58 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:58.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:58.826 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:58 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:58.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:58.907 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:58 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:58.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:58.943 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:58 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:58.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:58.976 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:58 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:58.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:57:59.010 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:58.406+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:57:59.068 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:58.406+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:57:59.102 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:58.406+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:57:59.135 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:58.406+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:57:59.168 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:58.406+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:28.875781+0000 front 2024-04-23T18:57:28.875827+0000 (oldest deadline 2024-04-23T18:57:54.775950+0000) 2024-04-23T18:57:59.202 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:58 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:58.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:57:59.247 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:58 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:58.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:57:59.285 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:58 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:58.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:57:59.327 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:58 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:58.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:57:59.368 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:58 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:58.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:57:59.402 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:58 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:58.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:57:59.436 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:58.870+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:57:59.477 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:58.870+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:57:59.518 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:58.870+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:57:59.621 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:58.870+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:57:59.660 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:58.870+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:57:59.693 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:58 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:58.870+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:57:59.764 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:59 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:59.343+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:57:59.794 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:59 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:59.343+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:57:59.835 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:59 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:59.343+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:57:59.869 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:59 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:59.343+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:57:59.902 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:59 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:59.343+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:57:59.944 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:57:59 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:57:59.343+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:57:59.997 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:59.416+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:54.271882+0000 front 2024-04-23T18:56:54.271899+0000 (oldest deadline 2024-04-23T18:57:20.171696+0000) 2024-04-23T18:58:00.036 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:59.416+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.472735+0000 front 2024-04-23T18:57:05.472837+0000 (oldest deadline 2024-04-23T18:57:31.372937+0000) 2024-04-23T18:58:00.077 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:59.416+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.373381+0000 front 2024-04-23T18:57:11.373546+0000 (oldest deadline 2024-04-23T18:57:37.273421+0000) 2024-04-23T18:58:00.119 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:59.416+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.874928+0000 front 2024-04-23T18:57:21.875041+0000 (oldest deadline 2024-04-23T18:57:43.574967+0000) 2024-04-23T18:58:00.152 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8[52027]: 2024-04-23T18:57:59.416+0000 7f92e2109700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:28.875781+0000 front 2024-04-23T18:57:28.875827+0000 (oldest deadline 2024-04-23T18:57:54.775950+0000) 2024-04-23T18:58:00.187 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:59 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:59.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:00.227 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:59 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:59.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:00.269 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:59 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:59.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:00.337 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:59 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:59.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:00.394 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:59 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:59.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:00.461 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:57:59 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:57:59.754+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:00.503 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:57:59 smithi155 podman[75582]: 2024-04-23 18:57:59.84291709 +0000 UTC m=+5.469342807 container died b170b60b50e953bc6a6eb8bf4da84b08ab15f7ecab8aae29a2f8511afc83ac57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:58:00.545 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:59.885+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:00.586 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:59.885+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:00.628 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:59.885+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:00.669 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:59.885+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:00.711 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:59.885+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:00.754 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:57:59 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:57:59.885+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:00.795 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:00 smithi155 podman[75582]: 2024-04-23 18:58:00.240395334 +0000 UTC m=+5.866821040 container cleanup b170b60b50e953bc6a6eb8bf4da84b08ab15f7ecab8aae29a2f8511afc83ac57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-23T18:58:00.836 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:00 smithi155 bash[75582]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8 2024-04-23T18:58:00.879 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:00.326+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:58:00.919 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:00.326+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:58:00.961 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:00.326+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:58:01.003 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:00.326+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:58:01.045 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:00.326+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:58:01.101 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:00 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:00.326+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:58:01.153 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:00 smithi155 podman[75598]: 2024-04-23 18:58:00.491665835 +0000 UTC m=+0.648748375 container remove b170b60b50e953bc6a6eb8bf4da84b08ab15f7ecab8aae29a2f8511afc83ac57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-23T18:58:01.196 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:00.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:01.237 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:00.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:01.278 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:00.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:01.320 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:00.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:01.361 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:00.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:01.430 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:00 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:00.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:01.486 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:00 smithi155 podman[75692]: 2024-04-23 18:58:00.709074544 +0000 UTC m=+0.017892628 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:58:01.556 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:00 smithi155 podman[75692]: 2024-04-23 18:58:00.834095296 +0000 UTC m=+0.142913375 container create 9b16c7b76adbfa6e32090e92e08d4ba8ef23ca442611caff92a717187c4e8efc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-23T18:58:01.595 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:01 smithi155 podman[75692]: 2024-04-23 18:58:01.078364156 +0000 UTC m=+0.387182239 container init 9b16c7b76adbfa6e32090e92e08d4ba8ef23ca442611caff92a717187c4e8efc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-23T18:58:01.637 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:00.930+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:01.678 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:00.930+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:01.720 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:00.930+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:01.753 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:00.930+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:01.787 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:00.930+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:01.829 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:00 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:00.930+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:01.891 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:01 smithi155 podman[75692]: 2024-04-23 18:58:01.081839829 +0000 UTC m=+0.390657916 container start 9b16c7b76adbfa6e32090e92e08d4ba8ef23ca442611caff92a717187c4e8efc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-23T18:58:01.928 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:01 smithi155 podman[75692]: 2024-04-23 18:58:01.178488781 +0000 UTC m=+0.487306868 container attach 9b16c7b76adbfa6e32090e92e08d4ba8ef23ca442611caff92a717187c4e8efc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-23T18:58:01.970 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:01 smithi155 podman[75692]: 2024-04-23 18:58:01.392263311 +0000 UTC m=+0.701081402 container died 9b16c7b76adbfa6e32090e92e08d4ba8ef23ca442611caff92a717187c4e8efc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-23T18:58:02.013 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:01.334+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:58:02.054 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:01.334+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:58:02.121 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:01.334+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:58:02.162 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:01.334+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:58:02.204 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:01.334+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:58:02.246 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:01 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:01.334+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:58:02.304 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.8.service' 2024-04-23T18:58:02.337 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:01 smithi155 podman[75718]: 2024-04-23 18:58:01.982491236 +0000 UTC m=+0.585432194 container remove 9b16c7b76adbfa6e32090e92e08d4ba8ef23ca442611caff92a717187c4e8efc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-23T18:58:02.370 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:02 smithi155 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.8.service: Deactivated successfully. 2024-04-23T18:58:02.412 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:02 smithi155 systemd[1]: Stopped Ceph osd.8 for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:58:02.454 INFO:journalctl@ceph.osd.8.smithi155.stdout:Apr 23 18:58:02 smithi155 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.8.service: Consumed 21.511s CPU time. 2024-04-23T18:58:02.488 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:01.939+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:02.529 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:01.939+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:02.596 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:01.939+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:02.668 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:01.939+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:02.732 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:01.939+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:02.771 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:01.939+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:02.805 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:01.786+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:02.862 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:01.786+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:02.896 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:01.786+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:02.938 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:01.786+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:02.979 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:01.786+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:03.013 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:01.786+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:03.054 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:01 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:01.786+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:03.114 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:02.356+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:58:03.154 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:02.356+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:58:03.196 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:02.356+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:58:03.238 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:02.356+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:58:03.279 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:02.356+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:58:03.365 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:02 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:02.356+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:58:03.422 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:02 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:02.768+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:03.487 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:02 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:02.768+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:03.521 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:02 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:02.768+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:03.563 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:02 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:02.768+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:03.605 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:02 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:02.768+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:03.646 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:02 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:02.768+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:03.688 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:02 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:02.768+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:03.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:03.772 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-23T18:58:03.813 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-23T18:58:03.855 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.9 2024-04-23T18:58:03.897 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:02 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:02.917+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:03.938 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:02 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:02.917+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:03.980 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:02 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:02.917+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:04.022 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:02 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:02.917+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:04.080 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:02 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:02.917+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:04.122 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:02 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:02.917+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:04.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:03.360+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:58:04.268 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:03.360+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:58:04.305 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:03.360+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:58:04.347 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:03.360+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:58:04.389 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:03.360+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:58:04.430 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:03.360+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:58:04.472 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:03 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:03.360+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:38.455646+0000 front 2024-04-23T18:57:38.455660+0000 (oldest deadline 2024-04-23T18:58:02.555751+0000) 2024-04-23T18:58:04.514 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:03.727+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:04.555 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:03.727+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:04.597 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:03.727+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:04.647 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:03.727+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:04.704 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:03.727+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:04.739 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:03.727+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:04.781 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:03 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:03.727+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:04.842 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:03.966+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:04.881 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:03.966+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:04.922 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:03.966+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:04.964 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:03.966+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:05.064 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:03.966+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:05.106 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:03 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:03.966+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:05.139 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:04.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:58:05.181 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:04.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:58:05.214 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:04.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:58:05.250 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:04.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:58:05.281 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:04.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:58:05.314 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:04.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:58:05.348 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:04.321+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:38.455646+0000 front 2024-04-23T18:57:38.455660+0000 (oldest deadline 2024-04-23T18:58:02.555751+0000) 2024-04-23T18:58:05.411 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 systemd[1]: Stopping Ceph osd.9 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:05.467 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:04.822+0000 7fe1a5e34700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:05.506 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:04.822+0000 7fe1a5e34700 -1 osd.9 143 *** Got signal Terminated *** 2024-04-23T18:58:05.573 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:04 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:04.822+0000 7fe1a5e34700 -1 osd.9 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:58:05.607 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:04 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:04.721+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:05.639 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:04 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:04.721+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:05.673 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:04 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:04.721+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:05.704 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:04 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:04.721+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:05.775 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:04 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:04.721+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:05.812 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:04 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:04.721+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:05.846 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:04 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:04.721+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:05.879 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:04 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:04.923+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:05.912 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:04 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:04.923+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:05.946 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:04 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:04.923+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:05.979 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:04 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:04.923+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:06.013 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:04 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:04.923+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:06.046 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:04 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:04.923+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:06.100 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:05.351+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:58:06.138 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:05.351+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:58:06.188 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:05.351+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:58:06.221 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:05.351+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:58:06.255 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:05.351+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:58:06.288 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:05.351+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:58:06.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:05 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:05.351+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:38.455646+0000 front 2024-04-23T18:57:38.455660+0000 (oldest deadline 2024-04-23T18:58:02.555751+0000) 2024-04-23T18:58:06.355 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:05.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:06.406 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:05.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:06.463 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:05.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:06.496 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:05.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:06.530 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:05.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:06.563 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:05.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:06.597 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:05 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:05.715+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:06.630 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:05.920+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:06.663 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:05.920+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:06.696 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:05.920+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:06.737 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:05.920+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:06.797 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:05.920+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:06.830 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:05.920+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:06.872 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:06.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:58:06.905 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:06.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:58:06.938 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:06.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:58:06.972 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:06.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:58:07.005 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:06.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:58:07.078 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:06.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:58:07.113 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:06 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:06.384+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:38.455646+0000 front 2024-04-23T18:57:38.455660+0000 (oldest deadline 2024-04-23T18:58:02.555751+0000) 2024-04-23T18:58:07.148 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:06.751+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:07.180 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:06.751+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:07.214 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:06.751+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:07.247 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:06.751+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:07.280 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:06.751+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:07.314 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:06.751+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:07.347 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:06 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:06.751+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:07.415 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:06.906+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:07.463 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:06.906+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:07.497 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:06.906+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:07.530 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:06.906+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:07.564 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:06.906+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:07.597 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:06 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:06.906+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:07.764 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:07.382+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:58:07.797 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:07.382+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:58:07.873 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:07.382+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:58:07.906 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:07.382+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:58:07.939 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:07.382+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:58:07.972 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:07.382+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:58:08.006 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:07 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:07.382+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:38.455646+0000 front 2024-04-23T18:57:38.455660+0000 (oldest deadline 2024-04-23T18:58:02.555751+0000) 2024-04-23T18:58:08.067 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:07 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:07.788+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:08.089 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:07 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:07.788+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:08.123 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:07 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:07.788+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:08.173 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:07 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:07.788+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:08.228 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:07 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:07.788+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:08.264 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:07 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:07.788+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:08.298 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:07 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:07.788+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:08.331 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:07 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:07.876+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:08.364 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:07 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:07.876+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:08.398 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:07 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:07.876+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:08.431 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:07 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:07.876+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:08.465 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:07 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:07.876+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:08.545 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:07 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:07.876+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:08.764 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:08.389+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:58:08.823 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:08.389+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:58:08.873 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:08.389+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:58:08.906 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:08.389+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:58:08.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:08.389+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:58:08.973 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:08.389+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:58:09.031 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:08 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:08.389+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:38.455646+0000 front 2024-04-23T18:57:38.455660+0000 (oldest deadline 2024-04-23T18:58:02.555751+0000) 2024-04-23T18:58:09.067 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:08 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:08.767+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:09.116 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:08 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:08.767+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:09.148 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:08 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:08.767+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:09.181 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:08 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:08.767+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:09.215 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:08 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:08.767+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:09.248 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:08 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:08.767+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:09.282 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:08 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:08.767+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:09.327 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:08 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:08.851+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:09.348 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:08 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:08.851+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:09.422 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:08 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:08.851+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:09.490 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:08 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:08.851+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:09.523 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:08 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:08.851+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:09.557 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:08 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:08.851+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:09.764 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:09.399+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:46.146395+0000 front 2024-04-23T18:56:46.146580+0000 (oldest deadline 2024-04-23T18:57:09.646574+0000) 2024-04-23T18:58:09.790 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:09.399+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:57.149019+0000 front 2024-04-23T18:56:57.148833+0000 (oldest deadline 2024-04-23T18:57:23.048777+0000) 2024-04-23T18:58:09.840 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:09.399+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.650463+0000 front 2024-04-23T18:57:04.650398+0000 (oldest deadline 2024-04-23T18:57:29.950139+0000) 2024-04-23T18:58:09.882 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:09.399+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.950486+0000 front 2024-04-23T18:57:09.950540+0000 (oldest deadline 2024-04-23T18:57:34.650617+0000) 2024-04-23T18:58:09.916 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:09.399+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:22.152945+0000 front 2024-04-23T18:57:22.152924+0000 (oldest deadline 2024-04-23T18:57:44.452738+0000) 2024-04-23T18:58:09.949 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:09.399+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:26.753987+0000 front 2024-04-23T18:57:26.754012+0000 (oldest deadline 2024-04-23T18:57:51.454039+0000) 2024-04-23T18:58:10.001 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:09 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9[63997]: 2024-04-23T18:58:09.399+0000 7fe19e420700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:38.455646+0000 front 2024-04-23T18:57:38.455660+0000 (oldest deadline 2024-04-23T18:58:02.555751+0000) 2024-04-23T18:58:10.067 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:09 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:09.791+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:10.091 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:09 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:09.791+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:10.124 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:09 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:09.791+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:10.157 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:09 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:09.791+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:10.191 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:09 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:09.791+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:10.224 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:09 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:09.791+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:10.302 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:09 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:09.791+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:10.341 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:09 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:09.825+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:10.374 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:09 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:09.825+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:10.408 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:09 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:09.825+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:10.441 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:09 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:09.825+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:10.500 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:09 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:09.825+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:10.533 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:09 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:09.825+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:10.567 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:10 smithi019 podman[101735]: 2024-04-23 18:58:10.257626547 +0000 UTC m=+5.532693856 container died 955d7752da791604ebcb7f863a531556188b95c849694eb1a429303383052058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-23T18:58:11.004 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:10 smithi019 podman[101735]: 2024-04-23 18:58:10.730801353 +0000 UTC m=+6.005868689 container cleanup 955d7752da791604ebcb7f863a531556188b95c849694eb1a429303383052058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-23T18:58:11.066 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:10 smithi019 bash[101735]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9 2024-04-23T18:58:11.109 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:10 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:10.820+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:11.141 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:10 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:10.820+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:11.175 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:10 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:10.820+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:11.208 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:10 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:10.820+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:11.241 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:10 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:10.820+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:11.274 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:10 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:10.820+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:11.308 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:10 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:10.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:11.392 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:10 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:10.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:11.425 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:10 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:10.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:11.458 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:10 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:10.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:11.491 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:10 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:10.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:11.542 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:10 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:10.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:11.575 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:10 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:10.798+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:11.609 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:11 smithi019 podman[101752]: 2024-04-23 18:58:11.007223875 +0000 UTC m=+0.748854506 container remove 955d7752da791604ebcb7f863a531556188b95c849694eb1a429303383052058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-23T18:58:11.642 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:11 smithi019 podman[101843]: 2024-04-23 18:58:11.221511009 +0000 UTC m=+0.017953315 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:58:11.675 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:11 smithi019 podman[101843]: 2024-04-23 18:58:11.333487471 +0000 UTC m=+0.129929779 container create 9472d11e83fcfc30448e7ef66bd63c8d84e9869b90bf9c373a54186440895921 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:58:11.728 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:11 smithi019 podman[101843]: 2024-04-23 18:58:11.544019186 +0000 UTC m=+0.340461491 container init 9472d11e83fcfc30448e7ef66bd63c8d84e9869b90bf9c373a54186440895921 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-04-23T18:58:11.767 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:11 smithi019 podman[101843]: 2024-04-23 18:58:11.547141377 +0000 UTC m=+0.343583683 container start 9472d11e83fcfc30448e7ef66bd63c8d84e9869b90bf9c373a54186440895921 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:58:11.800 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:11 smithi019 podman[101843]: 2024-04-23 18:58:11.618994261 +0000 UTC m=+0.415436566 container attach 9472d11e83fcfc30448e7ef66bd63c8d84e9869b90bf9c373a54186440895921 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:58:12.014 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:11 smithi019 podman[101843]: 2024-04-23 18:58:11.855030267 +0000 UTC m=+0.651472573 container died 9472d11e83fcfc30448e7ef66bd63c8d84e9869b90bf9c373a54186440895921 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-23T18:58:12.067 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:11 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:11.814+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:12.092 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:11 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:11.814+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:12.125 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:11 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:11.814+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:12.159 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:11 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:11.815+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:12.275 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:11 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:11.815+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:12.309 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:11 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:11.815+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:12.342 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:11 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:11.815+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:12.409 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:11 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:11.822+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:12.442 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:11 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:11.822+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:12.475 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:11 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:11.822+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:12.509 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:11 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:11.822+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:12.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:11 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:11.822+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:12.592 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:11 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:11.822+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:12.674 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.9.service' 2024-04-23T18:58:12.742 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:12 smithi019 podman[101868]: 2024-04-23 18:58:12.455667497 +0000 UTC m=+0.595479501 container remove 9472d11e83fcfc30448e7ef66bd63c8d84e9869b90bf9c373a54186440895921 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-23T18:58:12.776 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:12 smithi019 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.9.service: Deactivated successfully. 2024-04-23T18:58:12.809 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:12 smithi019 systemd[1]: Stopped Ceph osd.9 for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:58:12.842 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 23 18:58:12 smithi019 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.9.service: Consumed 24.641s CPU time. 2024-04-23T18:58:12.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:12.943 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-23T18:58:12.976 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-23T18:58:13.073 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.10 2024-04-23T18:58:13.134 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:12 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:12.814+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:13.184 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:12 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:12.814+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:13.217 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:12 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:12.814+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:13.251 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:12 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:12.814+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:13.284 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:12 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:12.814+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:13.343 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:12 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:12.814+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:13.376 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:12 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:12.814+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:13.410 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:12 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:12.855+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:13.473 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:12 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:12.855+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:13.509 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:12 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:12.855+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:13.543 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:12 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:12.855+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:13.576 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:12 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:12.855+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:13.610 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:12 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:12.855+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:13.643 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:12 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:12.855+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:13.677 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 systemd[1]: Stopping Ceph osd.10 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:13.811 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:13.458+0000 7f701c969700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:13.843 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:13.458+0000 7f701c969700 -1 osd.10 143 *** Got signal Terminated *** 2024-04-23T18:58:13.901 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:13.458+0000 7f701c969700 -1 osd.10 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:58:13.935 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:13.813+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:13.968 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:13.813+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:14.001 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:13.813+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:14.035 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:13.813+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:14.068 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:13.813+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:14.101 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:13.813+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:14.136 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:13.796+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:14.218 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:13.796+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:14.252 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:13.796+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:14.285 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:13.796+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:14.318 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:13.796+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:14.352 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:13.796+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:14.385 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:13.796+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:14.419 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:13 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:13.813+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:15.079 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:14 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:14.747+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:15.135 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:14 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:14.747+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:15.202 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:14 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:14.747+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:15.235 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:14 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:14.747+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:15.269 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:14 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:14.747+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:15.319 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:14 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:14.747+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:15.352 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:14 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:14.747+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:15.386 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:14 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:14.852+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:15.419 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:14 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:14.852+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:15.452 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:14 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:14.852+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:15.530 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:14 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:14.852+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:15.560 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:14 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:14.852+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:15.594 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:14 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:14.852+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:15.627 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:14 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:14.852+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:16.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:15 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:15.703+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:16.144 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:15 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:15.703+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:16.177 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:15 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:15.703+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:16.210 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:15 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:15.703+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:16.310 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:15 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:15.703+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:16.344 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:15 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:15.703+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:16.377 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:15 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:15.703+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:16.411 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:15 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:15.886+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:16.444 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:15 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:15.886+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:16.477 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:15 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:15.886+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:16.511 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:15 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:15.886+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:16.544 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:15 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:15.886+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:16.578 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:15 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:15.886+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:16.616 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:15 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:15.886+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:17.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:16.686+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:17.136 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:16.686+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:17.169 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:16.686+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:17.202 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:16.686+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:17.236 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:16.686+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:17.269 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:16.686+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:17.337 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:16 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:16.686+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:17.413 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:16.837+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:17.444 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:16.837+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:17.478 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:16.837+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:17.511 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:16.837+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:17.544 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:16.837+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:17.578 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:16.837+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:17.611 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:16 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:16.837+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:18.067 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:17.804+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:47.408560+0000 front 2024-04-23T18:56:47.408711+0000 (oldest deadline 2024-04-23T18:57:09.609067+0000) 2024-04-23T18:58:18.136 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:17.804+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.610738+0000 front 2024-04-23T18:56:53.610683+0000 (oldest deadline 2024-04-23T18:57:19.510917+0000) 2024-04-23T18:58:18.189 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:17.804+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:04.112523+0000 front 2024-04-23T18:57:04.112310+0000 (oldest deadline 2024-04-23T18:57:26.412436+0000) 2024-04-23T18:58:18.228 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:17.804+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:09.813871+0000 front 2024-04-23T18:57:09.814100+0000 (oldest deadline 2024-04-23T18:57:34.513829+0000) 2024-04-23T18:58:18.261 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:17.804+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:20.815728+0000 front 2024-04-23T18:57:20.815692+0000 (oldest deadline 2024-04-23T18:57:43.115796+0000) 2024-04-23T18:58:18.311 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:17.804+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:30.717333+0000 front 2024-04-23T18:57:30.717375+0000 (oldest deadline 2024-04-23T18:57:51.817196+0000) 2024-04-23T18:58:18.345 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:17 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10[57279]: 2024-04-23T18:58:17.804+0000 7f7014f55700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-23T18:57:36.919485+0000 front 2024-04-23T18:57:36.919693+0000 (oldest deadline 2024-04-23T18:58:01.619320+0000) 2024-04-23T18:58:18.378 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:17 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:17.677+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:18.428 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:17 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:17.677+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:18.461 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:17 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:17.677+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:18.531 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:17 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:17.677+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:18.570 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:17 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:17.677+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:18.603 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:17 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:17.677+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:18.636 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:17 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:17.677+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:19.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:18 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:18.643+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:19.103 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:18 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:18.643+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:19.137 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:18 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:18.643+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:19.170 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:18 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:18.643+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:19.203 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:18 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:18.643+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:19.245 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:18 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:18.643+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:19.270 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:18 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:18.643+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:19.304 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:18 smithi044 podman[76161]: 2024-04-23 18:58:18.835618539 +0000 UTC m=+5.497889201 container died 032af56efe04504824f9d05eadc9c13265f3e3771010e1bbccd3b41667a39600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True) 2024-04-23T18:58:19.337 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:19 smithi044 podman[76161]: 2024-04-23 18:58:19.218100068 +0000 UTC m=+5.880370719 container cleanup 032af56efe04504824f9d05eadc9c13265f3e3771010e1bbccd3b41667a39600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T18:58:19.474 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:19 smithi044 bash[76161]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10 2024-04-23T18:58:19.817 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:19 smithi044 podman[76179]: 2024-04-23 18:58:19.477681164 +0000 UTC m=+0.635471226 container remove 032af56efe04504824f9d05eadc9c13265f3e3771010e1bbccd3b41667a39600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:58:20.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:19 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:19.653+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:20.129 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:19 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:19.653+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:20.213 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:19 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:19.653+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:20.245 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:19 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:19.653+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:20.279 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:19 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:19.653+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:20.312 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:19 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:19.653+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:20.345 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:19 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:19.653+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:20.379 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:19 smithi044 podman[76270]: 2024-04-23 18:58:19.837011761 +0000 UTC m=+0.105970330 container create b9a68d4cb727859671e2c0902e26100a6f1d3081d84371a9101b5ce4cde045c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:58:20.414 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:19 smithi044 podman[76270]: 2024-04-23 18:58:19.749227787 +0000 UTC m=+0.018186381 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:58:20.445 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:20 smithi044 podman[76270]: 2024-04-23 18:58:20.04762549 +0000 UTC m=+0.316584086 container init b9a68d4cb727859671e2c0902e26100a6f1d3081d84371a9101b5ce4cde045c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T18:58:20.479 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:20 smithi044 podman[76270]: 2024-04-23 18:58:20.050723385 +0000 UTC m=+0.319681961 container start b9a68d4cb727859671e2c0902e26100a6f1d3081d84371a9101b5ce4cde045c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-23T18:58:20.512 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:20 smithi044 podman[76270]: 2024-04-23 18:58:20.122485423 +0000 UTC m=+0.391443996 container attach b9a68d4cb727859671e2c0902e26100a6f1d3081d84371a9101b5ce4cde045c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-23T18:58:20.567 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:20 smithi044 podman[76270]: 2024-04-23 18:58:20.353513139 +0000 UTC m=+0.622471720 container died b9a68d4cb727859671e2c0902e26100a6f1d3081d84371a9101b5ce4cde045c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-23T18:58:21.067 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:21 smithi044 podman[76296]: 2024-04-23 18:58:21.050864519 +0000 UTC m=+0.691611050 container remove b9a68d4cb727859671e2c0902e26100a6f1d3081d84371a9101b5ce4cde045c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-23T18:58:21.130 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:20.689+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:21.162 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:20.689+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:21.229 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:20.689+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:21.262 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:20.689+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:21.296 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:20.689+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:21.386 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:20.689+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:21.421 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:20 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:20.689+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:21.455 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.10.service' 2024-04-23T18:58:21.514 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:21 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-23T18:58:21.537 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:21 smithi044 systemd[1]: Stopped Ceph osd.10 for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:58:21.571 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 23 18:58:21 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@osd.10.service: Consumed 26.662s CPU time. 2024-04-23T18:58:21.606 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:21.638 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-23T18:58:21.671 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-23T18:58:21.709 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@osd.11 2024-04-23T18:58:22.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:21.640+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:22.129 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:21.640+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:22.163 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:21.640+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:22.213 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:21.640+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:22.246 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:21.640+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:22.296 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:21.640+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:22.330 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:21 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:21.640+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:22.416 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:21 smithi155 systemd[1]: Stopping Ceph osd.11 for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:22.577 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:22.084+0000 7fa8ee02f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:22.604 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:22.084+0000 7fa8ee02f700 -1 osd.11 143 *** Got signal Terminated *** 2024-04-23T18:58:22.638 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:22.084+0000 7fa8ee02f700 -1 osd.11 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:58:23.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:22.592+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:23.096 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:22.592+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:23.129 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:22.592+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:23.163 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:22.592+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:23.196 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:22.592+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:23.280 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:22.592+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:23.313 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:22 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:22.592+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:24.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:23.622+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:24.113 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:23.622+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:24.188 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:23.622+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:24.222 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:23.622+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:24.255 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:23.622+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:24.288 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:23.622+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:24.371 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:23 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:23.622+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:25.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:24.642+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:25.113 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:24.642+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:25.147 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:24.642+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:25.180 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:24.642+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:25.230 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:24.642+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:25.263 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:24.642+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:25.297 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:24.642+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:25.330 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:24 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:24.642+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6820 osd.8 since back 2024-04-23T18:57:59.069079+0000 front 2024-04-23T18:57:59.069121+0000 (oldest deadline 2024-04-23T18:58:23.769127+0000) 2024-04-23T18:58:26.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:25.632+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:26.122 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:25.632+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:26.155 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:25.632+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:26.206 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:25.632+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:26.239 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:25.632+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:26.272 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:25.632+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:26.306 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:25.632+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:26.339 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:25 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:25.632+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6820 osd.8 since back 2024-04-23T18:57:59.069079+0000 front 2024-04-23T18:57:59.069121+0000 (oldest deadline 2024-04-23T18:58:23.769127+0000) 2024-04-23T18:58:27.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:26.596+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-23T18:56:45.256227+0000 front 2024-04-23T18:56:45.256185+0000 (oldest deadline 2024-04-23T18:57:09.956285+0000) 2024-04-23T18:58:27.122 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:26.596+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-23T18:56:53.957901+0000 front 2024-04-23T18:56:53.958075+0000 (oldest deadline 2024-04-23T18:57:18.057759+0000) 2024-04-23T18:58:27.156 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:26.596+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-04-23T18:57:05.659460+0000 front 2024-04-23T18:57:05.659435+0000 (oldest deadline 2024-04-23T18:57:27.260198+0000) 2024-04-23T18:58:27.189 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:26.596+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-23T18:57:11.861648+0000 front 2024-04-23T18:57:11.861683+0000 (oldest deadline 2024-04-23T18:57:35.961770+0000) 2024-04-23T18:58:27.222 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:26.596+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-23T18:57:21.263119+0000 front 2024-04-23T18:57:21.263066+0000 (oldest deadline 2024-04-23T18:57:44.162978+0000) 2024-04-23T18:58:27.256 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:26.596+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-04-23T18:57:27.064147+0000 front 2024-04-23T18:57:27.064197+0000 (oldest deadline 2024-04-23T18:57:54.665067+0000) 2024-04-23T18:58:27.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:26.596+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-23T18:57:44.567216+0000 front 2024-04-23T18:57:44.567089+0000 (oldest deadline 2024-04-23T18:58:12.167474+0000) 2024-04-23T18:58:27.331 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:26 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11[58631]: 2024-04-23T18:58:26.596+0000 7fa8e661b700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.155:6820 osd.8 since back 2024-04-23T18:57:59.069079+0000 front 2024-04-23T18:57:59.069121+0000 (oldest deadline 2024-04-23T18:58:23.769127+0000) 2024-04-23T18:58:27.824 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:27 smithi155 podman[75809]: 2024-04-23 18:58:27.542091618 +0000 UTC m=+5.598668276 container died 3962e34ce10d0438e7588809d97cf827e391171281188193d5b5d1b9abbfeefe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-23T18:58:28.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:27 smithi155 podman[75809]: 2024-04-23 18:58:27.918129502 +0000 UTC m=+5.974706160 container cleanup 3962e34ce10d0438e7588809d97cf827e391171281188193d5b5d1b9abbfeefe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-23T18:58:28.097 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:27 smithi155 bash[75809]: ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11 2024-04-23T18:58:28.530 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:28 smithi155 podman[75826]: 2024-04-23 18:58:28.194432773 +0000 UTC m=+0.643590771 container remove 3962e34ce10d0438e7588809d97cf827e391171281188193d5b5d1b9abbfeefe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:58:28.802 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:28 smithi155 podman[75912]: 2024-04-23 18:58:28.436973885 +0000 UTC m=+0.018016952 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:58:28.831 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:28 smithi155 podman[75912]: 2024-04-23 18:58:28.536682744 +0000 UTC m=+0.117725808 container create b2e39ee2a4fe38a8fb896a732f3301fbc454da1f0c25362a11631ba307b9bf6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:58:28.864 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:28 smithi155 podman[75912]: 2024-04-23 18:58:28.730960409 +0000 UTC m=+0.312003460 container init b2e39ee2a4fe38a8fb896a732f3301fbc454da1f0c25362a11631ba307b9bf6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:58:28.897 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:28 smithi155 podman[75912]: 2024-04-23 18:58:28.734082225 +0000 UTC m=+0.315125277 container start b2e39ee2a4fe38a8fb896a732f3301fbc454da1f0c25362a11631ba307b9bf6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-23T18:58:28.931 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:28 smithi155 podman[75912]: 2024-04-23 18:58:28.80576718 +0000 UTC m=+0.386810233 container attach b2e39ee2a4fe38a8fb896a732f3301fbc454da1f0c25362a11631ba307b9bf6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-23T18:58:29.077 INFO:journalctl@ceph.osd.11.smithi155.stdout:Apr 23 18:58:29 smithi155 podman[75912]: 2024-04-23 18:58:29.025997779 +0000 UTC m=+0.607040824 container died b2e39ee2a4fe38a8fb896a732f3301fbc454da1f0c25362a11631ba307b9bf6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-23T18:58:29.652 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@osd.11.service' 2024-04-23T18:58:29.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:29.773 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-23T18:58:29.861 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-23T18:58:29.898 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.a 2024-04-23T18:58:30.222 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:58:29 smithi019 systemd[1]: Stopping Ceph mds.a for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:30.514 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:58:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a[70760]: 2024-04-23T18:58:30.224+0000 7ff35e771700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:30.539 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:58:30 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a[70760]: 2024-04-23T18:58:30.224+0000 7ff35e771700 -1 mds.a *** got signal Terminated *** 2024-04-23T18:58:32.347 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:58:32 smithi019 podman[101974]: 2024-04-23 18:58:32.046199602 +0000 UTC m=+1.909926484 container died 14eec03debed59c5f93cb6719a35caba19ca68d11a0090e5bc4664d02d0857a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:58:32.672 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:58:32 smithi019 podman[101974]: 2024-04-23 18:58:32.442503412 +0000 UTC m=+2.306230290 container cleanup 14eec03debed59c5f93cb6719a35caba19ca68d11a0090e5bc4664d02d0857a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:58:32.757 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 23 18:58:32 smithi019 bash[101974]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-a 2024-04-23T18:58:32.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.a.service' 2024-04-23T18:58:32.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:32.939 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-23T18:58:32.972 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-23T18:58:33.006 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.d 2024-04-23T18:58:33.334 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:58:33 smithi019 systemd[1]: Stopping Ceph mds.d for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:33.764 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:58:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-d[69751]: 2024-04-23T18:58:33.336+0000 7effb4292700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:33.798 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:58:33 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-d[69751]: 2024-04-23T18:58:33.336+0000 7effb4292700 -1 mds.d *** got signal Terminated *** 2024-04-23T18:58:35.111 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:58:34 smithi019 podman[102094]: 2024-04-23 18:58:34.834261098 +0000 UTC m=+1.610847178 container died 6ed95b660d4ec6bf9a0c1ed544866481b7e5e7948185e0778761f6cd838bf5b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:58:35.468 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:58:35 smithi019 podman[102094]: 2024-04-23 18:58:35.220108803 +0000 UTC m=+1.996694881 container cleanup 6ed95b660d4ec6bf9a0c1ed544866481b7e5e7948185e0778761f6cd838bf5b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T18:58:35.506 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 23 18:58:35 smithi019 bash[102094]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-d 2024-04-23T18:58:35.602 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.d.service' 2024-04-23T18:58:35.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:35.731 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-23T18:58:35.774 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-23T18:58:35.806 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.g 2024-04-23T18:58:36.162 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:58:35 smithi019 systemd[1]: Stopping Ceph mds.g for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:36.514 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:58:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g[70359]: 2024-04-23T18:58:36.165+0000 7f02d5b88700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:36.556 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:58:36 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g[70359]: 2024-04-23T18:58:36.165+0000 7f02d5b88700 -1 mds.g *** got signal Terminated *** 2024-04-23T18:58:40.413 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:58:40 smithi019 podman[102217]: 2024-04-23 18:58:40.04966009 +0000 UTC m=+3.993106627 container died dbbf0ce9b4e9357d1eb0a0c800178742557a4c2024f6435e803c51567989f678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-23T18:58:40.764 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:58:40 smithi019 podman[102217]: 2024-04-23 18:58:40.507062675 +0000 UTC m=+4.450509218 container cleanup dbbf0ce9b4e9357d1eb0a0c800178742557a4c2024f6435e803c51567989f678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-23T18:58:40.789 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 23 18:58:40 smithi019 bash[102217]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-g 2024-04-23T18:58:40.931 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.g.service' 2024-04-23T18:58:40.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:41.014 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-23T18:58:41.047 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-23T18:58:41.080 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.j 2024-04-23T18:58:41.407 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:58:41 smithi019 systemd[1]: Stopping Ceph mds.j for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:41.764 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:58:41 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-j[69397]: 2024-04-23T18:58:41.410+0000 7fab253f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:41.840 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:58:41 smithi019 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-j[69397]: 2024-04-23T18:58:41.410+0000 7fab253f8700 -1 mds.j *** got signal Terminated *** 2024-04-23T18:58:44.116 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:58:43 smithi019 podman[102338]: 2024-04-23 18:58:43.813862313 +0000 UTC m=+2.517925498 container died ae775ed8da7e113a304a7e9b2bd72bc5d6d759b57c0941b120ac121910330e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-23T18:58:44.450 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:58:44 smithi019 podman[102338]: 2024-04-23 18:58:44.219022599 +0000 UTC m=+2.923085773 container cleanup ae775ed8da7e113a304a7e9b2bd72bc5d6d759b57c0941b120ac121910330e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-j, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-23T18:58:44.472 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 23 18:58:44 smithi019 bash[102338]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-j 2024-04-23T18:58:44.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.j.service' 2024-04-23T18:58:44.654 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:44.672 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-23T18:58:44.705 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-23T18:58:44.808 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.b 2024-04-23T18:58:45.241 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:58:44 smithi044 systemd[1]: Stopping Ceph mds.b for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:45.567 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:58:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b[62807]: 2024-04-23T18:58:45.243+0000 7f9ce9cca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:45.589 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:58:45 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b[62807]: 2024-04-23T18:58:45.243+0000 7f9ce9cca700 -1 mds.b *** got signal Terminated *** 2024-04-23T18:58:48.299 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:58:48 smithi044 podman[76376]: 2024-04-23 18:58:48.043985051 +0000 UTC m=+2.899395540 container died bb88eed0142d8cb8372119ae9a0398184fa00587bb2eee45ccda7e5b94856bac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T18:58:48.567 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:58:48 smithi044 podman[76376]: 2024-04-23 18:58:48.459822305 +0000 UTC m=+3.315232822 container cleanup bb88eed0142d8cb8372119ae9a0398184fa00587bb2eee45ccda7e5b94856bac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:58:48.589 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:58:48 smithi044 bash[76376]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b 2024-04-23T18:58:48.905 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.b.service' 2024-04-23T18:58:48.952 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:58:48 smithi044 podman[76390]: 2024-04-23 18:58:48.677979371 +0000 UTC m=+0.635182137 container remove bb88eed0142d8cb8372119ae9a0398184fa00587bb2eee45ccda7e5b94856bac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True) 2024-04-23T18:58:48.980 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:58:48 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.b.service: Deactivated successfully. 2024-04-23T18:58:49.014 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:58:48 smithi044 systemd[1]: Stopped Ceph mds.b for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:58:49.047 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 23 18:58:48 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.b.service: Consumed 17.011s CPU time. 2024-04-23T18:58:49.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:49.114 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-23T18:58:49.147 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-23T18:58:49.181 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e 2024-04-23T18:58:49.503 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:58:49 smithi044 systemd[1]: Stopping Ceph mds.e for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:49.817 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:58:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69558]: 2024-04-23T18:58:49.505+0000 7f153194f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:49.847 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:58:49 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e[69558]: 2024-04-23T18:58:49.505+0000 7f153194f700 -1 mds.e *** got signal Terminated *** 2024-04-23T18:58:52.317 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:58:52 smithi044 podman[76500]: 2024-04-23 18:58:52.054517147 +0000 UTC m=+2.654798201 container died 77a9f69890bbf83c572470daa41d9cd61c522ef6f34a283dcdaa206d22a215ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-23T18:58:52.670 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:58:52 smithi044 podman[76500]: 2024-04-23 18:58:52.405415606 +0000 UTC m=+3.005696659 container cleanup 77a9f69890bbf83c572470daa41d9cd61c522ef6f34a283dcdaa206d22a215ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-23T18:58:52.747 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 23 18:58:52 smithi044 bash[76500]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-e 2024-04-23T18:58:52.797 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.e.service' 2024-04-23T18:58:52.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:52.889 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-23T18:58:52.922 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-23T18:58:52.955 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.h 2024-04-23T18:58:53.281 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:58:53 smithi044 systemd[1]: Stopping Ceph mds.h for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:53.567 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:58:53 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h[62501]: 2024-04-23T18:58:53.283+0000 7fe463022700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:53.589 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:58:53 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h[62501]: 2024-04-23T18:58:53.283+0000 7fe463022700 -1 mds.h *** got signal Terminated *** 2024-04-23T18:58:56.317 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:58:56 smithi044 podman[76634]: 2024-04-23 18:58:56.05520309 +0000 UTC m=+2.877862376 container died f624123f2dae4482930e460de7bcc647675fdb61ea7a892ddab2f940c9561f50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-23T18:58:56.817 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:58:56 smithi044 podman[76634]: 2024-04-23 18:58:56.400518174 +0000 UTC m=+3.223177468 container cleanup f624123f2dae4482930e460de7bcc647675fdb61ea7a892ddab2f940c9561f50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-23T18:58:56.920 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:58:56 smithi044 bash[76634]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h 2024-04-23T18:58:56.955 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 23 18:58:56 smithi044 podman[76648]: 2024-04-23 18:58:56.585050594 +0000 UTC m=+0.530780329 container remove f624123f2dae4482930e460de7bcc647675fdb61ea7a892ddab2f940c9561f50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:58:56.990 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.h.service' 2024-04-23T18:58:57.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:58:57.080 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-23T18:58:57.113 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-23T18:58:57.147 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.k 2024-04-23T18:58:57.477 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:58:57 smithi044 systemd[1]: Stopping Ceph mds.k for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:57.817 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:58:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k[61882]: 2024-04-23T18:58:57.479+0000 7f64da41a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:58:57.847 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:58:57 smithi044 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k[61882]: 2024-04-23T18:58:57.479+0000 7f64da41a700 -1 mds.k *** got signal Terminated *** 2024-04-23T18:59:00.310 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:59:00 smithi044 podman[76757]: 2024-04-23 18:59:00.036877806 +0000 UTC m=+2.668523740 container died 3fbb3b37d5decb19357278bceb1f7ddc170f4a7168427d0f5ba5368686427265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-23T18:59:00.567 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:59:00 smithi044 podman[76757]: 2024-04-23 18:59:00.496362443 +0000 UTC m=+3.128008387 container cleanup 3fbb3b37d5decb19357278bceb1f7ddc170f4a7168427d0f5ba5368686427265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-23T18:59:00.588 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:59:00 smithi044 bash[76757]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k 2024-04-23T18:59:00.845 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.k.service' 2024-04-23T18:59:00.949 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:59:00 smithi044 podman[76772]: 2024-04-23 18:59:00.730664465 +0000 UTC m=+0.683650710 container remove 3fbb3b37d5decb19357278bceb1f7ddc170f4a7168427d0f5ba5368686427265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2) 2024-04-23T18:59:00.980 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:59:00 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-23T18:59:01.013 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:59:00 smithi044 systemd[1]: Stopped Ceph mds.k for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:59:01.047 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 23 18:59:00 smithi044 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.k.service: Consumed 3.916s CPU time. 2024-04-23T18:59:01.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:59:01.113 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-23T18:59:01.147 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-23T18:59:01.181 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.c 2024-04-23T18:59:01.506 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:59:01 smithi155 systemd[1]: Stopping Ceph mds.c for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:59:01.827 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:59:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c[63613]: 2024-04-23T18:59:01.507+0000 7ff3ae25b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:59:01.847 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:59:01 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c[63613]: 2024-04-23T18:59:01.507+0000 7ff3ae25b700 -1 mds.c *** got signal Terminated *** 2024-04-23T18:59:04.315 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:59:04 smithi155 podman[76019]: 2024-04-23 18:59:04.036722259 +0000 UTC m=+2.630161221 container died ef0ea28b882bd0f028b235298ec42605b44b4938a1349aba55ca23a471d05623 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-23T18:59:04.577 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:59:04 smithi155 podman[76019]: 2024-04-23 18:59:04.492410987 +0000 UTC m=+3.085850003 container cleanup ef0ea28b882bd0f028b235298ec42605b44b4938a1349aba55ca23a471d05623 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-23T18:59:04.605 INFO:journalctl@ceph.mds.c.smithi155.stdout:Apr 23 18:59:04 smithi155 bash[76019]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-c 2024-04-23T18:59:04.920 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.c.service' 2024-04-23T18:59:04.983 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:59:05.005 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-23T18:59:05.039 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-23T18:59:05.072 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.f 2024-04-23T18:59:05.394 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:59:05 smithi155 systemd[1]: Stopping Ceph mds.f for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:59:05.827 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:59:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f[63308]: 2024-04-23T18:59:05.396+0000 7fed9d5f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:59:05.914 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:59:05 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f[63308]: 2024-04-23T18:59:05.396+0000 7fed9d5f6700 -1 mds.f *** got signal Terminated *** 2024-04-23T18:59:08.327 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:59:08 smithi155 podman[76153]: 2024-04-23 18:59:08.04393139 +0000 UTC m=+2.759634485 container died c9f647b7340718d0eafea3939530866e2278232007073c9a985c6dc1d0d97991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:59:08.661 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:59:08 smithi155 podman[76153]: 2024-04-23 18:59:08.480055101 +0000 UTC m=+3.195758194 container cleanup c9f647b7340718d0eafea3939530866e2278232007073c9a985c6dc1d0d97991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:59:08.680 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:59:08 smithi155 bash[76153]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f 2024-04-23T18:59:08.901 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.f.service' 2024-04-23T18:59:09.017 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:59:08 smithi155 podman[76167]: 2024-04-23 18:59:08.66438932 +0000 UTC m=+0.620784855 container remove c9f647b7340718d0eafea3939530866e2278232007073c9a985c6dc1d0d97991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-23T18:59:09.038 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:59:08 smithi155 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-23T18:59:09.072 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:59:08 smithi155 systemd[1]: Stopped Ceph mds.f for 46706838-019d-11ef-bc93-c7b262605968. 2024-04-23T18:59:09.105 INFO:journalctl@ceph.mds.f.smithi155.stdout:Apr 23 18:59:08 smithi155 systemd[1]: ceph-46706838-019d-11ef-bc93-c7b262605968@mds.f.service: Consumed 3.514s CPU time. 2024-04-23T18:59:09.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:59:09.194 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-23T18:59:09.230 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-23T18:59:09.263 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.i 2024-04-23T18:59:09.577 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:59:09 smithi155 systemd[1]: Stopping Ceph mds.i for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:59:10.077 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:59:09 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-i[63914]: 2024-04-23T18:59:09.583+0000 7f939b2a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:59:10.130 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:59:09 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-i[63914]: 2024-04-23T18:59:09.583+0000 7f939b2a7700 -1 mds.i *** got signal Terminated *** 2024-04-23T18:59:12.327 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:59:12 smithi155 podman[76274]: 2024-04-23 18:59:12.049920859 +0000 UTC m=+2.568448754 container died 9d59faab4de67aadffe99232d8ca37fe403ee955156782054c63e9acbc447f02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5) 2024-04-23T18:59:12.682 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:59:12 smithi155 podman[76274]: 2024-04-23 18:59:12.442168562 +0000 UTC m=+2.960696466 container cleanup 9d59faab4de67aadffe99232d8ca37fe403ee955156782054c63e9acbc447f02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:59:12.713 INFO:journalctl@ceph.mds.i.smithi155.stdout:Apr 23 18:59:12 smithi155 bash[76274]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-i 2024-04-23T18:59:12.813 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.i.service' 2024-04-23T18:59:12.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:59:12.914 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-23T18:59:12.947 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-23T18:59:12.980 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-46706838-019d-11ef-bc93-c7b262605968@mds.l 2024-04-23T18:59:13.301 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:59:13 smithi155 systemd[1]: Stopping Ceph mds.l for 46706838-019d-11ef-bc93-c7b262605968... 2024-04-23T18:59:13.577 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:59:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l[63000]: 2024-04-23T18:59:13.303+0000 7ff02594e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:59:13.622 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:59:13 smithi155 ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l[63000]: 2024-04-23T18:59:13.303+0000 7ff02594e700 -1 mds.l *** got signal Terminated *** 2024-04-23T18:59:16.327 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:59:16 smithi155 podman[76395]: 2024-04-23 18:59:16.051040118 +0000 UTC m=+2.863054001 container died c234d349ecda48453c437a2c1a9fbdd1ab54dfb5367500597b2ed31b8ece234d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-04-23T18:59:16.722 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:59:16 smithi155 podman[76395]: 2024-04-23 18:59:16.370855351 +0000 UTC m=+3.182869236 container cleanup c234d349ecda48453c437a2c1a9fbdd1ab54dfb5367500597b2ed31b8ece234d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-23T18:59:16.755 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:59:16 smithi155 bash[76395]: ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l 2024-04-23T18:59:16.805 INFO:journalctl@ceph.mds.l.smithi155.stdout:Apr 23 18:59:16 smithi155 podman[76409]: 2024-04-23 18:59:16.530191388 +0000 UTC m=+0.477812411 container remove c234d349ecda48453c437a2c1a9fbdd1ab54dfb5367500597b2ed31b8ece234d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-46706838-019d-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-23T18:59:16.850 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-46706838-019d-11ef-bc93-c7b262605968@mds.l.service' 2024-04-23T18:59:16.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:59:16.930 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-23T18:59:16.969 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 46706838-019d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T18:59:17.219 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:59:19.699 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 46706838-019d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T18:59:19.975 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:59:22.565 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 46706838-019d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T18:59:22.798 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:59:25.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:59:25.226 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:59:25.284 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:59:25.336 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-23T18:59:25.362 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112/remote/smithi019/crash 2024-04-23T18:59:25.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/crash -- . 2024-04-23T18:59:25.507 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T18:59:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-04-23T18:59:25.612 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112/remote/smithi044/crash 2024-04-23T18:59:25.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/crash -- . 2024-04-23T18:59:25.786 INFO:teuthology.orchestra.run.smithi044.stderr:tar: /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T18:59:25.827 INFO:teuthology.orchestra.run.smithi044.stderr:tar: Error is not recoverable: exiting now 2024-04-23T18:59:25.879 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112/remote/smithi155/crash 2024-04-23T18:59:25.928 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/crash -- . 2024-04-23T18:59:26.032 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/46706838-019d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T18:59:26.069 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-04-23T18:59:26.120 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-23T18:59:26.169 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-23T18:59:26.337 INFO:tasks.cephadm:Compressing logs... 2024-04-23T18:59:26.369 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T18:59:26.421 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T18:59:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-04-23T18:59:26.519 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-23T18:59:26.569 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.log 2024-04-23T18:59:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T18:59:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-23T18:59:26.719 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-23T18:59:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-23T18:59:26.827 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-23T18:59:26.877 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-23T18:59:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-23T18:59:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mgr.x.log: 90.5% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T18:59:27.027 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-23T18:59:27.077 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.log.gz/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mon.a.log: 2024-04-23T18:59:27.127 INFO:teuthology.orchestra.run.smithi019.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T18:59:27.220 INFO:teuthology.orchestra.run.smithi019.stderr: 94.7% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-23T18:59:27.269 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T18:59:27.319 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-23T18:59:27.369 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T18:59:27.419 INFO:teuthology.orchestra.run.smithi019.stderr: 84.6% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T18:59:27.472 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-23T18:59:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-23T18:59:27.571 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T18:59:27.639 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T18:59:27.686 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-23T18:59:27.736 INFO:teuthology.orchestra.run.smithi019.stderr: 94.8% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-23T18:59:27.786 INFO:teuthology.orchestra.run.smithi019.stderr: 94.8% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-23T18:59:27.836 INFO:teuthology.orchestra.run.smithi019.stderr: 90.4% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-23T18:59:27.902 INFO:teuthology.orchestra.run.smithi044.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T18:59:27.944 INFO:teuthology.orchestra.run.smithi044.stderr:: No such file or directory 2024-04-23T18:59:27.995 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.log 2024-04-23T18:59:28.044 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T18:59:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-23T18:59:28.186 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-23T18:59:28.236 INFO:teuthology.orchestra.run.smithi044.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T18:59:28.290 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-23T18:59:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-23T18:59:28.386 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-23T18:59:28.436 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-23T18:59:28.503 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-23T18:59:28.553 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.6% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T18:59:28.606 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T18:59:28.653 INFO:teuthology.orchestra.run.smithi044.stderr: 92.8% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-23T18:59:28.703 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T18:59:28.753 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-23T18:59:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T18:59:28.862 INFO:teuthology.orchestra.run.smithi044.stderr: 84.5% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T18:59:28.912 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-23T18:59:28.980 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-23T18:59:29.087 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T18:59:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.k.log: 93.4% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-23T18:59:29.187 INFO:teuthology.orchestra.run.smithi044.stderr: 94.9% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-23T18:59:29.239 INFO:teuthology.orchestra.run.smithi044.stderr: 93.6% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-23T18:59:29.287 INFO:teuthology.orchestra.run.smithi044.stderr: 92.3% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-23T18:59:29.339 INFO:teuthology.orchestra.run.smithi044.stderr: 91.7% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-23T18:59:29.420 INFO:teuthology.orchestra.run.smithi044.stderr: 93.0% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-23T18:59:29.470 INFO:teuthology.orchestra.run.smithi044.stderr: 92.5% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-23T18:59:29.522 INFO:teuthology.orchestra.run.smithi019.stderr: 90.3% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-23T18:59:29.570 INFO:teuthology.orchestra.run.smithi019.stderr: 92.9% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-23T18:59:29.620 INFO:teuthology.orchestra.run.smithi019.stderr: 92.6% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-23T18:59:29.670 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-23T18:59:29.720 INFO:teuthology.orchestra.run.smithi019.stderr: 91.9% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-23T18:59:29.770 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-23T18:59:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m2.984s 2024-04-23T18:59:29.870 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m12.738s 2024-04-23T18:59:29.921 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m0.393s 2024-04-23T18:59:30.009 INFO:teuthology.orchestra.run.smithi044.stderr: 92.4% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-23T18:59:30.054 INFO:teuthology.orchestra.run.smithi044.stderr: 92.3% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-23T18:59:30.104 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-23T18:59:30.155 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m3.223s 2024-04-23T18:59:30.204 INFO:teuthology.orchestra.run.smithi044.stderr:user 0m14.535s 2024-04-23T18:59:30.271 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m0.441s 2024-04-23T18:59:30.328 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T18:59:30.371 INFO:teuthology.orchestra.run.smithi155.stderr:: No such file or directory 2024-04-23T18:59:30.421 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.log 2024-04-23T18:59:30.471 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose/var/log/ceph/cephadm.log: -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-23T18:59:30.521 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-23T18:59:30.571 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T18:59:30.621 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-23T18:59:30.671 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-23T18:59:30.721 INFO:teuthology.orchestra.run.smithi155.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T18:59:30.771 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-23T18:59:30.821 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-23T18:59:30.927 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-23T18:59:30.996 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mgr.z.log: 88.2% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T18:59:31.046 INFO:teuthology.orchestra.run.smithi155.stderr: 90.6% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T18:59:31.096 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T18:59:31.155 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-23T18:59:31.205 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.4% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T18:59:31.254 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-23T18:59:31.304 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.9% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-23T18:59:31.355 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T18:59:31.405 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-23T18:59:31.455 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.1% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T18:59:31.505 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-23T18:59:31.555 INFO:teuthology.orchestra.run.smithi155.stderr: 93.5% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-23T18:59:31.605 INFO:teuthology.orchestra.run.smithi155.stderr: 93.5% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-23T18:59:31.655 INFO:teuthology.orchestra.run.smithi155.stderr: 94.8% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-23T18:59:31.705 INFO:teuthology.orchestra.run.smithi155.stderr: 92.1% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-23T18:59:31.755 INFO:teuthology.orchestra.run.smithi155.stderr: 92.8% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-23T18:59:31.872 INFO:teuthology.orchestra.run.smithi155.stderr: 92.5% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-23T18:59:31.913 INFO:teuthology.orchestra.run.smithi155.stderr: 92.4% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-23T18:59:31.965 INFO:teuthology.orchestra.run.smithi155.stderr: 92.4% -- replaced with /var/log/ceph/46706838-019d-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-23T18:59:32.030 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-04-23T18:59:32.083 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m3.019s 2024-04-23T18:59:32.130 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m10.473s 2024-04-23T18:59:32.205 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m0.380s 2024-04-23T18:59:32.255 INFO:tasks.cephadm:Archiving logs... 2024-04-23T18:59:32.305 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112/remote/smithi019/log 2024-04-23T18:59:32.356 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T18:59:34.250 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112/remote/smithi044/log 2024-04-23T18:59:34.331 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T18:59:36.306 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112/remote/smithi155/log 2024-04-23T18:59:36.356 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T18:59:38.039 INFO:tasks.cephadm:Removing cluster... 2024-04-23T18:59:38.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 46706838-019d-11ef-bc93-c7b262605968 --force 2024-04-23T18:59:38.330 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:59:38.612 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 46706838-019d-11ef-bc93-c7b262605968 --force 2024-04-23T18:59:38.862 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:59:39.139 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 46706838-019d-11ef-bc93-c7b262605968 --force 2024-04-23T18:59:39.352 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 46706838-019d-11ef-bc93-c7b262605968 2024-04-23T18:59:39.621 INFO:tasks.cephadm:Removing cephadm ... 2024-04-23T18:59:39.646 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T18:59:39.716 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T18:59:39.764 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T18:59:39.825 INFO:tasks.cephadm:Teardown complete 2024-04-23T18:59:39.855 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T18:59:39.920 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T18:59:39.955 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T18:59:40.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T18:59:40.058 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T18:59:40.200 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T18:59:40.286 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T18:59:40.322 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:59:40.374 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:59:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:59:40.489 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:59:40.539 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-23T18:59:40.589 INFO:teuthology.orchestra.run.smithi019.stdout:^- hv01.front.sepia.ceph.com 2 6 377 30 +344us[ +344us] +/- 582ms 2024-04-23T18:59:40.639 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv02.front.sepia.ceph.com 3 7 377 93 -323us[ -323us] +/- 64ms 2024-04-23T18:59:40.689 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 26 +444us[ +444us] +/- 74ms 2024-04-23T18:59:40.739 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:59:40.790 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:59:40.840 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:59:40.890 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:59:40.939 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-23T18:59:40.989 INFO:teuthology.orchestra.run.smithi044.stdout:^- hv01.front.sepia.ceph.com 2 6 377 28 +356us[ +356us] +/- 582ms 2024-04-23T18:59:41.039 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv02.front.sepia.ceph.com 3 7 377 92 -276us[ -276us] +/- 64ms 2024-04-23T18:59:41.089 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 89 +454us[ +454us] +/- 73ms 2024-04-23T18:59:41.164 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:59:41.231 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:59:41.282 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:59:41.331 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-04-23T18:59:41.381 INFO:teuthology.orchestra.run.smithi155.stdout:^- hv01.front.sepia.ceph.com 2 6 377 29 +347us[ +345us] +/- 582ms 2024-04-23T18:59:41.431 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -317us[ -319us] +/- 65ms 2024-04-23T18:59:41.481 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +400us[ +399us] +/- 74ms 2024-04-23T18:59:41.531 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:59:41.581 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T18:59:41.642 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T18:59:41.681 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T18:59:41.742 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T18:59:41.865 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T18:59:41.964 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T18:59:42.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:59:42.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:59:42.165 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:59:42.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:59:42.307 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:59:42.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:59:42.459 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T18:59:42.518 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T18:59:42.568 INFO:teuthology.task.internal:Duration was 3121.701661 seconds 2024-04-23T18:59:42.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T18:59:42.668 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T18:59:42.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T18:59:42.760 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T18:59:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:59:42.857 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T18:59:42.907 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:59:43.001 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:59:43.361 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T18:59:43.440 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-04-23T18:59:43.490 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T18:59:43.605 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-04-23T18:59:43.652 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T18:59:43.770 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-04-23T18:59:43.799 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T18:59:43.884 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T18:59:43.915 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T18:59:43.968 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T18:59:44.018 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T18:59:44.095 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T18:59:44.124 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T18:59:44.176 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T18:59:44.226 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T18:59:44.666 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T18:59:44.710 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T18:59:44.749 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T18:59:44.823 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T18:59:44.874 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T18:59:45.014 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T18:59:45.097 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:59:45.135 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:59:45.228 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-04-23T18:59:45.275 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:59:45.325 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-04-23T18:59:45.398 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-04-23T18:59:45.425 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:59:45.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:59:45.524 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:59:45.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:59:45.624 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:59:45.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:59:45.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T18:59:45.785 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T18:59:45.825 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112/remote/smithi019 2024-04-23T18:59:45.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T18:59:45.988 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112/remote/smithi044 2024-04-23T18:59:46.016 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T18:59:46.108 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112/remote/smithi155 2024-04-23T18:59:46.133 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T18:59:46.224 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T18:59:46.249 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T18:59:46.303 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T18:59:46.352 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T18:59:46.418 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T18:59:46.463 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T18:59:46.500 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T18:59:46.561 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T18:59:46.600 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T18:59:46.652 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T18:59:46.702 INFO:teuthology.orchestra.run.smithi019.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 18:59 /home/ubuntu/cephtest 2024-04-23T18:59:46.750 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T18:59:46.818 INFO:teuthology.orchestra.run.smithi044.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 18:59 /home/ubuntu/cephtest 2024-04-23T18:59:46.883 INFO:teuthology.orchestra.run.smithi155.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 18:59 /home/ubuntu/cephtest 2024-04-23T18:59:46.918 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T18:59:46.979 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T18:59:47.045 INFO:teuthology.nuke:Checking targets against current locks 2024-04-23T18:59:47.099 DEBUG:teuthology.nuke:shortname: smithi019 2024-04-23T18:59:47.125 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T18:59:47.197 DEBUG:teuthology.nuke:shortname: smithi044 2024-04-23T18:59:47.225 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T18:59:47.281 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:01:38.081400', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMtCpsB6fHHOWERpbqC88N84w83JhNicLksVn6GJEKBbciPrWOJ6s7iRJoaCO8meQ+jJVl/9z2r/kPIYr0WeTvI='} 2024-04-23T18:59:47.329 DEBUG:teuthology.nuke:shortname: smithi155 2024-04-23T18:59:47.375 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T18:59:47.429 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:01:38.083181', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSVB0DQq+JmjHsFTVlHc5+vgWW/txHUjTvJWMEyeQ4/gNLBLAMkjMc7qu1eq4qym1XvcczxFj1mWFEIbpRJ8='} 2024-04-23T18:59:47.498 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:01:38.084741', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDctVtOE1ycotqxHuvNAnBa3fEolSGV8vPpwsJsClYmYG3Fkg+E9SKzam0URIPIMw44k4iQ7itHePSrp1slMcE='} 2024-04-23T18:59:47.537 INFO:teuthology.orchestra.console.smithi019:Power off 2024-04-23T18:59:47.575 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T18:59:47.663 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-04-23T18:59:47.692 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:59:47.795 INFO:teuthology.orchestra.console.smithi044:Power off 2024-04-23T18:59:47.841 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T18:59:47.905 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-04-23T18:59:47.963 INFO:teuthology.orchestra.console.smithi155:Power off 2024-04-23T18:59:48.000 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T18:59:48.059 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2024-04-23T18:59:48.100 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:59:48.160 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2024-04-23T18:59:48.200 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:59:48.264 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-23T18:59:48.302 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-04-23T18:59:51.956 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:59:52.119 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-04-23T18:59:52.301 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:59:52.358 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:59:52.461 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-23T18:59:52.511 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-04-23T18:59:56.154 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:59:56.318 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-04-23T18:59:56.484 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:59:56.559 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:59:56.633 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-04-23T18:59:56.701 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2024-04-23T18:59:56.759 INFO:teuthology.orchestra.console.smithi044:Power off completed 2024-04-23T18:59:56.834 INFO:teuthology.orchestra.console.smithi155:Power off completed 2024-04-23T19:00:00.354 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T19:00:00.518 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-04-23T19:00:00.647 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-04-23T19:00:00.779 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 3121.7016611099243 failure_reason: 'error during quiesce thrashing: Error releasing set ''46eff5eb'': 1 (EPERM)' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-23T19:00:00.816 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T19:00:00.951 INFO:teuthology.run:FAIL